Event-based triggers
A trigger listens on a source you authorize (bucket notification, queue subscription, or event rule) and maps each event to a pipeline run. Configuration typically includes:- Resource ARN, bucket name, or project path
- Prefix and suffix filters (for example,
landing/orders/and.parquet) - Batch windowing (wait N seconds or M files before starting) to avoid one run per tiny object
- Optional payload passthrough so the pipeline reads the triggering key list
Exact event shapes differ by cloud. Use the connection test to validate that Planasonix receives notifications end to end.
Cloud event ingestion
- Amazon S3
- Google Cloud Storage
- Azure Blob Storage
Configure SNS or SQS notifications on
s3:ObjectCreated:* (and related events your policy allows). Grant the Planasonix integration principal permission to read affected objects and decrypt with KMS keys when used.Operational guidance
Idempotent loads
Idempotent loads
The same file may notify more than once. Use file ETags, manifest tables, or merge keys so duplicates do not double-count facts.
Ordering
Ordering
Object creation order is not guaranteed across partitions. If ordering matters, encode sequence metadata in paths or use a single partition strategy.
Security
Security
Scope triggers to prefixes used only by trusted producers. Public buckets should not feed production pipelines.
Related topics
Webhooks
HTTP-triggered runs from apps that do not emit cloud events.
Orchestration overview
How scheduling and triggers fit together.