How Ingestion Works
Learn how Ingestion works in Sprinto, including how entities, plans, and webhooks work together to receive, validate, process, and track incoming data from external systems.
Sprinto’s Ingestion framework helps you bring structured data from external systems into Sprinto using ingestion entities, plans, and webhooks. It acts as a configurable intake layer that receives incoming payloads, validates requests, maps data, and processes events into the appropriate destination workflows.
Ingestion is designed for teams that need to connect third-party systems, automate event delivery, or ingest operational data into Sprinto without relying on direct integrations.
The ingestion workflow is divided into three connected components:
Entities define the destination structure where incoming data is stored or interpreted.
Plans define how incoming data should be processed.
Webhooks act as the receiving endpoint for incoming payloads.
These components work together to ensure that external events are securely received, validated, processed, and tracked.
Core Components of Ingestion
Sprinto’s ingestion framework is built on three connected components that work together to receive, process, and organise incoming data.
Entities
Entities define the destination structure for incoming data.
An entity represents the schema or logical container where ingested information is mapped.
Entities help Sprinto understand:
What type of information is being received
Which fields should be mapped
How incoming records should be organised
Where processed data should be stored or referenced
Entities act as the data model within the ingestion pipeline.
Plans
Plans define the processing logic applied to incoming payloads.
A plan determines how Sprinto interprets, transforms, validates, or routes incoming data after it is received through a webhook.
Plans can include:
Mapping logic
Conditional processing rules
Filters
Runtime parameter mapping
Entity associations
A webhook can receive events independently, but processing only occurs when a plan is attached.
Webhooks
Webhooks act as the entry point for ingestion.
A webhook provides a unique endpoint URL that external systems can send requests to.
Webhooks handle:
Incoming payload reception
Authentication validation
Event identification
Rate limiting
Payload validation
Event tracking
Together, entities, plans, and webhooks create the ingestion pipeline.
How Ingestion Works
At a high level, ingestion follows a request-processing pipeline.
An external system sends data to a Sprinto webhook endpoint.
Sprinto validates the incoming request.
The webhook applies any configured rules and rate limits.
A processing plan determines how the payload should be interpreted.
The event is mapped to an ingestion entity.
Sprinto stores and processes the incoming event.
Event activity becomes visible in the webhook’s recent events log.
The following sections explain each stage of the ingestion lifecycle.
Step 1: Define an Entity
Before incoming data can be processed, Sprinto requires a destination structure.
Entities define how incoming information should be represented inside Sprinto.
An entity determines:
The type of record being ingested
Field structure and mapping expectations
The destination context for processed data
Entities provide the data model that later stages of ingestion reference.
Step 2: Create an Ingestion Plan
Once an entity exists, a plan defines how incoming data should be processed.
Plans connect business logic to ingestion.
A plan can determine:
Which entity should receive data
How fields should be mapped
Which transformations should occur
Which payload conditions should trigger processing
Plans serve as the orchestration layer between incoming requests and stored data.
Step 3: Create a Webhook Endpoint
A webhook acts as the entry point for incoming data.
Each webhook generates a unique endpoint URL that external systems can send data to. Once created, the webhook becomes available under the Webhooks tab within the Ingestion module.
A webhook includes:
Basic metadata such as name and description
Authentication configuration
Event identification rules
Optional challenge validation
Rate limits
Payload size restrictions
Each webhook operates independently and can be configured for different ingestion use cases.
Step 4: Configure Authentication and Validation
Before Sprinto accepts incoming data, it validates the request based on the authentication configuration attached to the webhook.
Authentication helps ensure that only trusted systems can submit data.
Depending on the use case, a webhook can support:
No authentication
Signature-based validation
HMAC SHA256 validation
Custom authentication rules
Authentication settings can include:
Signature header mapping
Secret configuration
Timestamp validation
Encoding rules
Payload signing templates
If the incoming request fails validation, Sprinto rejects the event before processing begins.
Step 5: Receive and Validate Incoming Payloads
Once a request reaches the webhook endpoint, Sprinto validates the payload.
During this stage, Sprinto checks:
Whether the webhook exists and is active
Authentication validity
Payload size limits
Request rate thresholds
Event identifier configuration
Each webhook can define an event ID extraction rule using JSON configuration.
This configuration determines where Sprinto should locate a unique identifier within the incoming request payload.
For example, Sprinto may extract an event ID from:
The request body
Headers
Nested JSON paths
Event IDs help prevent duplication and improve event tracking.
Step 6: Attach an Ingestion Plan
A webhook only receives events. Processing behaviour is controlled through an attached ingestion plan.
Plans define what Sprinto should do with incoming payloads.
Without an attached plan:
Incoming requests may still be received
Events may be stored for visibility
Payloads are not fully processed into downstream workflows
When a plan is attached, Sprinto applies the processing logic configured within that plan.
A webhook can attach a plan directly from its configuration page.
Plan attachment can include optional settings such as:
Filter expressions
Runtime parameter mappings
Conditional processing rules
These controls allow teams to determine which payloads should be processed and how values should be interpreted.
Step 7: Filter and Map Incoming Data
After a plan is attached, Sprinto evaluates the incoming payload against any configured filters.
Filter expressions help control whether an event should continue through processing.
For example, a filter may allow Sprinto to:
Process only specific event types
Ignore low-priority payloads
Accept events from selected systems
Restrict processing to matching attributes
Runtime parameter mapping allows values from the payload to be passed dynamically into downstream workflows.
This layer provides flexibility when working with varying payload formats.
Step 8: Process Events Through the Ingestion Pipeline
Once validation and filtering succeed, Sprinto begins processing the event.
The ingestion pipeline performs the following actions:
Accepts the payload from the webhook
Associates the payload with the configured plan
Maps payload data to ingestion entities
Executes processing logic
Stores event metadata
Updates event tracking information
Processing outcomes depend on the logic defined in the attached plan.
Step 9: Track Event Activity
Each webhook includes a recent events log that helps teams monitor incoming activity.
The event log provides visibility into webhook execution history.
This includes:
Event status
Event ID
Received timestamp
Processing timestamp
Payload preview
This allows teams to:
Verify whether requests were received
Confirm successful processing
Troubleshoot failed events
Validate external integrations
Recent events are visible directly within the webhook details page.
Webhook Lifecycle Management
Webhook configurations can evolve over time.
Sprinto allows you to manage webhook behaviour without recreating the entire ingestion flow.
Edit a Webhook
Webhook settings can be updated after creation.
You can modify:
Basic information
Authentication configuration
Event ID configuration
Challenge configuration
Rate limits
Maximum payload size
Changes take effect after saving the updated configuration.
Rotate Webhook ID
Webhook URLs can be rotated when endpoint security requires regeneration.
Rotating a webhook ID creates a new endpoint URL.
After rotation:
Existing webhook URLs stop working immediately
External systems must be updated with the new endpoint
Future requests must use the rotated webhook URL
Delete a Webhook
A webhook can be permanently removed if it is no longer required.
Deleting a webhook removes the endpoint and prevents future requests from being processed.
Deletion cannot be undone.
How the Components Work Together
The ingestion framework depends on the relationship between entities, plans, and webhooks.
Webhook
Receives incoming requests
Plan
Defines processing behaviour
Entity
Represents the destination structure for ingested data
Together, these components create a controlled data ingestion pipeline that enables Sprinto to receive, validate, process, and track external events.
Relationship Between Entities, Plans, and Webhooks
The ingestion workflow relies on the relationship between all three components.
An entity defines where data belongs.
A plan defines how data should be processed.
A webhook receives the incoming request.
Sprinto validates and processes the payload.
Processed data is mapped into the configured entity.
This separation allows teams to reuse entities and plans across multiple ingestion scenarios while maintaining flexible webhook configurations.
Typical Ingestion Flow
A typical ingestion workflow follows this sequence:
Create a webhook endpoint.
Configure authentication and validation rules.
Share the webhook URL with an external system.
Attach an ingestion plan.
Receive incoming events.
Validate payloads.
Apply filters and mappings.
Process events.
Track event activity through recent logs.
This architecture allows Sprinto to support flexible ingestion scenarios while maintaining validation, traceability, and processing control.
Last updated