# How Ingestion Works Sprinto’s Ingestion framework helps you bring structured data from external systems into Sprinto using ingestion entities, plans, and webhooks. It acts as a configurable intake layer that receives incoming payloads, validates requests, maps data, and processes events into the appropriate destination workflows. Ingestion is designed for teams that need to connect third-party systems, automate event delivery, or ingest operational data into Sprinto without relying on direct integrations. The ingestion workflow is divided into three connected components: * **Entities** define the destination structure where incoming data is stored or interpreted. * **Plans** define how incoming data should be processed. * **Webhooks** act as the receiving endpoint for incoming payloads. These components work together to ensure that external events are securely received, validated, processed, and tracked. *** ### Core Components of Ingestion Sprinto’s ingestion framework is built on three connected components that work together to receive, process, and organise incoming data. #### Entities Entities define the destination structure for incoming data. An entity represents the schema or logical container where ingested information is mapped. Entities help Sprinto understand: * What type of information is being received * Which fields should be mapped * How incoming records should be organised * Where processed data should be stored or referenced Entities act as the data model within the ingestion pipeline. #### Plans Plans define the processing logic applied to incoming payloads. A plan determines how Sprinto interprets, transforms, validates, or routes incoming data after it is received through a webhook. Plans can include: * Mapping logic * Conditional processing rules * Filters * Runtime parameter mapping * Entity associations A webhook can receive events independently, but processing only occurs when a plan is attached. #### Webhooks Webhooks act as the entry point for ingestion. A webhook provides a unique endpoint URL that external systems can send requests to. Webhooks handle: * Incoming payload reception * Authentication validation * Event identification * Rate limiting * Payload validation * Event tracking Together, entities, plans, and webhooks create the ingestion pipeline. *** ### How Ingestion Works At a high level, ingestion follows a request-processing pipeline. 1. An external system sends data to a Sprinto webhook endpoint. 2. Sprinto validates the incoming request. 3. The webhook applies any configured rules and rate limits. 4. A processing plan determines how the payload should be interpreted. 5. The event is mapped to an ingestion entity. 6. Sprinto stores and processes the incoming event. 7. Event activity becomes visible in the webhook’s recent events log. The following sections explain each stage of the ingestion lifecycle. *** ### Step 1: Define an Entity Before incoming data can be processed, Sprinto requires a destination structure. Entities define how incoming information should be represented inside Sprinto. An entity determines: * The type of record being ingested * Field structure and mapping expectations * The destination context for processed data Entities provide the data model that later stages of ingestion reference. *** ### Step 2: Create an Ingestion Plan Once an entity exists, a plan defines how incoming data should be processed. Plans connect business logic to ingestion. A plan can determine: * Which entity should receive data * How fields should be mapped * Which transformations should occur * Which payload conditions should trigger processing Plans serve as the orchestration layer between incoming requests and stored data. *** ### Step 3: Create a Webhook Endpoint A webhook acts as the entry point for incoming data. Each webhook generates a unique endpoint URL that external systems can send data to. Once created, the webhook becomes available under the **Webhooks** tab within the Ingestion module. A webhook includes: * Basic metadata such as name and description * Authentication configuration * Event identification rules * Optional challenge validation * Rate limits * Payload size restrictions Each webhook operates independently and can be configured for different ingestion use cases. *** ### Step 4: Configure Authentication and Validation Before Sprinto accepts incoming data, it validates the request based on the authentication configuration attached to the webhook. Authentication helps ensure that only trusted systems can submit data. Depending on the use case, a webhook can support: * No authentication * Signature-based validation * HMAC SHA256 validation * Custom authentication rules Authentication settings can include: * Signature header mapping * Secret configuration * Timestamp validation * Encoding rules * Payload signing templates If the incoming request fails validation, Sprinto rejects the event before processing begins. *** ### Step 5: Receive and Validate Incoming Payloads Once a request reaches the webhook endpoint, Sprinto validates the payload. During this stage, Sprinto checks: * Whether the webhook exists and is active * Authentication validity * Payload size limits * Request rate thresholds * Event identifier configuration Each webhook can define an event ID extraction rule using JSON configuration. This configuration determines where Sprinto should locate a unique identifier within the incoming request payload. For example, Sprinto may extract an event ID from: * The request body * Headers * Nested JSON paths Event IDs help prevent duplication and improve event tracking. *** ### Step 6: Attach an Ingestion Plan A webhook only receives events. Processing behaviour is controlled through an attached ingestion plan. Plans define what Sprinto should do with incoming payloads. Without an attached plan: * Incoming requests may still be received * Events may be stored for visibility * Payloads are not fully processed into downstream workflows When a plan is attached, Sprinto applies the processing logic configured within that plan. A webhook can attach a plan directly from its configuration page. Plan attachment can include optional settings such as: * Filter expressions * Runtime parameter mappings * Conditional processing rules These controls allow teams to determine which payloads should be processed and how values should be interpreted. *** ### Step 7: Filter and Map Incoming Data After a plan is attached, Sprinto evaluates the incoming payload against any configured filters. Filter expressions help control whether an event should continue through processing. For example, a filter may allow Sprinto to: * Process only specific event types * Ignore low-priority payloads * Accept events from selected systems * Restrict processing to matching attributes Runtime parameter mapping allows values from the payload to be passed dynamically into downstream workflows. This layer provides flexibility when working with varying payload formats. *** ### Step 8: Process Events Through the Ingestion Pipeline Once validation and filtering succeed, Sprinto begins processing the event. The ingestion pipeline performs the following actions: 1. Accepts the payload from the webhook 2. Associates the payload with the configured plan 3. Maps payload data to ingestion entities 4. Executes processing logic 5. Stores event metadata 6. Updates event tracking information Processing outcomes depend on the logic defined in the attached plan. *** ### Step 9: Track Event Activity Each webhook includes a recent events log that helps teams monitor incoming activity. The event log provides visibility into webhook execution history. This includes: * Event status * Event ID * Received timestamp * Processing timestamp * Payload preview This allows teams to: * Verify whether requests were received * Confirm successful processing * Troubleshoot failed events * Validate external integrations Recent events are visible directly within the webhook details page. *** ### Webhook Lifecycle Management Webhook configurations can evolve over time. Sprinto allows you to manage webhook behaviour without recreating the entire ingestion flow. #### Edit a Webhook Webhook settings can be updated after creation. You can modify: * Basic information * Authentication configuration * Event ID configuration * Challenge configuration * Rate limits * Maximum payload size Changes take effect after saving the updated configuration. #### Rotate Webhook ID Webhook URLs can be rotated when endpoint security requires regeneration. Rotating a webhook ID creates a new endpoint URL. After rotation: * Existing webhook URLs stop working immediately * External systems must be updated with the new endpoint * Future requests must use the rotated webhook URL #### Delete a Webhook A webhook can be permanently removed if it is no longer required. Deleting a webhook removes the endpoint and prevents future requests from being processed. Deletion cannot be undone. *** ### How the Components Work Together The ingestion framework depends on the relationship between entities, plans, and webhooks.
ComponentPurpose
WebhookReceives incoming requests
PlanDefines processing behaviour
EntityRepresents the destination structure for ingested data
Together, these components create a controlled data ingestion pipeline that enables Sprinto to receive, validate, process, and track external events. *** ### Relationship Between Entities, Plans, and Webhooks The ingestion workflow relies on the relationship between all three components. 1. An entity defines where data belongs. 2. A plan defines how data should be processed. 3. A webhook receives the incoming request. 4. Sprinto validates and processes the payload. 5. Processed data is mapped into the configured entity. This separation allows teams to reuse entities and plans across multiple ingestion scenarios while maintaining flexible webhook configurations. *** ### Typical Ingestion Flow A typical ingestion workflow follows this sequence: 1. Create a webhook endpoint. 2. Configure authentication and validation rules. 3. Share the webhook URL with an external system. 4. Attach an ingestion plan. 5. Receive incoming events. 6. Validate payloads. 7. Apply filters and mappings. 8. Process events. 9. Track event activity through recent logs. This architecture allows Sprinto to support flexible ingestion scenarios while maintaining validation, traceability, and processing control. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.sprinto.com/ingestion/how-ingestion-works.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.