Webhook configuration

What is a webhook?

A webhook is an asynchronous communication method that sends a one-way notification using an HTTP service. It is triggered by an event and sends information without an initial client or user request.

To function, a webhook must be subscribed to a specific topic. When the event occurs, the webhook receives and processes the information.

Example Use Case in OneStock: When an order changes state (e.g., from placed to shipped), instead of continuously polling an API, a webhook sends a notification of the state change directly to the user.

 

webhooks.png

 

 

Implementing a Webhook in OneStock

OneStock OMS requires certain configurations and parameters to be set before a webhook can be used. A legitimacy check can also be added as a final step. This is optional, but highly recommended. We discuss each of these steps and requirements below.

In order to configure webhooks in OneStock you will need to complete the 3 following steps:

  1. Configure your site for webhook notifications

  2. Set up a webhook

  3. Subscribe your webhook to a topic

 

1. Configure your site for webhooks

Your site configuration sets the following parameters:

  1. The retry delay before a temporary or complete stop

  2. A contact in case of an error

  3. The name of the notification to send in the error message

A template that can be used for configuring at Order Management Center/Configurations/Outbound Messages/Webhooks

{ "retry_intervals": [ 30, 60, 120, 240, 480, 840 ], "retries_until_failure": 3, "on_failure": { "contact_emails": [ "email@company.com" ], "contact_mobiles": [], "sms_notification_name": "", "email_notification_name": "webhook_failure" }, "on_deactivation": { "contact_emails": [ "email@company.com" ], "contact_mobiles": [], "sms_notification_name": "", "email_notification_name": "webhook_deactivation" }, "on_failure_recovered": { "contact_emails": [ "email@company.com" ], "contact_mobiles": [], "sms_notification_name": "", "email_notification_name": "webhook_failure_recovered" } }
image-20241007-162257.png

 

The default retry delay values are set as shown above in seconds. In this instance the webhook would make the first call and if no response or confirmation is received then it will retry 6 times, first after 30 seconds, then 60, then 2 minutes, 4 minutes, 8 minutes and finally 16 minutes. However, after the 3rd failed attempt, i.e. one with no response, it will send a notification of failure to the contact details in the configuration.

If a response or confirmation is received before the 6th call then the webhook will send another notification, as it is now classed as ‘recovered’. The retry counter will then be reset at 0.

2. Set Up a Webhook

Creating a webhook is mostly done via API (use the POST /webhooks route) however some options can be set via our new configuration screens. These are shown later.

The configuration of the webhook defines important information such as which topic to subscribe to and the HTTP method to connect.

There are three mandatory fields required for webhooks with OneStock, they are:

  1. Your Site ID

  2. The HTTP method

  3. The destination URL

Your OneStock contact will be able to provide with your Site ID if you do not know it. It will be a series of numbers following a lower case 'c', for example c404.

OneStock uses RESTful APIs so the HTTP method can be GET XGET POST PATCH PUT or DELETE.

The destination URL will be the address where the webhook will send its requests.

3. Subscribe your webhook to a topic

To listen to OneStock events you will subcribe your webhook to a topic. We distinguish to types of topics, standard and custom. Standard topics work out of the box, you just need to sucribe your webhook to one of them and you will start receving all events that go through the topic event stream. Custom topics need slightly more configuration and allow you to listen to specific events that you will trigger trough your workflow.

Standard topics

1. Subcribe a webhook to a standard topic

To listent to topics, you simply need to subscribe a webhook to one or multiple topics, through a

POST /webhooks call.

POST /webhooks

{ "site_id": "MySiteID", "token":"token", "webhook": { "hash_key": "personnal_hash_key", "http_method": "POST", "status": "enabled", "topics": [ "order_state_changed", "parcel_state_changed", "line_item_group_state_changed" ], "url": "https://myWebhookURL" //you can generate a disposbel one at https://webhook.site/ } }

 

Following a list of all topics we currently support. We may add more at any time, so in developing and maintaining your code, you should not assume that only these topics exist.

Topic

Displayed Name

Description

Message structure

Topic

Displayed Name

Description

Message structure

buffer_import_completed

Buffer Import Completed

Notifies when an async buffer import using API or file is completed

buffer_import_completed

import_id: string status: string

12.4

buffer_import_error_occurred

Buffer Import Error Occurred

Notifies when a synchronous buffer import using API POST method fails

buffer_import_error_occurred

12.4

async_buffer_import_error_occurred

Asynchronous Buffer Import Error Occurred

Notifies when an asynchronous buffer import fails

async_buffer_import_error_occurred

… full list bellow

 

 

 

Topic

Displayed Name

Description

Message structure

Topic

Displayed Name

Description

Message structure

buffer_import_completed

Buffer Import Completed

Notifies when an async buffer import using API or file is completed

buffer_import_completed

12.4

buffer_import_error_occurred

Buffer Import Error Occurred

Notifies when a synchronous buffer import using API POST method fails

buffer_import_error_occurred

12.4

async_buffer_import_error_occurred

Asynchronous Buffer Import Error Occurred

Notifies when an asynchronous buffer import fails

async_buffer_import_error_occurred

customer_import_completed

Customer Import Completed

Notifies when an async customer import using API or file is completed

customer_import_completed

12.4

customer_import_error_occurred

Customer Import Error Occurred

Notifies when a synchronous customer import using API POST method fails

customer_import_error_occurred

12.4

async_customer_import_error_occurred

Asynchronous Customer Import Error Occurred

Notifies when a asynchronous customer import fails

async_customer_import_error_occurred

transfer_import_completed

Transfer Import Completed

Notifies when an async transfer import using API or file is completed

transfer_import_completed

transfer_import_error_occurred

Transfer Import Error Occurred

Notifies when a synchronous transfer import using API POST method fails

transfer_import_error_occurred

12.4

async_customer_import_error_occurred

Asynchronous Transfer Import Error Occurred

Notifies when an asynchronous transfer import fails

async_transfer_import_error_occurred

stock_import_completed

Stock Import Completed

Notifies when an async stock import using API or file is completed

stock_import_completed

stock_import_error_occurred

Stock Import Error Occurred

Notifies when a synchronous stock import using API POST method fails

stock_import_error_occurred

12.4

async_stock_import_error_occurred

Asynchronous Stock Import Error Occurred

Notifies when an asynchronous stock import fails

async_stock_import_error_occurred

line_item_group_state_changed

Line Item Group State Change

Notifies of a state change for line item groups

line_item_group_state_changed

 

order_state_changed

Order State Change

Notifies of a state change for orders

order_state_changed

 

parcel_state_changed

Parcel State Change

Notifies of a state change for parcels

parcel_state_changed

 

entity_updated

Entity Update

Notifies of OMS entities information changes (parcels, line item groups and orders) POP (endpoint_orders, containers, piece_groups) and RM (return_parcels, and return_line_item_groups)

line_item_group_entity_updated, order_entity_updated, parcel_entity_updated

orchestration_rules_changed

Orchestration Rules Changed

Allows triggering of preparation order on time.

When there is an orchestration rule change with a warehouse custom claim action triggered, line item groups are retrieved and a preparation order is sent to the warehouse.

orchestration_rules_changed

 

candidates_added

Candidates Added

Notifies that an endpoint is candidate for an order.

candidates_added

 

candidates_removed

Candidates Removed

Notifies when an endpoint is no longer candidate for an order.

candidates_removed

 

rules_over Before 12.4

orchestration_rules_over 12.4

Rules Over (custom notification)

Notifies of a rules over transition for an order.

orchestration_rules_over 12.4 (new version more accurate than rules_over)

rules_over

back_from_rules_over

 

item_import_completed

Item Import Completed

Notifies of a completed item import

item_import_completed

12.4

item_import_error_occurred

Item Import Error Occurred

Notifies when a synchronous item import using API POST /items or a PATCH /items method fails

item_import_error_occurred

12.4

async_item_import_error_occurred

Asynchronous Item Import Error Occurred

Notifies when an asynchronous item import fails

async_item_import_error_occurred

psp_error_occurred

Payment Error

Notifies of errors during payment treatment

psp_error_occurred

container_state_changed

operator_state_changed

endpoint_order_state_changed

piece_group_state_changed

Order Preparation Entity Change

Notifies of order preparation entity state changes (creation, update, removal)

  • container_state_changed, operator_state_changed, endpoint_order_state_changedand piece_group_state_changed

return_parcel_created

Return Parcel Creation

Notifies of the creation of a return parcel

return_parcel_created :

return_parcel_created@v3 :

 

return_line_item_group_state_changed

return_parcel_state_changed

 

Notifies of a state change for a return return_line_item_group or return_parcel

return_line_item_group_state_changed and return_parcel_state_changed

 

line_items_reservations_updated

Line Items Reservations Update

Notifies of a reservation update on a line item group (global reservation to endpoint reservation, future stock to on hand, endpoint reservation removal)

line_items_reservations_updated

 

stock_coverage_import_completed

Stock Coverage Import Completed

Notifies of a completed stock coverage import

stock_coverage_import_completed

12.4

stock_coverage_import_error_occurred

Stock Coverage Import Error Occurred

Notifies when a synchronous stock coverage import using API POST method fails

stock_coverage_import_error_occurred

12.4

async_stock_coverage_import_error_occurred

Asynchronous Stock Coverage Import Error Occurred

Notifies when an asynchronous stock coverage import fails

async_stock_coverage_import_error_occurred

stock_disposition_import_completed

Stock Disposition Import Completed

Notifies of a completed stock disposition import

stock_disposition_import_completed

12.4

stock_disposition_import_error_occurred

Stock Disposition Import Error Occurred

Notifies when a synchronous stock disposition import using API POST method fails

stock_disposition_import_error_occurred

12.4

async_stock_disposition_import_error_occurred

Asynchronous Stock Disposition Import Error Occurred

Notifies when an asynchronous stock disposition import fails

async_stock_disposition_import_error_occurred

endpoint_import_completed

Endpoint Import Completed

Notifies of a completed stock import

endpoint_import_completed

12.4

endpoint_import_error_occurred

Endpoint Import Error Occurred

Notifies when a synchronous endpoint import using API POST method fails

endpoint_import_error_occurred

12.4

async_endpoint_import_error_occurred

Asynchronous Endpoint Import Error Occurred

Notifies when an asynchronous endpoint import fails

async_endpoint_import_error_occurred

user_import_completed

User Import Completed

Notifies of a completed user import

user_import_completed

12.4

user_import_error_occurred

User Import Error Occurred

Notifies when a synchronous user import using API POST method fails

user_import_error_occurred

12.4

async_user_import_error_occurred

Asynchronous User Import Error Occurred

Notifies when an asynchronous user import fails

async_user_import_error_occurred

carrier_error_occurred

Carrier Error

Relays of errors during carrier communication

carrier_error_occurred

tracking_link_created

 

Notifies of the creation of a tracking link

tracking_link_created

shipment_created

 

Notifies of the creation of a shipment

shipment_created

stock_export_completed

Stock Export Completed

Notifies the completion of a stock export by file (SFTP)

File name and other information regarding the import can be retrieved through a

2. Test that all works correctly

To do so, you can simply create a disposabel webhook endpoint going to https://webhook.site/ and use it as the url of your topic (by clicking on edit you can change the response code do 202, only 100 webhook receptions are allowed for free). In the case of our example, when a parcel transitions from newbagged , as when bagged in the Store App, we should receive a body as the following

Custom topics

If the event you wish to subscribe to is not listed above you can create a custom topic. The custom topic can either notify based on an existing OneStock events, or based on an event triggered in the workflow.

Custom topics can be created via API, or via the Back Office Configuration screens.

1. Create a custom notification

Go to Configuration > Outbound Messages > Notifications, and add a custom notification. Base yourself on the following code example and tailor it to your need by modifing the id and the topic id. Leave the rest unchanged.

2. Create the custom topic

In order to configure a webhook via API the topic must be set using the route POST /webhook_topics as shown below.

POST /webhook_topics

3. Subcribe a webhook to the custom topic

We must add a listener to the custom topic. To do so, we just need to create a webhook using the route POST /webhooks

POST /webhooks

4. Trigger the custom notification from the workflow

Add a send notification event in the transition that must trigger the custom notification, an set the custom name, in our case, email_sent_to_client

Video walkthrough

5. Test that all works correctly

Upon receiving a message via webhook, it is essential to return a 202 Accepted response promptly (within 15 seconds, but ideally under 300 ms) after verifying the message’s legitimacy (optional but recommended)

To do so, you can simply create a disposabel webhook endpoint going to https://webhook.site/ and use it as the url of your topic (by clicking on edit you can change the response code do 202, only 100 webhook receptions are allowed for free). In the case of our example, when a parcel transitions from newbagged , as when bagged in the Store App, we should receive a body as the following

4. Acknowledgment of Webhook Receipt

Upon receiving a message via webhook, it is essential to return a 202 Accepted response promptly (within 15 seconds, but ideally under 300 ms) after verifying the message’s legitimacy (optional but recommended)

Upon receiving a message via webhook, it is essential to return a 202 Accepted response promptly (within 15 seconds, but ideally under 300 ms) after verifying the message’s legitimacy (optional but recommended). If this acknowledgment is not received in time, the system will retry sending the message according to the retry_intervals configuration, if retries are not acknowledged under 15 seconds.

To maintain synchronization between OneStock and your service, please acknowledge incoming webhook messages immediately. Delays can cause desynchronization and data inconsistencies. We recommend processing messages asynchronously and sending acknowledgments promptly to ensure smooth integration.

Legitimacy Check

While optional, OneStock strongly recommends performing a legitimacy check to ensure that the webhook message is genuinely sent by the OneStock system.

Each webhook message includes a signature in the header with the following format: timestamp + "," + signature.

During webhook setup (POST /webhooks), a hash key is either provided by the client or generated by OneStock. This key, referred to as h0, is used to encrypt the signature. OneStock retains the three most recent hash keys (h0, h1, h2) to support ongoing signature verification.

The signature format is:
t=timestamp,h0=h[0],h1=h[1],h2=h[2], where:

  • timestamp: The current timestamp.

  • h[0]: The signature encrypted using the latest hash key.

  • h[1]: The signature encrypted using the previous hash key.

  • h[2]: The signature encrypted using the oldest hash key.

Examples of signature and verification code

Current timestamp = 1704092400

Encryption of h0 for 1704092400 = 1234 (encryption of the timestamp with the new key)

Encryption of h1 for 1704092400= 9876 (encryption of the timestamp with the previous key)

The signature will be the following : t=1704092400,h0=1234,h1=9876

Resulting the header : Onestock-Signature=t=1704092400,h0=1234,h1=9876

Webhook Acknowledgment Best Practices

To avoid blocking the entire message queue when processing webhooks, it is crucial to focus on the signature validation first and handle any additional checks asynchronously.

Recommended Acknowledgment Process:

  • Signature Validation:

    • Objective: Ensure the message is legitimately sent by OneStock.

    • Action: If the signature is valid, immediately return an HTTP 202 Accepted response.

  • Asynchronous Content Validation (if applicable):

    • Any additional checks (e.g., content validation) should be performed asynchronously after acknowledging the message. This prevents delays in processing subsequent webhooks and ensures a smooth message flow.

Key Benefits

  • Prevent Queue Blocking: By acknowledging based solely on signature validation, the message queue remains smooth and uninterrupted.

  • Efficient Processing: Additional content checks can be handled asynchronously, avoiding potential bottlenecks.

5. Retries, Error Management and Alerts

If acknowledgment is not received within the configured time frame, the system will retry sending the message. Notifications regarding errors can be sent via SMS, email, or both.

Events

  1. on_failure: Triggered after the maximum number of retries (retries_until_failure) is reached without acknowledgment. Retries count do not include the original message (the first one which was not ackowledged).

  2. on_failure_recovered: Triggered when a failed message is later acknowledged during a retry. The retry count resets to 0.

  3. on_deactivation: Triggered when the maximum retry attempts are exhausted without recovery. The webhook is automatically disabled and must be manually reactivated. Messages are stored while the webhook is in a disabled state upto a week. After a week, the webhook is considered dead, and all messages are erased.

To resolve these errors, ensure your web service is functional and restart the webhook by setting it’s status to enabled with a PATCH /webhooks/:id/status.

 

Alerts configuration

Emails and SMS are sent to notify webhook events stated above.

This are configured in the site configuration

Alert trigger example

 

With the configuration described, here's how the retry process works:

  1. Event 1 Fails Initial Acknowledgment: If Event 1 is not acknowledged within 15 seconds, the webhook status changes to paused, and a retry is scheduled for 30 seconds later.

  2. Retry 1: If the first retry is not acknowledged within 15 seconds, a second retry is triggered after 60 seconds.

  3. Retry 2: If the second retry fails to get acknowledged within 15 seconds, a third retry is scheduled for 120 seconds later.

  4. Retry 3 (Final Attempt Before Alert): If the third retry is also not acknowledged within 15 seconds, an "on failure" alert is triggered because the number of retries has reached the configured retries_until_failure. The webhook remains in paused status.

  5. Subsequent Retries:

    • A fourth retry is scheduled after 240 seconds. If acknowledged successfully, an "on failure recovered" alert is triggered, and the webhook status changes to enabled.

    • If retries continue to fail, and the sixtieth retry does not get acknowledged, the webhook will be disabled, and an "on deactivation" alert will be triggered.

 

6. Webhook Statuses

  • enabled: The default state after creation. Messages are processed normally, triggering the webhook.

  • paused: Messages are stored while the webhook is in paused state upto a week. After a week, the webhook is considered dead, and all messages are erased.
    The behavior of this state depends on the scenario:

    • Manual pause: No messages will be sent until the webhook is manually re-enabled.

    • Automatic pause: This occurs when a message is not acknowledged by the receiver within 15 seconds. In this case, the message will be retried. If the message fails to be acknowledged after the configured number of retries (retries_until_failure), an on_failure notification will be triggered (via email and/or SMS, depending on configuration). If the maximum number of retries is reached without success, the webhook will be disabled. However, if the message is acknowledged before reaching the retry limit, the webhook will automatically resume sending messages (enabled).

  • disabled: The webhook wont be triggered until enabled. Messages are stored while the webhook is in a disabled state upto a week. After a week, the webhook is considered dead, and all messages are erased.

7. Message storage

  • In the case of paused and disabled webhooks, messages are stored for a week; after a week, all messages will be automatically deleted. If a webhook is no re-enabled under a week, all messages will be lost.

  • In the case of enabled webhooks, messages are stored for a week; messages older than a week will get deleted. If desynchronization between OneStock and Client is greater than a week, old messages will be lost.

8. Integration Requirements

1. Process Messages as Quickly as Possible

Acknowledging webhook messages within 15 seconds is essential, but processing them as fast as possible ensures your system stays in sync with OneStock’s real-time events.

Why It Matters

  • Prevent Backlogs: If messages are processed just under the 15-second limit and events occur more frequently, a backlog will form, causing delays.

  • Avoid Desync: Since events are processed in order, slow processing can lead to discrepancies. For example, if events occur every 1 second and your system processes messages every 14 seconds, after an hour, only 257 out of 3600 events will be handled, creating a 55-minute delay.

  • Improve User Experience: Faster processing ensures that user-facing features and data remain accurate and up-to-date.

Tips for Faster Processing

  • Decouple Processing from Acknowledgment: Use a queuing system to instantly acknowledge messages and process them asynchronously.

  • Optimize Handling: Keep message processing efficient and lightweight.

  • Scale Dynamically: Use auto-scaling to handle traffic spikes.

  • Monitor Performance: Set alerts when processing times approach the 15-second limit.

2. Handling Idempotency in Webhook Integration

When integrating with OneStock's webhook system, it is crucial that your webhook client handles idempotency. OneStock expects all webhook clients to process messages in an idempotent manner to ensure consistency and reliability.

What Is Idempotency?

Idempotency refers to the property of certain operations where executing them multiple times yields the same result as executing them once. In the context of webhooks, this means that if the same message is received more than once, processing it multiple times does not cause unintended side effects or data inconsistencies.

Why Is Idempotency Important?

Due to network issues or other transient problems, a message sent by OneStock might not be acknowledged by your system. In such cases, OneStock will retry sending the same message to ensure it has been received and processed.

If your webhook client does not handle idempotency:

  • Duplicate processing: The same operation might be performed multiple times (e.g., applying the same order change repeatedly), leading to inconsistent data or errors.

  • Integration breakdowns: Critical systems like your ERP might encounter conflicts or errors due to repeated operations, potentially disrupting business processes.

How to Handle Idempotency

Use the message-id Header

Each webhook message sent by OneStock includes a unique message-id in the HTTP headers. This identifier is essential for ensuring idempotent processing.

Implement Message Queueing and Asynchronous Processing
  1. Acknowledge Immediately: Upon receiving a webhook message, your system should send an acknowledgement to OneStock immediately.

  2. Enqueue the Message: Store the message-id and the message payload in a reliable queue for asynchronous processing.

  3. Process Asynchronously: Use a separate worker or process to consume messages from the queue and handle the idempotency checks.

Ensure Idempotency in Asynchronous Processing
  • Store processed message IDs: Maintain a persistent record of all processed message-ids.

  • Check before processing: Before processing a message from the queue, check if its message-id exists in your records.

    • If it exists: Skip processing.

    • If it does not exist: Process the message and then store its message-id.

Ensure Atomic Operations

To prevent race conditions:

  • Atomic transactions: Combine the message processing and the storage of the message-id into a single atomic transaction.

  • Thread safety: If your application processes messages in parallel, ensure that access to the message ID store is thread-safe.

Example Implementation

Webhook Handler Function

Asynchronous Message Processor

Acknowledging Messages

  • Successful processing: Since you acknowledge messages immediately, ensure that your asynchronous processing handles failures internally without relying on OneStock to retry.

  • Failed processing: Implement retry mechanisms within your message processor for transient errors.

  • Network failures: Immediate acknowledgement reduces the chance of network issues causing message retries from OneStock.

Advantages of This Approach

  • Improved performance: Acknowledging immediately reduces response time and prevents timeouts.

  • Scalability: Asynchronous processing allows your system to handle a higher volume of messages efficiently.

  • Reliability: Queueing messages ensures they are not lost and can be retried in case of transient failures.

Summary

  • Handle retries gracefully: Design your webhook client to process duplicate messages without adverse effects.

  • Use message-id for idempotency: Leverage the unique message-id header provided by OneStock for tracking.

  • Process asynchronously: Acknowledge messages immediately and process them in a separate worker or process.

  • Prevent duplicate operations: Ensure that repeated messages do not cause inconsistent states or errors in your systems.

8. Integration Best practices

Use a Queuing System to Receive Messages

To ensure smooth and reliable communication, it's essential to acknowledge webhook messages swiftly—ideally within 15 seconds—by returning a 202 response code. This approach ensures that OneStock knows your system has received the message, allowing the process to continue without delays.

Why Use a Queuing System?

  • Immediate Acknowledgment: By decoupling message reception from processing, you can instantly acknowledge the receipt of the message. The queue receives and stores the message, allowing your system to process it asynchronously without holding up the response.

  • Enhanced Reliability: A queuing system provides a buffer, ensuring that even if your system is temporarily down or under heavy load, messages are not lost. They will be processed once your system is ready.

  • Easier Troubleshooting: Queuing services often come with tools that allow you to view and manage received messages, making it easier to diagnose and resolve any communication issues.

Recommended Queuing Solutions:

  • AWS SQS (Simple Queue Service)

  • Google Cloud Pub/Sub

  • Apache Kafka

Log Message Reception

If you choose not to use a queuing system, it's vital to implement robust logging of all received messages. Logging should include the message content and the response sent back to OneStock.

Benefits of Logging:

  • Monitoring and Auditing: Detailed logs provide a trail of received messages, which is crucial for monitoring system performance and ensuring that all messages are accounted for.

  • Incident Response: In the event of an issue, logs enable you to quickly identify and address the problem, minimizing downtime and maintaining communication integrity.

  • Compliance and Debugging: Logs can also serve as an essential resource for compliance and debugging purposes, helping you ensure that your system meets all necessary requirements and functions as expected.