Webhook configuration
What is a webhook?
A webhook is an asynchronous communication method that sends a one-way notification using an HTTP service. It is triggered by an event and sends information without an initial client or user request.
To function, a webhook must be subscribed to a specific topic. When the event occurs, the webhook receives and processes the information.
Example Use Case in OneStock: When an order changes state (e.g., from placed to shipped), instead of continuously polling an API, a webhook sends a notification of the state change directly to the user.
Implementing a Webhook in OneStock
OneStock OMS requires certain configurations and parameters to be set before a webhook can be used. A legitimacy check can also be added as a final step. This is optional, but highly recommended. We discuss each of these steps and requirements below.
In order to configure webhooks in OneStock you will need to complete the 3 following steps:
Configure your site for webhook notifications
Set up a webhook
Subscribe your webhook to a topic
- 1 1. Configure your site for webhooks
- 2 2. Set Up a Webhook
- 3 3. Subscribe your webhook to a topic
- 3.1 Standard topics
- 3.2 Custom topics
- 4 4. Acknowledgment of Webhook Receipt
- 5 5. Retries, Error Management and Alerts
- 5.1 Events
- 5.2 Alerts configuration
- 5.3 Alert trigger example
- 6 6. Webhook Statuses
- 7 7. Message storage
- 8 8. Integration Requirements
- 9 8. Integration Best practices
1. Configure your site for webhooks
Your site configuration sets the following parameters:
The retry delay before a temporary or complete stop
A contact in case of an error
The name of the notification to send in the error message
A template that can be used for configuring at Order Management Center/Configurations/Outbound Messages/Webhooks
The default retry delay values are set as shown above in seconds. In this instance the webhook would make the first call and if no response or confirmation is received then it will retry 6 times, first after 30 seconds, then 60, then 2 minutes, 4 minutes, 8 minutes and finally 16 minutes. However, after the 3rd failed attempt, i.e. one with no response, it will send a notification of failure to the contact details in the configuration.
If a response or confirmation is received before the 6th call then the webhook will send another notification, as it is now classed as ‘recovered’. The retry counter will then be reset at 0.
2. Set Up a Webhook
Creating a webhook is mostly done via API (use the POST /webhooks
route) however some options can be set via our new configuration screens. These are shown later.
The configuration of the webhook defines important information such as which topic to subscribe to and the HTTP method to connect.
There are three mandatory fields required for webhooks with OneStock, they are:
Your Site ID
The HTTP method
The destination URL
Your OneStock contact will be able to provide with your Site ID if you do not know it. It will be a series of numbers following a lower case 'c', for example c404.
OneStock uses RESTful APIs so the HTTP method can be GET
XGET
POST
PATCH
PUT
or DELETE
.
The destination URL will be the address where the webhook will send its requests.
3. Subscribe your webhook to a topic
To listen to OneStock events you will subcribe your webhook to a topic. We distinguish to types of topics, standard and custom. Standard topics work out of the box, you just need to sucribe your webhook to one of them and you will start receving all events that go through the topic event stream. Custom topics need slightly more configuration and allow you to listen to specific events that you will trigger trough your workflow.
Standard topics
1. Subcribe a webhook to a standard topic
To listent to topics, you simply need to subscribe a webhook to one or multiple topics, through a
POST /webhooks
call.
POST /webhooks
{
"site_id": "MySiteID",
"token":"token",
"webhook": {
"hash_key": "personnal_hash_key",
"http_method": "POST",
"status": "enabled",
"topics": [
"order_state_changed",
"parcel_state_changed",
"line_item_group_state_changed"
],
"url": "https://myWebhookURL" //you can generate a disposbel one at https://webhook.site/
}
}
Following a list of all topics we currently support. We may add more at any time, so in developing and maintaining your code, you should not assume that only these topics exist.
Topic | Displayed Name | Description | Message structure |
---|---|---|---|
buffer_import_completed | Buffer Import Completed | Notifies when an async buffer import using API or file is completed |
import_id: string
status: string |
12.4 buffer_import_error_occurred | Buffer Import Error Occurred | Notifies when a synchronous buffer import using API POST method fails |
|
12.4 async_buffer_import_error_occurred | Asynchronous Buffer Import Error Occurred | Notifies when an asynchronous buffer import fails |
|
… full list bellow |
|
|
|
2. Test that all works correctly
To do so, you can simply create a disposabel webhook endpoint going to https://webhook.site/ and use it as the url of your topic (by clicking on edit you can change the response code do 202
, only 100 webhook receptions are allowed for free). In the case of our example, when a parcel transitions from new
→ bagged
, as when bagged in the Store App, we should receive a body as the following
Custom topics
If the event you wish to subscribe to is not listed above you can create a custom topic. The custom topic can either notify based on an existing OneStock events, or based on an event triggered in the workflow.
Custom topics can be created via API, or via the Back Office Configuration screens.
1. Create a custom notification
Go to Configuration > Outbound Messages > Notifications, and add a custom notification. Base yourself on the following code example and tailor it to your need by modifing the id and the topic id. Leave the rest unchanged.
2. Create the custom topic
In order to configure a webhook via API the topic must be set using the route POST /webhook_topics
as shown below.
POST /webhook_topics
3. Subcribe a webhook to the custom topic
We must add a listener to the custom topic. To do so, we just need to create a webhook using the route POST /webhooks
POST /webhooks
4. Trigger the custom notification from the workflow
Add a send notification event in the transition that must trigger the custom notification, an set the custom name, in our case, email_sent_to_client
Video walkthrough
5. Test that all works correctly
Upon receiving a message via webhook, it is essential to return a 202 Accepted
response promptly (within 15 seconds, but ideally under 300 ms) after verifying the message’s legitimacy (optional but recommended)
To do so, you can simply create a disposabel webhook endpoint going to https://webhook.site/ and use it as the url of your topic (by clicking on edit you can change the response code do 202
, only 100 webhook receptions are allowed for free). In the case of our example, when a parcel transitions from new
→ bagged
, as when bagged in the Store App, we should receive a body as the following
4. Acknowledgment of Webhook Receipt
Upon receiving a message via webhook, it is essential to return a 202 Accepted
response promptly (within 15 seconds, but ideally under 300 ms) after verifying the message’s legitimacy (optional but recommended)
Upon receiving a message via webhook, it is essential to return a 202 Accepted
response promptly (within 15 seconds, but ideally under 300 ms) after verifying the message’s legitimacy (optional but recommended). If this acknowledgment is not received in time, the system will retry sending the message according to the retry_intervals
configuration, if retries are not acknowledged under 15 seconds.
To maintain synchronization between OneStock and your service, please acknowledge incoming webhook messages immediately. Delays can cause desynchronization and data inconsistencies. We recommend processing messages asynchronously and sending acknowledgments promptly to ensure smooth integration.
Legitimacy Check
While optional, OneStock strongly recommends performing a legitimacy check to ensure that the webhook message is genuinely sent by the OneStock system.
Each webhook message includes a signature in the header with the following format: timestamp + "," + signature
.
During webhook setup (POST /webhooks
), a hash key is either provided by the client or generated by OneStock. This key, referred to as h0
, is used to encrypt the signature. OneStock retains the three most recent hash keys (h0, h1, h2) to support ongoing signature verification.
The signature format is:t=timestamp,h0=h[0],h1=h[1],h2=h[2]
, where:
timestamp
: The current timestamp.h[0]
: The signature encrypted using the latest hash key.h[1]
: The signature encrypted using the previous hash key.h[2]
: The signature encrypted using the oldest hash key.
Examples of signature and verification code
Webhook Acknowledgment Best Practices
To avoid blocking the entire message queue when processing webhooks, it is crucial to focus on the signature validation first and handle any additional checks asynchronously.
Recommended Acknowledgment Process:
Signature Validation:
Objective: Ensure the message is legitimately sent by OneStock.
Action: If the signature is valid, immediately return an HTTP
202 Accepted
response.
Asynchronous Content Validation (if applicable):
Any additional checks (e.g., content validation) should be performed asynchronously after acknowledging the message. This prevents delays in processing subsequent webhooks and ensures a smooth message flow.
Key Benefits
Prevent Queue Blocking: By acknowledging based solely on signature validation, the message queue remains smooth and uninterrupted.
Efficient Processing: Additional content checks can be handled asynchronously, avoiding potential bottlenecks.
5. Retries, Error Management and Alerts
If acknowledgment is not received within the configured time frame, the system will retry sending the message. Notifications regarding errors can be sent via SMS, email, or both.
Events
on_failure
: Triggered after the maximum number of retries (retries_until_failure
) is reached without acknowledgment. Retries count do not include the original message (the first one which was not ackowledged).on_failure_recovered
: Triggered when a failed message is later acknowledged during a retry. The retry count resets to 0.on_deactivation
: Triggered when the maximum retry attempts are exhausted without recovery. The webhook is automatically disabled and must be manually reactivated. Messages are stored while the webhook is in a disabled state upto a week. After a week, the webhook is considered dead, and all messages are erased.
To resolve these errors, ensure your web service is functional and restart the webhook by setting it’s status to enabled
with a PATCH /webhooks/:id/status.
Alerts configuration
Emails and SMS are sent to notify webhook events stated above.
This are configured in the site configuration
Alert trigger example
With the configuration described, here's how the retry process works:
Event 1 Fails Initial Acknowledgment: If Event 1 is not acknowledged within 15 seconds, the webhook status changes to
paused
, and a retry is scheduled for 30 seconds later.Retry 1: If the first retry is not acknowledged within 15 seconds, a second retry is triggered after 60 seconds.
Retry 2: If the second retry fails to get acknowledged within 15 seconds, a third retry is scheduled for 120 seconds later.
Retry 3 (Final Attempt Before Alert): If the third retry is also not acknowledged within 15 seconds, an "on failure" alert is triggered because the number of retries has reached the configured
retries_until_failure
. The webhook remains inpaused
status.Subsequent Retries:
A fourth retry is scheduled after 240 seconds. If acknowledged successfully, an "on failure recovered" alert is triggered, and the webhook status changes to
enabled
.If retries continue to fail, and the sixtieth retry does not get acknowledged, the webhook will be disabled, and an "on deactivation" alert will be triggered.
6. Webhook Statuses
enabled
: The default state after creation. Messages are processed normally, triggering the webhook.paused
: Messages are stored while the webhook is in paused state upto a week. After a week, the webhook is considered dead, and all messages are erased.
The behavior of this state depends on the scenario:Manual pause: No messages will be sent until the webhook is manually re-enabled.
Automatic pause: This occurs when a message is not acknowledged by the receiver within 15 seconds. In this case, the message will be retried. If the message fails to be acknowledged after the configured number of retries (
retries_until_failure
), an on_failure notification will be triggered (via email and/or SMS, depending on configuration). If the maximum number of retries is reached without success, the webhook will bedisabled
. However, if the message is acknowledged before reaching the retry limit, the webhook will automatically resume sending messages (enabled
).
disabled
: The webhook wont be triggered untilenabled
. Messages are stored while the webhook is in a disabled state upto a week. After a week, the webhook is considered dead, and all messages are erased.
7. Message storage
In the case of paused and disabled webhooks, messages are stored for a week; after a week, all messages will be automatically deleted. If a webhook is no re-enabled under a week, all messages will be lost.
In the case of enabled webhooks, messages are stored for a week; messages older than a week will get deleted. If desynchronization between OneStock and Client is greater than a week, old messages will be lost.
8. Integration Requirements
1. Process Messages as Quickly as Possible
Acknowledging webhook messages within 15 seconds is essential, but processing them as fast as possible ensures your system stays in sync with OneStock’s real-time events.
Why It Matters
Prevent Backlogs: If messages are processed just under the 15-second limit and events occur more frequently, a backlog will form, causing delays.
Avoid Desync: Since events are processed in order, slow processing can lead to discrepancies. For example, if events occur every 1 second and your system processes messages every 14 seconds, after an hour, only 257 out of 3600 events will be handled, creating a 55-minute delay.
Improve User Experience: Faster processing ensures that user-facing features and data remain accurate and up-to-date.
Tips for Faster Processing
Decouple Processing from Acknowledgment: Use a queuing system to instantly acknowledge messages and process them asynchronously.
Optimize Handling: Keep message processing efficient and lightweight.
Scale Dynamically: Use auto-scaling to handle traffic spikes.
Monitor Performance: Set alerts when processing times approach the 15-second limit.
2. Handling Idempotency in Webhook Integration
When integrating with OneStock's webhook system, it is crucial that your webhook client handles idempotency. OneStock expects all webhook clients to process messages in an idempotent manner to ensure consistency and reliability.
What Is Idempotency?
Idempotency refers to the property of certain operations where executing them multiple times yields the same result as executing them once. In the context of webhooks, this means that if the same message is received more than once, processing it multiple times does not cause unintended side effects or data inconsistencies.
Why Is Idempotency Important?
Due to network issues or other transient problems, a message sent by OneStock might not be acknowledged by your system. In such cases, OneStock will retry sending the same message to ensure it has been received and processed.
If your webhook client does not handle idempotency:
Duplicate processing: The same operation might be performed multiple times (e.g., applying the same order change repeatedly), leading to inconsistent data or errors.
Integration breakdowns: Critical systems like your ERP might encounter conflicts or errors due to repeated operations, potentially disrupting business processes.
How to Handle Idempotency
Use the message-id
Header
Each webhook message sent by OneStock includes a unique message-id
in the HTTP headers. This identifier is essential for ensuring idempotent processing.
Implement Message Queueing and Asynchronous Processing
Acknowledge Immediately: Upon receiving a webhook message, your system should send an acknowledgement to OneStock immediately.
Enqueue the Message: Store the
message-id
and the message payload in a reliable queue for asynchronous processing.Process Asynchronously: Use a separate worker or process to consume messages from the queue and handle the idempotency checks.
Ensure Idempotency in Asynchronous Processing
Store processed message IDs: Maintain a persistent record of all processed
message-id
s.Check before processing: Before processing a message from the queue, check if its
message-id
exists in your records.If it exists: Skip processing.
If it does not exist: Process the message and then store its
message-id
.
Ensure Atomic Operations
To prevent race conditions:
Atomic transactions: Combine the message processing and the storage of the
message-id
into a single atomic transaction.Thread safety: If your application processes messages in parallel, ensure that access to the message ID store is thread-safe.
Example Implementation
Webhook Handler Function
Asynchronous Message Processor
Acknowledging Messages
Successful processing: Since you acknowledge messages immediately, ensure that your asynchronous processing handles failures internally without relying on OneStock to retry.
Failed processing: Implement retry mechanisms within your message processor for transient errors.
Network failures: Immediate acknowledgement reduces the chance of network issues causing message retries from OneStock.
Advantages of This Approach
Improved performance: Acknowledging immediately reduces response time and prevents timeouts.
Scalability: Asynchronous processing allows your system to handle a higher volume of messages efficiently.
Reliability: Queueing messages ensures they are not lost and can be retried in case of transient failures.
Summary
Handle retries gracefully: Design your webhook client to process duplicate messages without adverse effects.
Use
message-id
for idempotency: Leverage the uniquemessage-id
header provided by OneStock for tracking.Process asynchronously: Acknowledge messages immediately and process them in a separate worker or process.
Prevent duplicate operations: Ensure that repeated messages do not cause inconsistent states or errors in your systems.
8. Integration Best practices
Use a Queuing System to Receive Messages
To ensure smooth and reliable communication, it's essential to acknowledge webhook messages swiftly—ideally within 15 seconds—by returning a 202 response code. This approach ensures that OneStock knows your system has received the message, allowing the process to continue without delays.
Why Use a Queuing System?
Immediate Acknowledgment: By decoupling message reception from processing, you can instantly acknowledge the receipt of the message. The queue receives and stores the message, allowing your system to process it asynchronously without holding up the response.
Enhanced Reliability: A queuing system provides a buffer, ensuring that even if your system is temporarily down or under heavy load, messages are not lost. They will be processed once your system is ready.
Easier Troubleshooting: Queuing services often come with tools that allow you to view and manage received messages, making it easier to diagnose and resolve any communication issues.
Recommended Queuing Solutions:
AWS SQS (Simple Queue Service)
Google Cloud Pub/Sub
Apache Kafka
Log Message Reception
If you choose not to use a queuing system, it's vital to implement robust logging of all received messages. Logging should include the message content and the response sent back to OneStock.
Benefits of Logging:
Monitoring and Auditing: Detailed logs provide a trail of received messages, which is crucial for monitoring system performance and ensuring that all messages are accounted for.
Incident Response: In the event of an issue, logs enable you to quickly identify and address the problem, minimizing downtime and maintaining communication integrity.
Compliance and Debugging: Logs can also serve as an essential resource for compliance and debugging purposes, helping you ensure that your system meets all necessary requirements and functions as expected.