Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device. Atlassian cookies and tracking notice, (opens new window)
A webhook is an asynchronous communication method that sends a one-way notification using an HTTP service. It is triggered by an event and sends information without an initial client or user request.
To function, a webhook must be subscribed to a specific topic. When the event occurs, the webhook receives and processes the information.
Example Use Case in OneStock: When an order changes state (e.g., from placed to shipped), instead of continuously polling an API, a webhook sends a notification of the state change directly to the user.
Implementing a Webhook in OneStock
OneStock OMS requires certain configurations and parameters to be set before a webhook can be used. A legitimacy check can also be added as a final step. This is optional, but highly recommended. We discuss each of these steps and requirements below.
In order to configure webhooks in OneStock you will need to complete the 3 following steps:
The default retry delay values are set as shown above in seconds. In this instance the webhook would make the first call and if no response or confirmation is received then it will retry 6 times, first after 30 seconds, then 60, then 2 minutes, 4 minutes, 8 minutes and finally 16 minutes. However, after the 3rd failed attempt, i.e. one with no response, it will send a notification of failure to the contact details in the configuration.
If a response or confirmation is received before the 6th call then the webhook will send another notification, as it is now classed as ‘recovered’. The retry counter will then be reset at 0.
2. Set Up a Webhook
Creating a webhook is mostly done via API (use the POST /webhooksroute) however some options can be set via our new configuration screens. These are shown later.
The configuration of the webhook defines important information such as which topic to subscribe to and the HTTP method to connect.
There are three mandatory fields required for webhooks with OneStock, they are:
Your Site ID
The HTTP method
The destination URL
Your OneStock contact will be able to provide with your Site ID if you do not know it. It will be a series of numbers following a lower case 'c', for example c404.
OneStock uses RESTful APIs so the HTTP method can be GETXGETPOSTPATCHPUT or DELETE.
The destination URL will be the address where the webhook will send its requests.
3. Subscribe your webhook to a topic
To listen to OneStock events you will subcribe your webhook to a topic. We distinguish to types of topics, standard and custom. Standard topics work out of the box, you just need to sucribe your webhook to one of them and you will start receving all events that go through the topic event stream. Custom topics need slightly more configuration and allow you to listen to specific events that you will trigger trough your workflow.
Standard topics
1. Subcribe a webhook to a standard topic
To listent to topics, you simply need to subscribe a webhook to one or multiple topics, through a
POST /webhooks call.
POST /webhooks
{
"site_id": "MySiteID",
"token":"token",
"webhook": {
"hash_key": "personnal_hash_key",
"http_method": "POST",
"status": "enabled",
"topics": [
"order_state_changed",
"parcel_state_changed",
"line_item_group_state_changed"
],
"url": "https://myWebhookURL" //you can generate a disposbel one at https://webhook.site/
}
}
Following a list of all topics we currently support. We may add more at any time, so in developing and maintaining your code, you should not assume that only these topics exist.
Topic
Displayed Name
Description
Message structure
Topic
Displayed Name
Description
Message structure
buffer_import_completed
Buffer Import Completed
Notifies when an async buffer import using API or file is completed
buffer_import_completed
import_id: string
status: string
12.4
buffer_import_error_occurred
Buffer Import Error Occurred
Notifies when a synchronous buffer import using API POST method fails
Notifies of OMS entities information changes (parcels, line item groups and orders) POP (endpoint_orders, containers, piece_groups) and RM (return_parcels, and return_line_item_groups)
When there is an orchestration rule change with a warehouse custom claim action triggered, line item groups are retrieved and a preparation order is sent to the warehouse.
Notifies of a reservation update on a line item group (global reservation to endpoint reservation, future stock to on hand, endpoint reservation removal)
The notification is created using the transitions configured in the shipping instruction configuration page
shipping_instructions_computed
order_id: string
carrier:
name: string
information: map[string]any
option: string
endpoint_id: string
index_ranges: []
from: int
to: int
2. Test that all works correctly
To do so, you can simply create a disposabel webhook endpoint going to https://webhook.site/ and use it as the url of your topic (by clicking on edit you can change the response code do 202, only 100 webhook receptions are allowed for free). In the case of our example, when a parcel transitions from new → bagged , as when bagged in the Store App, we should receive a body as the following
If the event you wish to subscribe to is not listed above you can create a custom topic. The custom topic can either notify based on an existing OneStock events, or based on an event triggered in the workflow.
Custom topics can be created via API, or via the Back Office Configuration screens.
1. Create a custom notification
Go to Configuration > Outbound Messages > Notifications, and add a custom notification. Base yourself on the following code example and tailor it to your need by modifing the id and the topic id. Leave the rest unchanged.
"email_sent_to_client":{ //notification id, we will reference it in the workflow
"media":{
"webhook_topic":[
"email_sent_to_client_topic" //topic id, we will then create it through API
]
},
"time_type": "duration",
"time_value": "0s"
}
2. Create the custom topic
In order to configure a webhook via API the topic must be set using the route POST /webhook_topics as shown below.
POST /webhook_topics
{
"site_id": "My_Site_ID",
"token":"token",
"webhook_topic": {
"topic": "email_sent_to_client_topic", //custom topic id
"ordered": true //messages order will be respected
}
}
3. Subcribe a webhook to the custom topic
We must add a listener to the custom topic. To do so, we just need to create a webhook using the route POST /webhooks
POST /webhooks
{
"site_id": "MySiteID",
"token":"token",
"webhook": {
"hash_key": "personnal_hash_key",
"http_method": "POST",
"status": "enabled",
"topics": [
"email_sent_to_client_topic"
],
"url": "https://myWebhookURL" //you can generate a disposbel one at https://webhook.site/
}
}
4. Trigger the custom notification from the workflow
Add a send notification event in the transition that must trigger the custom notification, an set the custom name, in our case, email_sent_to_client
Video walkthrough
5. Test that all works correctly
Upon receiving a message via webhook, it is essential to return a 202 Accepted response promptly (within 15 seconds, but ideally under 300 ms) after verifying the message’s legitimacy (optional but recommended)
To do so, you can simply create a disposabel webhook endpoint going to https://webhook.site/ and use it as the url of your topic (by clicking on edit you can change the response code do 202, only 100 webhook receptions are allowed for free). In the case of our example, when a parcel transitions from new → bagged , as when bagged in the Store App, we should receive a body as the following
Upon receiving a message via webhook, it is essential to return a 202 Accepted response promptly (within 15 seconds, but ideally under 300 ms) after verifying the message’s legitimacy (optional but recommended)
Upon receiving a message via webhook, it is essential to return a 202 Accepted response promptly (within 15 seconds, but ideally under 300 ms) after verifying the message’s legitimacy (optional but recommended). If this acknowledgment is not received in time, the system will retry sending the message according to the retry_intervals configuration, if retries are not acknowledged under 15 seconds.
To maintain synchronization between OneStock and your service, please acknowledge incoming webhook messages immediately. Delays can cause desynchronization and data inconsistencies. We recommend processing messages asynchronously and sending acknowledgments promptly to ensure smooth integration.
Legitimacy Check
While optional, OneStock strongly recommends performing a legitimacy check to ensure that the webhook message is genuinely sent by the OneStock system.
Each webhook message includes a signature in the header with the following format: timestamp + "," + signature.
During webhook setup (POST /webhooks), a hash key is either provided by the client or generated by OneStock. This key, referred to as h0, is used to encrypt the signature. OneStock retains the three most recent hash keys (h0, h1, h2) to support ongoing signature verification.
The signature format is: t=timestamp,h0=h[0],h1=h[1],h2=h[2], where:
timestamp: The current timestamp.
h[0]: The signature encrypted using the latest hash key.
h[1]: The signature encrypted using the previous hash key.
h[2]: The signature encrypted using the oldest hash key.
Examples of signature and verification code
Current timestamp = 1704092400
Encryption of h0 for 1704092400 = 1234 (encryption of the timestamp with the new key)
Encryption of h1 for 1704092400= 9876 (encryption of the timestamp with the previous key)
The signature will be the following : t=1704092400,h0=1234,h1=9876
Resulting the header : Onestock-Signature=t=1704092400,h0=1234,h1=9876
<?php
// checkWebhookSignatureWithMultipleKeys check webhook signature with multiple secret keys.
function checkWebhookSignatureWithMultipleKeys($request, $keys) {
// Iterates through each provided secret key.
foreach ($keys as $key) {
// Calls the checkWebhookSignature function with the current request and secret key.
$isValid = checkWebhookSignature($request, $key);
// If the signature is valid with the current key, return true immediately.
if ($isValid) {
return true;
}
}
// If none of the keys validate the signature, return false.
return false;
}
// checkWebhookSignature check the validity of the webhook signature.
function checkWebhookSignature($request, $secretKey) {
// Extract the 'Onestock-Signature' header from the incoming request.
// This header contains the signature to be verified, formatted as "t=timestamp.h0=hash0,h1=hash1..."
$signatureHeader = $request->headers->get('Onestock-Signature');
// Read the raw body of the request. This is necessary to reconstruct the payload for hashing and verification.
$body = file_get_contents('php://input');
// Split the signature header into individual components based on the ',' delimiter.
// This separates the timestamp and each hash value for individual processing.
$signatureParts = explode(',', $signatureHeader);
// Ensure that the signature contains at least a timestamp and one hash value.
// If not, the signature is considered incomplete and thus invalid.
if (count($signatureParts) < 2) {
return false;
}
// Extract the timestamp from the first part of the signature, removing the 't=' prefix.
// The timestamp is used to verify the timeliness of the request to prevent replay attacks.
$timestamp = substr($signatureParts[0], 2);
// Check if the current time minus the provided timestamp exceeds 6 hours (21600 seconds).
// If so, consider the request too old and reject it to prevent replay attacks.
if (time() - intval($timestamp) > 60 * 60 * 6) {
return false;
}
// Concatenate the timestamp and the request body with a '.' to reconstruct the original payload.
// This payload mirrors what was used to generate the hash on the sender's side.
$payload = $timestamp . '.' . $body;
// Compute the expected signature by hashing the reconstructed payload with the provided secret key using SHA-256.
$expectedSignature = hash_hmac('sha256', $payload, $secretKey);
// Iterate over each hash value in the signature (excluding the timestamp).
for ($i = 1; $i < count($signatureParts); $i++) {
// Extract the hash value, removing the leading "hX=" where X is the hash index.
// This is done by cutting off the prefix based on its length.
$h = substr($signatureParts[$i], strlen("h{$i}="));
// Compare the extracted hash value with the expected signature.
// If any match is found, the signature is considered valid, and the function returns true.
if ($h === $expectedSignature) {
return true;
}
}
// If no hash values match the expected signature, the signature is deemed invalid.
return false;
}
// Example usage of the function with a webhook request and a list of previous keys.
$request = /* The webhook request to be verified */;
$previousKeys = ['key1', 'key2', 'key3']; // List of previous keys to check against.
// Calls the new function to check the signature against the list of previous keys.
$isSignatureValid = checkWebhookSignatureWithMultipleKeys($request, $previousKeys);
if ($isSignatureValid) {
echo "The webhook signature is valid.";
} else {
echo "The webhook signature is invalid.";
}
?>
import hmac
import hashlib
import time
# Function to check the signature of a webhook with multiple secret keys.
def check_webhook_signature_with_previous_keys(request, previous_keys):
"""
Attempts to verify the webhook signature against a list of previous secret keys.
:param request: The incoming webhook request object.
:param previous_keys: A list of previous secret keys to attempt verification with.
:return: True if the signature is verified successfully with any of the keys, otherwise False.
"""
for key in previous_keys:
# Attempt to verify the signature with the current key.
if check_webhook_signature(request, key):
# If verification is successful, return True immediately.
return True
# If none of the keys result in a successful verification, return False.
return False
def check_webhook_signature(request, secret_key):
"""
Verifies the webhook signature against a given secret key.
:param request: The incoming webhook request object, containing headers and data.
:param secret_key: The secret key used for verifying the signature.
:return: True if the signature is valid, otherwise False.
"""
signature_header = request.headers.get('Onestock-Signature')
body = request.data
signature_parts = signature_header.split(',')
if len(signature_parts) < 2:
return False
timestamp = signature_parts[0].removeprefix("t=")
if time.time() - int(timestamp) > 60 * 60 * 6:
return False
payload = f'{timestamp}.{body}'
expected_signature = compute_hash(secret_key, payload)
for i in range(1, len(signature_parts)):
h = signature_parts[i].removeprefix(f'h{i}=')
if hmac.compare_digest(h, expected_signature):
return True
return False
def compute_hash(secret_key, payload):
"""
Computes the HMAC SHA-256 hash of a payload using a secret key.
:param secret_key: The secret key as a string.
:param payload: The payload to hash, consisting of the timestamp and request body.
:return: The computed hash as a hexadecimal string.
"""
return hmac.new(bytes(secret_key, 'utf-8'), msg=bytes(payload, 'utf-8'), digestmod=hashlib.sha256).hexdigest()
# Example usage
request = ... # The webhook request object
previous_keys = ['old_key1', 'old_key2', 'current_key']
verified = check_webhook_signature_with_previous_keys(request, previous_keys)
if verified:
print("Webhook signature verified successfully.")
else:
print("Failed to verify webhook signature.")
const crypto = require('crypto');
/**
* Attempts to verify the webhook signature against a list of previous secret keys.
*
* @param {object} req - The request object from the webhook, containing headers and body.
* @param {array} previousKeys - An array of secret keys (including the current and any previous ones) to try for verification.
* @returns {boolean} - Returns true if the signature is valid with any of the provided keys, otherwise false.
*/
function verifyWithPreviousKeys(req, previousKeys) {
for (const key of previousKeys) {
// Attempt to verify the signature with the current key.
if (checkWebhookSignature(req, key)) {
// If verification succeeds with the current key, return true immediately.
return true;
}
}
// If verification fails with all keys, return false.
return false;
}
/**
* Checks the validity of the webhook signature to ensure the request is from a trusted source.
*
* @param {object} req - The request object from the webhook, containing headers and body.
* @param {string} secretKey - The secret key used to generate the signature for comparison.
* @returns {boolean} - Returns true if the signature is valid, otherwise false.
*/
function checkWebhookSignature(req, secretKey) {
const signatureHeader = req.headers['Onestock-Signature'];
const body = req.body;
const signatureParts = signatureHeader.split(',');
if (signatureParts.length < 2) {
return false;
}
const timestamp = signatureParts[0].replace('t=', '');
if (Date.now() / 1000 - parseInt(timestamp) > 60 * 60 * 6) {
return false;
}
const payload = `${timestamp}.${body}`;
const expectedSignature = crypto.createHmac('sha256', secretKey).update(payload).digest('hex');
console.log(expectedSignature)
for (let i = 1; i < signatureParts.length; i++) {
const h = signatureParts[i].replace(`h${i-1}=`, '');
if (h === expectedSignature) {
return true;
}
}
return false;
}
// Example usage
const req = {
headers: {
'Onestock-Signature': 't=timestamp.h0=hash0.h1=eb2a2ab3d29587fba099451925bdbd9637814711c97bc79c6a6e86f898884796'
},
body: 'request body'
};
const previousKeys = ['oldKey1', 'oldKey2', 'currentKey'];
const isVerified = verifyWithPreviousKeys(req, previousKeys);
console.log(`Webhook signature verification result: ${isVerified}`);
require 'openssl'
require 'time'
# Method to attempt verification of a webhook signature with multiple secret keys.
# @param request [Object] The incoming request object, containing headers and a body.
# @param previous_keys [Array<String>] An array of secret keys to attempt for signature verification.
# @return [Boolean] True if the signature is verified successfully with any of the keys, otherwise false.
def verify_signature_with_previous_keys(request, previous_keys)
# Iterates through each provided secret key to attempt signature verification.
previous_keys.each do |key|
# If the signature verification is successful with the current key, return true immediately.
if check_webhook_signature(request, key) {
return true
}
end
# If none of the keys result in a successful verification, return false.
return false
end
# Defines a method to check the validity of the webhook signature against the provided secret key.
# @param request [Object] The incoming request object, which contains headers and a body.
# @param secret_key [String] The secret key used for generating and comparing the signature.
# @return [Boolean] True if the signature is valid, otherwise false.
def check_webhook_signature(request, secret_key)
# Extracts the signature from the request's headers. Expected format: “t=timestamp.h0=h[0].h1=h[1].h2=h[2].h3=h[3]"
signature_header = request.headers['HTTP_ONESTOCK_SIGNATURE']
# Reads the request body as a string.
body = request.body.read
# Splits the signature header into its constituent parts: the timestamp and the hash values.
signature_parts = signature_header.split(',')
# Ensures that there are at least two parts in the signature (the timestamp and at least one hash).
if signature_parts.size < 2
return false
end
# Extracts and converts the timestamp from the first part of the signature, removing 't=' prefix.
timestamp = signature_parts[0].sub('t=', '').to_i
# Verifies that the request is not too old (more than 6 hours in this case) to avoid replay attacks.
if Time.now.to_i - timestamp > 60 * 60 * 6
return false
end
# Constructs the payload by concatenating the timestamp and the body, separated by a period.
# This payload is used as the basis for signature generation and comparison.
payload = "#{timestamp.to_s}.#{body}" # Format: "timestamp.body"
# Generates the expected signature by hashing the payload with the secret key using SHA-256.
expected_signature = OpenSSL::HMAC.hexdigest('SHA256', secret_key, payload)
# Iterates over each hash part in the signature, skipping the first part (the timestamp).
signature_parts.drop(1).each_with_index do |part, index|
# Removes the hash prefix ('hX=') to isolate the hash value for comparison.
h = part.sub("h#{index}=", '')
# If any of the hash values matches the expected signature, the signature is deemed valid.
if h == expected_signature
return true
end
end
# Example usage of the new method:
request = # Assume this is the incoming webhook request object.
previous_keys = ['old_secret_key1', 'old_secret_key2', 'current_secret_key']
if verify_signature_with_previous_keys(request, previous_keys)
puts "Webhook signature verified successfully."
else
puts "Failed to verify webhook signature."
end
Webhook Acknowledgment Best Practices
To avoid blocking the entire message queue when processing webhooks, it is crucial to focus on the signature validation first and handle any additional checks asynchronously.
Recommended Acknowledgment Process:
Signature Validation:
Objective: Ensure the message is legitimately sent by OneStock.
Action: If the signature is valid, immediately return an HTTP 202 Accepted response.
Asynchronous Content Validation (if applicable):
Any additional checks (e.g., content validation) should be performed asynchronously after acknowledging the message. This prevents delays in processing subsequent webhooks and ensures a smooth message flow.
Key Benefits
Prevent Queue Blocking: By acknowledging based solely on signature validation, the message queue remains smooth and uninterrupted.
Efficient Processing: Additional content checks can be handled asynchronously, avoiding potential bottlenecks.
5. Retries, Error Management and Alerts
If acknowledgment is not received within the configured time frame, the system will retry sending the message. Notifications regarding errors can be sent via SMS, email, or both.
Events
on_failure: Triggered after the maximum number of retries (retries_until_failure) is reached without acknowledgment. Retries count do not include the original message (the first one which was not ackowledged).
on_failure_recovered: Triggered when a failed message is later acknowledged during a retry. The retry count resets to 0.
on_deactivation: Triggered when the maximum retry attempts are exhausted without recovery. The webhook is automatically disabled and must be manually reactivated. Messages are stored while the webhook is in a disabled state upto a week. After a week, the webhook is considered dead, and all messages are erased.
To resolve these errors, ensure your web service is functional and restart the webhook by setting it’s status to enabled with a PATCH /webhooks/:id/status.
Emails and SMS are sent to notify webhook events stated above.
This are configured in the site configuration
{
"retry_intervals": [ //intervales in seconds between each retry
30, //between ackoweldgemnt failure and first retry
60, //between 1st retry failure and second...
120,
240,
480,
840
],
"retries_until_failure": 3, //number of failed retries in a row that need to happen for the on_failure email to be triggered
"on_failure": {
"contact_emails": [ //email alert recipients. In test environments emails are sent to test recipients set in Configuration/Outbound/Media/Test Emails
"email@company.com"
],
"contact_mobiles": [],
"sms_notification_name": "",
"email_notification_name": "webhook_failure" //do not change
},
"on_deactivation": {
"contact_emails": [ //email alert recipients. In test environments emails are sent to test recipients set in Configuration/Outbound/Media/Test Emails
"email@company.com"
],
"contact_mobiles": [],
"sms_notification_name": "",
"email_notification_name": "webhook_deactivation" //do not change
},
"on_failure_recovered": {
"contact_emails": [ //email alert recipients. In test environments emails are sent to test recipients set in Configuration/Outbound/Media/Test Emails
"email@company.com"
],
"contact_mobiles": [],
"sms_notification_name": "",
"email_notification_name": "webhook_failure_recovered" //do not change
}
}
Alert trigger example
With the configuration described, here's how the retry process works:
Event 1 Fails Initial Acknowledgment: If Event 1 is not acknowledged within 15 seconds, the webhook status changes to paused, and a retry is scheduled for 30 seconds later.
Retry 1: If the first retry is not acknowledged within 15 seconds, a second retry is triggered after 60 seconds.
Retry 2: If the second retry fails to get acknowledged within 15 seconds, a third retry is scheduled for 120 seconds later.
Retry 3 (Final Attempt Before Alert): If the third retry is also not acknowledged within 15 seconds, an "on failure" alert is triggered because the number of retries has reached the configured retries_until_failure. The webhook remains in paused status.
Subsequent Retries:
A fourth retry is scheduled after 240 seconds. If acknowledged successfully, an "on failure recovered" alert is triggered, and the webhook status changes to enabled.
If retries continue to fail, and the sixtieth retry does not get acknowledged, the webhook will be disabled, and an "on deactivation" alert will be triggered.
6. Webhook Statuses
enabled: The default state after creation. Messages are processed normally, triggering the webhook.
paused: Messages are stored while the webhook is in paused state upto a week. After a week, the webhook is considered dead, and all messages are erased. The behavior of this state depends on the scenario:
Manual pause: No messages will be sent until the webhook is manually re-enabled.
Automatic pause: This occurs when a message is not acknowledged by the receiver within 15 seconds. In this case, the message will be retried. If the message fails to be acknowledged after the configured number of retries (retries_until_failure), an on_failure notification will be triggered (via email and/or SMS, depending on configuration). If the maximum number of retries is reached without success, the webhook will be disabled. However, if the message is acknowledged before reaching the retry limit, the webhook will automatically resume sending messages (enabled).
disabled: The webhook wont be triggered until enabled. Messages are stored while the webhook is in a disabled state upto a week. After a week, the webhook is considered dead, and all messages are erased.
7. Message storage
For paused or disabled webhooks, messages are stored for one week; after that, all messages are deleted automatically. If a webhook is not re-enabled within one week, it becomes permanently disabled (cannot be re-enabled) and all stored messages are lost. You must create a new webhook to replace it.
For enabled webhooks, messages are stored for one week; any messages older than one week are deleted. If the synchronization between OneStock and the client is interrupted for more than one week, older messages will be lost.
8. Integration Requirements
1. Process Messages as Quickly as Possible
Acknowledging webhook messages within 15 seconds is essential, but processing them as fast as possible ensures your system stays in sync with OneStock’s real-time events.
Why It Matters
Prevent Backlogs: If messages are processed just under the 15-second limit and events occur more frequently, a backlog will form, causing delays.
Avoid Desync: Since events are processed in order, slow processing can lead to discrepancies. For example, if events occur every 1 second and your system processes messages every 14 seconds, after an hour, only 257 out of 3600 events will be handled, creating a 55-minute delay.
Improve User Experience: Faster processing ensures that user-facing features and data remain accurate and up-to-date.
Tips for Faster Processing
Decouple Processing from Acknowledgment: Use a queuing system to instantly acknowledge messages and process them asynchronously.
Optimize Handling: Keep message processing efficient and lightweight.
Scale Dynamically: Use auto-scaling to handle traffic spikes.
Monitor Performance: Set alerts when processing times approach the 15-second limit.
2. Handling Idempotency in Webhook Integration
When integrating with OneStock's webhook system, it is crucial that your webhook client handles idempotency. OneStock expects all webhook clients to process messages in an idempotent manner to ensure consistency and reliability.
What Is Idempotency?
Idempotency refers to the property of certain operations where executing them multiple times yields the same result as executing them once. In the context of webhooks, this means that if the same message is received more than once, processing it multiple times does not cause unintended side effects or data inconsistencies.
Why Is Idempotency Important?
Due to network issues or other transient problems, a message sent by OneStock might not be acknowledged by your system. In such cases, OneStock will retry sending the same message to ensure it has been received and processed.
If your webhook client does not handle idempotency:
Duplicate processing: The same operation might be performed multiple times (e.g., applying the same order change repeatedly), leading to inconsistent data or errors.
Integration breakdowns: Critical systems like your ERP might encounter conflicts or errors due to repeated operations, potentially disrupting business processes.
How to Handle Idempotency
Use the message_id Header
Each webhook message sent by OneStock includes a unique message_id in the HTTP headers. This identifier is essential for ensuring idempotent processing.
Implement Message Queueing and Asynchronous Processing
Acknowledge Immediately: Upon receiving a webhook message, your system should send an acknowledgement to OneStock immediately.
Enqueue the Message: Store the message_id and the message payload in a reliable queue for asynchronous processing.
Process Asynchronously: Use a separate worker or process to consume messages from the queue and handle the idempotency checks.
Ensure Idempotency in Asynchronous Processing
Store processed message IDs: Maintain a persistent record of all processed message_ids.
Check before processing: Before processing a message from the queue, check if its message_id exists in your records.
If it exists: Skip processing.
If it does not exist: Process the message and then store its message_id.
Ensure Atomic Operations
To prevent race conditions:
Atomic transactions: Combine the message processing and the storage of the message_id into a single atomic transaction.
Thread safety: If your application processes messages in parallel, ensure that access to the message ID store is thread-safe.
Example Implementation
Webhook Handler Function
function handleWebhookRequest(request):
messageId = request.headers['message_id']
messageBody = request.body
// Acknowledge the message immediately
sendAcknowledgement()
// Enqueue the message for asynchronous processing
enqueueMessage(messageId, messageBody)
Asynchronous Message Processor
function processQueuedMessage():
while true:
message = dequeueMessage()
if message is not null:
messageId = message.id
messageBody = message.body
if isMessageIdProcessed(messageId):
// Skip processing as this message has already been handled
continue
try:
beginTransaction()
// Process the message
processWebhookData(messageBody)
// Record the message_id as processed
storeProcessedMessageId(messageId)
commitTransaction()
except Exception as e:
rollbackTransaction()
logError(e)
// Optionally, re-enqueue the message or handle the failure accordingly
else:
// No messages in the queue, wait or sleep before retrying
wait()
Acknowledging Messages
Successful processing: Since you acknowledge messages immediately, ensure that your asynchronous processing handles failures internally without relying on OneStock to retry.
Failed processing: Implement retry mechanisms within your message processor for transient errors.
Network failures: Immediate acknowledgement reduces the chance of network issues causing message retries from OneStock.
Advantages of This Approach
Improved performance: Acknowledging immediately reduces response time and prevents timeouts.
Scalability: Asynchronous processing allows your system to handle a higher volume of messages efficiently.
Reliability: Queueing messages ensures they are not lost and can be retried in case of transient failures.
Summary
Handle retries gracefully: Design your webhook client to process duplicate messages without adverse effects.
Use message_id for idempotency: Leverage the unique message_id header provided by OneStock for tracking.
Process asynchronously: Acknowledge messages immediately and process them in a separate worker or process.
Prevent duplicate operations: Ensure that repeated messages do not cause inconsistent states or errors in your systems.
8. Integration Best practices
Use a Queuing System to Receive Messages
To ensure smooth and reliable communication, it's essential to acknowledge webhook messages swiftly—ideally within 15 seconds—by returning a 202 response code. This approach ensures that OneStock knows your system has received the message, allowing the process to continue without delays.
Why Use a Queuing System?
Immediate Acknowledgment: By decoupling message reception from processing, you can instantly acknowledge the receipt of the message. The queue receives and stores the message, allowing your system to process it asynchronously without holding up the response.
Enhanced Reliability: A queuing system provides a buffer, ensuring that even if your system is temporarily down or under heavy load, messages are not lost. They will be processed once your system is ready.
Easier Troubleshooting: Queuing services often come with tools that allow you to view and manage received messages, making it easier to diagnose and resolve any communication issues.
Recommended Queuing Solutions:
AWS SQS (Simple Queue Service)
Google Cloud Pub/Sub
Apache Kafka
Log Message Reception
If you choose not to use a queuing system, it's vital to implement robust logging of all received messages. Logging should include the message content and the response sent back to OneStock.
Benefits of Logging:
Monitoring and Auditing: Detailed logs provide a trail of received messages, which is crucial for monitoring system performance and ensuring that all messages are accounted for.
Incident Response: In the event of an issue, logs enable you to quickly identify and address the problem, minimizing downtime and maintaining communication integrity.
Compliance and Debugging: Logs can also serve as an essential resource for compliance and debugging purposes, helping you ensure that your system meets all necessary requirements and functions as expected.
