About This Recipe
This recipe ensures that messages that enter a system are resequenced and processed in a specific order.
The Sequencing Problem
Often, you have a stream of requests that must be executed in order, for example, create account, update account address, and update account contacts. The later two activities can't occur until the first has completed.
Before you can sequence messages, you need to know the order in which the messages should be processed. So, you need to use some sort of sequencing ID. This ID could be a timestamp or an actual sequence identifier. If you're using timestamps, then the closer to the message origin that the timestamp is applied the better. For example, if you take the timestamp from when the message arrives in Oracle Integration, then a network delay may have already caused our messages to be out of order. Typically, you don't want all messages to be in the same ordered sequence. In our account example, only messages for a given account need to be ordered. Messages for different accounts can execute in parallel. So, now you also need some sort of group ID to identify different sequence streams within your message stream.
After you have the messages and know their order, you can process them. Inherent in a resequencing solution is some sort of delay to allow messages to arrive out of order and then be sorted into order. The size of the delay specifies how much time you can accept a message to be delayed before you go ahead without it.
The Resequencing Solution
- Processes the input message based on the desired sequence ID rather than on the order in which the messages arrive.
- Parks each message in storage for a certain period of time (parking time) so that any out-of-sequence messages have a chance to be processed in the desired order.
- Lets you configure the maximum number of message groups being processed in parallel in order to throttle the outgoing calls.
- Handles all errors, including system errors, network errors, and bad requests.
To use the recipe, you must install the recipe and configure the connections and other resources within it. The recipe includes a set of integrations, connections, and scripts that use standard Oracle Integration features.
- The resequencer producer integration is the source integration that receives the messages.
- The resequencer producer integration pushes the data to the database.
- The resequencer consumer integration pulls data based on scheduled frequency from the database.
- The resequencer consumer integration calls the dispatcher integration which then sends messages to the target system.
-
The resequencer manager integration supervises the resequencer. It supports three operations.
Operation Path and Method Description Get configs Path: /configs
Method: GET
Returns the config of all the types. Example of invocation:
$ curl https://my.integration.cloud/ic/api/integration/v1/flows/rest/RSQMANAGER/1.0/configs -v -u username:passwordUpdate config Path: /configs/{type}
Method: PUT
Update the config for the given type. Example of invocation:
$ curl -X PUT https://my.integration.cloud/ic/api/integration/v1/flows/rest/RSQMANAGER/1.0/configs/employee -v -u username:password -H "Content-Type: application/json" -d@config.jsonconfig.json example:
{ "maxConcurrent": 5, "timeWindow": 11 }Recover Group Path: /types/{type}/groups/{group}/recover
Method: PUT
Deletes stuck messages in the message table and reactivates the group by setting its status to 'N'. Example of invocation:
$ curl -X PUT https://my.integration.cloud/ic/api/integration/v1/flows/rest/RSQMANAGER/1.0/types/employee/groups/eng/recover -v -u username:password -H "Content-length: 0"