Before-images Streaming

Learn about streaming before-images to your applications using the Oracle NoSQL Database Streams API.

Oracle NoSQL Database supports streaming before-images. Before-images enable applications to compute the change or delta made by write operations to tables in a data store.

The before-image of any write is the table row before it gets updated or deleted by a DML operation. By default, Oracle NoSQL Database does not generate before-images. After before-images are enabled using either the CREATE TABLE or ALTER TABLE DDL statement, you can configure your subscription to include before-images of subscribed tables in addition to after-images.

To explicitly generate before-images during any writes to a table and stream it to subscribers, you must:

  1. Enable before-images for the table using the ENABLE BEFORE IMAGE syntax. For more details, see Enabling Before-images During Table Creation and ALTER TABLE statement topics.
  2. Configure the subscription to stream before-images to the subscriber for the table of interest. For method details, see Using NoSQLSubscriptionConfig.
  3. Implement the subscriber to receive a stream with before-images events. For method details, see Using the StreamOperation Class.

Following table summarizes the content of a before-image for a DML operation on any user table, either local or Multi-Region:

Table 1-1 Before-images for DML Operations

DML operation Before-image content
INSERT NULL
UPDATE row before update
DELETE deleted row

For either an update or a delete operation, the before-image is the row before it is updated or deleted. For insert operations, before-images are empty.

Note:

  1. Enabling/disabling before-images is on a per table basis and can be done anytime during the lifetime of a table.

  2. A TTL definition sets the expiry for before-images. You can define before-images TTL either during the creation of a table or later using an ALTER TABLE statement. For more details, For more details, see Enabling Before-images During Table Creation and ALTER TABLE statement topics.

    After their expiration, the before-images will not appear in the stream.

  3. It is possible to change the TTL value of before-images during the lifetime of a table using the ALTER TABLE statement. The before-images that are generated after the TTL update reflect the new expiry. The existing before-images expiry remain unaffected.

    For example, if the before-images are already generated with a TTL of five days and you modify the before-images TTL to one day, then the before-images that are newly generated after the TTL update expire in one day. There is no impact to the expiry of existing before-images.

  4. If you have set a small TTL value for before-images and the streaming is lagging, the before-images can expire before the streaming becomes operative, resulting in a loss of the before-images from the stream.
  5. It is possible for a table to have both a table TTL and a before-images TTL. These two TTL values work independently of each other. As a result, it is possible for a data row to have expired, while the row's before-image may still be active. In such cases, the Streams API can still stream the before-image until its expiry. The reason being, even though the row has expired, it will still be available for streaming until the row is removed from the transaction log.
  6. Queries and DDL operations do not generate before-images.
  7. The streams can start or resume from any arbitrary location if the location is available. Therefore, streams may include events without before-images if the events were created prior to enabling before-images or after disabling them.
  8. A streamed event is self-describable. From any stream event, a subscriber can distinguish between the following cases by implementing the corresponding methods in the subscriber. For more information, oracle.kv.pubsub.StreamOperation class summary in the — Java Direct Driver API Reference
    • Before-images are not enabled for an event.
    • Before-images are enabled but the before-image has expired.
    • Before-images are enabled and has not expired, however, the event does not carry the before-image. For example, an INSERT operation.