Behavior on Inserts, Updates, and Deletes

The source trail file will contain insert, update. and delete operations for which the data can be pushed into Redis. The Redis Handler will process inserts, updates, and deletes as follows:

Inserts – The Redis Handler will create a new key in Redis the value of which is a hash map for which the hash map key is the column name and the hash map value is the column value.

Updates – The Redis Handler will update an existing hash map structure in Redis. The existing hash map will be updated with the column names and values from the update operation processed. Because hash map data is updated and not replace, full image updates are not required.

Primary Key Updates – The Redis Handler will move the old key to the new key name alone with the data structure, then an update will be performed on the hash map.

Deletes – The Redis Handler will delete the key and its corresponding data structure from Redis.

Handling of Null Values

Redis hash maps cannot store null as a value. A Redis hash map must have a non-null value. The default behavior is to omit columns with a null value from the generated hash map. If an update changes a column value from a non-null value to a null value, then the column key and value is removed from the hash map.

gg.handler.redis.omitNullValues=false
gg.handler.redis.nullValueRepresentation=null

The user will need to designate some value as null. But the following are legal too.

In this case the null value representation is an empty string or “”.

In this case the null value representation is set to a tab.

Support for Binary Values

The default functionality is to push all data into Redis hash maps as Java strings. Binary values must be converted to Base64 to be represented as a Java String. Consequently, binary values will be represented as Base64. Alternatively, users can push bytes into Redis hash maps to retain the original bytes values by setting the following configuration property.

Example hash map data in Redis:

127.0.0.1:6379> hgetall TCUSTMER:JANE
 1) "optype"
 2) "I"
 3) "CITY"
 4) "DENVER"
 5) "primarykeycolumns"
 6) "CUST_CODE"
 7) "STATE"
 8) "CO"
 9) "CUST_CODE"
10) "JANE"
11) "position"
12) "00000000000000002126"
13) "NAME"
14) "ROCKY FLYER INC."

Example Configuration

gg.handlerlist=redis
gg.handler.redis.type=redis
gg.handler.redis.hostPortList= localhost:6379
gg.handler.redis.createIndexes=true
gg.handler.redis.mode=op
gg.handler.redis.metacolumnsTemplate=${position},${optype},${primarykeycolumns}

Redis streams are analogs the Kafka topics. The Redis key is the stream name. The value of the stream are the individual messages pushed to the Redis stream. Individual messages are identified by a timestamp and offset of when the message was pushed to Redis. The value of each individual message is a hash map for which the key is the column name and value is the column value.

Behavior on Inserts, Updates, and Deletes

Each and every operation and its associated data is propagated to Redis Streams. Therefore, every operation will show up as a new message in Redis Streams.

Handling of Null Values

Redis streams stores hash maps as the value for each message. A Redis hash map cannot store null as a value. Null values work exactly as they do in hash maps functionality.

Support for Binary Values

The default functionality is to push all data into Redis hash maps as Java strings. Binary values must be converted to Base64 to be represented as a Java String. Consequently, binary values will be represented as Base64. Alternatively, users can push bytes into Redis hash maps to retain the original bytes values by setting the following configuration property.

Steam data appears in Redis as follows:

127.0.0.1:6379> xread STREAMS TCUSTMER 0-0
1) 1) "TCUSTMER"
   2) 1) 1) "1664399290398-0"
         2)  1) "optype"
             2) "I"
             3) "CITY"
             4) "SEATTLE"
             5) "primarykeycolumns"
             6) "CUST_CODE"
             7) "STATE"
             8) "WA"
             9) "CUST_CODE"
            10) "WILL"
            11) "position"
            12) "00000000000000001956"
            13) "NAME"
            14) "BG SOFTWARE CO."

2) 1) "1664399290398-1"
         2)  1) "optype"
             2) "I"
             3) "CITY"
             4) "DENVER"
             5) "primarykeycolumns"
             6) "CUST_CODE"
             7) "STATE"
             8) "CO"
             9) "CUST_CODE"
            10) "JANE"
            11) "position"
            12) "00000000000000002126"
            13) "NAME"
            14) "ROCKY FLYER INC."

Example Configuration

gg.handlerlist=redis
gg.handler.redis.type=redis
gg.handler.redis.hostportlist=localhost:6379
gg.handler.redis.mode=op
gg.handler.redis.integrationType=streams
gg.handler.redis.metacolumnsTemplate=${position},${optype},${primarykeycolumns}

Handling of Null Values

The JSON specification supports null values as JSON null. Therefore, null values in the data will be propagated as JSON null. Null value replacement is not supported since the JSON specification supports null values. Neither gg.handler.redis.omitNullValues nor gg.handler.redis.nullValueRepresentation configuration properties have any effect when the Redis Handler is configured to send JSONs. JSON per the specification is represented as follows: “fieldname”: null

Support for Binary Values

Per the JSON specification, binary values are represented as Base64. Therefore, all binary values will be converted and propagated as Base64. Setting the property gg.handler.redis.dataType has no effect. JSONs will generally appear in Redis as follows:

Example Configuration:

gg.handlerlist=redis
gg.handler.redis.type=redis
gg.handler.redis.hostportlist=localhost:6379
gg.handler.redis.mode=op
gg.handler.redis.integrationType=jsons
gg.handler.redis.createIndexes=true
gg.handler.redis.metacolumnsTemplate=${position},${optype},${primarykeycolumns}