Event Canal

The Event Canal microservice provides the foundation for streaming analytics of events. The Event Canal microservice is responsible for taking real-time events from the Event database and enqueueing them to an Apache Pulsar topic. Multiple microservices can read from this same topic to create analytics pipelines for enrichment, suppression, north-bound integrations, or machine learning.

This microservice is part of the Event microservice pipeline. See Understanding the Event Pipeline in Unified Assurance Concepts for conceptual information.

This microservice provides additional Prometheus monitoring metrics. See Event Canal Self-Monitoring Metrics.

Event Canal Prerequisites

Before deploying the microservice, confirm that the following prerequisites are met:

1. A microservice cluster is set up. See Microservice Cluster Setup.

2. The Pulsar microservice is deployed. See Pulsar.

Deploying Event Canal

To deploy the microservice, run the following commands:

su - assure1
export NAMESPACE=<namespace>
export WEBFQDN=<WebFQDN> 
a1helm install <microservice-release-name> assure1/event-canal -n $NAMESPACE --set global.imageRegistry=$WEBFQDN

In the commands:

• <namespace> is the namespace where you are deploying the microservice. The default namespace is a1-zone1-pri, but you can change the zone number and, when deploying to a redundant cluster, change pri to sec.

• <WebFQDN> is the fully-qualified domain name of the primary presentation server for the cluster.

• <microservice-release-name> is the name to use for the microservice instance. Oracle recommends using the microservice name (event-canal) and deploying only one instance of the Event Canal microservice for each Unified Assurance cluster.

You can also use the Unified Assurance UI to deploy microservices. See Deploying a Microservice by Using the UI for more information.

Changing Event Canal Configuration Parameters

When running the install command, you can optionally change default configuration parameter values by including them in the command with additional --set arguments. You can add as many additional --set arguments as you need.

For example, set a parameter described in Default Event Canal Configuration by adding --set configData.<parameter_name>=<parameter_value>. For example, --set configData.LOG_LEVEL=DEBUG.

Default Event Canal Configuration

The following table describes the default configuration parameters found in the Helm chart under configData for the microservice.

About Event Canal and the MySQL binlog

The MySQL Event database binary log files (binlog) record of all inserts, updates, and deletes on the MySQL server. The binlog supports replication and data recovery. Binlog entries include global transaction identifiers (GTIDs), which identify MySQL transactions.

The Event Canal microservice listens to binlog events for the Event database. When you first deploy it, it starts streaming from the most recent GTID and saves the GTID, binlog file name, and position in the file to the database. When a pod restarts, Event Canal gets the GTID from the database to determine the last binlog file position. If that GTID is still valid, it starts streaming events from that point on. If the GTID is invalid or does not exist anymore, Event Canal checks the REPL_GTID_INVALID_STREAM_ALL_EVENTS configuration parameter to determine where to start:

• If REPL_GTID_INVALID_STREAM_ALL_EVENTS is set to true, it streams from the oldest available binlog event.

• If REPL_GTID_INVALID_STREAM_ALL_EVENTS is set to false, it streams from the most recent binlog event.

For more information about binlog and using GTIDs, see The Binary Log and Replication with Global Transaction Identifiers in MySQL Reference Manual.

Event Canal Self-Monitoring Metrics

The Event Canal microservice exposes the self-monitoring metrics described in the following table to Prometheus.

Sample JSON Event Schema

Event data is normalized into a common JSON schema. The sample schema below shows the default fields, but any custom fields you add to the Event.Events table will automatically be included in the JSON.

Some fields are converted from their native format into ones appropriate for consumers of the output topic.

{
  "@timestamp": "2006-01-02T15:04:05Z",
  "_domain": "fault",
  "_type": "event",
  "_version": "5.0.0",
  "_agents": [
    {
      "app": "event-canal",
      "output": "persistent://assure1/event/canal",
      "cluster": "<name of the cluster>",
      "node": "<Host FQDN>",
      "input": "mysql:///Event",
      "@timestamp": "2024-08-13T15:12:33Z",
      "host": "<host inet address>",
      "id": "<pod id>",
      "version": "5.0.0"
    }
  ],
  "event": {
    "EventID": 0,
    "EventKey": "",
    "EventCategory": 3,
    "EventType": "",
    "Ack": 0,
    "Action": "",
    "Actor": "",
    "Count": 1,
    "Customer": "",
    "Department": "",
    "Details": {},
    "DeviceType": "",
    "Duration": 0,
    "EscalationFlag": 0,
    "ExpireTime": 0,
    "FirstReported": "2006-01-02T15:04:05.999999999Z",
    "GeoPath": {
      "type": "LineString",
      "coordinates": [
        [
          0.0,
          0.0
        ],
        [
          0.0,
          0.0
        ]
      ]
    },
    "GeoLocation": {
      "type": "Point",
      "coordinates": [
        0.0,
        0.0
      ]
    },
    "IPAddress": "",
    "LastChanged": "2006-01-02T15:04:05.999999999Z",
    "LastReported": "2006-01-02T15:04:05.999999999Z",
    "Location": "",
    "Method": "",
    "Node": "",
    "OrigSeverity": 1,
    "OwnerName": "",
    "RootCauseFlag": 0,
    "RootCauseID": 0,
    "Score": 0,
    "Service": "",
    "ServiceImpact": 0,
    "Severity": "Unknown",
    "SubDeviceType": "",
    "SubMethod": "",
    "SubNode": "",
    "Summary": "",
    "TicketFlag": 0,
    "TicketID": "",
    "ZoneID": 0
  }
}