Understanding the Event Pipeline

Learn about the Oracle Communications Unified Assurance Event microservice pipeline.

About the Event Pipeline

The Event pipeline collects events from discovered devices in the Unified Assurance database, processes and enriches or suppresses them, inserts them into the Event database, and optionally publishes them for consumption by other services.

The pipeline includes these flows, which all involve events in the Event database:

The following figure shows how events flow through the pipeline.

Event Microservice Pipeline

Description of illustration microservice-event-pipeline.png

Event Pipeline Components

The Event pipeline consists of the following components, sorted by component type and location in the pipeline:

Type Component Description
Microservice Trap Collector Collects SNMP trap events, converts them to JSON, and publishes them to the assure1/event/collection topic.
Microservice Syslog Collector Collects syslog events in RFC3164 and RFC5424 format, converts them to JSON, and publishes them to assure1/event/collection.
Microservice CORBA Collector Collects CORBA MTNM events and publishes them to assure1/event/collection. In the figure, this is grouped with the other collector microservices.
Microservice VMware Event Collector Collects events from VMware devices and publishes them to assure1/event/collection. In the figure, this is grouped with the other collector microservices.
Microservice Mist Event Collector Collects alarms and events from Juniper Mist clouds and publishes them to the assure1/event/sink topics. In the figure, it is grouped with other poller microservices in the non-FCOM flow.
Microservice Ciena MCP Poller Collects events from Ciena MCP devices and publishes them to assure1/event/sink.
Microservice Cisco Meraki Poller Collects events from Cisco Meraki devices and publishes them to assure1/event/sink.
Microservice RCA Availability Engine Reads events from assure1/discovery/rca, correlates and enriches outage events with root causes, and publishes them to assure1/event/collection.
Microservice FCOM Processor Reads events from assure1/event/collection, runs them through the fault common object model (FCOM) to normalize them, and publishes them to assure1/event/sink.
Microservice Trap Forwarder Reads events from the assure1/event/forward topic, converts them from JSON to SNMP traps, and forwards them to a UDP socket.
Microservice JavaScript Generic Event Processor Reads events from the assure1/event/js-generic-processor topic, enriches them according to custom JavaScript rules, and publishes them to assure1/event/sink.
Microservice Event Sink Reads events from assure1/event/sink and inserts them into the Event database.
Microservice Event Canal Gets events from the Event database and publishes them to the assure1/event/canal topic.
Supporting Microservice Prometheus Collects performance metrics from microservices and inserts them into the Metrics database. This is not shown in the figure. It runs in the background for all microservices.
Supporting Microservice Apache Pulsar Provides the messaging framework for pipeline microservices to publish and read data.
Supporting Microservice Kafka Bridge Reads events from the assure1/event/tmf688 topic and publishes them to an external Kafka bus.
Supporting Microservice ActiveMQ Bridge Optional integration for bidirectional communication between ActiveMQ and Pulsar. You can configure this microservice to read events from an ActiveMQ topic and publish them to a Pulsar topic, or the reverse. This is not shown in the figure.
Supporting Microservice ServiceNow Adapter Optional integration to read events from the Event database, publish them to a ServiceNow server for ticket creation, and keep the event journal and ServiceNow worklog synchronized. This is not shown in the figure.
Supporting Microservice Device Cache Warmup Optional service to perform a one-time preload devices from a set of zones into redis when redis is first installed. This is not shown in the figure.
Supporting Microservice redis Optional in-memory data structure storage. This is not shown in the figure.
Pulsar Topic assure1/event/collection The collectors and processors publish events here, FCOM Processor microservice subscribes.
Pulsar Topic assure1/event/sink The FCOM Processor microservice and pollers publish events here, Event Sink microservice subscribes.
Pulsar Topic assure1/event/js-generic-processor No microservices publish here by default, but you can configure an instance of the FCOM Processor microservice to do so if needed. JavaScript Generic Event Processor microservice subscribes.
Pulsar Topic assure1/event/canal The Event Canal microservice publishes events here. Event Analytics, the Vision application, and the TMF688 Event Processor microservice subscribe. In multi-server environments, ensure that microservices consume messages from only one cluster at a time.
Pulsar Topic assure1/event/forward No microservices publish here by default, but you can configure an instance of the FCOM Processor microservice to do so if needed. The Trap Forwarder microservice subscribes.
Database Event Database The Event Sink microservice inserts events here. The Event Canal gets events from here.

Event Pipeline Flows

This section describes the flows that make up the Event pipeline.

FCOM Flow

The FCOM flow is as follows:

  1. The collectors and pollers collect events and convert them to JSON.

    If the RCA Availability Engine microservice is also deployed, it reads ping and SNMP availability events collected by pollers in the Discovery pipeline and enriches them with root cause information.

  2. The collectors, pollers, and engine publish the events to assure1/event/collection.

  3. The FCOM Processor microservice reads the events from assure1/event/collection and processes them according to the FCOM rules and your custom overrides.

  4. The FCOM Processor microservice publishes the events to any of the following topics:

    1. assure1/event/sink: This is the default. The flow continues.

    2. assure1/event/js-generic-processor: The JavaScript Generic Event Processor microservice reads the events, applies custom enrichment, and publishes them to assure1/event/sink, where the flow continues.

    3. assure1/event/forward: The Trap Forwarder microservice reads the events, converts them from JSON, and forwards them to a UDP socket for external consumption.

  5. The Event Sink microservice reads the events from assure1/event/sink.

  6. The Event Sink microservice inserts the events into the Event database.

  7. The Event Canal microservice gets events from the Event database.

  8. The Event Canal microservice publishes the events to assure1/event/canal.

Non-FCOM Flow

The non-FCOM flow is as follows:

  1. The pollers poll events from discovered devices and convert the events to JSON.

  2. The pollers publish the events to assure1/event/sink.

  3. The Event Sink microservice reads the events from assure1/event/sink.

  4. The Event Sink microservice inserts them into the Event database.

  5. The Event Canal microservice gets events from the Event database.

  6. The Event Canal microservice publishes the events to assure1/event/canal.

TMF Event Flow

The TMF Event Flow publishes Unified Assurance events to a Kafka bus in a format compliant with the TM Forum (TMF) 688 Event Management API specification, with embedded schemas for alarm events and performance threshold events from the TMF 642 Alarm Management API specification.

Note:

Unified Assurance does not provide a TMF API and does not consume events in TMF formats. The TMF Event Flow translates Unified Assurance events into TMF-compliant format and publishes them to an output stream for external applications to consume.

Events are collected, processed, added to the Event database, and published to assure1/event/canal as described in FCOM Flow and Non-FCOM Flow. Then:

  1. The TMF688 Event Processor microservice reads the events, filters them for relevant alarms and thresholds, and converts the filtered events to TMF688 format. You can optionally add custom fields as TMF extensions using transformation rules within the TMF688 Event Processor.

  2. The TMF688 Event Processor microservice publishes the events to assure1/event/tmf688-event.

  3. The Kafka Bridge microservice reads the events.

  4. The Kafka Bridge microservice publishes the events to the external Kafka URI.

After the events are on the external Kafka bus, external inventory applications, such as Oracle Communications Unified Inventory Management, can read the events from the Kafka topic and act on them accordingly.

See TMF688 Event Processor in Unified Assurance Implementation Guide for information about filters, TMF formats, and how to add TMF extensions using transformation rules.

For general information about microservices and microservice pipelines, see Understanding Microservices and Understanding Microservice Pipelines

For more information about the Discovery microservice pipeline that feeds into this pipeline, see Understanding the Discovery Pipeline.

For a conceptual overview of root cause analysis, see Understanding Root Cause Analysis.

For information about deploying and configuring the microservices in this pipeline, see the following topics in Unified Assurance Implementation Guide: