18 Creating Kafka Topics for ECE Integration

You create Apache Kafka topics so Oracle Communications Elastic Charging Engine (ECE) can communicate with HTTP Gateway, Diameter Gateway, and RADIUS Gateway.

Topics in this document:

Setting Up ECE and Gateways for Apache Kafka

To set up ECE with your Kafka topics:

  1. When you install ECE, do this:

    • Specify to use Apache Kafka topics and enter the details for your ECE notification topics, Suspense topic, ECE failure topic, and ECE overage topic.

    • (HTTP Gateway Only) Enable Network Repository Function (NRF) registration in one of your HTTP Gateway servers.

    • (BRM Gateway Only) Specify the details for connecting to the BRM Gateway.

    For more information, see "Installing Elastic Charging Engine" in ECE Installation Guide.

  2. During the ECE postinstallation process, do this:

    • Run the kafka_post_install.sh script to create your ECE notification topics, Suspense topic, ECE failure topic, and ECE overage topic. See "Creating Kafka Topics for ECE" in ECE Installation Guide.

    • Run the post_Install.sh script and choose to create only the Acknowledgment queue. See "Creating WebLogic JMS Queues for BRM" in ECE Installation Guide.

    Note:

    It is strongly recommended that you use the kafka_post_install.sh script to create all ECE notification topics in advance instead of relying on topic auto-creation. This ensures that ECE notification consumers will be able to subscribe to topics even before the first notification message is produced.

    All ECE consumer groups are prefixed with Ece. Your consumers should never use a consumer group that starts with this prefix to avoid collisions.

  3. Connect ECE to your Kafka Server topics. See "Connecting ECE and Gateways to Kafka Topics".

  4. (Optional) Configure ECE to send information about failed usage requests to the ECE failure topic. See "Recording Failed ECE Usage Requests".

  5. (Optional) Configure ECE to generate CDRs for any prepaid usage overage and send them to the ECE overage topic. See "Configuring ECE to Support Prepaid Usage Overage".

Connecting ECE and Gateways to Kafka Topics

You connect ECE to the following Kafka topics so that ECE can publish notifications, failed usage requests, and CDRs with usage overage information to them.

  • ECE notification topics: Stores notifications from ECE.

  • Suspense topic: Stores failed notifications from ECE.

  • ECE failure topic: (HTTP Gateway and Diameter Gateway only) Stores details about failed ECE usage requests, such as the user ID and request payload.

  • ECE overage topic: (HTTP Gateway and Diameter Gateway only) Stores overage records, which contain details about usage overage amounts for prepaid customers.

You can also connect the HTTP Gateway, Diameter Gateway, and RADIUS Gateway to these Kafka topics so they can retrieve notifications, failed usage requests, and CDRs with usage overage information from them.

To connect ECE and gateways to your Kafka topics:

  1. Access the ECE configuration MBeans in a JMX editor, such as JConsole. See "Accessing ECE Configuration MBeans".

  2. Expand the ECE Configuration node.

  3. Expand charging.kafkaConfigurations.

  4. Expand Attributes.

  5. Specify the Kafka configuration values for the attributes in Table 18-1.

    Table 18-1 kafkaConfiguration Properties

    Property Name Description

    name

    The name of your ECE cluster.

    hostname

    The host name and port number of the machine in which Apache Kafka is installed.

    If it contains multiple Kafka brokers, create a comma-separated list.

    topicName

    The name of the Kafka topic where ECE will publish notifications.

    suspenseTopicName

    The name of the Kafka topic where failed notifications are published.

    failureTopicName

    The name of the Kafka topic where ECE will publish details about failed usage requests.

    overageTopicName

    The name of the Kafka topic where ECE will publish overage records with information about your prepaid customer's usage overage during online sessions.

    partitions

    The number of Kafka partitions in your topics ECE notification topic.

    The recommended number to create is calculated as follows:

    [(Max HTTP Gateway Nodes) + (Max Diameter Gateway Nodes * Max Diameter Clients) + (1 for BRM Gateway) + (1 for Internal Notifications)]

    For example, if you have 2 HTTP Gateway nodes, 4 Diameter Gateway nodes, 10 Diameter Gateway clients, and a BRM Gateway, you would need [(2 + (4 * 10) + 1 + 1) = 44 Kafka partitions.

    Arbitrarily, you can set this to a maximum value.

    failurePartitions

    The number of Kafka partitions in your ECE failure topic.

    kafkaProducerReconnectionInterval

    The amount of time, in milliseconds, the ECE Notification Publisher waits before attempting to reconnect to a Kafka topic.

    kafkaProducerReconnectionMax

    The maximum amount of time, in milliseconds, the ECE Notification Publisher waits before attempting to reconnect to a broker that has repeatedly failed to connect.

    The kafkaProducerReconnectionInterval will increase exponentially for each consecutive connection failure, up to this maximum.

    kafkaHTTPReconnectionInterval

    The amount of time, in milliseconds, HTTP Gateway waits before attempting to reconnect to the Kafka topic.

    kafkaHTTPReconnectionMax

    The maximum amount of time, in milliseconds, HTTP Gateway waits before attempting to reconnect to a broker that has repeatedly failed to connect.

    The kafkaHTTPReconnectionInterval will increase exponentially for each consecutive connection failure, up to this maximum.

    kafkaDGWReconnectionInterval

    The amount of time, in milliseconds, Diameter Gateway waits before attempting to reconnect to the Kafka topic.

    kafkaDGWReconnectionMax

    The maximum amount of time, in milliseconds, Diameter Gateway waits before attempting to reconnect to a broker that has repeatedly failed to connect.

    The kafkaDGWReconnectionInterval will increase exponentially for each consecutive connection failure, up to this maximum.

    kafkaBRMReconnectionInterval

    The amount of time, in milliseconds, BRM Gateway waits before attempting to reconnect to the Kafka topic.

    kafkaBRMReconnectionMax

    The maximum amount of time, in milliseconds, BRM Gateway waits before attempting to reconnect to a broker that has repeatedly failed to connect.

    The kafkaBRMReconnectionInterval will increase exponentially for each consecutive connection failure, up to this maximum.

    kafkaRGWReconnectionIntervalMillis

    The amount of time, in milliseconds, RADIUS Gateway waits before attempting to reconnect to the Kafka topic.

    kafkaRGWReconnectionMaxMillis

    The maximum amount of time, in milliseconds, RADIUS Gateway waits before attempting to reconnect to a broker that has repeatedly failed to connect.

    The kafkaRGWReconnectionIntervalMillis will increase exponentially for each consecutive connection failure up to this maximum.

Recording Failed ECE Usage Requests

ECE may occasionally fail to process usage requests. For example, a data usage request could fail because a customer has insufficient funds. You can configure ECE to publish details about failed usage requests, such as the user ID and request payload, to the ECE failure topic in your Kafka server. Later on, you can reprocess the usage requests or view the failure details for analysis and reporting.

To enable the recording of failed ECE usage requests:

  1. Access the ECE configuration MBeans in a JMX editor, such as JConsole.

  2. Expand the ECE Configuration node.

  3. Expand charging.kafkaConfigurations.

  4. Expand Attributes.

  5. Set the persistFailedRequestsToKafkaTopic property to true.

Note:

You can also examine the gateway log files to determine why a usage request failed. See "Troubleshooting Failed Usage Requests" in BRM System Administrator's Guide for more information.

Configuring ECE to Support Prepaid Usage Overage

You can configure ECE to capture any overage amounts by prepaid customers during an active session, which can help you prevent revenue leakage. If the network reports that the number of used units during a session is greater than a customer's available allowance and credit limit, ECE charges the customer up to the available allowance. It then creates an overage record with information about the overage amount and sends it to the ECE Overage topic. You can create a custom solution for reprocessing the overage amount later on.

Note:

If your system does not contain a Kafka server, you can configure ECE to publish overage usage amounts to a log file instead. See "Logging Prepaid Usage Overage".

For example, assume a customer has a prepaid balance of 100 minutes, but uses 130 minutes during a session. ECE would charge the customer for 100 minutes, create an overage record for the remaining 30 minutes of usage, and write the overage record to the ECE Overage topic.

To configure ECE to support prepaid usage overage, do the following:

  1. Ensure that you created an ECE Overage topic and connected ECE to your Kafka Server. See "Connecting ECE and Gateways to Kafka Topics".

  2. Enable ECE to check for and capture any usage overage:

    1. Access the ECE configuration MBeans in a JMX editor, such as JConsole. See "Accessing ECE Configuration MBeans".

    2. Expand the ECE Configuration node.

    3. Expand charging.server.

    4. Expand Attributes.

    5. Set the checkReservationOverImpact attribute to true. (The default is false.)

You can also customize ECE to include any existing unrated quantity in a CDR or notification using the ECE SDK post charging extension (postCharagingExtension). This can be done using the OverageDetailsImpl class. The unrated quantity can be then captured by the getOverageDetailsMap() method that is available in the extension.