TCPClientd
The Oracle Communications Unified Assurance Event TCP Client Aggregator is a generic integration that connects to a device over a TCP socket, reads the available messages, parses the results with customizable rules and creates de-duplicated events within Unified Assurance.
By default, Unified Assurance has the Event TCP Client Aggregator and Event TL1 Aggregator event services that utilizes the TCP Client Aggregator to collect events.
You can run this application as a service using the Services UI.
TCP Client Aggregator Setup
-
Determine the host and port settings needed to establish a client connection.
-
Review the logic in the rules files referenced in the configuration to see the processing that will be done when messages are retrieved:
-
LoadRules will be run during application startup to load data that might be needed during processing.
-
IncludeRules will be read during application startup to load additional files that might be called during processing.
-
BaseRules will be run for each device that is selected based on the configuration.
Update the logic as needed.
-
-
Create a clone of the default service, making changes to the configuration as needed.
-
Enable the Service.
Default Service
The following table shows the settings for the default service. Actual values are in bold, descriptions of values are in plaintext.
Field | Value |
---|---|
Package | coreCollection-app |
Name | Event TCP Client Aggregator |
Program | bin/core/collection/TCPClientd |
Arguments | This field is blank. There is no default value. |
Description | TCP Client Aggregator that collects from custom ports |
Failover Type | Standalone (Supported: Standalone, Primary, Redundant/Backup) |
Status | Disabled |
Privileged | This option is selected. |
See Services in Unified Assurance User's Guide for general information about the settings for services.
See Using Application Primary/Backup Failover for more information about the different failover types.
Default Configuration
The following table shows the default configurations for the application. Actual values are in bold, descriptions of values are in plaintext.
Name | Default Value | Possible Values | Notes |
---|---|---|---|
BaseRules | collection/event/tcpclient/base.rules | Text, 255 characters | The relative path to the application Base Rules file. |
BranchDir | core/default | Text, 255 characters | The relative path to the rules directory. |
IncludeRules | collection/event/tcpclient/base.includes | Text, 255 characters | The relative path to the application Include Rules file. |
LoadRules | collection/event/tcpclient/base.load | Text, 255 characters | The relative path to the application Load Rules file. |
LogFile | logs/EventTCPClient.log | Text, 255 characters | The relative path to the log file. |
LogLevel | ERROR | OFF, FATAL, ERROR, WARN, INFO, DEBUG | The logging level for the application. |
MetaData | This field is blank. There is no default value. | Text, 255 characters | Value to determine which devices should be monitored by this aggregator. (Requires MetaTag.) - NO RELOAD CONFIG SUPPORT |
MetaTag | This field is blank. There is no default value. | Text, 255 characters | Meta Tag to determine which devices should be monitored by this aggregator. Value should match one of the existing Meta Tag Types. (Requires MetaData.) - NO RELOAD CONFIG SUPPORT |
ShardID | 1 | An integer | Database shard to be used. |
Threads | 10 | An integer | Number of process threads created. The aggregator takes a third of this value (rounded up) for database threads unless overridden by the DBThreads application configuration. |
Buffer | This field is blank. There is no default value. | An integer | (Optional) If set, the amount of what has been read to can handle greater than 4096 message sizes up to a max buffer. |
Capture | Disabled | Enabled or Disabled | (Optional) If enabled, saves the raw message in the Log. |
DBThreads | This field is blank. There is no default value. | An integer | (Optional) Number of database threads to be created. If not specified, defaults to a third (rounded up) of Threads application configuration. |
DelimiterFile | This field is blank. There is no default value. | Text, 255 characters | (Optional) Path to file that contains the additional delimiters used to delineate records. If not specified, delimiter is newline (\n). |
EventBuffer | Enabled | Enabled or Disabled | (Optional) Enables the Event buffer |
FailoverBufferLimit | 0 | An integer | (Optional) Enables Failover Standby buffer that keeps N-seconds worth of messages and replays this buffer when becoming Failover Active. (0=off N=seconds to keep) See Tokens: $buffer and $received |
FieldSetFile | This field is blank. There is no default value. | Text, 255 characters | (Optional) Path to csv file containing custom list of fields that will be used when inserting data. (Requires InsertSQLFile.) |
Host | This field is blank. There is no default value. | Text, 255 characters | (Optional) (DEPRECATED: Use MetaTag and MetaData.) The DNS name or IP Address of the TCP Client. (Requires Port.) - NO RELOAD CONFIG SUPPORT |
InsertSQLFile | This field is blank. There is no default value. | Text, 255 characters | (Optional) Path to file containing custom SQL Insert statement for handling of event inserts. (Requires FieldSetFile.) |
Port | This field is blank. There is no default value. | An integer | (Optional) (DEPRECATED: Use MetaTag and MetaData.) The port for TCP Client collection. (Requires Host.) NO RELOAD CONFIG SUPPORT |
PreferIPv4 | Enabled | Enabled or Disabled | (Optional) Controls whether or not to prefer IPv4 transport to communicate with Devices. This option is only considered if both IPv4 and IPv6 are available for a device. If this config is missing, IPv6 will be preferred. |
ReconnectTime | 1200 | An integer | (Optional) Time to wait in seconds for TCP messages from a device before disconnecting and attempting to reconnect. Defaults to 1200 |
RetryTime | 1200 | An integer | (Optional) Time to wait in seconds before attempting to retry connecting to a device. Defaults to 1200 |
SocketTimeout | 2 | An integer | (Optional) Time to wait in seconds for a socket to become available before trying again. |
SplitCaptureByDevice | Disabled | Enabled or Disabled | (Optional) Outputs capture to separate logs by device (Requires Capture) |
Event TL1 Aggregator Service
Field | Value |
---|---|
Package | coreCollection-app |
Name | Event TL1 Aggregator |
Program | bin/core/collection/TCPClientd |
Arguments | This field is blank. There is no default value. |
Description | TCP Client Aggregator that collects from the TL1 Gateway |
Failover Type | Standalone (Supported: Standalone, Primary, Redundant/Backup) |
Status | Disabled |
Privileged | This option is selected. |
See Services in Unified Assurance User's Guide for general information about the settings for services.
See Using Application Primary/Backup Failover for more information about the different failover types.
Event TL1 Aggregator Configuration
Name | Default Value | Possible Values | Notes |
---|---|---|---|
BaseRules | collection/event/tl1g/base.rules | Text, 255 characters | The relative path to the application Base Rules file. |
BranchDir | core/default | Text, 255 characters | The relative path to the rules directory. |
IncludeRules | collection/event/tl1g/base.includes | Text, 255 characters | The relative path to the application Include Rules file. |
LoadRules | collection/event/tl1g/base.load | Text, 255 characters | The relative path to the application Load Rules file. |
LogFile | logs/EventTL1.log | Text, 255 characters | The relative path to the log file. |
LogLevel | ERROR | OFF, FATAL, ERROR, WARN, INFO, DEBUG | The logging level for the application. |
MetaData | This field is blank. There is no default value. | Text, 255 characters | Value to determine which devices should be monitored by this aggregator. (Requires MetaTag.) - NO RELOAD CONFIG SUPPORT |
MetaTag | This field is blank. There is no default value. | Text, 255 characters | Meta Tag to determine which devices should be monitored by this aggregator. Value should match one of the existing Meta Tag Types. (Requires MetaData.) - NO RELOAD CONFIG SUPPORT |
ShardID | 1 | An integer | Database shard to be used. |
Threads | 10 | An integer | Number of process threads created. The aggregator takes a third of this value (rounded up) for database threads unless overridden by the DBThreads application configuration. |
Buffer | 32 | An integer | (Optional) If set, the amount of what has been read to can handle greater than 4096 message sizes up to a max buffer. |
Capture | Disabled | Enabled or Disabled | (Optional) If enabled, saves the raw message in the Log. |
DBThreads | This field is blank. There is no default value. | An integer | (Optional) Number of database threads to be created. If not specified, defaults to a third (rounded up) of Threads application configuration. |
DelimiterFile | This field is blank. There is no default value. | Text, 255 characters | (Optional) Path to file that contains the additional delimiters used to delineate records. If not specified, delimiter is newline (\n). |
EventBuffer | Enabled | Enabled or Disabled | (Optional) Enables the Event buffer |
FailoverBufferLimit | 0 | An integer | (Optional) Enables Failover Standby buffer that keeps N-seconds worth of messages and replays this buffer when becoming Failover Active. (0=off N=seconds to keep) See Tokens: $buffer and $received |
FieldSetFile | This field is blank. There is no default value. | Text, 255 characters | (Optional) Path to csv file containing custom list of fields that will be used when inserting data. (Requires InsertSQLFile.) |
Host | This field is blank. There is no default value. | Text, 255 characters | (Optional) (DEPRECATED: Use MetaTag and MetaData) The DNS name or IP Address of the TCP Client. (Requires Port.) NO RELOAD CONFIG SUPPORT |
InsertSQLFile | This field is blank. There is no default value. | Text, 255 characters | (Optional) Path to file containing custom SQL Insert statement for handling of event inserts. (Requires FieldSetFile.) |
Port | This field is blank. There is no default value. | An integer | (Optional) (DEPRECATED: Use MetaTag and MetaData) The port for TCP Client collection. (Requires Host.) NO RELOAD CONFIG SUPPORT |
PreferIPv4 | This field is blank. There is no default value. | Enabled or Disabled | (Optional) Controls whether or not to prefer IPv4 transport to communicate with Devices. This option is only considered if both IPv4 and IPv6 are available for a device. If this config is missing, IPv6 will be preferred. |
ReconnectTime | 1200 | An integer | (Optional) Time to wait in seconds for TCP messages from a device before disconnecting and attempting to reconnect. Defaults to 1200 |
RetryTime | 1200 | An integer | (Optional) Time to wait in seconds before attempting to retry connecting to a device. Defaults to 1200 |
SocketTimeout | 2 | An integer | (Optional) Time to wait in seconds for a socket to become available before trying again. |
SplitCaptureByDevice | Disabled | Enabled or Disabled | (Optional) Outputs capture to separate logs by device (Requires Capture) |
Best Practices
The following list shows the best practices for working with this application:
- To let the aggregator run more efficiently and connect to multiple endpoints, a Meta Tag specific to the devices that will be connected should be used and then specified in the Application Configuration. In addition, the Meta Tag TCPAggregatorPort should be added to the devices with the value set to the port to be used.
Rules
This aggregator uses the Unified Assurance standard rules architecture in Perl syntax. For information about creating rules, see the following in Unified Assurance Developer's Guide:
-
Core for information about core rules functions.
-
Events for information about Event rules functions.
Tokens
The aggregator exposes the following tokens for rules processing.
Token | Description |
---|---|
$Event | Reference to the hash that is used to create and insert the Event data into the database. Keys map to the fields within the table used and values assigned are inserted in the database to that field. (e.g. $Event->{'IPAddress'} = '192.0.2.1' to assign the event IP address to '192.0.2.1') At least the 'Node' and 'Summary' fields must be set, or no event is inserted. |
$Packet | TCP Socket Message (hash reference) |
$Packet->{Received} | epoch time packet was received by the aggregator |
$Packet->{Buffer} | Flag for if was buffered during standby and was replayed (0 = No, 1 = Yes) |
$Packet->{Message} | Message Received |
$Packet->{Target} | Default connection point for Remote Client or Server. Will be $Packet->{DNS} if $Packet->{DNS} is populated, otherwise $Packet->{IP} |
$Packet->{DNS} | DNS of remote server |
$Packet->{DeviceName} | Device Name of remote server. |
$Packet->{Host} | (DEPRECATED: Use $Packet->{DeviceName}) Device Name of remote server. |
$Packet->{IP} | Preferred IP Address of Remote Client or Server calculated from $IPv4, $IPv6, and the PreferIPv4 application configuration. |
$Packet->{IPv4} | IPv4 Address of remote server. |
$Packet->{IPv6} | IPv6 Address of remote server. |
$discard_flag | Flag for discard (0=No, 1=Yes) |
$count | Message Counter |
$AppConfig | Hash reference to the application configuration name-value pairs that were configured. (i.e. use $AppConfig->{'Host'} to retrieve the set value for 'Host'.) |
$CustomHash | Custom key, value cache available across all rules. Contents commonly defined in Load Rules then used in Base or other rules. NOTE: This variable is a shared object and any additional sub hashes or arrays must be shared before use or it will cause the error: Invalid value for shared scalar. Instantiate the sub hash/array using '&share({})' e.g. $CustomHash->{SubObject} = &share({}); |
$StorageHash | Internal cache used as the StorageHash option when calling rules functions such as FindDeviceID(). NOTE: The structure of this cache is subject to change! Not recommended for custom global storage or manual manipulation; use $CustomHash. |
Example Integrations
Creating Custom Rules Files
The TCP Client Aggregator will split each message using the defined Delimiter, by default newline (\n). So if the message is:
This Device is Down\n This Device is Up\n
That will create two events in Unified Assurance.
These two events will be used as an example of how the messages could be parsed in the rules file.
Steps
-
Go to the Rules UI:
-
Expand the folder path: core -> default -> collection -> event -> tcpclient
-
Select the tcpclient folder, then click Add -> Add File. Enter the following:
-
File Name => newbase.rules
-
Logic
$Log->Message('DEBUG', "Ran Base Rules"); $Log->Message('DEBUG', [ '#===============================================', '# Received New TCP Client Message', '# DNS = ' . $Packet->{'DeviceName'}, '# IP = ' . $Packet->{'IP'}, '# Message = ' . $Packet->{'Message'}, '#===============================================' ]); $Event->{'Node'} = $Packet->{'DeviceName'}; $Event->{'IPAddress'} = $Packet->{'IP'}; $Event->{'Summary'} = $Packet->{'Message'}; $Event->{'Severity'} = 0; $Event->{'AlarmGroup'} = 'Unknown'; $Event->{'AlarmKey'} = 'GenericTCP-->' . $Event->{'Node'} . ':' . $Event->{'Summary'}; if ($Packet->{'Message'} eq '') { $Event->{'SubMethod'} = "Error"; $Log->Message('ERROR', "Blank message in packet"); $discard_flag = 1; } elsif ($Packet->{'Message'} =~ "Down") { $Event->{'AlarmGroup'} = "Down"; $Event->{'Severity'} = 5; } elsif ($Packet->{'Message'} =~ "Up") { $Event->{'AlarmGroup'} = "Up"; $Event->{'Severity'} = 0; } else { # NO Rules $Event->{'SubMethod'} = "Generic TCP"; $Log->Message('ERROR', "!ERROR! No Rules Defined for TCP"); $Log->Message('DEBUG', "No Rules for TCP Message [" . $Packet->{Message} . "]"); } $Log->Message('DEBUG', "Finished Base Rules");
When the event message is parsed, if the message contains Down, a Down Event is created with Severity set to 5 (Critical).
If the message contains Up, an event is passed in which clears the previous alarm by parsing the message and setting Severity to 0 (Normal).
-
Click Submit, then enter a commit message, then click OK.
-
-
Create a clone of the default service, making changes to the configuration as needed for connectivity, and change the BaseRules configuration to use the newbase.rules.
-
Verify the aggregator is processing the events correctly.
Administration Details
The following list shows the technical details you will need for advanced administration of the application:
-
Package: coreCollection-app
-
Package:
./TCPClientd [OPTIONS]
-
Options:
-c, --AppConfigID N Application Config ID (Service, Job, or Request ID) -?, -h, --Help Print usage and exit
-
Threaded: Multithreaded