8 Recycling EDRs in Pipeline-Only Systems
EDR recycling is used by systems that use Pipeline Manager but do not store suspended EDRs in the Oracle Communications Billing and Revenue Management (BRM) database. It uses FCT_PreRecycle to mark EDRs for recycling, and FCT_Recycle to send the rejected EDRs to a file for manual processing.
Topics in this document:
Systems using BRM with Pipeline Manager use either the standard recycling tools or the Suspense Manager service integration component for recycling and deleting EDRs. For details, see "About the EDR Recycling Features".
Note:
Before reading this document, you should be familiar with how Pipeline Manager works and how to configure it.
About Recycling EDRs
When processing a CDR file, there might be non-valid EDRs in the file, or your pipelines might not be set up correctly to handle certain EDRs. You use EDR recycling to fix configuration problems and re-process EDRs.
The recycling process uses these pipeline modules:
-
FCT_Reject
-
FCT_PreRecycle
-
FCT_Recycle
EDR recycling consists of the following high-level steps:
-
You start Pipeline Manager with active FCT_PreRecycle, FCT_Recycle, and FCT_Reject modules. (The FCT_PreRecycle and FCT_Recycle modules do nothing until you begin the recycling process using a semaphore.)
-
When an EDR is processed, a module may find an error in it. The module appends the error to the EDR and sets a flag to indicate that it has an error. The EDR is then sent to the next module. If any more errors are found, each module adds them.
-
The FCT_Reject module analyzes the errors in the EDR. If necessary, it moves the EDR to a reject file.
-
You examine the errors and determine how to reconfigure Pipeline Manager to prevent the errors.
-
You use a semaphore file entry to start the pre-recycling process, which sends the rejected EDRs through the pipeline again. The FCT_PreRecycle module adds a flag to the EDR to let the other modules know that it is being recycled.
You can pre-recycle and recycle EDRs in test mode or real mode. Typically, you run the pre-recycle and recycling processes in test mode to see if the errors have been fixed. When there are no longer any errors, you pre-recycle and recycle in real mode.
-
The FCT_Recycle module runs at the end of the pipeline. It does one of the following:
-
In test mode, the module creates a report about the processing but does not send the EDRs to an output file.
-
In recycle mode, the module sends the results to an output file and attaches a sequence number to it.
Note:
You can configure the output module to send an entire file to the error directory if it contains many errors. You can also configure the threshold for the number of errors allowed per file. See "Specifying the Maximum Errors Allowed in an Input File" in BRM Setting Up Pipeline Rating and Discounting.
-
How the FCT_PreRecycle Module Works
The FCT_PreRecycle module is always the first module in the pipeline. Although you can activate the FCT_PreRecycle module from the startup registry, you cannot run the FCT_PreRecycle module from the startup registry. You must run it by using a semaphore file.
The FCT_PreRecycle module does the following:
-
Gets the file of rejected EDRs from the reject output stream directory.
-
Puts the rejected EDR file into the input directory for recycling. It uses the same input directory as the incoming CDR files. It adds a recycle suffix to the file and a sequence number, so the original input file in the output directory cannot be overwritten.
You can recycle all EDRs in the reject directory or list specific files to recycle. See "Recycling EDRs".
-
For each EDR to recycle, sets a value in the INTERN_PROCESS_STATUS field to indicate that it is being recycled. This tells the FCT_Recycle module which EDRs to process, and allows the discounting modules to recalculate discount amounts correctly.
-
The value is set to 1 if the EDR is being recycled.
-
The value is set to 2 if the recycling is in test mode.
-
You can recycle all EDRs in the reject directory or list specific files to recycle.
How the FCT_Reject Module Works
Run the FCT_Reject module after all rating and enrichment modules and as the second-to-last function module in the pipeline. (The last function module is the FCT_Suspense module). You run it second-to-last because all potential errors must be found before the FCT_Reject module processes the EDRs.
You can run the FCT_Reject module from the registry or by using a semaphore file entry.
The FCT_Reject module does the following:
-
The FCT_Reject module checks the error status of the EDR. If the EDR contains an error status with a warning or critical severity, the EDR is rejected. The FCT_Reject module changes the value of the DETAIL.DISCARDING field from 0 to 1.
Note:
If the DETAIL.DISCARDING field is already set to 1, the EDR was rejected in a previous pass through the pipeline, and is rejected again.
If the error type in the EDR is not identified in the registry StreamMap entry, the EDR is sent to default reject stream.
-
By default, the EDR is moved to the reject stream, as identified in the UseRejectStream registry entry. The EDR is stored in a file that is used by the recycling modules. EDRs can also be rejected in the recycle process.
If the reject stream is not specified, the EDR is moved to the normal output stream, but the discard field is set to 1, indicating that the EDR has been rejected.
How the FCT_Recycle Module Works
The FCT_Recycle module is the last function module in the pipeline, before the output.
You activate the FCT_Recycle module from the startup registry, but it does nothing until the FCT_PreRecycle module starts the recycling process.
The FCT_Recycle module reads the INTERN_PROCESS_STATUS field for each EDR.
-
If the value is 2, recycling is in test mode. The FCT_Recycle module doesn't send the EDRs to an output directory. Instead, the FCT_Recycle module creates a report with the following data:
-
Stream name.
-
Total number of processed EDRs.
-
Number of EDRs that can be recycled without an error.
-
Number of EDRs that still generate an error.
-
List of all errors.
-
The wholesale charge amount from all successfully recycled EDRs. (This data is taken from the WHOLESALE_CHARGE field.)
-
The wholesale charge amount from all EDRs that still have errors. (This data is taken from the WHOLESALE_CHARGE field.)
-
The total duration for all successfully recycled EDRs. (This data is taken from the DURATION_MINUTES field.)
-
The total duration from all EDRs that still have errors. (This data is taken from the DURATION_MINUTES field.)
You can use this data to determine if the EDRs are worth further configuration and processing.
-
-
If the value is 1, recycling occurs. All EDRs are processed as usual, with the following differences in comparison to normal input file processing:
-
A sequence number is generated.
-
The sequence offset value is generated.
-
The sequence check is inactivated.
-
Recycling EDRs in Test Mode
After determining that EDRs have been rejected, you correct any pipeline problems that caused the problem. You then recycle the CDR file in test mode to ensure that the changes have had the desired affect.
To test recycle EDRs:
-
Configure the FCT_Reject module. See "Configuring FCT_Reject to Reject EDRs".
Typically, rejected EDRs are sent to the reject stream. You configure the reject stream in the registry in the following places:
-
In the FCT_Reject module pipeline configuration
-
In the Output stream
For a sample output stream configuration, see "Sample Output Configuration" in BRM Pipeline Manager Reference.
-
-
Configure the FCT_PreRecycle module. See "Configuring FCT_PreRecycle to Move Rejected EDRs to the Input Directory".
You configure FCT_PreRecycle in the following places:
-
In the FCT_PreRecycle module pipeline configuration
-
In the input stream
The module uses the same input configuration as the incoming CDR files, so you don't need to configure a separate input stream.
-
-
Configure the FCT_Recycle module. See "Configuring FCT_Recycle to Recycle or Test Recycle EDRs".
Configure the FCT_Recycle module's RecycleLog registry entry to specify the message file parameters. These settings are specified in the ProcessLog registry section. For more information, see "LOG" in BRM Pipeline Manager Reference.
-
Use a semaphore to inactivate the FCT_Reject module:
Module Reject.Module.Active = False
-
Use a semaphore to run the FCT_PreRecycle module in test mode:
Recycle.Module.RecycleTest {}
-
Review the log files that you configured in FCT_Recycle for errors, and repeat these steps as necessary.
Recycling EDRs
When you finish test recycling EDRs, follow these steps to do the actual recycling:
-
Configure the FCT_Reject module. See "Configuring FCT_Reject to Reject EDRs".
Typically, rejected EDRs are sent to the reject output stream. You configure the reject output stream in the following places:
-
In the FCT_Reject module pipeline configuration
-
In the Output stream
For a sample output stream configuration, see "Sample Output Configuration" in BRM Pipeline Manager Reference.
-
-
Configure the FCT_PreRecycle module. See "Configuring FCT_PreRecycle to Move Rejected EDRs to the Input Directory".
You configure the reject stream in the registry in the following places:
-
In the FCT_PreRecycle module pipeline configuration
-
In the input stream
The module uses the same input configuration as the incoming CDR files, so you don't need to configure a separate input stream.
-
-
Configure the FCT_Recycle module. See "Configuring FCT_Recycle to Recycle or Test Recycle EDRs".
Configure the FCT_Recycle module's RecycleLog registry entry to specify the message file parameters. You specify these settings in the ProcessLog registry section. For more information, see "LOG" in BRM Pipeline Manager Reference.
-
Use a semaphore to run the FCT_PreRecycle module:
-
To recycle all files:
ifw.Pipelines.PRE_RECYCLE.Functions.Processing.FunctionPool.PreRecycle.Module.Recycle {}
-
To recycle only specific files:
ifw.Pipelines.PRE_RECYCLE.Functions.Processing.FunctionPool.PreRecycle.Module.Recycle. File = ./format_a/abc.cdr
-
-
Review the log files that you configured in FCT_Reject for errors.
Configuring FCT_PreRecycle to Move Rejected EDRs to the Input Directory
Table 8-1 shows how to configure the FCT_PreRecycle pipeline module to retrieve a file of rejected EDRs, add a recycle suffix to the file name, and move the file to the recycling pipeline's input directory.
Table 8-1 Configuring FCT_PreRecycle
Registry Entry | Description | Required Value |
---|---|---|
Active |
Specifies if the module is active or inactive. |
True |
RecycleSuffix |
Specifies the suffix for the file that contains the EDRs that need recycling. The suffix is automatically appended when the file is moved from the reject directory to the input directory. If it is empty, no suffix is added. |
Suffix |
RecyFileName |
Specifies the file name and path for the file that contains the EDRs that need recycling. |
FileName |
For example, this FCT_PreRecycle registry configuration specifies to:
-
Save the EDRs that need recycling to the recycle.dat file
-
Add the RecycleFile suffix when moving files containing EDRs that need recycling to the input directory
PreRecycle { ModuleName = FCT_PreRecycle Module { Active = True RecycleSuffix = RecycleFile RecyFileName = ./recycle.dat } }
Configuring FCT_Reject to Reject EDRs
Table 8-2 shows how to configure the FCT_Reject pipeline module to determine whether to reject EDRs. If the EDR is rejected, the module sends it to the specified reject output stream.
Table 8-2 Configuring FCT_Reject for Recycling EDRs
Registry Entry | Description | Required Value |
---|---|---|
Active |
Specifies if the module is active or inactive. |
True |
NotifyonReject |
Specifies whether to notify other modules that an EDR has been rejected. |
True or False |
UseRejectStream |
Specifies whether to send rejected EDRs to the reject output stream. |
True |
CallAssemblingModule |
If you use both the FCT_CallAssembling module and the FCT_Reject module in a pipeline, use the CallAssemblingModule registry entry to ensure that the complete EDRs are recycled. Otherwise, only part of the EDR is recycled. |
Do not set or specify the registry path to FCT_CallAssembling |
MinErrorSeverity |
Specifies to reject EDRs that have a specified severity, allowing EDRs to be processed with warning or normal error messages without being rejected. For recycling EDRs in pipeline-only systems, set this entry to one of these values:
To allow warning and normal messages without rejecting the EDR, set this entry to 3. |
3, 4, or 5 |
StreamMap |
Specifies a list of error types mapped to output streams. To create
For a list of BRM error messages, see "EDR Suspense Reasons". |
BRMErrorMessage = OutputStreamName |
For example, this FCT_Reject registry configuration specifies to:
-
Notify other modules that an EDR has been rejected.
-
Use the FCT_CallAssembling module.
-
Reject EDRs with minor, major, and critical error messages.
-
Send rejected EDRs to one of three output streams: RapOutput, CiberOutput, and TadigOutput.
Reject { ModuleName = FCT_Reject { Active = True NotifyOnReject = True UseRejectStream = True CallAssemblingModule = ifw.Pipelines.Pipe.Functions.Standard.FunctionPool.CallAssembling MinErrorSeverity = 3 StreamMap { ERR_TAP3_RET = RapOutput ERR_CIBER_RET = CiberOutput ERR_TADIG_NOT_FOUND = TadigOutput } } }
Configuring FCT_Recycle to Recycle or Test Recycle EDRs
Table 8-3 shows how to configure the FCT_Reject pipeline module to:
-
(In test mode) Create a report with data about the processing but does not send the EDR to an output file.
-
(In recycle mode) Send the EDR file to an output stream and attach a sequence number the file.
Table 8-3 Configuring FCT_Recycle for Recycling EDRs
Registry Entry | Description |
---|---|
Active |
Specifies if the module is active or inactive. |
RecycleLog.MessageFilePath |
Specifies the path where the log file can find the message database. |
RecycleLog.MessageFilePrefix |
Specifies the prefix for collecting the files from the message file path. |
RecycleLog.MessageFileSuffix |
Specifies the suffix for collecting the files from the message file path. |
RecycleLog.FilePath |
Specifies the path in which the log file is written. |
RecycleLog.FilePrefix |
Specifies the prefix for the log file. |
RecycleLog.FileSuffix |
Specifies the suffix for the log file. |
For example, this FCT_Recycle registry configuration specifies to:
Recycle { ModuleName = FCT_Recycle Module { Active = True RecycleLog { MessageFilePath = .. MessageFilePrefix = Framework MessageFileSuffix = msg FilePath = ../tmp/log01 FilePrefix = rej_ FileSuffix = .log } } }