Debug Data Loads and Dimension Builds

Problems with Essbase data loads and dimension builds include incorrect loads or failed processes. The source may contain unknown members or missing / invalid fields, the load rule may be invalid, or there is no connection to the server or the source data. Error logs can help you debug.

If a data source does not correctly load into Essbase Server, ensure that you are connected to the appropriate application and database and that you are loading the correct data source.

If you still encounter problems, see these topics:

After you correct the problems, you can reload the records that did not load by reloading the error log. Refer to: Load from Dimension Build and Data Load Error Logs.

Verify that Essbase Server Is Available

If you are unable to load data, ensure that the Essbase Server is running. Try to access the Essbase Server for some other operations.

For example, verify that you can connect to your Essbase application for a retrieval: try running a query in Smart View.

Verify that the Source Data Is Available

If Essbase cannot access the source data that you want to load, ensure (if it's a file) that it's unlocked and that path and file extension are correct. If the source data is an external database, ensure that it is running and you can connect.

Check the following details about the source data.

  • The source data is not open (for example, is someone editing the source data?).

    Essbase can load only source datas that are not locked by another user or application.

  • The source data has the correct file extension.

    Text files must have a .txt extension; rules files must have an .rul extension.

  • The source data file name and the path name are correct.

    Check for misspellings.

  • The source data is in the specified location.

    Ensure that no one has moved or deleted the source data file.

  • If you are using a SQL source of data:

    • The connection details (such as the user name, password, and database name) are correct.

    • You can connect to the SQL source data without using Essbase.

Check Error Logs

It is a good idea to have Essbase write to an error log when performing data loads or dimension builds, as the log is a valuable debugging tool. It lists errors that occurred during a dimension build or a data load, and includes the records that failed to load.

After you fix the errors, you can reload from the error log. If there is no error log, check whether the following conditions exist:

  • The person running the data load or dimension build job had set up an error log. When performing a data load or dimension build, you can tell Essbase to either abort the data load upon errors, or write/append the errors to a log.

    When Essbase creates an error log for dimension build or data load, it's created in the cube directory. The file format is err_<databasename>_<number>.txt.

  • The source of data and Essbase Server are both available.

    See Verify that Essbase Server Is Available and Verify that the Source Data Is Available

  • Essbase Server crashed during the data load.

    If so, you probably received a network timeout error on the client.

  • The application log exists.

If the error log exists but is empty, Essbase does not think that an error occurred during loading. Check whether the following conditions exist:

Resolve Problems with Data Loaded Incorrectly

If the Essbase data loads without error but the data appears to be wrong, first check the source data, then perform checks for implied shares, data scaling, sign flipping, and other potential issues with the load rules.

Check the following items to troubleshoot the data that was loaded:

  • You loaded the correct data source.

    If so, check the data source again to make sure that it contains the correct values.

  • The source data is formatted correctly.

    • All ranges are set up properly.

    • The data is clean. For example, as it processes the data source, Essbase recognizes member names and knows the dimensions they belong to. If a data source record inadvertently includes a member from a dimension for which there is a member named in the header record, the new member name replaces the header record member for that dimension. In the following example data source, Essbase recognizes Florida as a member of the Market dimension. The values in the last four records are interpreted as Florida values instead of Texas values.

      Jan  Actual  Texas  Sales
         "100-10"   51.7
         "100-20"   102.5
         "100-20"   335.0
         Florida    96.7
         "200-20"   276.0
         "200-20"   113.1
         "200-10"   167.0
  • Are there any implicitly shared members (when a parent and child share the same data value) of which you were unaware?

    An implied shared member occurs when a parent and child share the same data value. It can happen if a parent has only one child, or only one child that rolls up into the parent.

  • You added incoming data to existing data instead of replacing incoming data with existing data.

    See Add to or Subtract from Existing Values.

  • You selected or rejected any records that you did not intend to select or reject.

    See Select and Reject Records.

  • The sign is reversed (for example, a minus sign instead of a plus sign) and whether you performed sign flips on any UDAs.

    See Flip Field Signs.

  • You cleared data combinations that you did not intend to clear.

    See Clear Existing Data Values During Data Load.

  • You scaled the incoming values incorrectly.

    See Scale Data Values.

  • All member and alias names are fewer than 79 characters long.

You can check the data by running a query from Smart View or Report Writer.

Create Rejection Criteria for End of File Markers

A SQL data source may have an end of file marker made up of special characters that cause an Essbase data load or dimension build to fail. To fix this problem, define a rejection criterion to reject the problem record.

To define rejection criteria:

  1. Find the end of file marker in the SQL data source.
  2. Determine how to search for the end of file marker using the Essbase search command.

    This task may be difficult, because the end of file marker may be composed of one or more special characters. Refer to Ignore Strings.

  3. Define a rejection criterion that rejects the end of file marker. Refer to Select and Reject Records.

How Essbase Processes a Load Rule

Sometimes, you can track down problems with dimension builds by understanding how Essbase initializes/validates the load rule, and in what order it performs operations while processing the records in the source data.

Essbase performs the following steps to initialize a load rule:

  1. Validates the load rule against the associated outline.

  2. Validates the dimensions. This process includes ensuring that the build method and field types are compatible and that each dimension name is unique. Member names must be either unique or shared.

  3. Adds new dimensions defined in the load rule to the outline.

  4. Reads header records specified in the source data.

Then Essbase performs the following operations on each record of the source data during a data load or dimension build:

  1. Sets the file delimiters for all records.

  2. Applies field operations to the data in the order in which the operations are defined in the load rule.

    Field operations include joins, moves, splits, and creating fields using text and joins. To learn the order in which field operations are defined in the load rule, refer to Perform Operations on Fields.

  3. Essbase applies all properties for each field, applying all properties to field1 before proceeding to field2. Essbase applies field properties in the following order:

    1. Ignores fields set to be ignored during data load.

    2. Ignores fields set to be ignored during dimension build.

    3. Flags the data field.

    4. Applies field names.

    5. Applies field generations.

    6. Performs all replaces in the order in which they are defined in the load rule.

    7. Drops leading and trailing spaces.

    8. Converts spaces to underscores.

    9. Applies suffix and prefix operations.

    10. Scales data values.

    11. Converts text to lowercase.

    12. Converts text to uppercase.

  4. Adds members, or member information, or both, to the outline.

  5. If you chose to skip lines, Essbase skips the number of lines that you specified; otherwise, Essbase proceeds to the first record.

  6. Essbase performs selection or rejection criteria in the order in which the criteria are defined in the load rule. Essbase loads or rejects individual records of the source data based on the specified criteria.

How Essbase Processes Missing or Invalid Fields During a Data Load

If a member field is missing during a data load, Essbase uses its previous value, but if there is none, stops loading. Unknown member names cause the record to be rejected. If there is an invalid data field, the data load stops, resulting in a partial data load.

For information on restarting the data load after such issues, refer to Load from Dimension Build and Data Load Error Logs.

Missing Dimension or Member Fields

If a dimension or member field is missing, Essbase uses the value that it used previously for that dimension or member field. If there is no previous value, Essbase stops the data load.

For example, when you load the following file into the Sample.Basic database, Essbase maps the Ohio member field into the Market dimension for all records, including the records that have Root Beer and Diet Cola in the Product dimension.

Jan  Sales  Actual  Ohio
                    Cola          25
                    "Root Beer"   50
                    "Diet Cola"   19

Essbase stops the data load if no prior record contains a value for the missing member field. For example, if you try to load the following file into the Sample.Basic database, the data load stops, because the Market dimension (Ohio, in the previous example) is not specified.

Jan  Sales  Actual
            Cola          25
           "Root Beer"    50
           "Diet Cola"    19

Unknown Member Fields

If you are performing a data load and an unknown member name is encountered, the entire record is rejected. If there is a prior record with a member name for the missing member field, Essbase continues to the next record. If there is no prior record, the data load stops. For example, when you load the following file into the Sample Basic database, the record containing Ginger Ale is rejected because it is not a valid member name. The records containing Cola, Root Beer, and Cream Soda are loaded. If Ginger Ale were in the first record, however, the data load would stop.

Jan, Sales, Actual
Ohio   Cola          2
       "Root Beer"   12
       "Ginger Ale"  15
       "Cream Soda"  11

Note:

If you are performing a dimension build, you can add the new member to the database. See Perform Data Loads or Dimension Builds.

Invalid Data Fields

If you are performing a data load, the data load stops if an invalid data field is encountered. All fields that are read before the invalid field are loaded into the cube, resulting in a partial data load. For example, in the following file, the data load stops when it encounters the 15- data value. The Jan and Feb Sales records are loaded but not the Mar and Apr Sales records.

East   Cola   Actual
Sales         Jan    $10
              Feb    $21
              Mar    $15-
              Apr    $16

Load from Dimension Build and Data Load Error Logs

If the Essbase dimension build or data load fails, you can load only the records that failed, by loading from the error log. Reloading only the failed records is much faster than reloading every record.

To reload from the error log,

  1. If you load from the server, change the file extension from .err to .txt. For example, change the dataload.err file to dataload.txt.

    If you load from the client, you can leave the .err extension.

  2. Fix the problem that caused the dimension build or data load to fail. Fixing the problem might involve changing the outline, changing some text in the error log, or editing the load rule.

    Check whether the following conditions are true:

  3. Load the error log using the appropriate load rule.