Migrating Data from Other Data Repositories

38 Migrating Data from Other Data Repositories

The following topics describe how to migrate data from both LDAP Version 3-compatible directories and application-specific data repositories into Oracle Internet Directory:

38.1 Understanding Data Migration from Other Data Repositories

During an Oracle Internet Directory installation, Oracle Identity Management 14c Installer creates a default schema and directory information tree (DIT).

Understanding the Concepts and Architecture of Oracle Internet Directory, and Planning, Deploying and Managing Realms, describe this default DIT framework. The framework is flexible and you can modify it to suit the needs of your deployment.

In Oracle Internet Directory, the following directory elements are created by default:

Root Oracle Context (cn=OracleContext): This is the container where Oracle products store enterprise-wide configuration data.
Default identity management realm (dc=dns_domain_of_host,dc=com): This is the container under which Oracle products expect to find enterprise users and groups. It approximates the enterprise DIT structure. For example, if Oracle Internet Directory is installed on a computer whose host name is: my_computer.us.my_company.com, then the default identity management realm created at installation of Oracle Internet Directory would be dc=us,dc=my_company,dc=com. Oracle products expect to find all users under the container cn=users,dc=us,dc=my_company,dc=com and all groups under cn=groups,dc=us,dc=my_company,dc=com. In addition to creating the default identity management realm entry, the Oracle Internet Directory Configuration Assistant stores a pointer to it in the Root Oracle Context so that other Oracle Internet Directory-enabled components can bootstrap themselves.

You can change this default identity management realm to suit your deployment requirements.

38.2 Migrating Data from LDAP-Compliant Directories

If you have a directory with an already-established structure, and you want to migrate the data from that directory into the default directory structure environment, then follow the instructions in this section.

The following topics provide practical information for migrating data from an LDAP-compliant, third-party directory to Oracle Internet Directory:

38.2.1 Understanding Data Migration Tools

The following topics describe the tools that are commonly used for migrating data and also provide a feature comparison between them:

38.2.1.1 Bulk Loader

The bulk loader, bulkload, is a command-line tool for loading a large number of entries into a directory server. It uses Oracle SQL*Loader to load the directory entries. The bulkload tool expects the input file to be in LDAP Data Interchange Format (LDIF). The bulkload tool can validate LDIF input for referential integrity, but it cannot perform any mapping or other transformation on the data.

When no translation is required and data is very large (500,000 or more), bulkload is the best choice for migrating data from a third-party directory to Oracle Internet Directory. It is fast and it can validate LDIF input.

For bulkload syntax information and examples, see Oracle Internet Directory Data Management Tools in Reference for Oracle Identity Management.

38.2.1.2 Directory Integration Assistant

The Directory Integration Assistant, syncProfileBootstrap, is a command-line tool for administering the synchronization profiles scheduled by the Oracle directory integration server. An administrator can use the syncProfileBootstrap operation to perform the initial migration of data between a connected directory and Oracle Internet Directory when configuring the Oracle directory integration server to perform ongoing synchronization. You also use it for a one-time data migration, without ongoing synchronization.

For more information about syncProfileBootstrap, see Directory Bootstrapping Using syncProfileBootstrap in Administering Oracle Directory Integration Platform.

38.2.1.3 Features Comparison Between Bulk Loader and Directory Integration Assistant

Table 38-1 lists the features of bulkload, as compared with syncProfileBootstrap.

Table 38-1 Features of bulkload and syncProfileBootstrap

Feature	bulkload	syncProfileBootstrap
Speed	Fast	Slow
Data transfer method	SQL	LDAP
Input types accepted	LDIF file only	LDIF file, LDAP directory, tagged file, CSV file
Transforms data	No	Yes
Validates LDIF input	Yes	No

38.2.1.4 LDIF File

LDIF is the IETF-sanctioned ASCII interchange format for representing LDAP-compliant directory data as a file. All LDAP-compliant directories should have tools to export their contents into one or more LDIF files representing the DIT at the time of export.

See Also:

RFC 2849 of the IETF, available for download at: http://www.ietf.org

38.2.2 Migrating LDAP Data Using LDIF File and Bulk Loader

To use this method, you must first export data from the third-party directory to an LDIF file.

See Figure 38-1.

Figure 38-1 Using an LDIF File and Bulk Loader

To migrate data to Oracle Internet Directory using an LDIF file and bulk loader:

Export data from the non-Oracle Internet Directory server into LDIF file format.
See the vendor-supplied documentation for instructions. If flags or options exist for exporting data from the foreign directory, be sure to select the method that:
- Produces LDIF output with the least amount of proprietary information included
- Provides maximum conformance to the IETF Request for Comments 2849 of the IETF, available for download at: http://www.ietf.org
Analyze the LDIF user data for any required schema additions referenced in the LDIF data.

Any attributes not found in the Oracle Internet Directory base schema require extension of the Oracle Internet Directory base schema before the importation of the LDIF file. Some directories may support the use of configuration files for defining extensions to their base schema (Oracle Internet Directory does not). If you have a configuration file you can use it as a guideline for extending the base schema in Oracle Internet Directory.
Extend the schema in Oracle Internet Directory.

See Managing Directory Schema for tips on how to extend the directory schema in Oracle Internet Directory. You can do this by using either Oracle Directory Services Manager or the SchemaSynch tool, which is documented in Reference for Oracle Identity Management.

If you have users who are using other Oracle products, you must create users with object class orclUserV2 and its required attributes. If you are integrating with Active Directory, you must create users with object class orclADUser and its required attributes. These object classes and their attributes are documented inOracle Identity Management Object Class Reference in Reference for Oracle Identity Management.
Remove any proprietary directory data from the LDIF file.

Certain elements of the LDAP v3 standard have not yet been formalized, such as ACI attributes. As a result, various directory vendors implement ACI policy objects in ways that do not translate well across vendor installations.

After the basic entry data has been imported from the cleaned up LDIF file to Oracle Internet Directory, you must explicitly reapply security policies in the Oracle Internet Directory environment. You can do this by using either Oracle Directory Services Manager, or command-line tools and LDIF files containing the desired ACP information.

There may be other proprietary metadata unrelated to access control. You should remove this as well. Understanding the various IETF RFCs can help you determine which directory metadata is proprietary to a given vendor and which complies with the LDAP standards, and is thus portable by way of an LDIF file.
Remove operational attributes from the LDIF file.

Four of the standard LDAP v3 operational attributes, namely, creatorsName, createTimestamp, modifiersName, and modifyTimestamp are automatically generated by Oracle Internet Directory whenever entries are created or imported. It is not possible to instantiate these values from existing directory data, for example by using LDIF file importation. Therefore you should remove these attributes from the file before attempting to import.
Remove incompatible userPassword attribute values from the LDIF file.

The userPassword attribute hash algorithms supported by Oracle Internet Directory are listed in the orclcryptoscheme entry in Attributes of the DSE.

The userPassword attribute hash values used by some vendor products are not compatible with Oracle Internet Directory. As a result, you must remove all lines corresponding to the userPassword attribute and value from the LDIF data file unless they are represented in plain text or contain no value. After importation of the LDIF data, you must manually reenter or upload hashed userPassword information separately into the directory. Be sure that the passwords comply with the Oracle Internet Directory password policies and are in clear text.
Run the bulkload check =TRUE" mode and determine any remaining schema violations or duplication errors.

Before generating and loading an LDIF file, always perform a check on it by using the bulkload utility check mode. The bulkload output reports any inconsistencies in the data.

See Also:

The bulkload command-line tool reference in Reference for Oracle Identity Management for instructions on how to use the bulkload check mode

38.2.3 Migrating LDAP Data Using Directory Integration Assistant Directly

The syncProfileBootstrap operation can take data either directly from a third-party LDAP-compliant directory or from an LDIF file, tagged file, or CSV file. You must provide mapping rules, either as a synchronization profile or in a configuration file.

For syncProfileBootstrap syntax information, configuration file properties, information about input file types, and examples, see Oracle Directory Integration Platform Tools in Reference for Oracle Identity Management and Directory Bootstrapping Using syncProfileBootstrap in Administering Oracle Directory Integration Platform.

If you must perform mapping when migrating the data from the third-party directory to Oracle Internet Directory, and if the data is small in size, you can use syncProfileBootstrap. As shown in Figure 38-2, you can use the third-party directory itself as input to syncProfileBootstrap.

Figure 38-2 Using syncProfileBootstrap Directly

38.2.4 Migrating LDAP Data Using LDIF File and Directory Integration Assistant

If you do not have direct access to the third-party directory, you can have the administrator export the data to an LDIF file.

SeeFigure 38-3, syncProfileBootstrap can take its input from an LDIF file. You could also use Oracle directory integration server to migrate the data.

Figure 38-3 Using an LDIF File and syncProfileBootstrap

Whenever you use an LDIF file and bulkload to migrate data to Oracle Internet Directory, you must perform certain tasks. In this scenario, you are using a mapping file with syncProfileBootstrap or Oracle Directory Integration Platform, so do not have to perform all the tasks listed in "Migrating LDAP Data Using LDIF File and Bulk Loader".

The following table lists the tasks to migrate data to Oracle Internet Directory using an LDIF file and Directory Integration Assistant:

Table 38-2 List of Tasks to Migrate Data to Oracle Internet Directory using LDIF File and Directory Integration Assistant

Task #	Task Description
Task 1	Export data from the non-Oracle Internet Directory Server into LDIF file format.
Task 2	Analyze the LDIF user data for any required schema additions referenced in the LDIF data.
Task 3	Extend the schema in Oracle Internet Directory.

38.2.5 Migrating LDAP Data Using Directory Integration Assistant, Bulk Loader, and LDIF Files

If you have a large amount of data and you must perform mapping on the data, you can use a combination of tools.

See Figure 38-4, you can export the data from the third-party directory to an LDIF file, then use syncProfileBootstrap to perform the mapping into another LDIF file, which you then load with bulkload.

Figure 38-4 Using syncProfileBootstrap, bulkload, and LDIF Files

The following table lists the tasks to migrate data to Oracle Internet Directory using an LDIF file, Bulk Loader and Directory Integration Assistant:

Table 38-3 List of Tasks to Migrate Data Using Directory Integration Assistant, Bulk Loader and LDIF Files

Task #	Task Description
Task 1	Export data from the non-Oracle Internet Directory server into LDIF file format.
Task 2	Analyze the LDIF user data for any required schema additions referenced in the LDIF data.
Task 3	Extend the schema in Oracle Internet Directory.

38.2.6 Migrating LDAP Data Using the Oracle Directory Integration Platform Server

Under some circumstances, an administrator might choose not to use syncProfileBootstrap when configuring the Oracle directory integration server. After it is configured, the Oracle directory integration server itself can migrate data from a connected directory to Oracle Internet Directory. You can also use the Oracle directory integration server for a one-time data migration.

The Oracle directory integration server enables you to configure bidirectional, ongoing integration between Oracle Internet Directory and a Third-party directory, as shown in Figure 38-5. For more information, see Understanding the Oracle Directory Synchronization Service in Administering Oracle Directory Integration Platform.

Figure 38-5 Using the Oracle Directory Integration Server

38.3 Migrating User Data from Application-Specific Repositories

This section given information on how to Migrate user data from an application-specific repository requires:

Migrating user data from an application-specific repository requires:

Collecting the user data from the application-specific repository and formatting it in a way that the directory can read it
Making that data available to the directory administrator who must then:
- Specify where to place it in the directory
- Import it into the directory

The following topics provide the generic instructions for enabling data migration from an application-specific repository:

38.3.1 Enabling Data Migration from Application-Specific Repositories

To enable this migration to happen, the DSPS requires the application-specific repository to export its data to an intermediate template file. Records in this template file are not in pure LDIF; they contain substitution variables that have to do with, for example, the location in the directory where the information is finally to reside. The application leaves these variables undefined, so that you, the directory administrator can define them later on.

To convert the user data from this intermediate template file into proper LDIF, you use the OID Migration Tool (ldifmigrator). After the data is converted to LDIF, you can load it into the directory.

To summarize, migrating data from application-specific repositories involves these general steps:

Exporting the application-specific data as an intermediate template file.
You, the directory administrator, using the OID Migration Tool (ldifmigrator) to read these partial LDIF entries and convert them to pure LDIF entries based on the deployment choices
You, the directory administrator, loading the data, now in pure LDIF, into Oracle Internet Directory.
The application completing the migration process according to its own specifications.

38.3.2 Reconciling Data in Application Repository with Existing Data in a Directory

The data you are migrating from an application-specific repository may already reside in Oracle Internet Directory. If this is the case, then you can reconcile differences between the two directories by using the reconciliation feature of the OID Migration Tool (ldifmigrator).

See ldifmigrator command-line tool reference in Reference for Oracle Identity Management for information about the reconciliation feature of the OID Migration Tool.

38.3.3 Managing Data Migration from Application-Specific Repositories

The following topics describe how to create an intermediate template file and how to run the migration tool while migrating data from application-specific repositories:

38.3.3.1 Creating an Intermediate Template File

The following topics describe the format and structure of the intermediate template file and also describe the attributes in a user entry:

38.3.3.1.1 Intermediate Template File

Applications generating data in national languages must store that data in AL32UTF8 in the intermediate template file as specified in the IETF RFC 2849, "The LDAP Data Interchange Format (LDIF) - Technical Specification" available at http://www.ietf.org.

When generating the intermediate template file, migrating applications must list all user records sequentially with a record separator as defined in RFC 2849. The OID Migration Tool (ldifmigrator) assigns all of these users to the default identity management realm, which corresponds to the enterprise itself.

Figure 38-6 shows the overall structure of the intermediate template file containing user entries.

Figure 38-6 Structure of the Intermediate User File

Description of "Figure 38-6 Structure of the Intermediate User File"

The intermediate template file uses the following format to generate a valid user entry. All of the strings in bold text are supplied from the application-specific repository.

dn: cn=UserID, %s_UserContainerDN%
sn: Last_Name
orclGlobalID: GUID_for_User
%s_UserNicknameAttribute%: UserID
objectClass: inetOrgPerson
objectClass: orclUserV2

In this template, the strings %s_UserContainerDN% and %s_UserNicknameAttribute% are substitution variables for which the OID Migration Tool provides values. The OID Migration Tool determines these values according to deployment-specific considerations. Either the application passes the arguments to the OID Migration Tool, or the tool retrieves them from the directory.

38.3.3.1.2 Example of User Entries in an Intermediate Template File

The following intermediate template file includes user entries generated by the application-specific migration logic. In this example, all of the data listed in bold text is from the application-specific user repository.

dn: cn=jdoe, %s_UserContainerDN%
sn: Doe
%s_UserNicknameAttribute%: jdoe
objectClass: inetOrgPerson
objectClass: orclUserV2
title: Member of Technical Staff
homePhone: 415-584-5670
homePostalAddress: 234 Lez Drive$ Redwood City$ CA$ 94402

dn: cn=jsmith, %s_UserContainerDN%
sn: Smith
%s_UserNicknameAttribute%: jsmith
objectClass: inetOrgPerson
objectClass: orclUserV2
title: Member of Technical Staff
homePhone: 650-584-5670
homePostalAddress: 232 Gonzalez Drive$ San Francisco$ CA$ 94404

dn: cn=lrider, %s_UserContainerDN%
sn: Rider
%s_UserNicknameAttribute%: lrider
objectClass: inetOrgPerson
objectClass: orclUserV2
title: Senior Member of Technical Staff
homePhone: 650-584-5670

After all of the user data is converted to the intermediate file format, the OID Migration Tool further converts it into a proper LDIF file that can be loaded into Oracle Internet Directory.

You can find examples of intermediate template files in $ORACLE_HOME/ldap/schema/oid.

38.3.3.1.3 Attributes in User Entries

Each user entry has mandatory and optional attributes.

Table 38-4 lists and describes the mandatory attributes in a user entry.

Table 38-4 Mandatory Attributes in a User Entry

Attribute	Description
`dn`	Distinguished name of the user entry with appropriate substitution variables. The relative distinguished name of the entry MUST contain the `cn` attribute.
`sn`	Surname—that is, the last name—of the user
`objectclass`	Object classes the entry should minimally belong to: `inetOrgPerson` and `orclUserV2`

See Also:

IETF Request for Comments 2798: "Definition of the inetOrgPerson LDAP Object Class," available at http://www.ietf.org, for a description of each attribute in the inetOrgPerson object class
orclUserV2object class in Reference for Oracle Identity Management for more information.

38.3.3.2 Running the OID Migration Tool

After you set up the intermediate template file, the OID Migration Tool enables you to bring all pertinent data from the application-specific repository into Oracle Internet Directory. After you have migrated the data, you can update whatever portion of it is relevant to the application by synchronizing that application with Oracle Internet Directory. You synchronize by using the Oracle Directory Synchronization Service.

See Also:

The ldifmigrator command-line tool reference in Reference for Oracle Identity Management for instructions about using the OID Migration Tool