Integrate with Flat File

Prerequisites

Before you install and configure a Flat File orchestrated system, you should consider the following prerequisites and tasks.

Certified Components

The system must be the following:

  • CSV Flat file located in Oracle Cloud Infrastructure (OCI) Object Storage in your tenancy

Supported Modes

Flat File orchestrated system supports the following modes:
  • Authoritative Source
  • Managed System

Supported Operations

The Flat File orchestrated system supports the following operations:
  • Create Account
  • Delete Account
  • Add Entitlement
  • Remove Entitlement

Create a bucket in the OCI Object Storage service for Flat File Orchestrated System Operations

In order to load a flat file into Oracle Access Governance you need to place the data files in a bucket created using the OCI Object Storage service. This bucket can be created in any compartment of your OCI tenancy. For details regarding OCI Object Storage, refer to Managing Buckets.

In order to access the bucket, you need to create a service user that has read, write, and delete access (manage privileges) to the bucket. Follow this process to create this service user:

  • Create a compartment, accessgovernance/
  • Create a local identity user, agcs_user in any domain in your tenancy.
  • Create an identity group, agcs_flatfilegroup in any domain in your tenancy.
  • Assign the identity user agcs_user to the identity group agcs_flatfilegroup.
  • Create a policy, agcs_flatfilepolicy, with the following policy statement: 
    allow group <groupname> to manage objects in compartment <compartmentname>
    where target.bucket.name = 'bucketname'

    For example:

    allow group agcs_flatfilegroup to manage objects in compartment accessgovernance
    where target.bucket.name = 'bucket-20231130-1143'

Configure

You can establish a connection between Flat File and Oracle Access Governance by entering connection details. To achieve this, use the Orchestrated Systems functionality available in the Oracle Access Governance Console.

Navigate to the Orchestrated Systems Page

Navigate to the Orchestrated Systems page of the Oracle Access Governance Console, by following these steps:
  1. From the Oracle Access Governance navigation menu icon Navigation menu, select Service Administration → Orchestrated Systems.
  2. Click the Add an orchestrated system button to start the workflow.

Select system

On the Select system step of the workflow, you can specify which type of system you would like to onboard.

  1. Select Flat File.
  2. Click Next.

Enter details

On the Add Details step of the workflow, enter the details for the orchestrated system:
  1. Enter a name for the system you want to connect to in the Name field.
  2. Enter a description for the system in the Description field.
  3. Determine if this orchestrated system is an authoritative source, and if Oracle Access Governance can manage permissions by setting the following check boxes.
    • This is the authoritative source for my identities
    • I want to manage permissions for this system

    The default value in each case is Unselected.

  4. Select Enable virtual systems to include multiple applications/systems within a single orchestrated system. You can define the virtual systems after integration settings. For more information, see Understanding Virtual Systems.
  5. Click Next.

Add Owners

You can associate resource ownership by adding primary and additional owners. This drives self-service as these owners can then manage (read, update or delete) the resources that they own. By default, the resource creator is designated as the resource owner. You can assign one primary owner and up to 20 additional owners for the resources.

Note:

When setting up the first Orchestrated System for your service instance, you can assign owners only after you enable the identities from the Manage Identities section.
To add owners:
  1. Select an Oracle Access Governance active user as the primary owner in the Who is the primary owner? field.
  2. Select one or more additional owners in the Who else owns it? list. You can add up to 20 additional owners for the resource.
You can view the Primary Owner in the list. All the owners can view and manage the resources that they own.

Account settings

On the Account settings step of the workflow, enter details of how you would like to manage accounts with Oracle Access Governance when configured as a managed system:
  1. Select to allow Oracle Access Governance to create new accounts when a permission is requested, if the account does not already exist. By default this option is selected meaning that an account will be created if it does not exist, when a permission is requested. If the option is unselected then permissions can only be provisioned where the account already exists in the orchestrated system. If permission is requested where no user exists then the provisioning operation will fail.
  2. Select where and who to send notification emails when an account is created. The default setting is User. You can select one, both, or none of these options. If you select no options then notifications will not be sent when an account is created.
    • User
    • User manager
  3. When an identity leaves your enterprise you should remove access to their accounts. You can select what to do with the account when this happens. Select one of the following options:
    • Delete
    • Disable
    • No action

    Note:

    The options above are only displayed if supported in the orchestrated system type being configured. For example, if Delete is not supported, then you will only see the Disable and No action options.
  4. When all permissions for an account are removed, for example when moving from one department to another, you may need to adjust what accounts the identity has access to. You can select what to do with the account when this happens. Select one of the following options:
    • Delete
    • Disable
    • No action

    Note:

    The options above are only displayed if supported in the orchestrated system type being configured. For example, if Delete is not supported, then you will only see the Disable and No action options.
  5. If you want Oracle Access Governance to manage accounts created directly in the orchestrated system you can select the Manage accounts that are not created by Access Governance option. This will reconcile accounts created in the managed system and will allow you to manage them from Oracle Access Governance.

Note:

If you do not configure your system as a managed system then this step in the workflow will display but is not enabled. In this case you proceed directly to the Integration settings step of the workflow.

Note:

If your orchestrated system requires dynamic schema discovery, as with the Generic REST and Database Application Tables integrations, then only the notification email destination can be set (User, Usermanager) when creating the orchestrated system. You cannot set the disable/delete rules for movers and leavers. To do this you need to create the orchestrated system, and then update the account settings as described in Configure Orchestrated System Account Settings.

Integration settings

On the Integration settings step of the workflow, enter the configuration details required to allow Oracle Access Governance to connect to the Flat File.

  1. In the What is the OCI user's OCID? field, add the OCID for the OCI user owning the bucket containing the flat files you want to integrate.
  2. In the What is the fingerprint of the OCI user's API key?. Enter the fingerprint for the OCU user's API key. Consult Required Keys and OCIDs in the OCI documentation for details on how to obtain the value for this.
  3. Enter the user's private API key, in PEM format into the What is the OCI user's private API key in PEM format? field. Consult Required Keys and OCIDs in the OCI documentation for details on how to obtain the value for this.
  4. Enter the tenancy into the What is the OCI tenancy of the OCI user? field.
  5. Enter the home region code of the tenancy into the What us the OCI tenancy's home region code? field. Details of region codes can be found in Regions and Availability Domains OCI documentation.
  6. Enter the bucket namespace of the tenancy in the What is the namespace for the bucket? field.
  7. In the Enter the name of the bucket where your flat file is stored in OCI object storage field, enter the name of the bucket where your flat file is stored in OCI object storage.
  8. Enter the encoding into the Encoding field. Default is UTF-8.
  9. In the Field Delimiter field, enter the field delimiter character used in the Flat File. Default is ,.
  10. In the Sub Field Delimiter field, enter the sub field delimiter character used in the Flat File. Default is #.
  11. In the MultiValue Delimiter field, enter the multivalue delimiter character used in the Flat File. Default is ;.
  12. In the Text Qualifier field, enter the character used in the Flat File to act as a text qualifier. Default is ".
  13. In the Date Format field, enter the Java data format in which date type fields are included in the Flat File, for example dd/MM/yyyy. If no date format is specified, the date field will be assumed to be of data type Long.
  14. If you want to check the connectivity to your Flat File, click the Test integration button.
  15. Click Add to create the orchestrated system.

Finish up

Finally, if you have enabled Virtual Systems, you first need to define and upload the subsystems CSV file and then activate the orchestrated system. Select I'm done.

If virtual systems are disabled, then you can either activate the orchestrated system or save it as draft only.

Upload the CSV File

If you have enabled virtual systems, then upload a CSV file with ID and Name for the systems. You could add up to 100 virtual systems.

For example:

ID Name
virtual_ad_123 Alpha
virtual_ad_456 Beta
virtual_ad_789 Gamma
  • Virtual system name must not contain the following special characters `~!@#$%^&*><".
  • Special characters are not allowed for ID or Name.
  • Virtual Systems name should be unique across all orchestrated systems.
  • IDs should be unique for that orchestrated system.

Select the Update button to add a latest version of the virtual systems.

Existing virtual systems cannot be deleted but can be updated. To manage virtual systems after the creation, see Manage Virtual Systems.

Post Configuration

Check Bucket Folder Structure

After creation of the orchestrated system, the following folder structure should be created in the defined bucket. 

<ServiceInstanceName>/<OrchestratedSystemName>
    failed //Same sub-folders to be created as inbox
    inbox/
	IDENTITY/
		virtual-sys-1
		virtual-sys-2
		virtual-sys-3
	PERMISSION/
		virtual-sys-1
		virtual-sys-2
		virtual-sys-3
	TARGETACCOUNT/
		virtual-sys-1
		virtual-sys-2
		virtual-sys-3
    outbox/
	Same sub-folders to be created as inbox
    sample/
    schema/

Note:

Sub folders such as, virtual-sys-1, virtual-sys-2, and so on are created only when virtual systems are enabled.
These folders fulfill the following purposes:
  • failed: Files with any kind of data issue will be moved to this folder under the respective entity folder, in the event of a data load operation failure.
  • inbox: Contains IDENTITY, PERMISSION and TARGETACCOUNT folders, each of which contains virtual systems folders, referenced by ID. Place the CSV files within the virtual systems folder to include in the data load operation. If you have not selected virtual systems during your configuration, then directly place your data files under the IDENTITY, PERMISSION and TARGETACCOUNT folders.
  • outbox: Provisioning events for each entity of the orchestrated system.
  • sample: Contains example CSVs with the expected header. These can be used as a reference for generating data and putting in the inbox for data load. These files should not be altered.
  • schema: Contains the JSON representation of each entity's schema. This can be referred to for understanding details like:
    • dataType
    • Mandatory attributes
    • Whether an attribute is multivalued or not
    • If the attribute is complex and has nested attributes (dataType will be CUSTOM)
    • Supported dataTypes are:
      • TEXT
      • NUMBER
      • DECIMAL_NUMBER
      • DATE
      • FLAG
      • CUSTOM

Define Custom Attributes

Custom attributes are supported for the IDENTITY entity. If you want to include custom attributes in your dataload then you need to add them in the <ServiceInstanceName>/<OrchestratedSystemName>/schema/IDENTITY.json file.

Custom attribute names should meet the following requirements:
  • start with a character: A-Z or a-z
  • contain only characters or numbers: A-Z or a-z or 0-9
  • For the DATE type attribute, only long value is supported
  • Custom attributes can only be added, they cannot be deleted
  • A custom attribute of CUSTOM type cannot be added

Once you have added any custom attributes in the IDENTITY.json file, you will need to include them in Oracle Access Governance as described in Fetch Latest Custom Attributes. After this is completed, the sample CSV will be updated with the newly added custom attribute(s). Update the data files in the inbox to include the custom attribute(s) in your next data load.

Run Dataload

Data load is on demand. Always run the data load after you have defined custom attributes or added the relevant CSV data files into the inbox folder. Each time you run a data load it is a full data load, there is no incremental load. UTF-8-BOM encoding is not supported.

If there is any kind of failure (single record or complete file failure), the data load operation will be marked as failed. The files that have been processed successfully will stay in the inbox while the failed files will be moved to the failed folder. Fix the data issue and place the files again in the inbox folder.

Data integrity issues, such as a permission being assigned to an account that is missing in the CSV, can also cause the data load operation to fail. However, in such cases the CSV files is not moved to the failed folder. Files are moved to the failed folder only when there are issues reading the data itself, such as missing mandatory data.

Schema Extension - Adding Custom Account Attributes

You can configure account attributes for your Flat File orchestrated system in addition to the default account attributes supported out-of-the box. Details of account attributes and how they are managed can be found in Account Attributes and Configure Account Attributes.

Schema Extension - Simple Attribute Example

To demonstrate how to add a simple account attribute let's look at the example of adding a phone number attribute to your Flat File schema.

Assume that you have a Day 0 configuration of your Flat File orchestrated system which has the following attributes by default:
  • commonName
  • displayName
  • email
  • firstName
  • lastName
  • middleName
  • __NAME__
  • permissions
  • __ENABLE__
  • title
  • __UID__
A sample TARGETACCOUNT.csv would be:
__NAME__,firstName,lastName,middleName,displayName,commonName,title,__ENABLE__,email,permissions
A sample TARGETACCOUNT.json would be:
[
    {
        "name": "displayName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "permissions",
        "dataType": "CUSTOM",
        "required": false,
        "multiValued": true,
        "subAttributes": [
            {
                "name": "__NAME__",
                "dataType": "TEXT",
                "required": true,
                "idAttribute": true
            }
        ]
    },
    {
        "name": "middleName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "lastName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "commonName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "firstName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "__NAME__",
        "dataType": "TEXT",
        "required": true,
        "multiValued": false
    },
    {
        "name": "title",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "email",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "__ENABLE__",
        "dataType": "FLAG",
        "required": false,
        "multiValued": false
    }
]
To add a custom account attribute, phoneNumber, complete the steps detailed in Configure Account Attributes, for example:
  1. Create a system provided attribute, phoneNumber, and select the options include in inbound and outbound data, and support multiple values.
  2. Add the new attribute to the TARGETACCOUNT.csv and TARGETACCOUNT.json.
    __NAME__,firstName,lastName,middleName,displayName,commonName,title,__ENABLE__,email,permissions,phoneNumber
    [
    {
        "name": "displayName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "permissions",
        "dataType": "CUSTOM",
        "required": false,
        "multiValued": true,
        "subAttributes": [
            {
                "name": "__NAME__",
                "dataType": "TEXT",
                "required": true,
                "idAttribute": true
            }
        ]
    },
    {
        "name": "phoneNumber",
        "dataType": "TEXT",
        "required": false,
        "multiValued": true
    },
    {
        "name": "firstName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "email",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "__ENABLE__",
        "dataType": "FLAG",
        "required": false,
        "multiValued": false
    },
    {
        "name": "middleName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "lastName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "commonName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "__NAME__",
        "dataType": "TEXT",
        "required": true,
        "multiValued": false
    },
    {
        "name": "title",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    }
]
  3. Trigger a full data load from Oracle Access Governance Console. The new custom attribute should be loaded and is visible in the Enterprise Wide Browser.

Schema Extension - Complex Attribute Example

To demonstrate how to add a complex account attribute let's look at the example of adding an other contact details attribute to your Flat File schema.

Assume that you have a Day 0 configuration of your Flat File orchestrated system which has the following attributes by default:
  • commonName
  • displayName
  • email
  • firstName
  • lastName
  • middleName
  • __NAME__
  • permissions
  • __ENABLE__
  • title
  • __UID__
A sample TARGETACCOUNT.csv would be:
__NAME__,firstName,lastName,middleName,displayName,commonName,title,__ENABLE__,email,permissions
A sample TARGETACCOUNT.json would be:
[
    {
        "name": "displayName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "permissions",
        "dataType": "CUSTOM",
        "required": false,
        "multiValued": true,
        "subAttributes": [
            {
                "name": "__NAME__",
                "dataType": "TEXT",
                "required": true,
                "idAttribute": true
            }
        ]
    },
    {
        "name": "middleName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "lastName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "commonName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "firstName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "__NAME__",
        "dataType": "TEXT",
        "required": true,
        "multiValued": false
    },
    {
        "name": "title",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "email",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "__ENABLE__",
        "dataType": "FLAG",
        "required": false,
        "multiValued": false
    }
]
To add a custom account attribute, otherContactDetails with , complete the steps detailed in Configure Account Attributes, for example:
  1. Create a system provided attribute, otherContactDetails, with the following child attributes:
    • contactDetailsId
    • faxNumber
    • website
    and select the options include in inbound and outbound data, and support multiple values.
  2. Add the new attribute to the TARGETACCOUNT.csv and TARGETACCOUNT.json.
    __NAME__,firstName,lastName,middleName,displayName,commonName,title,__ENABLE__,email,permissions,phoneNumber,otherContactDetails.contactDetailsId,otherContactDetails.faxNumber,otherContactDetails.website
    [
    {
        "name": "displayName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "permissions",
        "dataType": "CUSTOM",
        "required": false,
        "multiValued": true,
        "subAttributes": [
            {
                "name": "__NAME__",
                "dataType": "TEXT",
                "required": true,
                "idAttribute": true
            }
        ]
    },
    {
        "name": "firstName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "email",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "__ENABLE__",
        "dataType": "FLAG",
        "required": false,
        "multiValued": false
    },
    {
        "name": "middleName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "lastName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "commonName",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "title",
        "dataType": "TEXT",
        "required": false,
        "multiValued": false
    },
    {
        "name": "__NAME__",
        "dataType": "TEXT",
        "required": true,
        "multiValued": false
    },
    {
        "name": "otherContactDetails",
        "dataType": "CUSTOM",
        "required": false,
        "multiValued": true,
        "subAttributes": [
            {
                "name": "faxNumber",
                "dataType": "NUMBER",
                "required": false
            },
            {
                "name": "contactDetailsId",
                "dataType": "TEXT",
                "required": true,
                "idAttribute": true
            },
            {
                "name": "website",
                "dataType": "TEXT",
                "required": false
            }
        ]
    }
]
  3. Trigger a full data load from Oracle Access Governance Console. The new custom attribute should be loaded and is visible in the Enterprise Wide Browser.