Integrate with Flat File
Prerequisites
Before you install and configure a Flat File orchestrated system, you should consider the following prerequisites and tasks.
Certified Components
The system must be the following:
- CSV Flat file located in Oracle Cloud Infrastructure (OCI) Object Storage in your tenancy
Supported Modes
- Authoritative Source
- Managed System
Supported Operations
- Create Account
- Delete Account
- Add Entitlement
- Remove Entitlement
Create a bucket in the OCI Object Storage service for Flat File Orchestrated System Operations
In order to load a flat file into Oracle Access Governance you need to place the data files in a bucket created using the OCI Object Storage service. This bucket can be created in any compartment of your OCI tenancy. For details regarding OCI Object Storage, refer to Managing Buckets.
In order to access the bucket, you need to create a service user that has read, write, and delete access (manage privileges) to the bucket. Follow this process to create this service user:
- Create a compartment, accessgovernance/
- Create a local identity user, agcs_user in any domain in your tenancy.
- Create an identity group, agcs_flatfilegroup in any domain in your tenancy.
- Assign the identity user agcs_user to the identity group agcs_flatfilegroup.
- Create a policy, agcs_flatfilepolicy, with the following policy statement:
allow group <groupname> to manage objects in compartment <compartmentname> where target.bucket.name = 'bucketname'
For example:
allow group agcs_flatfilegroup to manage objects in compartment accessgovernance where target.bucket.name = 'bucket-20231130-1143'
Configure
You can establish a connection between Flat File and Oracle Access Governance by entering connection details. To achieve this, use the Orchestrated Systems functionality available in the Oracle Access Governance Console.
Navigate to the Orchestrated Systems Page
- From the Oracle Access Governance navigation menu icon
, select Service Administration → Orchestrated Systems.
- Click the Add an orchestrated system button to start the workflow.
Select system
On the Select system step of the workflow, you can specify which type of system you would like to onboard.
- Select Flat File.
- Click Next.
Enter details
- Enter a name for the system you want to connect to in the What do you want to call this system? field.
- Enter a description for the system in the How do you want to describe this system? field.
- Determine if this orchestrated system is an authoritative source, and if Oracle Access Governance can manage permissions by setting the following checkboxes.
- This is the authoritative source for my identities
- I want to manage permissions for this system
- Click Next.
Add Owners
Note:
When setting up the first Orchestrated System for your service instance, you can assign owners only after you enable the identities from the Manage Identities section.- Select an Oracle Access Governance active user as the primary owner in the Who is the primary owner? field.
- Select one or more additional owners in the Who else owns it? list. You can add up to 20 additional owners for the resource.
Account settings
-
Select where to send notification emails when an account is created. The default setting is User. You can select one, both, or none of these options. If you select no options then notifications will not be sent when an account is created.
- User
- User manager
- When an identity moves within your enterprise, for example when
moving from one department to another, you may need to adjust what accounts the
identity has access to. In some cases the identity will no longer require
certain accounts which are not relevant to their new role in the enterprise. You
can select what to do with the account when this happens. Select one of the
following options:
- Disable
- Delete
- When an identity leaves your enterprise you should remove access to
their accounts. You can select what to do with the account when this happens.
Select one of the following options:
- Disable
- Delete
Note:
If you do not configure your system as a managed system then this step in the workflow will display but is not enabled. In this case you proceed directly to the Integration settings step of the workflow.Note:
If your orchestrated system requires dynamic schema discovery, as with the and integrations, then only the notification email destination can be set (User, Usermanager) when creating the orchestrated system. You cannot set the disable/delete rules for movers and leavers. To do this you need to create the orchestrated system, and then update the account settings as described in Configure Orchestrated System Account Settings.Integration settings
On the Integration settings step of the workflow, enter the configuration details required to allow Oracle Access Governance to connect to the Flat File.
- In the What is the OCI user's OCID? field, add the OCID for the OCI user owning the bucket containing the flat files you want to integrate.
- In the What is the fingerprint of the OCI user's API key?. Enter the fingerprint for the OCU user's API key. Consult Required Keys and OCIDs in the OCI documentation for details on how to obtain the value for this.
- Enter the user's private API key, in PEM format into the What is the OCI user's private API key in PEM format? field. Consult Required Keys and OCIDs in the OCI documentation for details on how to obtain the value for this.
- Enter the tenancy into the What is the OCI tenancy of the OCI user? field.
- Enter the home region code of the tenancy into the What us the OCI tenancy's home region code? field. Details of region codes can be found in Regions and Availability Domains OCI documentation.
- Enter the bucket namespace of the tenancy in the What is the namespace for the bucket? field.
- In the Enter the name of the bucket where your flat file is stored in OCI object storage field, enter the name of the bucket where your flat file is stored in OCI object storage.
- Enter the encoding into the Encoding field. Default is UTF-8.
- In the Field Delimiter field, enter the field delimiter
character used in the Flat File. Default is
,
. - In the Sub Field Delimiter field, enter the sub field delimiter character
used in the Flat File. Default is
#
. - In the MultiValue Delimiter field, enter the multivalue delimiter
character used in the Flat File. Default is
;
. - In the Text Qualifier field, enter the character used in the Flat File to
act as a text qualifier. Default is
"
. - In the Date Format field, enter the Java data format in
which date type fields are included in the Flat File, for example
dd/MM/yyyy
. If no date format is specified, the date field will be assumed to be of data type Long. - If you want to check the connectivity to your Flat File, click the Test integration button.
- Click Add to create the orchestrated system.
Finish up
- Customize before enabling the system for data loads
- Activate and prepare the data load with the provided defaults
Post Configuration
Check Bucket Folder Structure
After creation of the orchestrated system, the following folder structure should be created in the defined bucket.
<ServiceInstanceName>/<OrchestratedSystemName>
failed
inbox
outbox
sample
schema
failed
: Files with any kind of data issue will be moved to this folder under the respective entity folder, in the event of a dataload operation failure.inbox
: Contains the entity subfolders in which CSV files should be placed to be included in the dataload operation.outbox
: Used to output the provisioning events for each entity.sample
: Contains example CSVs with the expected header. These can be used as a reference for generating data and putting in the inbox for data load. These files should not be altered.schema
: contains the JSON representation of each entity's schema.This can be referred to for understanding details like:dataType
- Mandatory attributes
- Whether an attribute is multivalued or not
- If the attribute is complex and has nested attributes (dataType will be CUSTOM)
- Supported
dataTypes
are:- TEXT
- NUMBER
- DECIMAL_NUMBER
- DATE
- FLAG
- CUSTOM
Define Custom Attributes
Custom attributes are supported for the IDENTITY
entity.
If you want to include custom attributes in your dataload then you need to add them
in the
<ServiceInstanceName>/<OrchestratedSystemName>/schema/IDENTITY.json
file.
- start with a character: A-Z or a-z
- contain only characters or numbers: A-Z or a-z or 0-9
- For the DATE type attribute, only long value is supported
- Custom attributes can only be added, they cannot be deleted
- A custom attribute of CUSTOM type cannot be added
Once you have added any custom attributes in the IDENTITY.json
file,
you will need to include them in Oracle Access Governance as described in Fetch Latest Custom Attributes. Once this is completed,
the sample CSV will be updated with the newly added custom attribute(s). Update the
data files in the inbox to include the custom attribute(s) in your next data
load.
Run Dataload
Dataload is on demand. You should run the dataload when you have defined any custom
attributes, and have added the relevant CSV data files into the
inbox
folder. Each time you run a dataload it is a full data
load, there is no incremental load. UTF-8-BOM encoding is not supported.
If there is any kind of failure (single record or complete file failure),
the data load operation will be marked as failed. The files that have been processed
successfully will stay in the inbox
while the failed files will be
moved to the failed
folder. After fixing the data issue, the
customer is expected to put the files back in the inbox
again and
retry the dataload. Data integrity issues, such as a permission being assigned to an
account that is missing in the CSV), can cause the dataload operation to fail.
However, in such cases the CSV files will not be moved to the
failed
folder. Files will be moved to the
failed
folder only when there are issues reading the data
itself, such as missing mandatory data.
For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.
Access to Oracle Support
Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.