Oracle Cloud Infrastructure Documentation

Pipeline Policies

Data Science Pipelines require specific IAM policies to function correctly.

These policies ensure access to essential OCI services such as Networking, Logging, Object Storage, and more. This document provides a comprehensive and concise guide to setting up these policies,

Policies

Pipeline Run Access to OCI Services
Allow pipeline runs to access OCI services such as Logging, Networking, and Object Storage:
allow any-user to use object-family in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
allow any-user to use virtual-network-family in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
allow any-user to use log-content in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
Accessing Container Images in OCIR
Grant permissions for pipeline runs to read container images in OCIR:
allow any-user to read repos in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
Custom Networking
Allow Data Science to use custom networking:
allow service datascience to use virtual-network-family in compartment <COMPARMENT_NAME>
Using Object Storage Buckets as Storage Mounts
Enable pipeline runs to access Object Storage buckets:
allow any-user to use object-family in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
Using File Storage and Storage Mounts
Provide access to file storage:
allow any-user to use file-systems in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
allow any-user to use mount-targets in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
Data Flow and Pipelines Integration
If Data Flow applications are used in pipelines, policies that grant the pipeline run privileges to access Data Flow are required, as follows:
allow any-user to read object-family in compartment <YOUR_COMPARTMENT> where any { request.principal.type = 'datasciencepipelinerun' }
allow any-user to manage dataflow-run in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
allow any-user to read dataflow-application in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
Ensure that users working with pipelines are granted the appropriate privileges. The following policies are needed to establish the integration between Data Flow and the Data Science on behalf of the user. Also on behalf of the user, the service needs to inspect compartments to manage resources there and manage event rules to establish communication between services. The following policy assumes that the users belong to the datascienceusers group:
allow group datascienceusers to inspect compartments in tenancy
allow group datascienceusers to manage cloudevents-rules in tenancy where all {target.rule.type='managed', target.event.source in ('dataflow')}
allow group datascienceusers to read dataflow-application in compartment <YOUR_COMPARTMENT>
General Access for Data Science Resources
Provide comprehensive access to Data Science resources:
allow any-user to manage data-science-family in compartment <COMPARTMENT_NAME> where ALL { request.principal.type = 'datasciencepipelinerun' }
These statements don't use dynamic groups, but it's easy to create equal policies using dynamic groups. For example, instead of writing:
allow any-user to read object-family in compartment <COMPARTMENT_NAME> where ALL { request.principal.type='datasciencepipelinerun' }
you can write:
allow dynamic-group sample_group to read object-family in compartment <COMPARTMENT_NAME>
where sample_group is defined as:
ALL { resource.type = 'datasciencepipelinerun' }