Oracle Cloud Infrastructure Documentation

Creating a Private Endpoint

Create a private endpoint for a custom or pretrained model on a hosting dedicated AI cluster in OCI Generative AI.

Note

Before you create a private endpoint, perform the Prerequisites for Private Endpoints and have the following details:

  • The name of the Virtual Cloud Network (VCN)
  • The name of the private subnet in the VCN

By default, a tenancy has a maximum limit of 5 private endpoints. For more Generative AI private endpoints, request an increase in the limit, private-endpoint-count for the Generative AI service.

  • On the Private Endpoints list page, select Create private endpoint. If you need help finding the list page, see Listing Private Endpoints.

    General Information

    1. Select a compartment to create the private endpoint in. The default compartment is the same as the list page, but you can select any compartment that you have permission to work in.
      We recommend that you create the private endpoint in the same compartment as the model that will use this endpoint.
    2. (Optional) Enter a name for the endpoint. Start the name with a letter or underscore, followed by letters, numbers, hyphens, or underscores. The length can be 1 to 255 characters. If you don't enter a name, the system generates a name that you can change later.
      The generated name has the format generativeaiprivateendpoint<timestamp>. Example: generativeaiprivateendpoint20250929212918
    3. (Optional) Enter a description for the model.

    VCN and Subnet

    Select the following information:

    • VCN compartment
    • VCN
    • Private subnet compartment
    • Private subnet

    DNS and Network Security Groups

    1. Enter a domain namespace (DNS) prefix for the FQDN.
      A preview displays the FQDN with this DNS prefix. A fully qualified domain name is a complete unique name for a network resource, that's resolved to a specific IP address. For example, 
      <DNS-prefix>.pe.inference.generativeai.us-chicago-1.oci.oraclecloud.com
    2. (Optional) Select whether to add one or more network security groups by selecting Add network security group for each group that you want to add.
      Learn about Security Rules.
    3. (Optional) Select a network security group from then list.
    4. (Optional) Add more network security groups.
    5. (Optional) Select Add tag and assign tags to this private endpoint. See Resource Tags.
    6. Select Create.

    Use this Endpoint for On-Demand Models

    By default, this private endpoint is available for models hosted on dedicated AI clusters. If you want this endpoint to also be available for on-demand models that are offered in Generative AI service, then perform this step:
    1. Select Allow Usage In On-Demand Mode.
    2. See the Tip at the end of this section on how to reach the on-demand model.
    Important

    To access a Generative AI model through this private endpoint, see Adding a Model to a Private Endpoint.

    Create the Endpoint

    1. (Optional) Select Add tag and assign tags to this private endpoint. See Resource Tags.
    2. Select Create.
    Tip

    To use this private endpoint to reach an on-demand model, create a Compute instance in the private subnet allocated for the private endpoint, add your code to the Compute instance, use the FQDN for the private endpoint, and access the model from that Compute instance.

  • Use the generative-ai-private-endpoint create command and required parameters to create a private endpoint:

    oci generative-ai generative-ai-private-endpoint create [OPTIONS] 
[OPTIONS]

    For a complete list of parameters and values for CLI commands, see the CLI Command Reference.

    Note

    For pretrained models, instead of an OCID, you can use the model name exactly as listed in the Console's playground. You can also find this OCI model name, in the model's detail page in Offered Pretrained Foundational Models in Generative AI.

  • Run the CreateGenerativeAiPrivateEndpoint operation to create a private endpoint.