Package com.oracle.bmc.generativeaiinference

Interface GenerativeAiInference

  • All Superinterfaces:
    AutoCloseable
    All Known Implementing Classes:
    GenerativeAiInferenceClient
    @Generated(value="OracleSDKGenerator",
           comments="API Version: 20231130")
public interface GenerativeAiInference
extends AutoCloseable
    OCI Generative AI is a fully managed service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover a wide range of use cases for text generation, summarization, and text embeddings.

    Use the Generative AI service inference API to access your custom model endpoints, or to try the out-of-the-box models to {@link #eNGenerative-ai-inferenceLatestChatResultChat(ENGenerative-ai-inferenceLatestChatResultChatRequest) eNGenerative-ai-inferenceLatestChatResultChat}, {@link #eNGenerative-ai-inferenceLatestGenerateTextResultGenerateText(ENGenerative-ai-inferenceLatestGenerateTextResultGenerateTextRequest) eNGenerative-ai-inferenceLatestGenerateTextResultGenerateText}, {@link #eNGenerative-ai-inferenceLatestSummarizeTextResultSummarizeText(ENGenerative-ai-inferenceLatestSummarizeTextResultSummarizeTextRequest) eNGenerative-ai-inferenceLatestSummarizeTextResultSummarizeText}, and {@link #eNGenerative-ai-inferenceLatestEmbedTextResultEmbedText(ENGenerative-ai-inferenceLatestEmbedTextResultEmbedTextRequest) eNGenerative-ai-inferenceLatestEmbedTextResultEmbedText}.

    To use a Generative AI custom model for inference, you must first create an endpoint for that model. Use the {@link #eNGenerative-aiLatest(ENGenerative-aiLatestRequest) eNGenerative-aiLatest} to {@link #eNGenerative-aiLatestModel(ENGenerative-aiLatestModelRequest) eNGenerative-aiLatestModel} by fine-tuning an out-of-the-box model, or a previous version of a custom model, using your own data. Fine-tune the custom model on a {@link #eNGenerative-aiLatestDedicatedAiCluster(ENGenerative-aiLatestDedicatedAiClusterRequest) eNGenerative-aiLatestDedicatedAiCluster}. Then, create a {@link #eNGenerative-aiLatestDedicatedAiCluster(ENGenerative-aiLatestDedicatedAiClusterRequest) eNGenerative-aiLatestDedicatedAiCluster} with an Endpoint to host your custom model. For resource management in the Generative AI service, use the {@link #eNGenerative-aiLatest(ENGenerative-aiLatestRequest) eNGenerative-aiLatest}.

    To learn more about the service, see the [Generative AI documentation](https://docs.oracle.com/iaas/Content/generative-ai/home.htm).

    This service client uses CircuitBreakerUtils.DEFAULT_CIRCUIT_BREAKER for all the operations by default if no circuit breaker configuration is defined by the user.

    • Method Detail

      • refreshClient

        void refreshClient()
        Rebuilds the client from scratch.

        Useful to refresh certificates.

      • setEndpoint

        void setEndpoint​(String endpoint)
        Sets the endpoint to call (ex, https://www.example.com).
        Parameters:
        endpoint - The endpoint of the service.

      • getEndpoint

        String getEndpoint()
        Gets the set endpoint for REST call (ex, https://www.example.com)

      • setRegion

        void setRegion​(Region region)
        Sets the region to call (ex, Region.US_PHOENIX_1).

        Note, this will call setEndpoint after resolving the endpoint. If the service is not available in this Region, however, an IllegalArgumentException will be raised.

        Parameters:
        region - The region of the service.

      • setRegion

        void setRegion​(String regionId)
        Sets the region to call (ex, ‘us-phoenix-1’).

        Note, this will first try to map the region ID to a known Region and call setRegion.

        If no known Region could be determined, it will create an endpoint based on the default endpoint format (Region.formatDefaultRegionEndpoint(Service, String) and then call setEndpoint.

        Parameters:
        regionId - The public region ID.

      • useRealmSpecificEndpointTemplate

        void useRealmSpecificEndpointTemplate​(boolean realmSpecificEndpointTemplateEnabled)
        Determines whether realm specific endpoint should be used or not.

        Set realmSpecificEndpointTemplateEnabled to “true” if the user wants to enable use of realm specific endpoint template, otherwise set it to “false”

        Parameters:
        realmSpecificEndpointTemplateEnabled - flag to enable the use of realm specific endpoint template

      • applyGuardrails

        ApplyGuardrailsResponse applyGuardrails​(ApplyGuardrailsRequest request)
        Applies guardrails to the input text, including content moderation, PII detection, and prompt injection protection.
        Parameters:
        request - The request object containing the details to send
        Returns:
        A response object containing details about the completed operation
        Throws:
        BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

        Example: Click <a href=“https://docs.oracle.com/en-us/iaas/tools/java-sdk-examples/3.75.0/generativeaiinference/ApplyGuardrailsExample.java.html"target=”_blank"rel=“noopener noreferrer”>here to see how to use ApplyGuardrails API.

      • chat

        ChatResponse chat​(ChatRequest request)
        Creates a response for the given conversation.
        Parameters:
        request - The request object containing the details to send
        Returns:
        A response object containing details about the completed operation
        Throws:
        BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

        Example: Click <a href=“https://docs.oracle.com/en-us/iaas/tools/java-sdk-examples/3.75.0/generativeaiinference/ChatExample.java.html"target=”_blank"rel=“noopener noreferrer”>here to see how to use Chat API.

      • embedText

        EmbedTextResponse embedText​(EmbedTextRequest request)
        Produces embeddings for the inputs.

        An embedding is numeric representation of a piece of text. This text can be a phrase, a sentence, or one or more paragraphs. The Generative AI embedding model transforms each phrase, sentence, or paragraph that you input, into an array with 1024 numbers. You can use these embeddings for finding similarity in your input text such as finding phrases that are similar in context or category. Embeddings are mostly used for semantic searches where the search function focuses on the meaning of the text that it's searching through rather than finding results based on keywords.

        Parameters:
        request - The request object containing the details to send
        Returns:
        A response object containing details about the completed operation
        Throws:
        BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

        Example: Click <a href=“https://docs.oracle.com/en-us/iaas/tools/java-sdk-examples/3.75.0/generativeaiinference/EmbedTextExample.java.html"target=”_blank"rel=“noopener noreferrer”>here to see how to use EmbedText API.

      • generateText

        GenerateTextResponse generateText​(GenerateTextRequest request)
        Generates a text response based on the user prompt.
        Parameters:
        request - The request object containing the details to send
        Returns:
        A response object containing details about the completed operation
        Throws:
        BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

        Example: Click <a href=“https://docs.oracle.com/en-us/iaas/tools/java-sdk-examples/3.75.0/generativeaiinference/GenerateTextExample.java.html"target=”_blank"rel=“noopener noreferrer”>here to see how to use GenerateText API.

      • rerankText

        RerankTextResponse rerankText​(RerankTextRequest request)
        Reranks the text responses based on the input documents and a prompt.

        Rerank assigns an index and a relevance score to each document, indicating which document is most related to the prompt.

        Parameters:
        request - The request object containing the details to send
        Returns:
        A response object containing details about the completed operation
        Throws:
        BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

        Example: Click <a href=“https://docs.oracle.com/en-us/iaas/tools/java-sdk-examples/3.75.0/generativeaiinference/RerankTextExample.java.html"target=”_blank"rel=“noopener noreferrer”>here to see how to use RerankText API.

      • summarizeText

        SummarizeTextResponse summarizeText​(SummarizeTextRequest request)
        Summarizes the input text.
        Parameters:
        request - The request object containing the details to send
        Returns:
        A response object containing details about the completed operation
        Throws:
        BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

        Example: Click <a href=“https://docs.oracle.com/en-us/iaas/tools/java-sdk-examples/3.75.0/generativeaiinference/SummarizeTextExample.java.html"target=”_blank"rel=“noopener noreferrer”>here to see how to use SummarizeText API.