Purpose

To communicate with Large Language Models (LLMs) through natural language conversations. You can generate a textual answer, description, or summary for prompts and images, given as input to LLM-powered chat interfaces.

• Prompt to Text:

  A prompt can be an input text string, such as a question that you ask an LLM. For example, "What is Oracle Text?". A prompt can also be a command, such as "Summarize the following ...", "Draft an email asking for ...", or "Rewrite the following ...", and can include results from a search. The LLM responds with a textual answer or description based on the specified task in the prompt.

  For this operation, this API makes a REST call to your chosen remote third-party provider (Cohere, Generative AI, Google AI, Hugging Face, OpenAI, or Vertex AI) or local third-party provider (Ollama).

• Image to Text:

  You can also prompt with a media file, such as an image, to extract text from pictures or photos. You supply a text question as the prompt (such as "What is this image about?" or "How many birds are there in this painting?") along with the image. The LLM responds with a textual analysis or description of the contents of the image.

  For this operation, this API makes a REST call to your chosen remote third-party provider (Google AI, Hugging Face, OpenAI, or Vertex AI) or local third-party provider (Ollama).

Syntax

This function accepts the input as CLOB containing text data (for textual prompts) or as BLOB containing media data (for media files such as images). It then processes this information to generate a new CLOB containing the generated text.

• Prompt to Text:

  DBMS_VECTOR.UTL_TO_GENERATE_TEXT (
      	DATA          IN CLOB,
      	PARAMS        IN JSON default NULL
  ) return CLOB;

• Image to Text:

  DBMS_VECTOR.UTL_TO_GENERATE_TEXT(
            TEXT_DATA      IN CLOB,
            MEDIA_DATA     IN BLOB,
            MEDIA_TYPE     IN VARCHAR2 default 'image/jpeg',
            PARAMS         IN JSON default NULL
  ) return CLOB;

DATA and TEXT_DATA

Specify the textual prompt as CLOB for the DATA or TEXT_DATA clause.

MEDIA_DATA

Specify the BLOB file, such as an image or a visual PDF file.

MEDIA_TYPE

Specify the image format for the given image or visual PDF file (BLOB file) in one of the supported image data MIME types. For example:

• For PNG: image/png

• For JPEG: image/jpeg

• For PDF: application/pdf

PARAMS

Specify the following input parameters in JSON format, depending on the service provider that you want to access for text generation:

{
  "provider"         : "<AI service provider>",
  "credential_name"  : "<credential name>",  
  "url"              : "<REST endpoint URL for text generation service>",
  "model"            : "<text generation model name>",
  "transfer_timeout" : <maximum wait time for the request to complete>,
  "max_count": "<maximum calls to the AI service provider>",
  "<additional REST provider parameter>": "<REST provider parameter value>"
}

Additional third-party provider parameters:

Optionally, specify additional provider-specific parameters.

Let us look at some example configurations for all third-party providers:

{
  "provider"       : "cohere", 
  "credential_name": "COHERE_CRED",
  "url"            : "https://api.cohere.example.com/chat",
  "model"          : "command"
}

Generative AI example:

{
  "provider"       : "ocigenai",
  "credential_name": "OCI_CRED",
  "url"            : "https://inference.generativeai.us-example.com/chat",
  "model"          : "cohere.command-r-16k",
  "chatRequest"    : {
                      "maxTokens"    : 256
                     }
}

{
  "provider"        : "googleai",
  "credential_name" : "GOOGLEAI_CRED",
  "url"             : "https://googleapis.example.com/models/",
  "model"           : "gemini-pro:generateContent"
}

{
  "provider"        : "huggingface",
  "credential_name" : "HF_CRED",
  "url"             : "https://api.huggingface.example.com/models/",
  "model"           : "gpt2"
}

{
  "provider"       : "ollama", 
  "host"           : "local", 
  "url"            : "http://localhost:11434/api/generate", 
  "model"          : "phi3:mini"
}

{
  "provider"        : "openai",
  "credential_name" : "OPENAI_CRED",
  "url"             : "https://api.openai.example.com",
  "model"           : "gpt-4o-mini",
  "max_tokens"      : 60,
  "temperature"     : 1.0
}

{
  "provider"         : "vertexai",
  "credential_name"  : "VERTEXAI_CRED",
  "url"              : "https://googleapis.example.com/models/",
  "model"            : "gemini-1.0-pro:generateContent",
  "generation_config": {
                        "temperature"    : 0.9,
                        "topP"           : 1,
                        "candidateCount" : 1,
                        "maxOutputTokens": 256
                       }
}

Examples

• Prompt to Text:

  The following statements generate a text response by making a REST call to Generative AI. The prompt given here is "What is Oracle Text?".

  Here, the cohere.command-r-16k and meta.llama-3.1-70b-instruct models are used. You can replace the model value with any other supported model that you want to use with Generative AI, as listed in Supported Third-Party Provider Operations and Endpoints.

  Using the cohere.command-r-16k model:

  -- select example
  
  var params clob;
  exec :params := '
  {
    "provider"       : "ocigenai",
    "credential_name": "OCI_CRED",
    "url"            : "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/chat",
    "model"          : "cohere.command-r-16k",
    "chatRequest"    : {
                        "maxTokens": 256
                       }
  }';
  
  select dbms_vector.utl_to_generate_text(
   'What is Oracle Text?',
   json(:params)) from dual;
  
  -- PL/SQL example
  
  declare
    input clob;
    params clob;
    output clob;
  begin
    input := 'What is Oracle Text?';
  
    params := '
  {
    "provider"       : "ocigenai",
    "credential_name": "OCI_CRED",
    "url"            : "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/chat",
    "model"          : "cohere.command-r-16k",
    "chatRequest"    : {
                        "maxTokens": 256
                       }
  }';
  
    output := dbms_vector.utl_to_generate_text(input, json(params));
    dbms_output.put_line(output);
    if output is not null then
      dbms_lob.freetemporary(output);
    end if;
  exception
    when OTHERS THEN
      DBMS_OUTPUT.PUT_LINE (SQLERRM);
      DBMS_OUTPUT.PUT_LINE (SQLCODE);
  end;
  /

  Using the meta.llama-3.1-70b-instruct model:

  -- select example
  
  var params clob;
  exec :params := '
  {
     "provider"       : "ocigenai",
     "credential_name": "OCI_CRED",
     "url"            : "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/chat",
     "model"          : "meta.llama-3.1-70b-instruct",
     "chatRequest"    : {
                         "topK" : 1
                        }
  }';
  
  select dbms_vector.utl_to_generate_text(
   'What is Oracle Text?',
   json(:params)) from dual;
  
  -- PL/SQL example
  
  declare
    input clob;
    params clob;
    output clob;
  begin
    input := 'What is Oracle Text?';
  
    params := '
  {
     "provider"       : "ocigenai",
     "credential_name": "OCI_CRED",
     "url"            : "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/chat",
     "model"          : "meta.llama-3.1-70b-instruct",
     "chatRequest"    : {
                         "topK" : 1
                        }
  }';
  
    output := dbms_vector.utl_to_generate_text(input, json(params));
    dbms_output.put_line(output);
    if output is not null then
      dbms_lob.freetemporary(output);
    end if;
  exception
    when OTHERS THEN
      DBMS_OUTPUT.PUT_LINE (SQLERRM);
      DBMS_OUTPUT.PUT_LINE (SQLCODE);
  end;
  /

  End-to-end examples:

  To run end-to-end example scenarios, see Generate Text Response.

• Image to Text:

  The following statements generate a text response by making a REST call to OpenAI. Here, the input is an image (sample_image.jpeg) along with the prompt "Describe this image?".

  -- select example
  
  var input clob;
  var media_data blob;
  var media_type clob;
  var params clob;
  
  begin
    :input := 'Describe this image';
    :media_data := load_blob_from_file('DEMO_DIR', 'sample_image.jpeg');
    :media_type := 'image/jpeg';
    :params := '
  {
    "provider"       : "openai",
    "credential_name": "OPENAI_CRED",
    "url"            : "https://api.openai.com/v1/chat/completions",
    "model"          : "gpt-4o-mini",
    "max_tokens"     : 60
  }';
  end;
  /
  
  select dbms_vector.utl_to_generate_text(:input, :media_data, :media_type, json(:params));
  
  -- PL/SQL example
  
  declare
    input clob;
    media_data blob;
    media_type varchar2(32);
    params clob;
    output clob;
  
  begin
    input := 'Describe this image';
    media_data := load_blob_from_file('DEMO_DIR', 'image_file');
    media_type := 'image/jpeg';
    params := '
  {
    "provider"       : "openai",
    "credential_name": "OPENAI_CRED",
    "url"            : "https://api.openai.com/v1/chat/completions",
    "model"          : "gpt-4o-mini",
    "max_tokens"     : 60
  }';
  
    output := dbms_vector.utl_to_generate_text(
      input, media_data, media_type, json(params));
    dbms_output.put_line(output);
  
    if output is not null then
      dbms_lob.freetemporary(output);
    end if;
    if media_data is not null then
      dbms_lob.freetemporary(media_data);
    end if;
  exception
    when OTHERS THEN
      DBMS_OUTPUT.PUT_LINE (SQLERRM);
      DBMS_OUTPUT.PUT_LINE (SQLCODE);
  end;
  /

  End-to-end examples:

  To run end-to-end example scenarios, see Describe Image Content.