27.3.9.1 LLM Class

This class represents a large language model. Members accessible from instances of this class are listed here:

name (String): The name of the model
options (Object): The model's configuration options
isLoaded (Boolean): true if the model is loaded, false if it is not

LLM class constructor

LLM LLM(
  String name,
  Object options
)

Arguments

name: the name of the model
options (Object) (default {}): an object containing the options used by this instance

Return type

An instance of LLM

Usage

let model = MLL("cohere.command", {max_tokens: 10})

LLM provides methods for creating embeddings, generating responses, and performing Retrieval Augmented Generation. The API also provides convenience methods under the ml namespace. Both the LLM and the ml versions of these methods support variants of the methods for performing single jobs and batch processing.

LLM.unload()

Unloads the model that was loaded in the constructor. This is optional, but recommended, since doing so can reduce memory usage; after unloading, any subsequent attempt to use the instance raises an error.

Signature

```
undefined LLM.unload()
```

Arguments

None

Return type

undefined

Usage

```
model.unload()
```

LLM.generate()

This method acts as a wrapper for ML_GENERATE, and generates a response using the prompt and options provided for the loaded model. It supports two variants, one for a single invocation, and one for batch processing; both of these are described in the next few paragraphs.

Signature (single job)

Object LLM.generate(
  String prompt,
  Object options
)

Arguments

prompt (String): prompt to be used for text generation
options (Object) (default {}): an object containing the options used by this instance; see the description of ML_GENERATE for available options

Return type

Object: The structure is similar to that of ML_GENERATE.

Usage

let response = model.generate("What is MySql?", {"top_k": 2, "task": "generation"})

Signature (batch processing)

undefined LLM.generate(
  Table inputTable,
  String inputColumn,
  String outputColumn,
  Object options
)

Arguments

inputTable (Table): Table to use for operations
inputColumn (String): Name of column from inputTable to be embedded
outputColumn (String): Name of column in which to store embeddings; this can be either a fully-qualified name of a column or the name of the column only; in the latter case, the input table and its schema are used to construct the fully-qualified name
options (Object) (optional; default {}): An object containing the options used for embedding; see the description of ML_EMBED_ROW for available options

Return type

undefined

Usage

let schema = session.getSchema("mlcorpus")
let table = schema.getTable("genai_table")

model.generate(table, "input", "mlcorpus.predictions.response")

LLM.embed()

This method is a wrapper for ML_EMBED_ROW, and generates an embedding whose type corresponds to the MySQL VECTOR type. It supports two variants, one for a single invocation, and one for batch processing; both of these are described in the next few paragraphs.

Signature (single job)

Float32Array LLM.embed(
  String query,
  Object options
)

Arguments

query (String): The text of the query to be embedded
options (Object) (optional; default {}): An object containing the options used for embedding; see the description of ML_EMBED_ROW for available options

Return type

Float32Array (MySQL VECTOR): The embedding

Usage

let embedding = model.embed("What is MySql?")

Signature (batch processing)

undefined LLM.embed(
  Table inputTable, 
  String inputColumn, 
  String outputColumn, 
  Object options
)

Arguments

inputTable (Table): Table to use for operations
inputColumn (String): Name of column from inputTable to be embedded
outputColumn (String): Name of column in which to store embeddings; this can be either a fully-qualified name of a column or the name of the column only; in the latter case, the input table and its schema are used to construct the fully-qualified name
options (Object) (optional; default {}): An object containing the options used for embedding; see the description of ML_EMBED_ROW for available options

Return type

undefined

Usage

// Using fully-qualified output column name

let schema = session.getSchema("mlcorpus")
let table = schema.getTable("genai_table")
model.embed(table, "input", "mlcorpus.predictions.response")

// Using output column name only; constructs the fully-qualfied name
// "mlcorpus.genai_table.response"

let schema = session.getSchema("mlcorpus")
let table = schema.getTable("genai_table")
model.embed(table, "input", "response")

LLM.rag()

This method performs Retrieval Augmented Generation for a given query using the loaded genAI model, acting as a wrapper for ML_RAG. It supports two variants, one for a single invocation, and one for batch processing; both of these are described in the next few paragraphs.

Signature (single job)

Object LLM.rag(
  String query,
  Object options
)

Arguments

query (String): The text of the query to be used for generation
options (Object) (default {}): The options employed for generation; these follow the same rules as the options used with LLM.generate()

Return type

Object: The structure is similar to that of the object returned by ML_RAG.

Usage

let result = model.rag("What is MySql?", {schema: ["vector_store"], n_citations: 1})

Signature (batch processing)

undefined LLM.rag(
  Table inputTable, 
  String inputColumn, 
  String outputColumn, 
  Object options
)

Arguments

inputTable (Table): Table to use for operations
inputColumn (String): Name of column from inputTable to be embedded
outputColumn (String): Name of column in which to store embeddings; this can be either a fully-qualified name of a column or the name of the column only; in the latter case, the input table and its schema are used to construct the fully-qualified name
options (Object) (optional; default {}): An object containing the options used for embedding; see the description of ML_EMBED_ROW for available options

Return type

undefined

Usage

let schema = session.getSchema("mlcorpus")
let table = schema.getTable("genai_table")

model.rag(table, "input", "mlcorpus.predictions.response", 
  {schema: ["vector_store"], n_citations: 1})