1. SuiteCloud Platform
  2. SuiteScript
  3. SuiteScript 2.x API Reference
  4. SuiteScript 2.x Modules
  5. N/llm Module
  6. llm.generateTextStreamed(options)

llm.generateTextStreamed(options)

Note:

The content in this help topic pertains to SuiteScript 2.1.

Method Description

Returns the streamed response from the LLM for a given prompt.

This method is similar to llm.generateText(options) but returns the LLM response as a stream. After calling this method, you can access the partial response (using the StreamedResponse.text property of the returned llm.StreamedResponse object) before the entire response has been generated. You can also use an iterator to examine each token returned by the LLM, as the following example shows:

                    var response = llm.generateTextStreamed("Write a 500 word pitch for a TV show about bears");
var iter = response.iterator();
iter.each(function(token){
    log.debug("token.value: " + token.value);
    log.debug("response.text: " + response.text);
    return true;
})

In this example, token.value contains the values of each token returned by the LLM, and response.text contains the partial response up to and including that token. For more information about iterators in SuiteScript, see Iterator.

When unlimited usage mode is used, this method accepts the OCI configuration parameters. You can also specify OCI configuration parameters on the SuiteScript tab of the AI Preferences page. For more information, see Using Your Own OCI Configuration for SuiteScript Generative AI APIs.

Returns

llm.StreamedResponse

Aliases

llm.chatStreamed(options)

Note:

These aliases use the same parameters and can throw the same errors as the llm.generateTextStreamed(options) method.

Supported Script Types

Server scripts

For more information, see SuiteScript 2.x Script Types.

Governance

100

Module

N/llm Module

Since

2025.1

Parameters

Parameter

Type

Required / Optional

Description

Since

options.prompt

string

required

Prompt for the LLM.

2024.1

options.chatHistory

llm.ChatMessage[]

optional

Chat history to be taken into consideration.

2024.1

options.documents

llm.Document[]

optional

A list of documents to provide additional context for the LLM to generate the response.

2025.1

options.image

file.File

optional

An image to query.

You can send an image (as a file.File object) to the LLM and ask questions about the image and get text outputs, such as:

  • Advanced image captions

  • Detailed description of an image

  • Answers to questions about an image

  • Information about charts and graphs in an image

Note:

Image processing is available only when using the Meta Llama 3.2 model (meta.llama-3.2-90b-vision-instruct).

For more information about creating and loading files, see N/file Module.

2025.1

options.modelFamily

enum

optional

Specifies the LLM to use. Use llm.ModelFamily to set the value. If not specified, the Cohere Command R LLM is used.

Note:

JavaScript does not include an enumeration type. The SuiteScript 2.x documentation uses the term enumeration (or enum) to describe a plain JavaScript object with a flat, map-like structure. In this object, each key points to a read-only string value.

2024.2

options.modelParameters

Object

optional

Parameters of the model. For more information about the model parameters, refer to the Chat Model Parameters topic in the Oracle Cloud Infrastructure Documentation.

2024.1

options.modelParameters.frequencyPenalty

number

optional

A penalty that is assigned to a token when that token appears frequently. The higher the value, the stronger a penalty is applied to previously present tokens, proportional to how many times they have already appeared in the prompt or prior generation. See Model Parameter Values by LLM for valid values.

2024.1

options.modelParameters.maxTokens

number

optional

The maximum number of tokens the LLM is allowed to generate. The average number of tokens per word is 3. See Model Parameter Values by LLM for valid values.

2024.1

options.modelParameters.presencePenalty

number

optional

A penalty that is assigned to each token when it appears in the output to encourage generating outputs with tokens that haven't been used. Similar to frequencyPenalty, except that this penalty is applied equally to all tokens that have already appeared, regardless of their exact frequencies. See Model Parameter Values by LLM for valid values.

2024.1

options.modelParameters.temperature

number

optional

Defines a range of randomness for the response. A lower temperature will lean toward the highest probability tokens and expected answers, while a higher temperature will deviate toward random and unconventional responses. A lower value works best for responses that must be more factual or accurate, and a higher value works best for getting more creative responses. See Model Parameter Values by LLM for valid values.

2024.1

options.modelParameters.topK

number

optional

Determines how many tokens are considered for generation at each step. See Model Parameter Values by LLM for valid values.

2024.1

options.modelParameters.topP

number

optional

Sets the probability, which ensures that only the most likely tokens with total probability mass of p are considered for generation at each step. If both topK and topP are set, topP acts after topK. See Model Parameter Values by LLM for valid values.

2024.1

options.ociConfig

Object

optional

Configuration needed for unlimited usage through OCI Generative AI Service. Required only when accessing the LLM through an Oracle Cloud Account and the OCI Generative AI Service. SuiteApps installed to target accounts are prevented from using the free usage pool for N/llm and must use the OCI configuration.

2024.1

options.ociConfig.compartmentId

string

optional

Compartment OCID. For more information, refer to Managing Compartments in the Oracle Cloud Infrastructure Documentation.

2024.1

options.ociConfig.endpointId

string

optional

Endpoint ID. This value is needed only when a custom OCI DAC (dedicated AI cluster) is to be used. For more information, refer to Managing an Endpoint in Generative AI in the Oracle Cloud Infrastructure Documentation.

2024.1

options.ociConfig.fingerprint

string

optional

Fingerprint of the public key (only a NetSuite secret is accepted—see Creating Secrets ). For more information, refer to Required Keys and OCIDs in the Oracle Cloud Infrastructure Documentation.

2024.1

options.ociConfig.privateKey

string

optional

Private key of the OCI user (only a NetSuite secret is accepted—see Creating Secrets ). For more information, refer to Required Keys and OCIDs in the Oracle Cloud Infrastructure Documentation.

2024.1

options.ociConfig.tenancyId

string

optional

Tenancy OCID. For more information, refer to Managing the Tenancy in the Oracle Cloud Infrastructure Documentation.

2024.1

options.ociConfig.userId

string

optional

User OCID. For more information, refer to Managing Users in the Oracle Cloud Infrastructure Documentation.

2024.1

options.preamble

string

optional

Preamble override for the LLM. A preamble is the Initial context or guiding message for an LLM. For more details about using a preamble, refer to About the Chat Models in Generative AI (Chat Model Parameters section) in the Oracle Cloud Infrastructure Documentation.

Note:

Only valid for the Cohere Command R model.

2024.1

options.timeout

number

optional

Timeout in milliseconds, defaults to 30,000.

2024.1

Errors

Error Code

Thrown If

SSS_MISSING_REQD_ARGUMENT

The options.prompt parameter is missing.

MUTUALLY_EXCLUSIVE_ARGUMENTS

Both options.modelParameters.presencePenalty and options.modelParameters.frequencyPenalty parameters are used when options.modelFamily is set to COHERE_COMMAND_R. You can use a value greater than zero for one or the other, but not both.

UNRECOGNIZED_MODEL_PARAMETERS

One or more unrecognized model parameters have been used.

UNRECOGNIZED_OCI_CONFIG_PARAMETERS

One or more unrecognized parameters for OCI configuration have been used.

ONLY_API_SECRET_IS_ACCEPTED

The options.ociConfig. privateKey or options.ociConfig.fingerprint parameters are not NetSuite API secrets.

INVALID_MODEL_FAMILY_VALUE

The options.modelFamily parameter was not set to a valid options. (For example, there may have been a typo in the value or in the enum.)

MODEL_1_DOES_NOT_ACCEPT_PREAMBLE

The options.preamble parameter was provided but the model does not support a preamble (for example, when a Meta Llama model is used).

MODEL_1_DOES_NOT_ACCEPT_DOCUMENTS

The options.documents parameter was provided but the model does not support retrieval-augmented generation (RAG).

MODEL_1_DOES_NOT_ACCEPT_IMAGE

The options.image parameter was provided but the model does not support image processing.

DOCUMENT_IDS_MUST_BE_UNIQUE

Documents provided using the options.documents parameter have duplicate IDs.

INVALID_MAX_TOKENS_VALUE

The options.modelParameters.maxTokens parameter value is incorrect for the model. See Model Parameter Values by LLM for valid values.

INVALID_TEMPERATURE_VALUE

The options.modelParameters.temperature parameter value is incorrect for the model. See Model Parameter Values by LLM for valid values.

INVALID_TOP_K_VALUE

The options.modelParameters.topK parameter value is incorrect for the model. See Model Parameter Values by LLM for valid values.

INVALID_TOP_P_VALUE

The options.modelParameters.topP parameter value is incorrect for the model. See Model Parameter Values by LLM for valid values.

INVALID_FREQUENCY_PENALTY_VALUE

The options.modelParameters.frequencyPenalty parameter value is incorrect for the model. See Model Parameter Values by LLM for valid values.

INVALID_PRESENCE_PENALTY_VALUE

The options.modelParameters.presencePenalty parameter value is incorrect for the model. See Model Parameter Values by LLM for valid values.

MAXIMUM_PARALLEL_REQUESTS_LIMIT_EXCEEDED

The number of parallel requests to the LLM is greater than 5.

Syntax

Important:

The following code sample shows the syntax for this member. It is not a functional example. For a complete script example, see N/llm Module Script Samples.

            // Add additional code
...

const response = llm.generateTextStreamed({
   // preamble is optional for Cohere and must not be used for Meta Llama
   preamble: "You are a successful salesperson. Answer in an enthusiastic, professional tone.", 
   prompt: "Hello World!",
   documents: [doc1, doc2], // create documents using llm.createDocument(options)
   modelFamily: llm.ModelFamily.COHERE_COMMAND_R, // uses COHERE_COMMAND_R when modelFamily is omitted
   modelParameters: {
      maxTokens: 1000,
      temperature: 0.2,
      topK: 3,
      topP: 0.7,
      frequencyPenalty: 0.4,
      presencePenalty: 0
   },
   ociConfig: {
      // Replace ociConfig values with your Oracle Cloud Account values
      userId: 'ocid1.user.oc1..aaaaaaaanld….exampleuserid',
      tenancyId: 'ocid1.tenancy.oc1..aaaaaaaabt….exampletenancyid',
      compartmentId: 'ocid1.compartment.oc1..aaaaaaaaph….examplecompartmentid',
   // Replace fingerprint and privateKey with your NetSuite API secret ID values
      fingerprint: 'custsecret_oci_fingerprint',
      privateKey: 'custsecret_oci_private_key'
   }
});

var iter = response.iterator();
iter.each(function(token){
    log.debug("token.value: " + token.value);
    log.debug("response.text: " + response.text);
    return true;
})

...
// Add additional code

Related Topics

General Notices