Class: OCI::GenerativeAiInference::GenerativeAiInferenceClient

Inherits:
Object
  • Object
show all
Defined in:
lib/oci/generative_ai_inference/generative_ai_inference_client.rb

Overview

OCI Generative AI is a fully managed service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover a wide range of use cases for text generation, summarization, and text embeddings.

Use the Generative AI service inference API to access your custom model endpoints, or to try the out-of-the-box models to chat, generate text, summarize, and create text embeddings.

To use a Generative AI custom model for inference, you must first create an endpoint for that model. Use the Generative AI service management API to create a custom model by fine-tuning an out-of-the-box model, or a previous version of a custom model, using your own data. Fine-tune the custom model on a fine-tuning dedicated AI cluster. Then, create a hosting dedicated AI cluster with an endpoint to host your custom model. For resource management in the Generative AI service, use the Generative AI service management API.

To learn more about the service, see the Generative AI documentation.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(config: nil, region: nil, endpoint: nil, signer: nil, proxy_settings: nil, retry_config: nil) ⇒ GenerativeAiInferenceClient

Creates a new GenerativeAiInferenceClient. Notes: If a config is not specified, then the global OCI.config will be used.

This client is not thread-safe

Either a region or an endpoint must be specified. If an endpoint is specified, it will be used instead of the region. A region may be specified in the config or via or the region parameter. If specified in both, then the region parameter will be used.

Parameters:

  • config (Config) (defaults to: nil) —

    A Config object.

  • region (String) (defaults to: nil) —

    A region used to determine the service endpoint. This will usually correspond to a value in Regions::REGION_ENUM, but may be an arbitrary string.

  • endpoint (String) (defaults to: nil) —

    The fully qualified endpoint URL

  • signer (OCI::BaseSigner) (defaults to: nil) —

    A signer implementation which can be used by this client. If this is not provided then a signer will be constructed via the provided config. One use case of this parameter is instance principals authentication, so that the instance principals signer can be provided to the client

  • proxy_settings (OCI::ApiClientProxySettings) (defaults to: nil) —

    If your environment requires you to use a proxy server for outgoing HTTP requests the details for the proxy can be provided in this parameter

  • retry_config (OCI::Retry::RetryConfig) (defaults to: nil) —

    The retry configuration for this service client. This represents the default retry configuration to apply across all operations. This can be overridden on a per-operation basis. The default retry configuration value is nil, which means that an operation will not perform any retries



61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 61

def initialize(config: nil, region: nil, endpoint: nil, signer: nil, proxy_settings: nil, retry_config: nil)
  # If the signer is an InstancePrincipalsSecurityTokenSigner or SecurityTokenSigner and no config was supplied (they are self-sufficient signers)
  # then create a dummy config to pass to the ApiClient constructor. If customers wish to create a client which uses instance principals
  # and has config (either populated programmatically or loaded from a file), they must construct that config themselves and then
  # pass it to this constructor.
  #
  # If there is no signer (or the signer is not an instance principals signer) and no config was supplied, this is not valid
  # so try and load the config from the default file.
  config = OCI::Config.validate_and_build_config_with_signer(config, signer)

  signer = OCI::Signer.config_file_auth_builder(config) if signer.nil?

  @api_client = OCI::ApiClient.new(config, signer, proxy_settings: proxy_settings)
  @retry_config = retry_config

  if endpoint
    @endpoint = endpoint + '/20231130'
  else
    region ||= config.region
    region ||= signer.region if signer.respond_to?(:region)
    self.region = region
  end
  logger.info "GenerativeAiInferenceClient endpoint set to '#{@endpoint}'." if logger
end

Instance Attribute Details

#api_client ⇒ OCI::ApiClient (readonly)

Client used to make HTTP requests.

Returns:



21
22
23
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 21

def api_client
  @api_client
end

#endpoint ⇒ String (readonly)

Fully qualified endpoint URL

Returns:

  • (String) 


25
26
27
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 25

def endpoint
  @endpoint
end

#region ⇒ String

The region, which will usually correspond to a value in Regions::REGION_ENUM.

Returns:

  • (String) 


35
36
37
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 35

def region
  @region
end

#retry_config ⇒ OCI::Retry::RetryConfig (readonly)

The default retry configuration to apply to all operations in this service client. This can be overridden on a per-operation basis. The default retry configuration value is nil, which means that an operation will not perform any retries

Returns:



31
32
33
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 31

def retry_config
  @retry_config
end

Instance Method Details

#apply_guardrails(apply_guardrails_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use apply_guardrails API.

Applies guardrails to the input text, including content moderation, PII detection, and prompt injection protection.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 123

def apply_guardrails(apply_guardrails_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#apply_guardrails.' if logger

  raise "Missing the required parameter 'apply_guardrails_details' when calling apply_guardrails." if apply_guardrails_details.nil?

  path = '/actions/applyGuardrails'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(apply_guardrails_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#apply_guardrails') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::ApplyGuardrailsResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#chat(chat_details, opts = {}, &block) ⇒ Response

Note:

Click here to see an example of how to use chat API.

Creates a response for the given conversation.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 184

def chat(chat_details, opts = {}, &block)
  logger.debug 'Calling operation GenerativeAiInferenceClient#chat.' if logger

  raise "Missing the required parameter 'chat_details' when calling chat." if chat_details.nil?

  path = '/actions/chat'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json, text/event-stream'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(chat_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#chat') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::ChatResult',
      &block
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#embed_text(embed_text_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use embed_text API.

Produces embeddings for the inputs.

An embedding is numeric representation of a piece of text. This text can be a phrase, a sentence, or one or more paragraphs. The Generative AI embedding model transforms each phrase, sentence, or paragraph that you input, into an array with 1024 numbers. You can use these embeddings for finding similarity in your input text such as finding phrases that are similar in context or category. Embeddings are mostly used for semantic searches where the search function focuses on the meaning of the text that it's searching through rather than finding results based on keywords.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 248

def embed_text(embed_text_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#embed_text.' if logger

  raise "Missing the required parameter 'embed_text_details' when calling embed_text." if embed_text_details.nil?

  path = '/actions/embedText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(embed_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#embed_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::EmbedTextResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#generate_text(generate_text_details, opts = {}, &block) ⇒ Response

Note:

Click here to see an example of how to use generate_text API.

Generates a text response based on the user prompt.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 309

def generate_text(generate_text_details, opts = {}, &block)
  logger.debug 'Calling operation GenerativeAiInferenceClient#generate_text.' if logger

  raise "Missing the required parameter 'generate_text_details' when calling generate_text." if generate_text_details.nil?

  path = '/actions/generateText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json, text/event-stream'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(generate_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#generate_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::GenerateTextResult',
      &block
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#logger ⇒ Logger

Returns The logger for this client. May be nil.

Returns:

  • (Logger) —

    The logger for this client. May be nil.



100
101
102
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 100

def logger
  @api_client.config.logger
end

#rerank_text(rerank_text_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use rerank_text API.

Reranks the text responses based on the input documents and a prompt.

Rerank assigns an index and a relevance score to each document, indicating which document is most related to the prompt.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 373

def rerank_text(rerank_text_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#rerank_text.' if logger

  raise "Missing the required parameter 'rerank_text_details' when calling rerank_text." if rerank_text_details.nil?

  path = '/actions/rerankText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(rerank_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#rerank_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::RerankTextResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#summarize_text(summarize_text_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use summarize_text API.

Summarizes the input text.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 434

def summarize_text(summarize_text_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#summarize_text.' if logger

  raise "Missing the required parameter 'summarize_text_details' when calling summarize_text." if summarize_text_details.nil?

  path = '/actions/summarizeText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(summarize_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#summarize_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::SummarizeTextResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end