11.1.2 Configure Gen AI service with OBRH

This topic describes the systematic instructions to configure the Gen AI service with OBRH.

Calls to the LLM can be routed via OBRH. Follow the below steps to configure OBRH to use the Gen AI Service.

Note:

Before configuring the OBRH, make sure that all the common dependencies are installed except the LLM dependencies.

Note:

For detailed information on the OBRH configuration, refer to Routing Hub Configuration User Guide.
  1. On Home screen, click Core Maintenance. Under Core Maintenance, click Routing Hub. Under Routing Hub, click Service Consumers.
  2. Create a new service consumer with default name GENAI.
  3. Click on GENAI service consumer and add a new service provider with the following details:

    Table 11-1 Add Service Provider

    Field Value
    Product Name COHERE
    Type EXTERNAL
    Version 0.1
    Headers Add the below headers as Name - Value:
    • accept - application/json
    • content-type - application/json
    Service Update the below values under service
    Type OTHERS
    Name Add the below services:
    • CHAT_ENDPOINT
    • EMBED_ENDPOINT
    Http Method POST
    Endpoint Add the below endpoints for the respective services:
    • For CHAT_ENDPOINT- v1/chat
    • For EMBED_ENDPOINT - v1/embed
    Service Header Add the service header for both the services:
    • Name - Authorization
    • Value - Bearer $body.token
  4. Click the COHERE service provider and add the implementation parameters.

    Table 11-2 Add Implementation

    Field Value
    Name COHERE_Default
    Description Default Implementation
    Type DEFAULT
    Default Toggle Enabled
    Scheme https
    Host api.cohere.com
    Port 0
  5. Click Consumer Services and add the following service IDs.
    • COHERE_CHAT_ENDPOINT
    • COHERE_EMBED_ENDPOINT
  6. Click the consumer service IDs and add the transformation, and routing details.
    1. COHERE_CHAT_ENDPOINT

      Table 11-3 Add Transformation

      Field Value
      Name Chat
      Active Toggle Enabled
      Product Processor COHERE 0.1
      Implementation COHERE_Default
      Service CHAT_ENDPOINT - v1/chat
      Request Transformation Update the below values under request transformation
      Body Type RAW
      Template Type JSLT
      Template {"model": .model,"message": .message, "temperature": .temperature, "chat_history": .chat_history,"prompt_truncation": "AUTO","stream": false,"connectors": []}

      Table 11-4 Add Route

      Field Value
      Name chat
      Start/Stop START
      Auto Shutdown Toggle Enabled
      Default/Custom Rule Default Rule
      Transformations Click Add and select the transformation created.
    1. COHERE_EMBED_ENDPOINT

      Table 11-5 Add Transformation

      Field Value
      Name embed
      Active Toggle Enabled
      Product Processor COHERE 0.1
      Implementation COHERE_Default
      Service EMBED_ENDPOINT - v1/embed
      Request Transformation Update the below values under request transformation
      Body Type RAW
      Template Type JSLT
      Template {"texts":.texts,"model": .model,"input_type": .input_type}

      Table 11-6 Add Route

      Field Value
      Name route1
      Start/Stop START
      Auto Shutdown Toggle Enabled
      Default/Custom Rule Default Rule
      Transformations Click Add and select the transformation created.