Oracle Cloud Infrastructure Documentation

Paying for Dedicated AI Clusters

You get the following benefits for using dedicated AI clusters in OCI Generative AI:

  • Predictable pricing that doesn’t fluctuate with demand.
  • Great for fine-tuning or hosting models.
  • Minimum hosting commitment: 744 unit-hours per hosting cluster.
  • Minimum fine-tuning commitment: 1 unit-hour per fine-tuning job. (Depending on the model, fine-tuning requires at least 2 units to run).

The following examples calculate dedicated AI cluster cost in OCI Generative AI. For calculating on-demand inferencing cost, see Paying for On-Demand Inferencing.

Matching Models to Dedicated Cluster Unit Prices

If you're hosting foundational models or fine-tuning them on dedicated AI clusters, you're charged by the unit hour rather than by transaction.

Go to the pretrained models page and select the model that you want to work with. In the Dedicated AI Cluster for the Model section, find the unit size for the dedicated AI cluster that matches the model and the Pricing Page Information. Then, review the examples in this section to learn how to calculate the cost for using these models.

Important

Some OCI Generative AI foundational pretrained base models supported for the dedicated serving mode are now deprecated and will retire no sooner than 6 months after the release of the 1st replacement model. You can host a base model, or fine-tune a base model and host the fine-tuned model on a dedicated AI cluster (dedicated serving mode) until the base model is retired. For dedicated serving mode retirement dates, see Retiring the Models.

Hosting a Foundational Model Example 1

John wants to host an instance of the Command R+ 08-2024 (cohere.command-r-plus-08-2024) model on dedicated infrastructure. John deletes the cluster after 40 days and wants to know cost of the cluster. To host a cohere.command-r-plus-08-2024 model, John first needs to identify the unit size that can host the cohere.command-r-plus-08-2024 model. The unit size for cohere.command-r-plus-08-2024 model is a Large Cohere V2_2 unit. See Dedicated AI Cluster for the Model.

John needs a minimum of one Large Cohere V2_2 unit to host the cohere.command-r-plus-08-2024 model. Here are the steps to calculate the cost of a hosting cluster with one Large Cohere V2_2 unit.

  1. Calculate the unit hours for 40 days. 
    40 days x 24 hours per day x 1 unit = 960 unit hours.
  2. Ensure that the unit hours exceed the minimum commitment for hosting the models. 
    960 unit hours > 744 minimum unit hours
  3. Go to AI Pricing and under OCI Generative AI, for Oracle Cloud Infrastructure Generative AI- Large Cohere - Dedicated, find the <Large-Cohere-dedicated-unit-per-hour-price>.
  4. From the dedicated AI cluster section of Dedicated AI Cluster for the Model section, find the multiplier for the cohere.command-r-plus-08-2024 model: 
    For Hosting, Multiply the Unit Price: x 2
  5. Calculate the price for 40 days. 
    price = (960 unit hours) x $<Large-Cohere-dedicated-unit-per-hour-price> x 2

Hosting a Foundational Model Example 2

Alice wants to host an instance of the Command R 08-2024 (cohere.command-r-08-2024) model on dedicated infrastructure. To host a cohere.command-r-08-2024 model, Alice first needs to identify the unit size that can host the Command R 08-2024 model. The unit size for Command R 08-2024 is a Small Cohere V2 unit. See Dedicated AI Cluster for the Model.

Alice decides to buy three units of Small Cohere V2 to handle a higher call volume to the model than a single unit would provide. Alice plans to delete the cluster after five days. Here are the steps to calculate the cost of a hosting cluster with three Small Cohere V2 units for five days.

  1. Calculate the unit hours. 
    5 days x 24 hours per day x 3 units = 360 unit hours.
  2. Compare the unit hours to the minimum commitment for hosting the models. 
    360 unit hours < 744 minimum unit hours
Alice is charged for 744 unit hours.
  3. Go to AI Pricing and under OCI Generative AI, for Oracle Cloud Infrastructure Generative AI- Small Cohere - Dedicated, find the <Small-Cohere-dedicated-unit-per-hour-price>.
  4. From the Dedicated AI Cluster for the Model section, find the multiplier for the cohere.command-r-08-2024 model.

    You don't need to multiply the price for hosting cohere.command-r-08-2024 model.

  5. Calculate the cost for five days. 
    price = (744 unit hours) x $<Small-Cohere-dedicated-unit-per-hour-price>

Fine-Tuning and Hosting a Model Example

Bob wants to fine-tune a Command R 08-2024 (cohere.command-r-08-2024) model. Bob creates a fine-tuning dedicated AI cluster with the preset value of eight Small Cohere V2 units. Bob creates a custom model on the fine-tuning dedicated AI cluster and fine-tunes the Command R 08-2024 foundational model with training data. The fine-tuning job takes 5 hours to complete. Bob creates a fine-tuning cluster every week.

To host a cohere.command-r-08-2024 model, Bob needs to identify the unit size that can host the cohere.command-r-08-2024 model. The unit size for cohere.command-r-08-2024 model is a Small Cohere V2 unit. See Dedicated AI Cluster for the Model. Bob can host up to 50 fine-tuned models on a single hosting cluster. Here are the steps to calculate the monthly cost for fine-tuning and hosting the models.

  1. Calculate the unit hours for each fine-tuning. 
    Each fine-tuning cluster requires 8 units and each cluster is active for 5 hours
fine-tuning per cluster = 40 unit-hours
  2. Compare the unit hours to the minimum commitment for fine-tuning the models. 
    40 unit hours > 1 unit hour
  3. Calculate the unit hours for hosting. 
    31 days x 24 hours per day x 1 unit = 744 unit hours
  4. Compare the unit hours to the minimum commitment for hosting the models. 
    744 unit hours = 744 minimum unit hours
  5. Go to AI Pricing and under OCI Generative AI, for Oracle Cloud Infrastructure Generative AI- Small Cohere - Dedicated, find the <Small-Cohere-dedicated-unit-per-hour-price>.
  6. Find the total monthly price. 
    fine-tuning price = (40 unit hours) per week x (4 weeks) x $<Small-Cohere-dedicated-unit-per-hour-price> 
                            
fine-tuning price = 160 x <Small-Cohere-dedicated-unit-per-hour-price>
    hosting price = (744 unit hours) x $<Small-Cohere-dedicated-unit-per-hour-price>
    total monthly price = (160 + 744 unit hours) x $<Small-Cohere-dedicated-unit-per-hour-price>
Tip

In addition to calculating the price, you can estimate the cost by selecting the AI and Machine Learning category and loading the cost estimator for OCI Generative AI.