Release Update 23.7 Features
AI Vector Search: Arithmetic and Aggregate Operations
Just as you can add (+), subtract (-), or multiply (*) dates, timestamps, intervals, and numbers, you can now apply these arithmetic operators to vectors. The arithmetic operation is performed at each dimensional element of the vectors. It's also possible to calculate the SUM or AVG of a set of vectors.
Arithmetic operations on vectors allow AI systems to manipulate and combine abstract concepts, which enhances their ability to understand and process language or data in more sophisticated ways.
Additional Flexibility Defining JSON-Relational Duality Views
You can:
- Use
field _idin document subobjects to identify a column that selects document. - Use an identity column as an identifying column.
These additional possibilities when defining a duality view provide more kinds of documents that can be supported, and thus allow for more application use cases.
Change Compatible to 23.6.0 to Use New AI Vector Search Features in 23.6 or Later Releases
Starting with Oracle Database 23ai (23.6), the ability to use the new AI Vector Search features is introduced in Release Update (RU) 23.6 or later releases.
Regarding the COMPATIBLE parameter in Release Update 23.6:
- New customers installing Oracle Database 23ai Release Update 23.6 directly:
- The 23.6 RU fresh install and using db create comes with the
COMPATIBLEparameter set to 23.6.0 by default. - After this installation, you will be unable to downgrade
COMPATIBLEparameter to a lower value later.
- The 23.6 RU fresh install and using db create comes with the
- Customers on Oracle Database 23ai Release Update 23.4 and 23.5:
- Before patching, the
COMPATIBLEparameter can be 23.0.0 or 23.4.0 or 23.5.0. - When you install the 23.6 release update (that is, when you patch to RU 23.6), to access the new Vector DB features, you must manually set the
COMPATIBLEparameter to 23.6.0. - Note: In RU 23.6.0. updating the
COMPATIBLEparameter requires downtime. It is not automatically done as part of patching. You must choose to update the databaseCOMPATIBLEsetting.
- Before patching, the
- Customers on Oracle Database 19c or 21c:
- When you upgrade the database to Oracle Database 23ai, the
COMPATIBLEparameter remains as previously set in the source database. - If you want to use the new Vector DB features, then you must manually set the
COMPATIBLEparameter to 23.6.0. - Note: The minimum
COMPATIBLEsetting permitted for upgrading to 23ai is 19.0.0. In other words, you cannot upgrade directly from Oracle Database 12c or 18c to Oracle Database 23ai.
- When you upgrade the database to Oracle Database 23ai, the
Cloud Developer Packages
The Cloud Developer packages provide built-in tools to connect, process, and integrate with cloud services seamlessly. These packages, namely DBMS_CLOUD, DBMS_CLOUD_PIPELINE, DBMS_CLOUD_REPO, DBMS_CLOUD_NOTIFICATION, and DBMS_CLOUD_AI, enable easy data access to cloud storage, automate workflows through pipelines, trigger event notifications, and integrate AI/ML capabilities directly within the database, using these packages or SELECT AI for SQL access to generative AI using Large Language Models (LLMs) and embedding models. Together, they empower modern, cloud-ready database applications while simplifying complex cloud interactions.
The Cloud Developer packages enable businesses to work with data stored in the cloud, process real-time data, and use advanced tools like AI/ML, benefiting from cloud capabilities. This approach helps simplify operations and supports specifically a mix of on-premises and cloud systems.
DBMS_DEVELOPER Package
The DBMS_DEVELOPER package provides an efficient way for developers to retrieve metadata for database objects, such as tables, views, and indexes.
In previous releases, object metadata had to be retrieved using the DBMS_METADATA package, and then queries had to be performed on various data dictionary tables and views. This method returned metadata in XML format.
The DBMS_DEVELOPER package returns metadata in JSON format, which can be easily integrated with applications and services, such as developer tools. You can adjust the level of detail to control the amount of metadata returned, tailoring it to your application's needs. Additionally, metadata retrieval time is significantly improved, enhancing integration with applications and services.
Dimension-Wise Arithmetic Support in PL/SQL
Addition (+), subtraction (-), and multiplication (*) can now be applied to vectors in PL/SQL. The arithmetic operation is performed at each dimensional element of the vectors.
Arithmetic operations on vectors allow AI systems to manipulate and combine abstract concepts, enhancing their ability to understand and process language or data in more sophisticated ways. PL/SQL support of vector arithmetic provides developers a means to apply these operations within PL/SQL blocks and functions without calling out to SQL.
Foreign Function Interface for JavaScript to Call PL/SQL Code Units
The Foreign Function Interface (FFI) allows JavaScript developers to use a more familiar syntax to call code units written in PL/SQL.
Rather than using PL/SQL blocks, it is possible to use native JavaScript constructs to interact with most code written in PL/SQL. Using the FFI provides JavaScript developers with a much improved experience by streamlining the process of integrating PL/SQL code into JavaScript functions.
Hybrid Vector Index for JSON
Hybrid Vector Indexes allow document retrieval by integrating full-text search capabilities with semantic vector search techniques, resulting in higher-quality search results. This powerful feature has now been extended to support JSON columns, offering greater flexibility in data indexing and querying.
Creating a Hybrid Vector Index on a JSON column offers a unified query API that enables users to execute various types of searches:
- Textual queries
- Vector similarity queries
- Hybrid queries that leverage both approaches
This versatile functionality allows users to:
- Easily customize the search experience
- Significantly enhance the quality and relevance of search results
In-Database Algorithms Support for VECTOR Data Type Predictors
This feature enables users to include one or more columns of VECTOR data type as predictors along with structured enterprise data to in-database machine learning algorithms.
Vector representations of unstructured data can be a powerful input to traditional machine learning algorithms. They enable efficient data processing on text and image data, helping to speed data-driven decision making. Providing vectors as input to machine learning models enables handling a broader class of use cases.
Included Columns in Neighbor Partition Vector Indexes
Included columns in vector indexes facilitate faster searches with attribute filters by incorporating non-vector columns within a Neighbor Partition Vector Index. This feature optimizes query execution by removing the need to access these columns from the base table.
Sophisticated workloads often combine business data search on relational columns with vector similarity search. Having included columns in Neighbor Partition Vector Indexes significantly enhances enterprise search capabilities by integrating attribute filters with vector-based similarity searches.
This integration facilitates efficient execution of complex queries by:
- Directly evaluating attribute filters in tandem with vector searches
- Eliminating the need for base table access through expensive join operations.
Moreover, when the index includes all columns required for a query as covering columns, data can be retrieved directly from the index, thereby accelerating query performance.
JSON to Duality Migrator: Multi-Collection Import API
DBMS_JSON_DUALITY.IMPORT_ALL is a PL/SQL procedure that is designed to import multiple document collections into a JSON-relational duality view.
While the existing DBMS_JSON_DUALITY.IMPORT procedure is limited to importing a single collection, the ability to import multiple collections in a single PL/SQL call simplifies the process and helps prevent constraint violation errors. The IMPORT_ALL procedure enables the efficient import of multiple document collections into a JSON-relational duality view.
JSON to Duality Migrator: Validation of Schema and Data
The PL/SQL functions DBMS_JSON_DUALITY.VALIDATE_SCHEMA_REPORT and VALIDATE_IMPORT_REPORT are provided to validate the relational schema and data that are created and imported by the JSON-To-Duality Migrator.
There is a growing need for functions that would validate the recommended relational schema. To address these needs, the validation APIs have been developed that:
- Help users verify accuracy of the recommended relational schema
- Confirms that no data is lost when they migrate their document collections to duality views.
Materialized Expression Columns
Expression columns, also known as virtual columns, are additional columns derived (computed) from existing columns. They can be persisted (materialized) on disk, complementing the existing default functionality of computing the results at runtime only.
Choosing between computation at runtime and computation at DML time for expression columns provides more flexibility in choosing the right approach for an application.
Materializing expression columns trades disk storage for the need to compute the same expression over and over again.
Move Data Chunks Between Shardspaces
With the Oracle Globally Distributed Database composite sharding method, data is organized into different shardspaces, allowing you to differentiate subsets of data; however, any automatic chunk moves for load balancing occurred within a shardspace. With this release, Oracle Globally Distributed Database provides you with the ability to move data chunks from an existing shardspace to another shardspace.
This feature makes it possible to move data between existing shardspaces, or move some data to a newly added shardspace. You can arrange sharded data for your new business needs, such as providing new levels of services or resources for certain customers, or move customers from one class of service to another while maintaining regional data sovereignty.
PL/SQL BINARY Vector Support
PL/SQL supports BINARY as a new dimension format for the vector type, in line with SQL.
BINARY vectors are frequently used to represent whether some entity, such as a textual document, does or does not include certain features, list words, or terms. The advantages of the BINARY format are two-fold. The storage footprint of vectors can be reduced 32X compared to the default FLOAT32 vectors and distance computations on BINARY vectors are up to 40X faster.
Support for BINARY vectors means that PL/SQL is able to handle a binary vector in the same way that it supports a vector of any other dimension format.
PL/SQL JACCARD Distance Support
The JACCARD distance metric is now supported in PL/SQL, which provides a similarity measure as a value from 0 to 1 between two BINARY vectors.
Jaccard distance is a common similarity measure for binary vectors. Support for JACCARD as a new metric in the PL/SQL VECTOR_DISTANCE operator means that PL/SQL code can make use of the JACCARD distance metric from within PL/SQL instead of calling out into SQL.
SQL Time Bucketing
Time bucketing is a common operation when processing time series or event streaming data where a series of data points within an arbitrarily defined time window need to be mapped to a specific fixed time interval (bucket) for aggregated analysis.
With the new SQL operator TIME_BUCKET, Oracle provides native and performant support for time bucketing of time-based data for DATETIMES.
Providing a native SQL operator for common fixed-time interval bucketing for time series data significantly simplifies application development and data analysis of such information. In addition to simpler and less error prone code, the native operator increases the performance of time series analysis.
Sharding Support for AI Vector Search
With Globally Distributed Database support for Vector Search, tables containing vectors are automatically distributed and replicated across a pool of Oracle databases that share no hardware. Similarity searches are automatically parallelized across shards or directed to a specific shard if the sharding key is provided.
Globally Distributed Database AI Vector Search offers several benefits, including greater scalability by allowing vectors to be distributed across multiple machines, improved performance by parallelizing vector searches across shards, and improved data resilience because if one shard goes down, the other shards can continue to operate. It also allows vector search to be deployed as part of a distributed database, where a single logical database is distributed over multiple geographies.
Smallfile Tablespace Shrink
This feature supplies the capability to reliably shrink a smallfile tablespace.
In earlier releases, organizations may find the datafiles of a smallfile tablespace grow larger despite the actual used space being much smaller. This can happen after a user drops segments or objects in the tablespace, but depending on where data was located in the datafiles, users were not always able to use datafile resize to recover the freed space.
By using Smallfile Tablespace Shrink, you can now expect your smallfile tablespace size to be close to the sum of the size of all segments,and objects in that tablespace. You can now reliably shrink a smallfile tablespace, which means improved storage optimization and reduced storage costs.
Support for Image Transformer Models with AI Vector Search Using the In-Database ONNX Runtime
This feature enables the import and utilization of image transformer models with the in-database ONNX Runtime engine available in 23ai. Image transformer models must be in ONNX format and include the required image decoding and preprocessing as part of the ONNX pipeline.
Oracle Database 23ai has expanded its capabilities beyond text-based vector generation. Initially supporting sentence transformers for use with the in-database ONNX Runtime and AI Vector Search, the system now incorporates image transformer models. This enhancement enables users to leverage AI Vector Search for semantic similarity searches involving images, significantly broadening the range of supported use cases.
Vector Data Type Support in External Tables
Vector data stored outside the database (on a file system or cloud object store) can be easily accessed to perform similarity searches using external tables.
Vector embeddings created outside the database can be loaded quickly and easily using an external table and standard SQL. It is also possible to run similarity searches on vector embeddings stored outside the database and seamlessly combine those searches with data that is stored inside the database.