10.2 Optimizing Queries for Response Time

By default, Oracle Text optimizes queries for throughput so that queries return all rows in the shortest time possible.

However, in many cases, especially in a web application, you must optimize queries for response time, because you are only interested in obtaining the first few hits of a potentially large hitlist in the shortest time possible.

The following sections describe some ways to optimize CONTAINS queries for response time:

10.2.1 Other Factors That Influence Query Response Time

The following factors can influence query response time:

10.2.2 Improved Response Time with the FIRST_ROWS(n) Hint for ORDER BY Queries

When you need the first rows of an ORDER BY query, Oracle recommends that you use the cost-based FIRST_ROWS(n) hint.

Note:

As the FIRST_ROWS(n) hint is cost-based, Oracle recommends that you collect statistics on your tables before you use this hint.

You use the FIRST_ROWS(n) hint in cases where you want the first n number of rows in the shortest possible time. For example, consider the following PL/SQL block that uses a cursor to retrieve the first 10 hits of a query and the FIRST_ROWS(n) hint to optimize the response time:

declare
cursor c is 

select /*+ FIRST_ROWS(10) */ article_id from articles_tab
   where contains(article, 'Omophagia')>0 order by pub_date desc;

begin
for i in c
loop
insert into t_s values(i.pk, i.col);
exit when c%rowcount > 11;
end loop;
end;
/

The c cursor is a SELECT statement that returns the rowids that contain the word omophagia in sorted order. The code loops through the cursor to extract the first 10 rows. These rows are stored in the temporary t_s table.

With the FIRST_ROWS(n) hint, the optimizer instructs the Oracle Text index to return rowids in score-sorted order when the cost of returning the top-N hits is lower.

Without the hint, Oracle Database sorts the rowids after the Oracle Text index returns all rows in unsorted order that satisfy the CONTAINS predicate. Retrieving the entire result set takes time.

Because only the first 10 hits are needed in this query, using the hint results in better performance.

Note:

Use the FIRST_ROWS(n) hint when you need only the first few hits of a query. When you need the entire result set, do not use this hint as it might result in poor performance.

10.2.3 Improved Response Time Using the DOMAIN_INDEX_SORT Hint

You can also optimize for response time by using the related DOMAIN_INDEX_SORT hint. Like FIRST_ROWS(n), when queries are optimized for response time, Oracle Text returns the first rows in the shortest time possible.

For example, you can use this hint:

select /*+ DOMAIN_INDEX_SORT */ pk, score(1), col from ctx_tab 
            where contains(txt_col, 'test', 1) > 0 order by score(1) desc;

However, this hint is only rule-based. This means that Oracle Text always chooses the index which satisfies the ORDER BY clause. This hint might result in suboptimal performance for queries where the CONTAINS clause is very selective. In these cases, Oracle recommends that you use the FIRST_ROWS(n) hint, which is fully cost-based.

10.2.4 Improved Response Time Using the Local Partitioned CONTEXT Index

Partitioning your data and creating local partitioned indexes can improve your query performance. On a partitioned table, each partition has its own set of index tables. Effectively, there are multiple indexes, but the results are combined as necessary to produce the final result set.

Create the CONTEXT index with the LOCAL keyword:

CREATE INDEX index_name ON table_name (column_name) 
INDEXTYPE IS ctxsys.context
PARAMETERS ('...')
LOCAL

With partitioned tables and indexes, you can improve performance of the following types of queries:

  • Range Search on Partition Key Column: This query restricts the search to a particular range of values on a column that is also the partition key. For example, consider a query on a date range:

    SELECT storyid FROM storytab WHERE CONTAINS(story, 'oliver')>0 and pub_date BETWEEN '1-OCT-93' AND '1-NOV-93';

    If the date range is quite restrictive, it is very likely that the query can be satisfied by only looking in a single partition.

  • ORDER BY Partition Key Column: This query requires only the first n hits, and the ORDER BY clause names the partition key. Consider an ORDER BY query on a price column to fetch the first 20 hits:

    SELECT * FROM (
    SELECT itemid FROM item_tab WHERE CONTAINS(item_desc, 'cd player')
      >0 ORDER BY price)
      WHERE ROWNUM < 20;

    In this example, with the table partitioned by price, the query might only need to get hits from the first partition to satisfy the query.

10.2.5 Improved Response Time with the Local Partitioned Index for Order by Score

The DOMAIN_INDEX_SORT hint on a local partitioned index might result in poor performance, especially when you order by score. All hits to the query across all partitions must be obtained before the results can be sorted.

Instead, use an inline view when you use the DOMAIN_INDEX_SORT hint. Specifically, use the DOMAIN_INDEX_SORT hint to improve query performance on a local partitioned index under the following conditions:

  • The Oracle Text query itself, including the order by SCORE() clause, is expressed as an in-line view.

  • The Oracle Text query inside the in-line view contains the DOMAIN_INDEX_SORT hint.

  • The query on the in-line view has a ROWNUM predicate that limits the number of rows to fetch from the view.

For example, the following Oracle Text query and local Oracle Text index are created on a partitioned doc_tab table:

     select doc_id, score(1) from doc_tab 
        where contains(doc, 'oracle', 1)>0 
        order by score(1) desc;

If you are interested in fetching only the top 20 rows, you can rewrite the query as follows:

     select * from 
          (select /*+ DOMAIN_INDEX_SORT */ doc_id, score(1) from doc_tab 
              where contains(doc, 'oracle', 1)>0 order by score(1) desc) 
      where rownum < 21;

See Also:

Oracle Database SQL Language Reference for more information about the EXPLAIN PLAN statement

10.2.6 Improved Response Time with the Query Filter Cache

Oracle Text provides a cache layer called the query filter cache that you can use to cache the query results. The query filter cache is sharable across queries. Multiple queries can reuse cached query results to improve the query response time.

Use the ctxfiltercache operator to specify which query results to cache. The following example uses the operator to store the results of the common_predicate query in the cache:

select * from docs where contains(txt, 'ctxfiltercache((common_predicate), FALSE)')>0;

In this example, the cached results of the common_predicate query are reused by the new_query query, to improve the query response time.

select * from docs where contains(txt, 'new_query & ctxfiltercache((common_predicate), FALSE)')>0; 

Note:

  • You can specify the size of the query filter cache by using the basic query_filter_cache_size storage attribute.

  • The ctx_filter_cache_statistics view provides various statistics about the query filter cache.

Note:

The CTXFILTERCACHE query operator was designed to speed up commonly-used expressions in queries. In Oracle Database Release 21c, this function is replaced by other internal improvements. The CTXFILTERCACHE operator is deprecated (and will pass through its operands to be run as a normal query). Because they no longer have a function, the view CTX_FILTER_CACHE_STATISTICS is also deprecated, and also the storage attribute QUERY_FILTER_CACHE_SIZE.

See Also:

Oracle Text Reference for more information about:

  • ctxfiltercache operator

  • query_filter_cache_size basic storage attribute

  • ctx_filter_cache_statistics view

10.2.7 Improved Response Time Using the BIG_IO Option of CONTEXT Index

Oracle Text provides the BIG_IO option for improving the query performance for the CONTEXT indexes that extensively use IO operations.

The query performance improvement is mainly for data stored on rotating disks, not for data stored on solid state disks.

When you enable the BIG_IO option, a CONTEXT index creates token type pairs with one large object (LOB) data type for each unique token text. Tokens with the same text but different token types correspond to different rows in the $I table.

Note:

The BIG_IO attribute of the CONTEXT indextype is deprecated with Oracle Database 23ai, and can be disabled or removed in a future release.

Oracle recommends that you allow this value to be set to its default value of N. BIG_IO was introduced to reduce the cost of seeks when index postings exceeded 4KB in length. However, the internal code is relatively inefficient, and the attribute cannot be combined with newer index options. Seek cost is much less relevant for solid state disks or non-volatile memory devices (NVMe), and seek cost is irrelevant when postings are cached. This setting is therefore of little benefit for most indexes.

The indexes with the BIG_IO option enabled should have the token LOBs created as SecureFile LOBs, so that the data is stored sequentially in multiple blocks. This method improves the response time of the queries, because the queries can now perform longer sequential reads instead of many short reads.

Note:

If you use SecureFiles, you must set the COMPATIBLE setting to 11.0 or higher. In addition, you must create the LOB on an automatic segment space management (ASSM) tablespace. When you migrate the existing Oracle Text indexes to SecureFiles, use an ASSM tablespace. To help migrate the existing indexes to SecureFiles, you can extend ALTER INDEX REBUILD to provide storage preferences that only affect the $I table.

To create a CONTEXT index with the BIG_IO index option, first create a basic storage preference by setting the value of its BIG_IO storage attribute to YES, and then specify this storage preference while creating the CONTEXT index.

The following example creates a basic mystore storage preference and sets the value of its BIG_IO storage attribute to YES:

exec ctx_ddl.create_preference('mystore', 'BASIC_STORAGE');
exec ctx_ddl.set_attribute('mystore', 'BIG_IO', 'YES');

To disable the BIG_IO option, update the existing storage preference (mystore) by setting the value of its BIG_IO storage attribute to NO, and then rebuild the index.

exec ctx_ddl.set_attribute('mystore', 'BIG_IO', 'NO');
alter index idx rebuild('replace storage mystore');

WARNING:

Do not use the replace metadata operation to disable the BIG_IO index option. It can leave the index in an inconsistent state.

To enable the BIG_IO option for a partitioned index without rebuilding the index, modify the basic storage preference by setting the value of its BIG_IO storage attribute to YES, replace the global index metadata using ctx_ddl.replace_index_metadata, and then call optimize_index in REBUILD mode for each partition of the partitioned index table.

The following example enables the BIG_IO option for the idx partitioned index:

exec ctx_ddl.set_attribute('mystore', 'BIG_IO', 'YES');
exec ctx_ddl.replace_index_metadata('idx', 'replace metadata storage mystore');
exec ctx_ddl.optimize_index('idx', 'rebuild', part_name=>'part1');

Note:

If a procedure modifies the existing index tables with only the BIG_IO option enabled, then it will not result in reindexing of the data.

Note:

Because the BIG_IO index option performs longer sequential reads, the queries that use the BIG_IO index option require a large program global area (PGA) memory.

10.2.8 Improved Response Time Using the SEPARATE_OFFSETS Option of the CONTEXT Index

Oracle Text provides the SEPARATE_OFFSETS option to improve the query performance for the CONTEXT indexes that use IO operations, and whose queries are mainly single-word or Boolean queries.

The SEPARATE_OFFSETS option creates a different postings list structure for the tokens of type TEXT. Instead of interspersing docids, frequencies, info-length (length of the offsets information), and the offsets in the postings list, the SEPARATE_OFFSETS option stores all docids and frequencies at the beginning of the postings list, and all info-lengths and offsets at the end of the postings list. The header at the beginning of the posting contains the information about the boundary points between docids and offsets. Because separation of docids and offsets reduces the time for the queries to read the data, it improves the query response time.

To create a CONTEXT index with the SEPARATE_OFFSETS option, first create a basic storage preference by setting the value of its SEPARATE_OFFSETS storage attribute to T. Next, specify this storage preference when you create the CONTEXT index.

The following example creates a basic mystore storage preference and sets the value of its SEPARATE_OFFSETS storage attribute to T:

exec ctx_ddl.create_preference('mystore', 'BASIC_STORAGE');
exec ctx_ddl.set_attribute('mystore', 'SEPARATE_OFFSETS', 'T');

To disable the SEPARATE_OFFSETS option, update the existing storage preference (mystore) by setting the value of its SEPARATE_OFFSETS storage attribute to F, and then rebuild the index.

exec ctx_ddl.set_attribute('mystore', 'SEPARATE_OFFSETS', 'F');
alter index idx rebuild('replace storage mystore');

WARNING:

Do not use replace metadata operation to disable the SEPARATE_OFFSETS index option, as it can leave the index in an inconsistent state.

To enable the SEPARATE_OFFSETS option for a partitioned index without rebuilding the index, modify the basic storage preference by setting the value of its SEPARATE_OFFSETS storage attribute to T, replace the global index metadata by using ctx_ddl.replace_index_metadata, and then call optimize_index in REBUILD mode for each partition in the partitioned index table.

The following example enables the SEPARATE_OFFSETS option for the partitioned idx index:

exec ctx_ddl.set_attribute('mystore', 'SEPARATE_OFFSETS', 'T');
exec ctx_ddl.replace_index_metadata('idx', 'replace storage mystore');
exec ctx_ddl.optimize_index('idx', 'rebuild', part_name=>'part1');

Note:

If a procedure modifies the existing index tables with only the SEPARATE_OFFSETS option enabled, then the data is not reindexed.

10.2.9 Improved Response Time Using the STAGE_ITAB, STAGE_ITAB_MAX_ROWS, and STAGE_ITAB_PARALLEL Options of CONTEXT Index

Oracle Text provides the STAGE_ITAB option for improving the query performance for CONTEXT and search indexes that extensively use insert, update, and delete operations for near real-time indexing.

The STAGE_ITAB option is the default index option only for search indexes.

If you do not use the STAGE_ITAB index option, then when you add a new document to the CONTEXT index, SYNC_INDEX is called to make the documents searchable. This call creates new rows in the $I table, and increases the fragmentation in the $I table. The result is deterioration of the query performance.

When you enable the STAGE_ITAB index option, the following happens:

  • Information about the new documents is stored in the $G staging table, not in the $I table. This storage ensures that the $I table is not fragmented and does not deteriorate the query performance.

  • The $H b-tree index is created on the $G table. The $G table and $H b-tree index are equivalent to the $I table and $X b-tree index.

Rows are merged automatically from the $G table to the $I table when the number of rows in $G exceeds the storage setting, STAGE_ITAB_MAX_ROWS (10K by default). You can also force an immediate merge of the rows from $G to $I by running index optimization in MERGE optimization mode.

Note:

The $G table is stored in the KEEP pool. To improve query performance, you should allocate sufficient KEEP pool memory and maintain a large enough $G table size by using the new stage_itab_max_rows option.

To create a CONTEXT index with the STAGE_ITAB index option, first create a basic storage preference by setting the value of its STAGE_ITAB storage attribute to YES. Next, specify this storage preference when you create the CONTEXT index.

The following example creates a basic mystore storage preference and sets the value of its STAGE_ITAB storage attribute to YES:

exec ctx_ddl.create_preference('mystore', 'BASIC_STORAGE');
exec ctx_ddl.set_attribute('mystore', 'STAGE_ITAB', 'YES');

You can also enable the STAGE_ITAB index option for an existing nonpartitioned CONTEXT index by using the rebuild option of the ALTER INDEX statement.

alter index IDX rebuild parameters('replace storage mystore');

To disable the STAGE_ITAB option for a nonpartitioned CONTEXT index, update the existing storage preference (mystore) by setting the value of its STAGE_ITAB storage attribute to NO, and then rebuild the index.

exec ctx_ddl.set_attribute('mystore', 'STAGE_ITAB', 'NO');
alter index idx rebuild parameters('replace storage mystore');

This operation runs the optimization process by using the MERGE optimization mode and then drops the $G table.

The rebuild option of the ALTER INDEX statement does not work with the partitioned CONTEXT index for enabling and disabling the STAGE_ITAB option.

The following example enables the STAGE_ITAB option for the partitioned CONTEXT idx index:

alter index idx parameters('add stage_itab');

The following example disables the STAGE_ITAB option for the partitioned CONTEXT idx index:

alter index idx parameters('remove stage_itab');

The contents of $G were automatically moved to $I during index synchronization when $G had more than 1 million rows in Oracle Database 12c Release 2 (12.2) or 100K rows in Oracle Database Release 18c. Starting with Oracle Database Release 21c, the contents of $G are automatically moved to $I during index synchronization when $G has more than 10K rows by default. This value is controlled by the STAGE_ITAB_MAX_ROWS attribute of the STORAGE preference.

Note:

To use the STAGE_ITAB index option for a CONTEXT index, you must specify the g_index_clause and g_table_clause BASIC_STORAGE preferences.

The query performance is deteriorated when $G table is too fragmented. To avoid deterioration, starting with Oracle Database Release 18c, Oracle Text provides automatic background optimize merge for every index or partition. To enable automatic background optimize merge, you must set the STAGE_ITAB storage preference attribute to TRUE, and you must create the index with a storage preference which uses the STAGE_ITAB attribute.

By default, if you had enabled STAGE_ITAB in indexes before you upgraded to Oracle Database Release 18c, then STAGE_ITAB_AUTO_OPT is not enabled. If STAGE_ITAB and AUTO_OPTIMIZE are enabled in existing indexes, then you must disable AUTO_OPTIMIZE before you enable STAGE_ITAB_AUTO_OPT. Starting with Oracle Database Release 19c, STAGE_ITAB_AUTO_OPT is set to TRUE by default for automatic background optimize merge. If you set STAGE_ITAB_AUTO_OPT to FALSE, the merge is run as part of SYNC INDEX. It is recommended to set STAGE_ITAB and STAGE_ITAB_AUTO_OPT to TRUE instead of using AUTO_OPTIMIZE.

Note:

In Oracle Database Release 21c, the procedures ADD_AUTO_OPTIMIZE and REMOVE_AUTO_OPTIMIZE, and the views CTX_AUTO_OPTIMIZE_INDEXES, CTX_USER_AUTO_OPTIMIZE_INDEXES and CTX_AUTO_OPTIMIZE_STATUS are deprecated.

The following example creates a basic mystore storage preference and sets the value of its STAGE_ITAB_AUTO_OPT storage attribute to TRUE:

exec ctx_ddl.create_preference('mystore', 'basic_storage');
exec ctx_ddl.set_attribute('mystore', 'stage_itab', 'TRUE');
exec ctx_ddl.set_attribute('mystore', 'stage_itab_auto_opt', 'TRUE');
exec ctx_ddl.set_attribute('mystore', 'stage_itab_parallel', 16);

Related Topics