24.8 Adding and Dropping Virtual Columns For JSON Fields Based on Data-Guide Information

Based on data-guide information for a JSON column, you can project scalar fields from that JSON data as virtual columns in the same table. The scalar fields projected are those that are not under an array.

You can do all of the following with a virtual column, with the aim of improving performance:

  • Build an index on it.

  • Gather statistics on it for the optimizer.

  • Load it into the In-Memory Column Store (IM column store).

Note:

The number of virtual columns per table is limited by the value of initialization parameter MAX_COLUMNS. By default that value is STANDARD, which means 1000 columns maximum. See MAX_COLUMNS in Oracle Database Reference.

You use PL/SQL procedure DBMS_JSON.add_virtual_columns to add virtual columns based on data-guide information for a JSON column. Before it adds virtual columns, procedure add_virtual_columns first drops any existing virtual columns that were projected from fields in the same JSON column by a previous invocation of add_virtual_columns or by data-guide change-trigger procedure add_vc (in effect, it does what procedure DBMS_JSON.drop_virtual_columns does).

There are two alternative sources of the data-guide information that you provide to procedure add_virtual_columns:

  • It can come from a hierarchical or schema data guide that you pass as an argument. All scalar fields in the data guide that are not under an array are projected as virtual columns. All other fields in the data guide are ignored (not projected).

    In this case, you can edit the data guide before passing it, so that it specifies the scalar fields (not under an array) that you want projected. You do not need a data guide-enabled search index in this case.

  • It can come from a data guide-enabled JSON search index.

    In this case, you can specify, as the value of argument FREQUENCY to procedure add_virtual_columns, a minimum frequency of occurrence for the scalar fields to be projected. You need a data guide-enabled search index in this case, but you do not need a data guide.

You can also specify that added virtual columns be hidden. The SQL describe command does not list hidden columns.

  • If you pass a hierarchical or schema data guide to add_virtual_columns then you can specify projection of particular scalar fields (not under an array) as hidden virtual columns by adding "o:hidden": true to their descriptions in the data guide.

  • If you use a data guide-enabled JSON search index with add_virtual_columns then you can specify a PL/SQL TRUE value for argument HIDDEN, to make all of the added virtual columns be hidden. (The default value of HIDDEN is FALSE, meaning that the added virtual columns are not hidden.)

_________________________________________________________

See Also:

Related Topics

24.8.1 Adding Virtual Columns For JSON Fields Based on a Hierarchical or Schema Data Guide

You can use a hierarchical or schema data guide to project scalar fields from JSON data as virtual columns in the same table.

All scalar fields in the data guide that are not under an array are projected as virtual columns. All other fields in the data guide are ignored (not projected).

You can obtain a hierarchical or schema data guide using Oracle SQL function json_dataguide with argument DBMS_JSON.FORMAT_HIERARCHICAL or DBMS_JSON.FORMAT_SCHEMA, respectively.

You can edit the data guide obtained, to include only specific scalar fields (that are not under an array), rename those fields, or change the lengths of their types. The resulting data guide specifies which such fields to project as new virtual columns. Any fields in the data guide that are not scalar fields not under an array are ignored (not projected).

Note:

When you use a schema data guide to add a virtual column for a given field, if that field has values of different scalar types in the document set then a column is added for each of those scalar types. The names of such multiple columns other than the first have _N appended to the field name (N = 1, 2,…).

For example, if field a has a number value in one document and a string value in another document, then two virtual columns are created, one of type NUMBER and the other of type VARCHAR2. One column is named A; the other is named A_1.

You use PL/SQL procedure DBMS_JSON.add_virtual_columns to add the virtual columns to the table that contains the JSON column containing the projected fields. That procedure first drops any existing virtual columns that were projected from fields in the same JSON column by a previous invocation of add_virtual_columns or by data-guide change-trigger procedure add_vc (in effect, it does what procedure DBMS_JSON.drop_virtual_columns does).

Example 24-9 illustrates this. It projects scalar fields that are not under an array, from the data in JSON column data of table j_purchaseorder. The fields projected are those that are indicated in the data guide.

Example 24-10 illustrates passing a data-guide argument that specifies the projection of two fields as virtual columns. Data-guide field o:hidden is used to hide one of these columns.

See Also:

Example 24-9 Adding Virtual Columns That Project JSON Fields Using a Data Guide Obtained With JSON_DATAGUIDE

This example uses a hierarchical data guide obtained using function json_dataguide with JSON column data.

The added virtual columns are all of the columns in table j_purchaseorder except for ID, DATE_LOADED, and PODOCUMENT.

  • Parameter resolveNameConflicts is TRUE, to ensure that any name conflicts get resolved. (Optional, for clarity; this is anyway the default value.)

  • Parameter colNamePrefix is 'DATA$', to use that as the default prefix for column names.

  • Parameter mixedCaseColumns is TRUE, to make column names be case-sensitive, that is, to distinguish uppercase and lowercase letters.

DECLARE
  dg CLOB;
BEGIN
  SELECT json_dataguide(data, DBMS_JSON.FORMAT_HIERARCHICAL) INTO dg
    FROM j_purchaseorder;
  DBMS_JSON.add_virtual_columns('J_PURCHASEORDER',
                                'DATA',
                                dg,
                                resolveNameConflicts=>TRUE,
                                colNamePrefix=>'DATA$',
                                mixedCaseColumns=>TRUE);
END;
/

DESCRIBE j_purchaseorder;
 Name                             Null?    Type
 -------------------------------- -------- ---------------------------
 ID                               NOT NULL RAW(16)
 DATE_LOADED                               TIMESTAMP(6) WITH TIME ZONE
 DATA                                      CLOB
 DATA$User                                 VARCHAR2(8)
 DATA$PONumber                             NUMBER
 DATA$Reference                            VARCHAR2(16)
 DATA$Requestor                            VARCHAR2(16)
 DATA$CostCenter                           VARCHAR2(4)
 DATA$AllowPartialShipment                 VARCHAR2(4)
 DATA$name                                 VARCHAR2(16)
 DATA$Phone                                VARCHAR2(16)
 DATA$city                                 VARCHAR2(32)
 DATA$state                                VARCHAR2(2)
 DATA$street                               VARCHAR2(32)
 DATA$country                              VARCHAR2(32)
 DATA$zipCode                              NUMBER
 DATA$SpecialInstructions                  VARCHAR2(8)

Example 24-10 Adding Virtual Columns, Hidden and Visible

In this example only two fields are projected as virtual columns: PO_Number and PO_Reference. The data guide is defined locally as a literal string. Data-guide field o:hidden is used here to hide the virtual column for PO_Reference. (For PO_Number the o:hidden: false entry is not needed, as false is the default value.)

DECLARE
  dg CLOB;
BEGIN
  dg := '{"type" : "object",
          "properties" :
            {"PO_Number"    : {"type" : "number",
                               "o:length" : 4,
                               "o:preferred_column_name" : "PO_Number",
                               "o:hidden" : false},
             "PO_Reference" : {"type" : "string",
                               "o:length" : 16,
                               "o:preferred_column_name" : "PO_Reference",
                               "o:hidden" : true}}}';
  DBMS_JSON.add_virtual_columns('J_PURCHASEORDER', 'DATA', dg);
END;
/

DESCRIBE j_purchaseorder;
 Name        Null?    Type
 ----------- -------- ---------------------------
 ID          NOT NULL RAW(16)
 DATE_LOADED          TIMESTAMP(6) WITH TIME ZONE
 DATA                 CLOB
 PO_Number            NUMBER

SELECT column_name FROM user_tab_columns
  WHERE table_name = 'J_PURCHASEORDER' ORDER BY 1;
COLUMN_NAME
-----------
DATE_LOADED
ID
DATA
PO_Number
PO_Reference

5 rows selected.

Related Topics

24.8.2 Adding Virtual Columns For JSON Fields Based on a Data Guide-Enabled Search Index

You can use a data guide-enabled search index for a JSON column to project scalar fields from that JSON data as virtual columns in the same table. Only scalar fields not under an array are projected. You can specify a minimum frequency of occurrence for the fields to be projected.

You use procedure DBMS_JSON.add_virtual_columns to add the virtual columns.

Example 24-11 illustrates this. It projects all scalar fields that are not under an array to table j_purchaseorder as virtual columns.

If you gather statistics on the documents in the JSON column where you want to project fields then the data-guide information in the data guide-enabled JSON search index records the frequency of occurrence, across that document set, of each field in a document.

When you add virtual columns you can specify that only those fields with a given minimum frequency of occurrence are to be projected.

You do this by specifying a non-zero value for parameter FREQUENCY of procedure add_virtual_columns. Zero is the default value, so if you do not include argument FREQUENCY then all scalar fields (not under an array) are projected. The frequency of a given field is the number of documents containing that field divided by the total number of documents in the JSON column, expressed as a percentage.

Example 24-12 projects all scalars (not under an array) that occur in all (100%) of the documents as virtual columns.

If you want to hide all of the added virtual columns then specify a TRUE value for argument HIDDEN. (The default value of parameter HIDDEN is FALSE, meaning that the added virtual columns are not hidden.)

Example 24-13 projects, as hidden virtual columns, the scalar fields (not under an array) that occur in all (100%) of the documents.

See Also:

Example 24-11 Projecting All Scalar Fields Not Under an Array as Virtual Columns

The added virtual columns are all of the columns in table j_purchaseorder except for ID, DATE_LOADED, and PODOCUMENT. This is because no FREQUENCY argument is passed to add_virtual_columns, so all scalar fields (that are not under an array) are projected.

(Columns whose names are italic in the describe command output are those that have been renamed using PL/SQL procedure DBMS_JSON.rename_column.)

EXEC DBMS_JSON.add_virtual_columns('J_PURCHASEORDER', 'DATA');

DESCRIBE j_purchaseorder;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 ID                                        NOT NULL RAW(16)
 DATE_LOADED                                        TIMESTAMP(6) WITH TIME ZONE
 DATA                                               CLOB
 DATA$User                                          VARCHAR2(8)
 PONumber                                           NUMBER
 DATA$Reference                                     VARCHAR2(16)
 DATA$Requestor                                     VARCHAR2(16)
 DATA$CostCenter                                    VARCHAR2(4)
 DATA$AllowPartialShipment                          VARCHAR2(4)
 DATA$name                                          VARCHAR2(16)
 Phone                                              VARCHAR2(16)
 DATA$city                                          VARCHAR2(32)
 DATA$state                                         VARCHAR2(2)
 DATA$street                                        VARCHAR2(32)
 DATA$country                                       VARCHAR2(32)
 DATA$zipCode                                       NUMBER
 DATA$SpecialInstructions                           VARCHAR2(8)

Example 24-12 Projecting Scalar Fields With a Minimum Frequency as Virtual Columns

All scalar fields that occur in all (100%) of the documents are projected as virtual columns. The result is the same as that for Example 24-11, except that fields AllowPartialShipment and Phone are not projected, because they do not occur in 100% of the documents.

(Columns whose names are italic in the describe command output are those that have been renamed using PL/SQL procedure DBMS_JSON.rename_column.)

EXEC DBMS_JSON.add_virtual_columns('J_PURCHASEORDER', 'DATA', 100);

DESCRIBE j_purchaseorder;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 ID                                        NOT NULL RAW(16)
 DATE_LOADED                                        TIMESTAMP(6) WITH TIME ZONE
 DATA                                               CLOB
 DATA$User                                          VARCHAR2(8)
 PONumber                                           NUMBER
 DATA$Reference                                     VARCHAR2(16)
 DATA$Requestor                                     VARCHAR2(16)
 DATA$CostCenter                                    VARCHAR2(4)
 DATA$name                                          VARCHAR2(16)
 DATA$city                                          VARCHAR2(32)
 DATA$state                                         VARCHAR2(2)
 DATA$street                                        VARCHAR2(32)
 DATA$country                                       VARCHAR2(32)
 DATA$zipCode                                       NUMBER
 DATA$SpecialInstructions                           VARCHAR2(8)

Example 24-13 Projecting Scalar Fields With a Minimum Frequency as Hidden Virtual Columns

The result is the same as that for Example 24-12, except that all of the added virtual columns are hidden. (The query of view USER_TAB_COLUMNS shows that the virtual columns were in fact added.)

EXEC DBMS_JSON.add_virtual_columns('J_PURCHASEORDER', 'DATA', 100, TRUE);

DESCRIBE j_purchaseorder;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 ID                                        NOT NULL RAW(16)
 DATE_LOADED                                        TIMESTAMP(6) WITH TIME ZONE
 DATA                                               CLOB

SELECT column_name FROM user_tab_columns
  WHERE table_name = 'J_PURCHASEORDER'
  ORDER BY 1;

COLUMN_NAME
-----------
DATE_LOADED
ID
PONumber
DATA
DATA$CostCenter
DATA$Reference
DATA$Requestor
DATA$SpecialInstructions
DATA$User
DATA$city
DATA$country
DATA$name
DATA$state
DATA$street
DATA$zipCode

24.8.3 Dropping Virtual Columns for JSON Fields Based on Data-Guide Information

You can use procedure DBMS_JSON.drop_virtual_columns to drop all virtual columns that were added for JSON fields in a column of JSON data.

Procedure DBMS_JSON.drop_virtual_columns drops all virtual columns that were projected from fields in a given JSON column by an invocation of add_virtual_columns or by data-guide change-trigger procedure add_vc. Example 24-14 illustrates this for fields projected from column data of table j_purchaseorder.

See Also:

Example 24-14 Dropping Virtual Columns Projected From JSON Fields

EXEC DBMS_JSON.drop_virtual_columns('J_PURCHASEORDER', 'DATA');