24.8 Adding and Dropping Virtual Columns For JSON Fields Based on Data-Guide Information
Based on data-guide information for a JSON column, you can project scalar fields from that JSON data as virtual columns in the same table. The scalar fields projected are those that are not under an array.
You can do all of the following with a virtual column, with the aim of improving performance:
-
Build an index on it.
-
Gather statistics on it for the optimizer.
-
Load it into the In-Memory Column Store (IM column store).
Note:
The number of virtual columns per table is limited by the value of
initialization parameter MAX_COLUMNS
. By default that value is
STANDARD
, which means 1000 columns maximum. See MAX_COLUMNS in Oracle Database
Reference.
You use PL/SQL procedure DBMS_JSON.add_virtual_columns
to add virtual columns based on data-guide information for a JSON column. Before it adds virtual columns, procedure add_virtual_columns
first drops any existing virtual columns that were projected from fields in the same JSON column by a previous invocation of add_virtual_columns
or by data-guide change-trigger procedure add_vc
(in effect, it does what procedure DBMS_JSON.drop_virtual_columns
does).
There are two alternative sources of the data-guide information that you provide to procedure add_virtual_columns
:
-
It can come from a hierarchical or schema data guide that you pass as an argument. All scalar fields in the data guide that are not under an array are projected as virtual columns. All other fields in the data guide are ignored (not projected).
In this case, you can edit the data guide before passing it, so that it specifies the scalar fields (not under an array) that you want projected. You do not need a data guide-enabled search index in this case.
-
It can come from a data guide-enabled JSON search index.
In this case, you can specify, as the value of argument
FREQUENCY
to procedureadd_virtual_columns
, a minimum frequency of occurrence for the scalar fields to be projected. You need a data guide-enabled search index in this case, but you do not need a data guide.
You can also specify that added virtual columns be hidden. The SQL describe
command does not list hidden columns.
-
If you pass a hierarchical or schema data guide to
add_virtual_columns
then you can specify projection of particular scalar fields (not under an array) as hidden virtual columns by adding"o:hidden": true
to their descriptions in the data guide. -
If you use a data guide-enabled JSON search index with
add_virtual_columns
then you can specify a PL/SQLTRUE
value for argumentHIDDEN
, to make all of the added virtual columns be hidden. (The default value ofHIDDEN
isFALSE
, meaning that the added virtual columns are not hidden.)
_________________________________________________________
See Also:
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.add_virtual_columns
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.create_view_on_path
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.drop_virtual_columns
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.rename_column
- Adding Virtual Columns For JSON Fields Based on a Hierarchical or Schema Data Guide
You can use a hierarchical or schema data guide to project scalar fields from JSON data as virtual columns in the same table. - Adding Virtual Columns For JSON Fields Based on a Data Guide-Enabled Search Index
You can use a data guide-enabled search index for a JSON column to project scalar fields from that JSON data as virtual columns in the same table. Only scalar fields not under an array are projected. You can specify a minimum frequency of occurrence for the fields to be projected. - Dropping Virtual Columns for JSON Fields Based on Data-Guide Information
You can use procedureDBMS_JSON.drop_virtual_columns
to drop all virtual columns that were added for JSON fields in a column of JSON data.
Related Topics
Parent topic: JSON Data Guide
24.8.1 Adding Virtual Columns For JSON Fields Based on a Hierarchical or Schema Data Guide
You can use a hierarchical or schema data guide to project scalar fields from JSON data as virtual columns in the same table.
All scalar fields in the data guide that are not under an array are projected as virtual columns. All other fields in the data guide are ignored (not projected).
You can obtain a hierarchical or schema data guide using Oracle SQL function
json_dataguide
with argument
DBMS_JSON.FORMAT_HIERARCHICAL
or
DBMS_JSON.FORMAT_SCHEMA
, respectively.
You can edit the data guide obtained, to include only specific scalar fields (that are not under an array), rename those fields, or change the lengths of their types. The resulting data guide specifies which such fields to project as new virtual columns. Any fields in the data guide that are not scalar fields not under an array are ignored (not projected).
Note:
When you use a schema data guide to add a virtual column for a given field, if that field
has values of different scalar types in the document set then a column is added for each
of those scalar types. The names of such multiple columns other than the first have
_N
appended to the field name (N
= 1,
2,…).
For example, if field a
has a number value in one document and a string
value in another document, then two virtual columns are created, one of type
NUMBER
and the other of type VARCHAR2
. One column is
named A
; the other is named A_1
.
You use PL/SQL procedure
DBMS_JSON.add_virtual_columns
to add the virtual columns to the table
that contains the JSON column containing the projected fields. That procedure first drops
any existing virtual columns that were projected from fields in the same JSON column by a
previous invocation of add_virtual_columns
or by data-guide change-trigger
procedure add_vc
(in effect, it does what procedure
DBMS_JSON.drop_virtual_columns
does).
Example 24-9 illustrates this. It projects scalar fields that are not
under an array, from the data in JSON column data
of table
j_purchaseorder
. The fields projected are those that are indicated in the
data guide.
Example 24-10 illustrates passing a data-guide argument that specifies
the projection of two fields as virtual columns. Data-guide field o:hidden
is used to hide one of these columns.
See Also:
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.add_virtual_columns
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.drop_virtual_columns
-
Oracle Database SQL Language Reference for information about SQL function
json_dataguide
-
Oracle Database SQL Language Reference for information about PL/SQL constants
DBMS_JSON.FORMAT_HIERARCHICAL
andDBMS_JSON.FORMAT_SCHEMA
Example 24-9 Adding Virtual Columns That Project JSON Fields Using a Data Guide Obtained With JSON_DATAGUIDE
This example uses a hierarchical data guide obtained using function
json_dataguide
with JSON column data
.
The added virtual columns are all of the columns in table
j_purchaseorder
except for ID
,
DATE_LOADED
, and PODOCUMENT
.
-
Parameter
resolveNameConflicts
isTRUE
, to ensure that any name conflicts get resolved. (Optional, for clarity; this is anyway the default value.) -
Parameter
colNamePrefix
is'DATA$'
, to use that as the default prefix for column names. -
Parameter
mixedCaseColumns
isTRUE
, to make column names be case-sensitive, that is, to distinguish uppercase and lowercase letters.
DECLARE
dg CLOB;
BEGIN
SELECT json_dataguide(data, DBMS_JSON.FORMAT_HIERARCHICAL) INTO dg
FROM j_purchaseorder;
DBMS_JSON.add_virtual_columns('J_PURCHASEORDER',
'DATA',
dg,
resolveNameConflicts=>TRUE,
colNamePrefix=>'DATA$',
mixedCaseColumns=>TRUE);
END;
/
DESCRIBE j_purchaseorder;
Name Null? Type
-------------------------------- -------- ---------------------------
ID NOT NULL RAW(16)
DATE_LOADED TIMESTAMP(6) WITH TIME ZONE
DATA CLOB
DATA$User VARCHAR2(8)
DATA$PONumber NUMBER
DATA$Reference VARCHAR2(16)
DATA$Requestor VARCHAR2(16)
DATA$CostCenter VARCHAR2(4)
DATA$AllowPartialShipment VARCHAR2(4)
DATA$name VARCHAR2(16)
DATA$Phone VARCHAR2(16)
DATA$city VARCHAR2(32)
DATA$state VARCHAR2(2)
DATA$street VARCHAR2(32)
DATA$country VARCHAR2(32)
DATA$zipCode NUMBER
DATA$SpecialInstructions VARCHAR2(8)
Example 24-10 Adding Virtual Columns, Hidden and Visible
In this example only two fields are projected as
virtual columns: PO_Number
and PO_Reference
. The data
guide is defined locally as a literal string. Data-guide field o:hidden
is
used here to hide the virtual column for PO_Reference
. (For
PO_Number
the o:hidden: false
entry is not needed, as
false
is the default
value.)
DECLARE
dg CLOB;
BEGIN
dg := '{"type" : "object",
"properties" :
{"PO_Number" : {"type" : "number",
"o:length" : 4,
"o:preferred_column_name" : "PO_Number",
"o:hidden" : false},
"PO_Reference" : {"type" : "string",
"o:length" : 16,
"o:preferred_column_name" : "PO_Reference",
"o:hidden" : true}}}';
DBMS_JSON.add_virtual_columns('J_PURCHASEORDER', 'DATA', dg);
END;
/
DESCRIBE j_purchaseorder;
Name Null? Type
----------- -------- ---------------------------
ID NOT NULL RAW(16)
DATE_LOADED TIMESTAMP(6) WITH TIME ZONE
DATA CLOB
PO_Number NUMBER
SELECT column_name FROM user_tab_columns
WHERE table_name = 'J_PURCHASEORDER' ORDER BY 1;
COLUMN_NAME
-----------
DATE_LOADED
ID
DATA
PO_Number
PO_Reference
5 rows selected.
Related Topics
24.8.2 Adding Virtual Columns For JSON Fields Based on a Data Guide-Enabled Search Index
You can use a data guide-enabled search index for a JSON column to project scalar fields from that JSON data as virtual columns in the same table. Only scalar fields not under an array are projected. You can specify a minimum frequency of occurrence for the fields to be projected.
You use procedure
DBMS_JSON.add_virtual_columns
to add the virtual columns.
Example 24-11 illustrates this. It projects all scalar fields that are
not under an array to table j_purchaseorder
as virtual columns.
If you gather statistics on the documents in the JSON column where you want to project fields then the data-guide information in the data guide-enabled JSON search index records the frequency of occurrence, across that document set, of each field in a document.
When you add virtual columns you can specify that only those fields with a given minimum frequency of occurrence are to be projected.
You do this by specifying a non-zero value for parameter FREQUENCY
of procedure add_virtual_columns
. Zero is the default value, so if you
do not include argument FREQUENCY
then all scalar fields (not under an array) are projected. The frequency of a given
field is the number of documents containing that field divided by the total number of
documents in the JSON column, expressed as a percentage.
Example 24-12 projects all scalars (not under an array) that occur in all (100%) of the documents as virtual columns.
If you want to hide all
of the added virtual columns then specify a TRUE
value for argument
HIDDEN
. (The default value of parameter
HIDDEN
is FALSE
, meaning that the added virtual
columns are not hidden.)
Example 24-13 projects, as hidden virtual columns, the scalar fields (not under an array) that occur in all (100%) of the documents.
See Also:
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.add_virtual_columns
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.rename_column
Example 24-11 Projecting All Scalar Fields Not Under an Array as Virtual Columns
The added virtual columns are all of the columns in table j_purchaseorder
except for ID
, DATE_LOADED
, and
PODOCUMENT
. This is because no FREQUENCY
argument
is passed to add_virtual_columns
, so all scalar fields (that are not under
an array) are projected.
(Columns whose names are
italic
in the describe
command output are those
that have been renamed using PL/SQL procedure
DBMS_JSON.rename_column
.)
EXEC DBMS_JSON.add_virtual_columns('J_PURCHASEORDER', 'DATA');
DESCRIBE j_purchaseorder;
Name Null? Type
----------------------------------------- -------- ----------------------------
ID NOT NULL RAW(16)
DATE_LOADED TIMESTAMP(6) WITH TIME ZONE
DATA CLOB
DATA$User VARCHAR2(8)
PONumber NUMBER
DATA$Reference VARCHAR2(16)
DATA$Requestor VARCHAR2(16)
DATA$CostCenter VARCHAR2(4)
DATA$AllowPartialShipment VARCHAR2(4)
DATA$name VARCHAR2(16)
Phone VARCHAR2(16)
DATA$city VARCHAR2(32)
DATA$state VARCHAR2(2)
DATA$street VARCHAR2(32)
DATA$country VARCHAR2(32)
DATA$zipCode NUMBER
DATA$SpecialInstructions VARCHAR2(8)
Example 24-12 Projecting Scalar Fields With a Minimum Frequency as Virtual Columns
All scalar fields that occur in all (100%) of the documents are projected as virtual
columns. The result is the same as that for Example 24-11, except that fields AllowPartialShipment
and Phone
are not projected, because they do not occur in 100% of the
documents.
(Columns whose names are italic
in the
describe
command output are those that have been renamed using PL/SQL
procedure
DBMS_JSON.rename_column
.)
EXEC DBMS_JSON.add_virtual_columns('J_PURCHASEORDER', 'DATA', 100);
DESCRIBE j_purchaseorder;
Name Null? Type
----------------------------------------- -------- ----------------------------
ID NOT NULL RAW(16)
DATE_LOADED TIMESTAMP(6) WITH TIME ZONE
DATA CLOB
DATA$User VARCHAR2(8)
PONumber NUMBER
DATA$Reference VARCHAR2(16)
DATA$Requestor VARCHAR2(16)
DATA$CostCenter VARCHAR2(4)
DATA$name VARCHAR2(16)
DATA$city VARCHAR2(32)
DATA$state VARCHAR2(2)
DATA$street VARCHAR2(32)
DATA$country VARCHAR2(32)
DATA$zipCode NUMBER
DATA$SpecialInstructions VARCHAR2(8)
Example 24-13 Projecting Scalar Fields With a Minimum Frequency as Hidden Virtual Columns
The result is the same as that for Example 24-12, except that all of the added virtual columns are
hidden. (The query of view USER_TAB_COLUMNS
shows that the virtual
columns were in fact
added.)
EXEC DBMS_JSON.add_virtual_columns('J_PURCHASEORDER', 'DATA', 100, TRUE);
DESCRIBE j_purchaseorder;
Name Null? Type
----------------------------------------- -------- ----------------------------
ID NOT NULL RAW(16)
DATE_LOADED TIMESTAMP(6) WITH TIME ZONE
DATA CLOB
SELECT column_name FROM user_tab_columns
WHERE table_name = 'J_PURCHASEORDER'
ORDER BY 1;
COLUMN_NAME
-----------
DATE_LOADED
ID
PONumber
DATA
DATA$CostCenter
DATA$Reference
DATA$Requestor
DATA$SpecialInstructions
DATA$User
DATA$city
DATA$country
DATA$name
DATA$state
DATA$street
DATA$zipCode
24.8.3 Dropping Virtual Columns for JSON Fields Based on Data-Guide Information
You can use procedure DBMS_JSON.drop_virtual_columns
to drop all virtual columns that were added for JSON fields in a column of JSON data.
DBMS_JSON.drop_virtual_columns
drops all virtual columns that were
projected from fields in a given JSON column by an invocation of
add_virtual_columns
or by data-guide change-trigger procedure
add_vc
. Example 24-14 illustrates this for fields projected from column
data
of table j_purchaseorder
.
See Also:
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.add_virtual_columns
-
Oracle Database PL/SQL Packages and Types Reference for information about
DBMS_JSON.drop_virtual_columns
Example 24-14 Dropping Virtual Columns Projected From JSON Fields
EXEC DBMS_JSON.drop_virtual_columns('J_PURCHASEORDER', 'DATA');