112 DBMS_JSON_DUALITY
The package DBMS_JSON_DUALITY
simplifies the creation and
access of JSON-relational duality views.
- Migrate or convert a set of JSON collections to JSON-relational duality views
- Perform multi-operation non-interactive transactions on duality views
This chapter provides an overview of the package, outlines any security restrictions associated with it, and details the subprograms available within the package.
DBMS_JSON_DUALITY Overview
The DBMS_JSON_DUALITY
package enables seamless integration
between JSON documents and relational databases. The package provides subprograms that
converts JSON collections into JSON relational duality views, imports data into duality
views, and supports multi-operational transactions on the duality views.
DBMS_JSON_DUALITY
package contains a set of
procedures and functions that allows you to find and create relational schema,
import JSON documents to duality views and perform non-interactive transactions on
the imported duality views.
- JSON-To-Duality Migrator : Execute one of the appropriate JSON-To-Duality
Migrator PL/SQL functions listed below to find and create the relational
schema for the input collections (JSON tables).
-
INFER_SCHEMA
: Infers the JSON schema that represents all of the input document sets. -
GENERATE_SCHEMA
: Produces the DDL code to create the required database objects for each duality view. -
INFER_AND_GENERATE_SCHEMA
: performs both operations ofINFER_SCHEMA
andGENERATE_SCHEMA
.
-
- JSON-To-Duality Importer :
This feature is a PL/SQL procedure
IMPORT
to import application data from sets of JSON documents into duality views defined using the JSON-To-Duality Converter feature. Based on the relational schema created by the JSON-To-Duality Converter, input data gets decomposed and normalized into relational tables automatically. Any data that cannot be successfully imported is logged in an error log with the reason for the error and suggestions to fix the issue, if possible.Note:
Use the importer feature only when you are satisfied with the relational schema recommended by the JSON Relational Duality Converter feature. - Non-Interactive TransactionsThis feature is a set of PL/SQL procedures that allows you to perform a multi-operational transaction on duality views. The following are the non-interactive procedures from package
DBMS_JSON_DUALITY
:-
BEGIN_TRANSACTION
: Begins a multi-operational transaction. -
REGISTER
: Check that the ETAG value of a document as last read matches that of the document in the database at the start of the transaction. -
COMMIT_TRANSACTION
: Commits a multi-operational transaction.
-
DBMS_JSON_DUALITY Security
All subprograms in the DBMS_JSON_DUALITY
package are
executed with invoker rights. You must have SELECT
privileges on tables and
views referenced in a subprogram to use it.
INFER_SCHEMA
,
GENERATE_SCHEMA
and INFER_AND_GENERATE_SCHEMA
subprograms, is subject to the following restrictions:
-
They cannot be run in
SYS
schema. -
They require "select any table" privilege if
targetSchema
is not current schema. Otherwise, duality view creation fails while attempting to access the tables intargetSchema
.Example:grant select any table to targetSchema;
Summary of DBMS_JSON_Duality Subprograms
This table lists the DBMS_JSON_DUALITY
subprograms and
briefly describes them.
DBMS_JSON_DUALITY Package Subprograms
Subprogram | Description |
---|---|
|
Takes as input a set of JSON collections (tables with a single JSON column) and configuration parameters formatted as a JSON document. Returns the validated and normalized relational schema representing the collections as a JSON document |
|
Takes as input a normalized relational schema represented as a JSON document and returns the DDL script to generate the schema. |
|
Takes as input a set of JSON collections (tables with a single JSON column) and configuration parameters formatted as a JSON document. Returns the DDL script to generate the validated and normalized relational schema representing the collections as a CLOB. |
|
A table function which returns a report for schema validation. It
validates the schema generated by
Note: To validate a schema, it is assumed that you have created the schema by running an appropriate DDL script. |
|
Imports data into the tables underlying the duality view(s). Logs any documents that fail import into an error log. |
|
A procedure to import data from multiple tables into corresponding duality views. |
|
A table function which return a report for import validation. It compares documents in the input JSON collection (tables with a single JSON column) with the documents of the corresponding duality view and generates a report for documents that fail validation. Note: To validate an import, it is assumed that you have imported the data using IMPORT Procedure or IMPORT_ALL procedure. |
|
Starts a special multi-op transaction. |
|
Commits the multi-op transaction. |
|
Registers each document to be modified in the transaction, ensuring that the document has not been modified since the last read. |
INFER_SCHEMA Function
This section describes the syntax, input and output formats of the
INFER_SCHEMA
function.
Overview
This function takes tables with a single JSON column and their configuration parameters formatted as a JSON document as input. The input fields of the JSON configuration document are described in the section Input Configuration Parameters. It returns the validated and normalized relational schema representing the collections as a JSON document.
Syntax
FUNCTION INFER_SCHEMA(config in JSON)
RETURN JSON;
Example 112-1 Example for INFER_SCHEMA Function
DECLARE
schema_json json;
schema_sql clob;
BEGIN
-- infer schema
schema_json :=
dbms_json_duality.infer_schema(
json('{
"tableNames" : ["EMPLOYEE_TAB", "DEPARTMENT_TAB"],
"viewNames" : ["EMPLOYEES", "DEPARTMENTS"],
"hints" : json({"type" : "key",
"table" : "EMP",
"path" : "$",
"value" : "empName"}),
"useFlexFields" : true,
"updatability" : true,
"sourceSchema" : "HR",
"targetSchema" : "HR",
"tablespace" : "HR_TBS",
"outputFormat" : "executable"
}'));
Input Configuration Fields
Note:
INFER_SCHEMA
, INFER_AND_GENERATE_SCHEMA
and IMPORT_ALL
take a JSON configuration document as
input. The parameter list for these functions are different but has some overlap.
However, you can create a common configuration document that can be used for both
functions and any parameters that do not apply to that function are ignored.
Field | Value |
---|---|
tableNames |
JSON array of strings of the table names of JSON collection to be processed |
viewNames |
JSON array of strings of the output duality view names to be generated |
useFlexFields |
Boolean field which indicates whether flex fields
should be used in the duality views.
|
updatability |
Boolean field which indicates whether the duality
views created by the migrator should be updatable by default.
|
normalize |
Boolean field which indicates whether the migrator will attempt to normalize the relational tables it infers for de-duplicating data. The default value for this parameter is 'true' |
sourceSchema |
This string field defines the schema name for the
input tables.
|
targetSchema |
This string field defines the schema name for the
output tables and duality views.
|
tablespace |
This string field defines the
tablespace name to use for all tables
created by the migrator.
|
ingestLimit |
This numeric field defines the maximum number of
documents that will be analyzed in each JSON collection to infer
the relational schema.
|
outputFormat |
This string field defines the format of the output
DDL script.
|
minFieldFrequency |
This numeric field defines the threshold in
percentage for identifying high entropy fields.
|
minTypeFrequency |
This numeric field defines the threshold in
percentage for identifying high entropy types of a given
field.
|
softnessThreshold |
This numeric field defines the threshold in
percentage for allowable 'softness' in identifying functional
dependencies in the input data.
|
hints |
A JSON array with elements that are JSON objects whose fields
specify overrides for the behavior of the converter in
generating a relational schema. A hint object must have these
fields (otherwise, an error is raised):
type ,
table , path , and
value . All parameters except
value are string type.
Note: Detailed information abouthints field could
be found here : Configuration Fields for
JSON-To-Duality Migrator |
GENERATE_SCHEMA Function
This section describes the syntax, input and output formats of the
GENERATE_SCHEMA
function.
Overview
This function takes a normalized relational schema represented as a JSON document as input and returns the DDL script to generate the schema.
Syntax
FUNCTION GENERATE_SCHEMA(er_schema in JSON)
RETURN CLOB;
Example 112-2 Example for GENERATE_SCHEMA Function
DECLARE
schema_json json;
schema_sql clob;
BEGIN
-- infer schema
schema_json :=
dbms_json_duality.infer_schema(
json('{
"tableNames" : ["EMPLOYEE_TAB", "DEPARTMENT_TAB"],
"viewNames" : ["EMPLOYEES", "DEPARTMENTS"],
"useFlexFields" : true,
"updatability" : true,
"sourceSchema" : "HR",
"targetSchema" : "HR",
"tablespace" : "HR_TBS",
"outputFormat" : "executable"
}'));
-- modify schema as needed
...
-- generate schema
schema_sql = dbms_json_duality.generate_schema(schema_json);
-- create schema
execute immediate schema_sql;
END;
/
Note:
Theschema_sql
is
the output from INFER_SCHEMA
function. The details of
INFER_SCHEMA
function are available here.
INFER_AND_GENERATE_SCHEMA Function
This section describes the syntax, input and output formats of the
INFER_AND_GENERATE_SCHEMA
function.
Overview
This function takes a set of JSON collections (tables with a single JSON column) and
configuration parameters formatted as a JSON document as input. The input fields of
the JSON configuration document are identical to the INFER_SCHEMA
function and listed here. It returns the DDL script to generate
the validated and normalized relational schema representing the collections as a
CLOB.
Syntax
FUNCTION INFER_AND_GENERATE_SCHEMA(config in JSON)
RETURN CLOB;
Example 112-3 Example for INFER_AND_GENERATE_SCHEMA Function
DECLARE
schema_sql clob;
BEGIN
-- infer and generate schema
schema_sql :=
dbms_json_duality.infer_and_generate_schema(
json('{
"tableNames" : ["EMPLOYEE_TAB", "DEPARTMENT_TAB"],
"viewNames" : ["EMPLOYEES", "DEPARTMENTS"],
"useFlexFields" : true,
"updatability" : true,
"sourceSchema" : "HR",
"targetSchema" : "HR",
"tablespace" : "HR_TBS",
"outputFormat" : "executable"
}'));
-- create schema
execute immediate schema_sql;
END;
/
VALIDATE_SCHEMA_REPORT Function
This section describes the syntax, input and output formats of the
VALIDATE_SCHEMA_REPORT
function.
Overview
VALIDATE_SCHEMA_REPORT
is a table function which return
a report for schema validation. It validates the schema generated by GENERATE_SCHEMA
or INFER_AND_GENERATE_SCHEMA
functions and
generates a report for documents that fail validation. Each row in the output
corresponds to validation failure for a single JSON document. If no invalid document
is found, the function does not return any rows. The output format for
VALIDATE_SCHEMA_REPORT
is a table with two CLOB columns,
data
and errors
. The column
data
contains the erroneous JSON document, and the column
errors
contains an array of schema errors. The value of errors
for VALIDATE_SCHEMA_REPORT
is a JSON array containing JSON schema
validation errors. The format of the array is identical to the format of the
'errors' field in the output of DBMS_JSON_SCHEMA.VALIDATE_REPORT
.
Syntax
FUNCTION VALIDATE_SCHEMA_REPORT(
table_owner_name in VARCHAR2 DEFAULT NULL,
table_name in VARCHAR2,
view_owner_name in VARCHAR2 DEFAULT NULL,
view_name in VARCHAR2
) return validate_report_tab_t PIPELINED;
Input Parameters
Table 112-1 Input Parameters for DBMS_JSON_DUALITY.VALIDATE_SCHEMA_REPORT
Parameter | Description |
---|---|
|
The name of the owner of the input table with a single JSON column. The value of this field should be identical to the table owner name as present in the data dictionary. Otherwise, the schema of the current connected user is used. |
|
The name of the input table with a single JSON column. The value of this field should be identical to the table name as present in the data dictionary. |
|
The name of the owner of the output duality view. The value of this field should be identical to the view owner name as present in the data dictionary. Otherwise, the schema of the current connected user is used. |
|
The name of the output duality view. The value of this field should be identical to the view name as present in the data dictionary. |
Example 112-4 Example for VALIDATE_SCHEMA_REPORT Function
Note:
To validate a schema, it is assumed that you have generated a schema using GENERATE_SCHEMA Function or INFER_AND_GENERATE_SCHEMA Function.select * from dbms_json_duality.validate_schema_report('HR, 'EMPLOYEE_TAB', 'HR', 'EMPLOYEES');
select * from dbms_json_duality.validate_schema_report('HR, 'DEPARTMENT_TAB', 'HR', 'DEPARTMENTS');
IMPORT Procedure
This section describes the syntax, input and output formats of the
IMPORT
procedure.
Overview
This procedure imports data from a table with a single JSON column into a Duality View. It also logs any documents that fail import into an error log.
Note:
In the process of normalization, data may be transformed, cast to different data types, and truncated to honor maximum size limits. In addition, any data not conforming to the resulting schema may be rejected during the import phase. Oracle recommends to verify that all data has been successfully imported by running verification tests and looking at the error log for resolving import errors.Syntax
PROCEDURE IMPORT(table_owner_name in VARCHAR2 DEFAULT NULL,
table_name in VARCHAR2,
view_owner_name in VARCHAR2 DEFAULT NULL,
view_name in VARCHAR2,
err_log_owner_name in VARCHAR2 DEFAULT NULL,
err_log_name in VARCHAR2 DEFAULT NULL,
reject_limit in NUMBER DEFAULT NULL);
Input Parameter Fields
Parameter | Value |
---|---|
|
The name of the owner of the input table with a single JSON column. You
can specify the owner in |
|
The name of the input table with a single JSON column. |
|
The name of the owner of the output duality view. You can specify the
owner in |
|
The name of the output duality view. |
|
The name of the owner of the error log to use for document that cannot be
imported successfully. You can specify the owner in
|
|
The name of the error log to use.The default value is NULL, so no errors will be logged by default. |
|
Specifies the maximum number of errors that can be logged before the
operation is aborted. The default (NULL) is unlimited, so the operation
will never be aborted by default when |
Example 112-5 Example for IMPORT Procedure
BEGIN
dbms_errlog.create_error_log(dml_table_name => 'EMPLOYEES', err_log_table_name => 'ERR$_EMP', skip_unsupported => TRUE);
dbms_errlog.create_error_log(dml_table_name => 'DEPARTMENTS', err_log_table_name => 'ERR$_DEPT', skip_unsupported => TRUE);
dbms_json_duality.import('HR', 'EMPLOYEE_TAB', 'HR', 'EMPLOYEES', 'HR', 'ERR$_EMP', 100);
dbms_json_duality.import('HR', 'DEPARTMENT_TAB', 'HR' 'DEPARTMENTS', 'HR', 'ERR$_DEPT', 100);
END;
/
IMPORT_ALL Procedure
This section describes the syntax, input and output formats of the
IMPORT_ALL
procedure.
Overview
IMPORT_ALL
is a procedure to import data from multiple
tables into corresponding duality views. When multiple imports are needed, then this
procedure should be used instead of the IMPORT
procedure to avoid risk of foreign
key constraint violation. The input for the IMPORT_ALL
procedure is
set of configuration options provided in the form of a JSON document.
IMPORT_ALL
has no output, since it is a procedure.
PROCEDURE IMPORT_ALL(config in JSON);
Table 112-2 Input Fields of the JSON Configuration Document
Field | Description |
---|---|
|
Array of the names of the tables with a single JSON column to be imported. The value of this field should be identical to the table name as present in the data dictionary. |
|
Array of the names of the duality view. The order
must be maintained between |
|
This string parameter defines the quoted schema name for the input tables. The value of this field should be identical to the source schema as present in the data dictionary. Default value for this parameter is the current user schema. |
|
This string parameter defines the quoted schema name for duality views. The value of this field should be identical to the target schema as present in the data dictionary. Default value for this parameter is the current user schema. |
|
|
|
The name of the schema of the error log to use
for document that cannot be imported successfully. You can
specify the owner in |
|
Specifies the maximum number of errors that can
be logged before the operation is aborted. The default
(NULL) is unlimited, so the operation will never be aborted
by default when |
Note:
All parameters excepttableNames
are optional. INFER_SCHEMA
, INFER_AND_GENERATE_SCHEMA
and IMPORT_ALL
take a JSON configuration document
as input. The parameter list for these functions are different but has some
overlap. However, you can create a common configuration document that can be
used for both functions and any parameters that do not apply to that function
are ignored.
Example 112-6 Example for IMPORT_ALL Procedure
BEGIN
dbms_errlog.create_error_log(dml_table_name => 'EMPLOYEES', err_log_table_name => 'ERR$_EMP', skip_unsupported => TRUE);
dbms_errlog.create_error_log(dml_table_name => 'DEPARTMENTS', err_log_table_name => 'ERR$_DEPT', skip_unsupported => TRUE);
dbms_json_duality.import_all(json('{"tableNames" : ["EMPLOYEE_TAB","DEPARTMENT_TAB"],
"viewNames" : ["EMPLOYEES","DEPARTMENTS"],
"sourceSchema" : "HR",
"targetSchema" : "HR",
"errorLog" : ["ERR$_EMP","ERR$_DEPT"]
}'));
END;
/
VALIDATE_IMPORT_REPORT Function
This section describes the syntax, input and output formats of the
VALIDATE_IMPORT_REPORT
function.
Overview
VALIDATE_IMPORT_REPORT
is a table function which return
a report for import validation. It compares the documents passed as an input to
IMPORT
and IMPORT_ALL
functions with the documents of the
corresponding duality view. and generates a report for documents that fail
validation. Each row in the output corresponds to validation failure for a single
JSON document. If no invalid document is found, the function does not return any
rows. The output format for VALIDATE_IMPORT_REPORT
is a table with
two CLOB columns, data
and errors
. The column
data
contains the erroneous JSON document, and the column
errors
contains an array of import errors. The value of errors
for VALIDATE_IMPORT_REPORT
is a JSON diff between the input JSON
document in the table and the output JSON document in the duality view. The format
of the diff is the same as the format of the JSON patch.
Syntax
FUNCTION VALIDATE_IMPORT_REPORT(
table_owner_name in VARCHAR2 DEFAULT NULL,
table_name in VARCHAR2,
view_owner_name in VARCHAR2 DEFAULT NULL,
view_name in VARCHAR2
) return validate_report_tab_t PIPELINED;
Input Parameters
Table 112-3 Input Parameters for DBMS_JSON_DUALITY.VALIDATE_IMPORT_REPORT
Parameter | Description |
---|---|
|
The name of the owner of the input table with a single JSON column. The value of this field should be identical to the table owner name as present in the data dictionary. Otherwise, the schema of the current connected user is used. |
|
The name of the input table with a single JSON column. The value of this field should be identical to the table name as present in the data dictionary. |
|
The name of the owner of the output duality view. The value of this field should be identical to the view owner name as present in the data dictionary. Otherwise, the schema of the current connected user is used. |
|
The name of the output duality view. The value of this field should be identical to the view name as present in the data dictionary. |
Example 112-7 Example for VALIDATE_IMPORT_REPORT Function
Note:
To validate an import, it is assumed that you have imported the data using IMPORT Procedure or IMPORT_ALL procedure.select * from dbms_json_duality.validate_import_report('HR, 'EMPLOYEE_TAB', 'HR', 'EMPLOYEES');
select * from dbms_json_duality.validate_import_report('HR, 'DEPARTMENT_TAB', 'HR', 'DEPARTMENTS');
BEGIN_TRANSACTION Procedure
This section describes the syntax of the non-interactive procedure named
BEGIN_TRANSACTION
Overview
-
Begin the transaction
-
Register the documents
-
Perform queries or DMLs on the duality views
-
Commit the transaction
BEGIN_TRANSACTION
procedure starts a special multi-operation
transaction with the following properties:
-
The multi-operation transaction provides repeatable reads, i.e., all reads run as of the snapshot when the transaction began.
-
Locks are only taken for rows that are modified, unmodified rows remain unlocked throughout the transaction.
Syntax
procedure begin_transaction;
Example 112-8 Example for BEGIN_TRANSACTION Procedure
dbms_json_duality.begin_transaction()
Note:
TheBEGIN_TRANSACTION
procedure is normally followed by
REGISTER
procedure. The detailed information about
REGISTER
procedure can be found here.
COMMIT_TRANSACTION Procedure
This section describes the syntax of the non-interactive procedure
named COMMIT_TRANSACTION
Overview
This procedure commits the special multi-operation transaction. It validates all modified documents in the transaction, i.e., ensures that no other session has modified these documents.
Syntax
procedure commit_transaction;
Example 112-9 Example for COMMIT_TRANSACTION Procedure
COMMIT_TRANSACTION
procedure is normally the last step in the
non-interactive transaction after queries are performed on the duality views.
dbms_json_duality.commit_transaction();
REGISTER Procedure
This section describes the syntax of the non-interactive procedure named
REGISTER
Overview
This procedure verifies that the current etag
of the
object is unchanged from a prior read. If the etag
doesn’t match
the expected etag
, and error is thrown from this function. The
register procedure relies on object ids (oid) to identify objects/documents to
register and validate. The object id can be obtained by querying the
resid
hidden column from the duality view. Its value is simply
a concatenated binary encoding of all the primary key columns of the root table in
the duality view.
Syntax
procedure register(view_name in VARCHAR2, /* duality view name */
oid in RAW, /* duality view obj identifier */
etag in RAW); /* document etag */
Example 112-10 Example for REGISTER Procedure
This example assumes that a duality view team_dv
is
already created. The detailed information about creating the duality views are
explained under Creating Duality Views in
Oracle Database JSON-Relational Duality Developer's Guide. Obtaining the
RESID
and ETAG
is mandatory step prior to
calling the REGISTER
procedure. The following shows an example code
to obtain the RESID
and ETAG
of the duality view
team_dv
:
SELECT RESID, DATA FROM team_dv dv
WHERE dv.DATA.name LIKE 'Mercedes%';
This produces an output that includes the oid
(RESID
) and the ETAG
information as shown
below:
RESID
-----
DATA
----
FB03C2040400
{"_id" : 303,
"_metadata":
{"etag" : "039A7874ACEE6B6709E06E42E4DC6355",
"asof" : "00000000001BE239"},
"name" : "Mercedes",
...}
Note:
REGISTER
procedure
is normally performed after executing the BEGIN_TRANSACTION
procedure. The details of BEGIN_TRANSACTION
procedure are provided
here.
BEGIN
DBMS_JSON_DUALITY.begin_transaction();
DBMS_JSON_DUALITY.register('team_dv',
hextoraw('FB03C2040400'),
hextoraw('039A7874ACEE6B6709E06E42E4DC6355'));
.............................................
****Perform the updating (DML) operations****
.............................................
DBMS_JSON_DUALITY.commit_transaction();
END