6 Data Monitoring
Data Monitoring evaluates how your data evolves over time. It helps you with insights on trends and multivariate dependencies in the data. It also gives you an early warning about data drift.
Data drift occurs when data diverges from the original baseline data over time. Data drift can happen for a variety of reasons, such as a changing business environment, evolving user behavior and interest, data modifications from third-party sources, data quality issues, or issues with upstream data processing pipelines.
The key to accurately interpret your models and to ensure that the models are able to solve business problems is to understand how data evolves over time. Data monitoring is complementary to successful model monitoring, as understanding the changes in data is critical in understanding the changes in the efficacy of the models. The ability to quickly and reliably detect changes in the statistical properties of your data ensures that your machine learning models are able to meet business objectives.
Figure 6-1 Data Monitors page
- Create: Create a data monitor.
Note:
The supported data types for data monitoring are NUMERIC and CATEGORICAL. - Edit: Select a data monitor and click Edit to edit a data monitor.
- Duplicate: Select a data monitor and click Duplicate to create a copy of the monitor.
- Delete: Select a data monitor and click Delete to delete a data monitor.
- History: Select a data monitor and click History to view the runtime details. Click Back to Monitors to go back to the Data Monitoring page.
- Start: Start a data monitor.
- Stop: Stop a data monitor that is running.
- More: Click More for additional
options to:
Figure 6-2 More option under Data Monitors
- Enable: Select a data monitor and
click Enable to enable a monitor that is
disabled. By default, a data monitor is enabled. The status is displayed
as
SCHEDULED. - Disable: Select a data monitor and
click Disable to disable a data monitor. The
status is displayed as
DISABLED. - Show Managed Monitors: Click this
option to view the data monitors that are created and managed by OML
Services REST API and Model Monitors in Oracle Machine Learning UI. The
data monitors that are managed by these two components have a system
generated name, and are indicated by specific icons against its name.
- Click on the link icon against a managed data
monitor name to view the details of the associated model
monitor. The associated model monitor details are displayed on a
separate pane that slides in. The slide-in pane displays the
model monitor name with links to view the model monitor results
and settings. Clicking on the link icon also displays the data
drift details on the lower pane of the Data Monitors page. Click
on the X on the top left corner to close the pane.
Figure 6-3 Data Monitors page displaying the associated model monitor results and settings
In this example, the slide-in pane displays the details of the model monitor Power Consumption. On the slide-in pane:
- Click Model Monitor Results to view the results computed by the model monitor — settings, models, model drift, metric, and prediction statistics. Click Monitors to return to the Data Monitors page. See View Model Monitor Results.
- Click Model Monitor Settings to view and edit the settings, details, and models monitored by the model monitor on the Edit Model Monitor page. Click Cancel to return to the Data Monitors page. Click Save to save any changes.
- Click on the check box against the data monitor
name to view the data drift values on the lower pane.
Figure 6-4 Select a managed data monitor
- Click on the data monitor name to view the details
of the data monitor — settings, data drift values and monitored
features.
Figure 6-5 Data monitor click
- Click on the link icon against a managed data
monitor name to view the details of the associated model
monitor. The associated model monitor details are displayed on a
separate pane that slides in. The slide-in pane displays the
model monitor name with links to view the model monitor results
and settings. Clicking on the link icon also displays the data
drift details on the lower pane of the Data Monitors page. Click
on the X on the top left corner to close the pane.
- Enable: Select a data monitor and
click Enable to enable a monitor that is
disabled. By default, a data monitor is enabled. The status is displayed
as
The Data Monitors page displays the information about the selected monitor: Monitor name, Baseline Data, New Data, Last Start Date, Last Status, Next Run Data, Status, and Schedule. The page also displays the data drift, if the data monitor has run successfully. To view data drift:
Figure 6-6 Data Drift preview on Data Monitors page
Select a data monitor that has run successfully, as shown in the screenshot. On the lower pane, the data drift of the selected monitor is displayed. The X axis depicts the analysis period, and the Y axis depicts the data drift values. The horizontal dotted line is the threshold value, and the line depicts the drift value for each point in time for the analysis period. Hover your mouse over the line to view the drift values. For more information on this example, see View Data Monitor Results.
- Create a Data Monitor
Data Monitoring allows you to detect data drift over time and the potentially negative impact on the performance of your machine learning models. On the Data Monitor page, you can create, run, and track data monitors and the results. - View Data Monitor Results
The Data Monitor Results page displays the information on the selected data monitor that have run successfully, along with data drift details for each monitored feature. - View History
The History page displays the runtime details of data monitors.
Related Topics