19.12 Execute and Monitor

RuntimeClient class provides API for execution of Data Loads, Data Flows and Workflows. Assuming that workbench is connected, you can use the get_runtime_client method to perform execution and status check operations.

Successful execution will return a dictionary as (key,value), where key is job reference and value is status of the job.

Example {"job123-abc-xyz":"WAITING"}.

DataTransformsException is thrown for failures with status and message.

Executing Data Flow

To execute a data flow, use the run_dataflow method.

# Execute a data flow
runtime_client = workbench.get_runtime_client()

try:
    job_status = runtime_client.run_dataflow(project_name, dataflow_name)
    job_session_id = list(job_status.keys())[0]
except DataTransformsException as e:
    logging.error("DataTransformsException: %s", str(e))

Executing Data Load

To execute a data load, use the run_dataload method.

runtime_client = workbench.get_runtime_client()
# Execute a data load
try:
    job_status = runtime_client.run_dataload(project_name, dataload_name)
    job_session_id = list(job_status.keys())[0]
except DataTransformsException as e:
    logging.error("DataTransformsException: %s", str(e))

Executing Workflow

To execute a workflow, use the run_workflow method.

runtime_client = workbench.get_runtime_client()
# Execute a workflow
try:
    job_status = runtime_client.run_workflow(project_name, workflow_name)
    job_session_id = list(job_status.keys())[0]
except DataTransformsException as e:
    logging.error("DataTransformsException: %s", str(e))

Check the status of job

runtime_client = workbench.get_runtime_client()
# Get the job ID from the response of the execute method
job_session_id = ... # from job execution

# Get the job status
response = runtime_client.get_job_status(job_session_id)
print(response)