Logging Errors
With any map/reduce script, you should include logic that checks for errors that may have occurred during the getInputData, map, and reduce stages. You can access data about errors using the context objects made available to each entry point. Use the properties shown in the following table.
Stage where error occurred |
Property that contains data |
---|---|
Get Input Data |
|
Map |
|
Reduce |
|
Syntax
The following snippets shows how you can capture data about errors in various stages.
Map Stage
You can use this snippet in a map function. The snippet logs data only if an error was thrown during a previous invocation of the map function for the same key-value pair currently being processed.
// Create a log entry showing each full serialized error thrown
// during previous attempts to process the current key-value pair.
mapContext.errors.iterator().each(function (key, error, executionNo){
log.error({
title: 'Map error for key: ' + key + ', execution no ' + executionNo,
details: error
});
return true;
});
Reduce Stage
You can use this snippet in a reduce function. The snippet logs data only if an error was thrown during a previous invocation of the reduce function for the same key-value pair currently being processed.
// Create a log entry showing each full serialized error thrown
// during previous attempts to process the current key-value pair.
reduceContext.errors.iterator().each(function (key, error, executionNo){
log.error({
title: 'Reduce error for key: ' + key + ', execution no ' + executionNo,
details: error
});
return true;
});
Summarize Stage
You can use these snippets in a summarize function. They log data about errors thrown during previous stages.
// If an error was thrown during the input stage, log the error.
if (summary.inputSummary.error)
{
log.error({
title: 'Input Error',
details: summary.inputSummary.error)
});
// For each error thrown during the map stage, log the error, the corresponding key,
// and the execution number. The execution number indicates whether the error was
// thrown during the the first attempt to process the key, or during a
// subsequent attempt.
summary.mapSummary.errors.iterator().each(function (key, error, executionNo){
log.error({
title: 'Map error for key: ' + key + ', execution no. ' + executionNo,
details: error
});
return true;
});
// For each error thrown during the reduce stage, log the error, the corresponding
// key, and the execution number. The execution number indicates whether the error was
// thrown during the the first attempt to process the key, or during a
// subsequent attempt.
summary.reduceSummary.errors.iterator().each(function (key, error, executionNo){
log.error({
title: 'Reduce error for key: ' + key + ', execution no. ' + executionNo,
details: error
});
return true;
});