Best Practices

To ensure the best possible performance and prevent the known "Code too large" error message during export execution, follow these best practices:

Simplify Query Logic:

Reduce unnecessary computations and transformations within the SQL query. For example, in a query that needs aggregated values, a HAVING clause can be changed to a WHERE clause to filter records before doing the aggregation. Complex text transformations and formatting can also be handled within the optional JavaScript step.

Before:

SELECT column1, COUNT(*) as count FROM step1() GROUP BY column1 HAVING column1 = 'sample'

After:

SELECT column1, COUNT(*) FROM step1() WHERE column1 = 'sample' GROUP BY column1

Break Down Complex Expressions

Split lengthy expressions into smaller, manageable segments. One option includes splitting a large CASE logic into multiple steps and combining the output of multiple steps in a subsequent step of the configuration:

SELECT column1, column2, column3 FROM step1() UNION ALL SELECT column1, column2, column3 FROM step2()

Optimize SELECT Clause

Minimize nested calculations and redundant operations, and use Joins instead of IN clauses.

Before:

SELECT column1, column2, column3 FROM step1() WHERE column3 IN (SELECT col FROM step2() WHERE name = 'Hello')

After:

SELECT column1, column2, column3 FROM step1() a JOIN step2() b ON a.column3 = b.column3 WHERE b.name = 'Hello'

Also consider using data mapping instead of hard coding computations or large CASE statements.