If query execution isn’t meeting your service level agreements, or you’re encountering errors due to resource exhaustion or timeout, consider:
- Using the results from previous queries instead of recomputing. For example, your weekly total could be the sum computed in BigQuery of 7 single day aggregate queries.
- Decomposing queries into logical subqueries (such as splitting multiple joins into multiple queries), or otherwise restricting the set of data being processed. You can combine results from individual jobs into a single dataset in BigQuery. Although this may help with resource exhaustion, it may slow down your query.
- If you’re running into resources exceeded errors in BigQuery, try using temp tables to split your query into multiple BigQuery queries.
- Referencing fewer tables in a single query, as this uses large amounts of memory and can cause your query to fail.
- Rewriting your queries such that they join fewer user tables.
- Rewriting your queries to avoid joining the same table back on itself.