The library / Analytics & warehouses

BigQuery: scan less data before adding more compute

Use explicit columns and partition filters to reduce unnecessary work, then validate the bytes your query will process.

Start with what the query reads

In an analytical workload, reducing unnecessary input is often a useful first optimization. Select only the columns needed and constrain the relevant partitions when the table supports it.

The example assumes a table partitioned by a DATE column named order_date. Replace the project and dataset identifiers with your own development environment.

Make the date boundary explicit

SELECT
  customer_id,
  SUM(total_cents) AS revenue_cents
FROM `your_project.analytics.orders`
WHERE order_date >= DATE '2026-08-01'
  AND order_date < DATE '2026-09-01'
GROUP BY customer_id;

The half-open range covers August without relying on an end-of-day timestamp. A qualifying partition filter can allow BigQuery to avoid reading unrelated partitions.

A filter on a different field is not automatically equivalent to a usable partition filter.

Check before running

Use a dry run or the query editor's processing estimate to inspect expected bytes. Where supported for the chosen billing mode, set a maximum-bytes-billed limit as an additional guard.

A LIMIT on the final result is not a general guarantee of a small scan.

Distinguish price models

Bytes processed are directly relevant to on-demand query billing. Capacity-based pricing has a different model, although reducing work can still improve efficiency.

Storage, ingestion, and other operations may have separate costs. Verify the current configuration and region.

Consider clustering next

Within partitions, clustering can improve some queries by organizing data around commonly filtered fields. Its value depends on data layout and access patterns.

Do not add every plausible clustering field by habit. Measure the queries you actually run.

An experiment to document

Compare a query with the partition filter to the same query without it using dry-run estimates. Record the table's partitioning, selected columns, estimated bytes, and expected result range. Avoid executing an unbounded production scan merely to demonstrate the difference.

Keep exploring

Go deeper with the original documentation.

Official documentation
D
DBMinutes Editorial

Practical explanations of database systems, cloud services, and the engineering decisions between them.

AI-assisted content · Our editorial process

Keep the curiosity going.

Back to the library
A little learning goes a long way

Make room for a few good minutes.

Join the list for practical guides, thoughtful comparisons,
and ideas worth bringing to your next project.

Find your next answer

Search concepts, tools, and practical guides.