Integrated Reasoning
How we use Isofold to reduce BigQuery scan volume for IRX.
Overview
Integrated Reasoning runs IRX, a large-scale LP/MIP solver-as-a-service application. IRX ingests high-dimensional models, computes solutions, and logs solver behavior across thousands of structured fields. We built our backend on BigQuery for its scale and performance — but as usage grew, query efficiency didn’t.
We built Isofold to fix that.
The Problem
Our core schema includes dense relational structures:
mip_models
– uploaded model metadatamip_model_stats
– per-variable and per-constraint diagnosticssolver_sessions
– optimization jobssession_log_lines
– fine-grained solver logs
Every user-facing request compiles into a SQL query that joins these tables — sometimes filtering by bound ranges, sometimes aggregating constraint structure. These queries are dynamic. They reflect model size, solver config, and diagnostic context. No two are quite the same.
Even with tight SELECT lists, proper clustering, and partition filters, BigQuery would frequently default to wide scans. Join order, filter placement, and subquery structure all affected cost — and BigQuery’s planner didn’t always choose well.
We saw scan volumes drift upward over time, even when the queries looked “clean.”
Why We Built Isofold
We needed a system that could:
- Analyze SQL queries at runtime
- Apply cost-reducing rewrites safely
- Verify semantic equivalence between original and optimized queries
- Fall back automatically if verification failed
So we built one.
Isofold sits between our application and BigQuery. It rewrites incoming queries, dry-runs both versions to estimate cost, then executes the cheaper one — if and only if the results match.
We integrated Isofold in a single line of configuration. Everything else happened at the proxy layer.
What It Did
In the first week:
- Rewrite coverage: 96.4% of queries
- Average scan size reduction: 38.1%
- Verification failure rate: 0%
- Application changes: none
Isofold didn’t just reduce cost — it gave us visibility. We tracked rewritten queries, measured their impact, and confirmed they produced identical results.
Example Rewrite
This seemingly useless QUALIFY
forced BigQuery to use scan fusion logic, which reduced slot time by over 40% on large partitions.
Why It Worked
We didn’t need another query linter. We needed a system that could reason about execution cost in context, rewrite queries safely, and confirm that rewrites didn’t change semantics.
Isofold did exactly that — automatically, reliably, and without slowing anything down.