fix: execute INSERTs with fetch=false - #1049
Conversation
…torage-Read destination tables dbt.run_query always fetches results. Fusion's BigQuery path then reads the job destination table, which OOMs on large elementary tables during on-run-end. Callers already discarded those INSERT results.
…se-specific fetch bug Callers that ignore INSERT results should not use run_query, which always fetches.
|
👋 @github-christophe-oudar |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (3)
Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review. 📝 WalkthroughWalkthroughThe change adds ChangesQuery execution
Estimated code review effort: 1 (Trivial) | ~5 minutes Merge Risk: ⚪ Minimal · up to The change only adjusts INSERT execution to avoid fetching unused results, with no actionable merge-blocking risk remaining after normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (3 skipped: 3 unsupported.) ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Summary
elementary.execute_no_fetch, which callsadapter.execute(..., fetch=false)after the usual metadata comment wrapping.insert_rowsand theon-run-endtest-result INSERTs inhandle_tests_resultsto that helper instead ofdbt.run_query.Why
Callers already discarded INSERT results.
dbt.run_queryalways fetches (statement(..., fetch_result=true)).adapter.execute(sql, fetch=false)is the existing dbt adapter API (fetch=Falseis the default on dbt-core 1.x). This is not a Fusion-only flag and should not change 1.x behavior: run the INSERT, do not materialize a result table.On Fusion this becomes effective once dbt-labs/dbt-core#16064 lands (Fusion currently still drains the Arrow reader on BigQuery even when
fetch=false).Test plan
dbt run/dbt testwith Elementary enabledon-run-endstill writes run/test resultsSummary by CodeRabbit