Add per-provenance counters to dataflow and spanner

[vish-cs]

No description provided.

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 16 complexity

Metric	Results
Complexity	16

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}

[gemini-code-assist]

Code Review

This pull request updates the ingestion pipeline to track metrics (node, edge, observation, and time series counts) on a per-import basis by suffixing counter names with the import name. It also introduces a get_import_counts method in the Spanner client to query these counts directly from Spanner when they are not provided in the metrics, and updates the version history tables accordingly. The feedback highlights a logical issue in update_version_history where Spanner is not queried if metrics are only partially populated, suggests qualifying the provenance column in a SQL JOIN query to prevent ambiguity, and recommends filtering metrics by the specific import name suffix in get_ingestion_metrics to avoid incorrect aggregation across different imports.

[gmechali]

sweet thank you Vishal, have you tested this and deployed it via DCP as well to ensure it doesnt' break?