Skip to content

Stop known per-question bot errors from failing runs - #393

Merged
CodexVeritas merged 2 commits into
mainfrom
claude/forecasting-tools-bot-failures-6b1e93
Nov 6, 2026
Merged

CodexVeritas merged 2 commits into
mainfrom
claude/forecasting-tools-bot-failures-6b1e93

Conversation

@CodexVeritas

@CodexVeritas CodexVeritas commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds questions to POST_IDS_TO_NOT_RAISE_ERRORS_FOR that fail every main-site run for one or two bots while the other bots forecast them. The errors are still logged; they just no longer fail the workflow.

  • 20845: DeepSeek 3.2 reasoning and Inkling forecast dates entirely past the 2027 upper bound, which the out-of-range check rejects.
  • 45463: Qwen 3.7 Plus treats it as already resolved, so the parser refuses its forecasts.
  • 6142: Muse Wonder repeats percentile values at the open lower bound, which produces a non-increasing CDF that fails standardization. No forecast is submitted in this case.
  • 43119, 44880, 44883, 45414, 45415, 45876, 46442: US-China and Russia questions rejected by Moonshot's content filter for the Kimi bots.

In the last main-site window, no other bot errored on these questions, so the entries do not currently hide other errors.

Test plan

  • Unit suite passes (TZ=UTC).
  • Next main-site window run passes for the affected bots.

🤖 Generated with Claude Code

CodexVeritas and others added 2 commits November 4, 2026 22:39
…ors failing runs

- NumericDistribution._get_cdf_at matched segments with a 1e-10 location
  tolerance but did not clamp the interpolation fraction. Repeated
  percentile values at the lower bound are nudged into points only ~1e-12
  apart, so location 0 extrapolated to a CDF height of ~1.0 and
  standardization failed with a non-increasing CDF. The fraction is now
  clamped to the segment.
- Add questions to POST_IDS_TO_NOT_RAISE_ERRORS_FOR that fail every
  main-site run for one or two bots while other bots forecast them: 20845
  (forecasts entirely past the upper bound), 45463 (treated as resolved),
  and seven US-China/Russia questions rejected by Moonshot's content filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CodexVeritas CodexVeritas changed the title Fix CDF extrapolation on repeated bound values and stop known bot errors failing runs Stop known per-question bot errors from failing runs Nov 6, 2026
@CodexVeritas
CodexVeritas merged commit 91e6d1c into main Nov 6, 2026
2 checks passed
@CodexVeritas
CodexVeritas deleted the claude/forecasting-tools-bot-failures-6b1e93 branch November 6, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant