Skip to content

Add skill evaluation dataset for cuopt-lp-milp-api-python #1172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rapids-bot merged 5 commits into from
May 5, 2026
+1,131 −0
Merged

Add skill evaluation dataset for cuopt-lp-milp-api-python #1172

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1d61c3f
Add LP/MILP skill evaluation dataset (10 IndustryOR problems)
rgsl888prabhu May 1, 2026
76bc39b
Make exact-precision objective-match requirement explicit in rubric
rgsl888prabhu May 1, 2026
d28ae61
Expand LP/MILP evals dataset to all 99 IndustryOR problems
rgsl888prabhu May 1, 2026
992cba2
Trim expected_behavior to the exact-match tolerance bullet only
rgsl888prabhu May 1, 2026
9933e43
Merge branch 'main' into add-lp-milp-api-evals
rgsl888prabhu May 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions skills/cuopt-lp-milp-api-python/evals/SOURCES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Sources

Eval prompts in `evals.json` for the `cuopt-lp-milp-api-python` skill are
adapted from the **OptiGuide / OptiMind IndustryOR** dataset:

- Repository: [microsoft/OptiGuide](https://github.com/microsoft/OptiGuide)
- File: [`optimind/data/optimind_cleaned_classified_industryor.csv`](https://github.com/microsoft/OptiGuide/blob/main/optimind/data/optimind_cleaned_classified_industryor.csv)
- License: MIT (Copyright (c) Microsoft Corporation)

Each entry's `source` field references the original row index. Problem
statements are quoted verbatim; ground-truth values are the dataset's
optimal objective values.

## License

The MIT license under which the source dataset is distributed:

```
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add a language tag to the fenced license block to satisfy markdownlint.

Line 18 opens a fenced block without a language, which triggers MD040.

Proposed fix 
-```
+```text
 MIT License
@@
 SOFTWARE
</details>

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)

[warning] 18-18: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@skills/cuopt-lp-milp-api-python/evals/SOURCES.md` at line 18, The fenced
license block in SOURCES.md is missing a language tag (MD040); update the
opening fence used for the MIT license block from ``` to ```text so the block
becomes a labeled text code fence (e.g., change the license block's opening
fence in the MIT License section to ```text).

MIT License

Copyright (c) Microsoft Corporation.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE
```
Loading