[camera_android_camerax] Add the ability to unit test skills. - #12285
[camera_android_camerax] Add the ability to unit test skills. #12285reidbaker-agent wants to merge 12 commits into
Conversation
There was a problem hiding this comment.
I think this is no longer used. remove
| DisplayOrientedMeteringPointFactory(CameraInfo cameraInfo, double width, double height); | ||
| } | ||
|
|
||
| // dummy eval modification |
There was a problem hiding this comment.
This file should not be checked in. This is leaked state from an eval.
0d0aa4d to
68f87b7
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds the run-evals skill, including its evaluation configurations, templates, and scripts, and introduces a test suite in packages/camera/camera_android_camerax to verify the structural consistency of evaluation files. Feedback on these changes highlights issues with hardcoded repository paths in the skill documentation, a missing reference to code_quality_rubric.json in the evaluation configuration, and the use of Directory.current.path in tests, which may cause failures when executed from the repository root.
68f87b7 to
c8f0176
Compare
camsim99
left a comment
camsim99
left a comment
There was a problem hiding this comment.
LGTM! Just a question about dart_skills_lint references in the run-evals skill
|
|
||
| # Run Skill Evals | ||
|
|
||
| 1. **Read Framework**: Read `tool/dart_skills_lint/evals/README.md` for understanding the difference between per-skill evals and cross-skill evals. |
There was a problem hiding this comment.
What if tools/dart_skills_lint is not in the context? Should the references to it be links instead?
There was a problem hiding this comment.
flutter/agent-plugins#196. Will update this pr when 196 lands.
reidbaker
changed the title
3 participants
Add the concept of evals for skills to camera_android_camerax this is part of project-oneshot.
This is basically the equivalent of llm unit tests.
The idea and harness comes from dart_skills_lint see flutter/agent-plugins#189
The infra for running evals can and should be replaced by a tool that is designed for this use case when it is available.
In the meantime, the valuable parts are the evals.jsons files. They document the expectations for the skills under test in this repo. - @reidbaker
Agent authored description
Updates agent evals for check-readiness and run-evals, including fixing missing license blocks and enforcing relative paths.
Evaluation Results:
cross_skill_rubricEval ID 1: Failure Scenario
Eval ID 2: Success Scenario