Skip to content

Make tests and sim/gen directories agent ready #1691

@shuds13

Description

@shuds13

Make tests 'agent ready'

An agent or script creates an index file of test names with content from docstring (all docstring or top paragraph).

The Ensemble agent will use this when generating/fixing scripts:

  • First reads index file (or creates it if not there)
  • Second pulls relevant test examples into context
  • Helps agent modify scripts to serve the user's goal.
  • This also may be sufficient to replace the generation from templates.

To make tests (also sims and gens) ready:

  • Top docstring should say what it does and with what in a structured way
    • e.g This test finds multiple minima of a smooth function using APOSMM...
    • Review the run line examples in the docstring (or we just take top paragraph).
    • May be worth mentioning if it is gest-api(vocs) format, what sim is, and whether runs application or pure python.
  • Consider whether we want parse_args usage. May be better to have nworkers in the script
  • This can extend to Optimas/Xopt examples/modules.
  • Tests should be clean and structured examples in the format we want for libE 2+.
  • Either agent grabs all in a directory OR only ones that are marked as exemplars in the docstring or test name.
  • If former - move to separate dir tests that are 'messy' e.g. multiple loops for testing....
  • Convert most to vocs style (requires discussion)?

To see current index files produced get the script and run e.g,:

python create_examples_index.py tests --first-paragraph
and
python create_examples_index.py sim_funcs --first-paragraph

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions