Community package: azure-ai-evaluation-openeval-adapter (evaluate() data/results <-> EvalPort interchange)

[adhabnr-ux]

Hi Azure AI Evaluation team — I maintain EvalPort (Apache 2.0), an open, schema-validated JSON interchange format for portable LLM evaluation test cases, graders, suites, and results. It already has independently-tested adapter packages for ~20 other eval/observability frameworks (MLflow, LangSmith, Ragas, Vertex AI Gen AI Evaluation, Hugging Face evaluate, and others), and I built one for azure-ai-evaluation the same way.

This isn't a request for a change in this repo — I'm not proposing new API surface or asking for a design review, just flagging a working, tested community package in case it's useful to know about or link from docs.

azure-ai-evaluation-openeval-adapter

from azure.ai.evaluation import F1ScoreEvaluator, evaluate
from azure_ai_evaluation_openeval_adapter import to_openeval, evaluation_result_to_openeval
from openeval.validate import validate_suite, validate_result_set

suite = to_openeval(data="my_eval_data.jsonl", evaluators={"f1": F1ScoreEvaluator()}, suite_id="my_eval_suite")
assert validate_suite(suite).valid

result = evaluate(data="my_eval_data.jsonl", evaluators={"f1": F1ScoreEvaluator()})
result_set = evaluation_result_to_openeval(result, suite_id="my_eval_suite")
assert validate_result_set(result_set).valid

to_openeval() accepts exactly what evaluate() itself accepts for data/evaluators, so it's a pure format bridge rather than new infrastructure. The one design choice worth flagging: every evaluator (local NLP metrics like F1/BLEU/ROUGE, AI-assisted evaluators needing a live model_config, and the content-safety evaluators needing a live Foundry project) maps to EvalPort's custom grader type rather than being force-fit into semantic_similarity or llm_judge — those types require params (threshold, prompt) this adapter can't honestly fabricate from the outside. Full mapping table and the flat-row parsing logic (recovering per-metric score/passed/reason from evaluate()'s real outputs.<evaluator>.* column convention) are in the README.

21 tests, all passing locally against the real installed azure-ai-evaluation package and EvalPort's real validate_suite()/validate_result_set() — not mocked.

No action needed — this lives entirely outside azure-sdk-for-python as an independent package (pip install via git+, not yet on PyPI). Flagging mainly for discoverability; happy to adjust the mapping if the evaluation module's public API shifts, or to send a one-line docs PR if there's a community-packages list this belongs on.

Spec: https://github.com/adhabnr-ux/evalport/blob/main/spec/SPEC.md