Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions sdk/ai/azure-ai-projects/.env.template
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ OPTIMIZATION_MODEL=
JOB_ID=
WORK_IQ_PROJECT_CONNECTION_ID=
WORK_IQ_USER_INPUT=
WEB_IQ_PROJECT_CONNECTION_ID=
WEB_IQ_USER_INPUT=
FABRIC_IQ_PROJECT_CONNECTION_ID=
FABRIC_IQ_USER_INPUT=
AI_SEARCH_CONNECTION_NAME=
Expand Down
5 changes: 4 additions & 1 deletion sdk/ai/azure-ai-projects/.github/skills/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,10 @@ This skill creates a new topic branch, emits SDK from TypeSpec, runs some post-p

This skill updates the file CHANGELOG.md, comparing the source in the current branch which the source of the latest public release. It does not create a new topic branch or a PR.

### azure-ai-projects-author-samples

This skill compares a newly emitted or merged public API surface with its selected base, then creates or updates idiomatic synchronous and asynchronous Python samples. It also keeps unrecorded samples excluded from the recorded sample harness.

### azure-ai-projects-author-tests


This skill updates existing pytest coverage and authors complete sync/async Test Proxy tests for new behavior. New recorded service tests remain explicitly skipped until a human adds recordings.
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
name: azure-ai-projects-author-samples
description: 'Create or update idiomatic azure-ai-projects Python samples for newly emitted or merged API changes. WHEN: author azure-ai-projects samples; update azure-ai-projects samples after TypeSpec emission; add azure-ai-projects feature samples; fix azure-ai-projects samples after API changes. DO NOT USE FOR: other packages; Test Proxy recording. INVOKES: git, azpysdk, Python validation commands.'
---

# Author samples for azure-ai-projects API changes

Run from `sdk\ai\azure-ai-projects`. This workflow handles both uncommitted emission output and already-merged changes.

## 1. Establish the API delta

Regenerate `api.md` with `azpysdk apistub .`. Choose the comparison base explicitly:

```powershell
# Uncommitted emission/customization changes
git diff HEAD -- api.md azure\ai\projects

# Committed or merged changes
git diff <base-ref>...HEAD -- api.md azure\ai\projects
```

Use [prompts\diff-api-surface.prompt.md](prompts/diff-api-surface.prompt.md) to classify additions, signature changes, renames, and removals. Do not use the latest release as the base when the requested scope is a particular emission or merge.

## 2. Map changes to existing coverage

Search `samples\` and `tests\` for every affected public symbol. Update all stale call sites for renames, removals, and changed parameters. Add a new sample only for a meaningful end-to-end workflow, not for every new model or overload.

Bucket work beside the closest existing feature. Read the neighboring sync/async pair before editing; see [references\sample-conventions.md](references/sample-conventions.md).

## 3. Author or update samples

- Keep sync and async samples behaviorally equivalent when both clients expose the feature.
- Start new files from [templates\sample-skeleton.py](templates/sample-skeleton.py) and [templates\sample-skeleton_async.py](templates/sample-skeleton_async.py), then replace every placeholder.
- Use only public imports and the final customized API surface from `api.md`; never import generated internals.
- Make the workflow runnable against a real Foundry project, narrate useful results, and clean up created resources in `finally` where failure could leak them.
- Preserve nearby sample voice, environment-variable names, authentication style, and preview wording. Python samples already cover beta APIs, so do not exclude a feature merely because it is under `.beta`.

## 4. Keep recorded sample tests green

Check `tests\samples\test_samples.py` and `test_samples_async.py`. Existing folders are often auto-discovered by `get_sample_paths` or `get_async_sample_paths`; when no recording exists, add each new filename to that test's `samples_to_skip` with a concrete recording-needed reason. For a new sample folder, add matching sync/async harness coverage but keep the new cases excluded until recordings are supplied.

Do not create recordings or modify `assets.json`.

## 5. Validate

Run targeted syntax and formatting checks for every edited sample, then package checks:

```powershell
python -m compileall -q <edited-sample-paths>
python -m black --check <edited-sample-paths>
azpysdk pylint .
azpysdk mypy .
```

Fix source problems; do not weaken lint or type-check settings. Do not run samples live unless the user explicitly requests it.

## 6. Hand off

Run `azure-ai-projects-author-tests` for affected behavior, then `azure-ai-projects-update-changelog`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Diff the azure-ai-projects Python API surface

Classify the public API changes introduced by one emission or merge. The report drives sample and test authoring.

## Inputs

- `api.md` at the current worktree or `HEAD`.
- `api.md` at the explicitly selected comparison base.
- The same-range diff under `azure\ai\projects` when `api.md` lacks enough context to distinguish a rename from add/remove.

Use one range consistently:

```powershell
# Uncommitted changes
git diff --unified=80 HEAD -- api.md azure\ai\projects

# Committed or merged changes
git diff --unified=80 <base-ref>...HEAD -- api.md azure\ai\projects
```

## Task

Return additions, behaviorally relevant signature changes, likely renames, and removals:

```json
{
"addedNamespaces": [
{"name": "azure.ai.projects.operations.ExampleOperations", "isBeta": false}
],
"addedClasses": [
{"name": "Example", "namespace": "azure.ai.projects.models", "isBeta": false}
],
"addedMethods": [
{
"name": "create",
"owner": "ExampleOperations",
"namespace": "azure.ai.projects.operations",
"signature": "create(name: str, *, description: str | None = ...) -> Example",
"isBeta": false
}
],
"changedMethods": [
{
"name": "update",
"owner": "ExampleOperations",
"before": "update(name: str, body: JSON) -> Example",
"after": "update(name: str, *, description: str) -> Example",
"changeKinds": ["parameter-removed", "keyword-added"],
"isBeta": false
}
],
"renamedSymbols": [
{"oldName": "OldExample", "newName": "Example", "kind": "class", "confidence": "high"}
],
"removedSymbols": [
{"name": "delete_legacy", "owner": "ExampleOperations", "kind": "method", "isBeta": false}
],
"affectedAreas": ["examples"]
}
```

Rules:

1. Include public surface only; exclude underscore-prefixed symbols and generated implementation details absent from `api.md`.
2. Collapse overload-only formatting noise. Report a method once with its effective public signatures.
3. Mark anything owned by `BetaOperations`, a `Beta*Operations` type, or exposed through `.beta` as `isBeta: true`.
4. Use source context and matching owners/shapes to identify likely renames. If uncertain, emit separate add/remove entries rather than claiming a rename.
5. Include changed defaults, required/optional transitions, sync/async exposure changes, and return-type changes.
6. Do not treat the expected sync/async duplicate as two product features.
7. Set `affectedAreas` to likely existing `samples\`/`tests\` feature folders; do not invent a folder when no close match exists.

## Output

Return only the JSON report.
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# azure-ai-projects sample conventions

Use nearby files as the final authority. Good anchors are [`sample_agent_basic.py`](../../../../samples/agents/sample_agent_basic.py), its [async mirror](../../../../samples/agents/sample_agent_basic_async.py), and the [skills CRUD pair](../../../../samples/skills/).

## Shape and naming

- Put `sample_<workflow>.py` beside related features and use `sample_<workflow>_async.py` for the async mirror.
- Start with the package copyright header and a module docstring containing `DESCRIPTION`, `USAGE`, install prerequisites, and every required/optional environment variable.
- Load `.env` with `python-dotenv`; read required values with `os.environ[...]` and optional values with `os.environ.get(...)`.
- Import sync credentials/client from `azure.identity` and `azure.ai.projects`; async variants come from their `.aio` namespaces. Import models from the public `azure.ai.projects.models` surface.
- Prefer one coherent workflow with useful progress/result output. Iterate pageable results with `for` or `async for`, and `await` every async service call.

## Lifetime and parity

Use credential and client context managers. When creating service resources, retain identifiers and clean them up in `finally` so partial failures do not leak resources. Sync and async samples should demonstrate the same scenario, parameters, output, and cleanup unless the API exists in only one client.

The sample executor imports the module and calls `main()` when present. A guarded sync `main()` and an `asyncio.run(main())` async entry point are both supported. Match the closest neighboring pair rather than restyling an existing sample unnecessarily.

## Updating existing samples

For a renamed/changed API, search every sample call site and update it in place. Preserve the scenario unless the old behavior no longer exists. Do not import `_operations`, `_models`, `_patch`, or other implementation modules to bypass the public surface.

Python samples intentionally include preview features; state preview status in the description and use the existing `.beta` access pattern when applicable.

## Recorded sample harness

[`test_samples.py`](../../../../tests/samples/test_samples.py) and [`test_samples_async.py`](../../../../tests/samples/test_samples_async.py) auto-discover many folders. Add unrecorded files to the appropriate `samples_to_skip` list with a specific reason. Whitelist-based `samples_to_test` blocks require explicit opt-in instead.

If a new folder needs output validation, also update [`llm_instructions.py`](../../../../tests/samples/llm_instructions.py) and its folder mapping. Add required sanitized environment defaults through the narrowest preparer and mapping helper; never put live values in source.
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# ------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------

"""
DESCRIPTION:
This sample demonstrates how to use <feature> with the synchronous
AIProjectClient.

USAGE:
python sample_<feature>.py

Before running the sample:

pip install "azure-ai-projects" python-dotenv

Set these environment variables with your own values:
1) FOUNDRY_PROJECT_ENDPOINT - The Microsoft Foundry project endpoint.
"""

import os

from dotenv import load_dotenv

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

load_dotenv()

endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"]


def main() -> None:
created_resource = None

with (
DefaultAzureCredential() as credential,
AIProjectClient(endpoint=endpoint, credential=credential) as project_client,
):
try:
# TODO(<feature>): call the public operation and print meaningful results.
created_resource = project_client
print("Completed <feature> workflow")
finally:
if created_resource is not None:
# TODO(<feature>): delete service resources created by this sample.
pass


if __name__ == "__main__":
main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# ------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------

"""
DESCRIPTION:
This sample demonstrates how to use <feature> with the asynchronous
AIProjectClient.

USAGE:
python sample_<feature>_async.py

Before running the sample:

pip install "azure-ai-projects" python-dotenv aiohttp

Set these environment variables with your own values:
1) FOUNDRY_PROJECT_ENDPOINT - The Microsoft Foundry project endpoint.
"""

import asyncio
import os

from dotenv import load_dotenv

from azure.identity.aio import DefaultAzureCredential
from azure.ai.projects.aio import AIProjectClient

load_dotenv()

endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"]


async def main() -> None:
created_resource = None

async with (
DefaultAzureCredential() as credential,
AIProjectClient(endpoint=endpoint, credential=credential) as project_client,
):
try:
# TODO(<feature>): await the public operation and print meaningful results.
created_resource = project_client
print("Completed <feature> workflow")
finally:
if created_resource is not None:
# TODO(<feature>): await deletion of service resources created by this sample.
pass


if __name__ == "__main__":
asyncio.run(main())
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
name: azure-ai-projects-author-tests
description: 'Create or update idiomatic azure-ai-projects pytest coverage for newly emitted or merged API changes, leaving new service tests skipped until Test Proxy recordings exist. WHEN: author azure-ai-projects tests; update azure-ai-projects tests after TypeSpec emission; add azure-ai-projects feature tests; fix azure-ai-projects tests after API changes. DO NOT USE FOR: other packages; recording or rerecording tests. INVOKES: git, pytest collection, azpysdk, Python validation commands.'
---

# Author tests for azure-ai-projects API changes

Run from `sdk\ai\azure-ai-projects`. Consume the API-delta report from `azure-ai-projects-author-samples`, or recompute it with [..\azure-ai-projects-author-samples\prompts\diff-api-surface.prompt.md](../azure-ai-projects-author-samples/prompts/diff-api-surface.prompt.md).

## 1. Map the delta to tests

Search `tests\` for every affected symbol. Update existing tests and assertions for changed signatures, renames, and removals while preserving their current enabled/skipped state. Add new behavior coverage beside the closest feature tests; Python coverage includes both GA and beta APIs.

Prefer extending an existing sync/async pair. Read its fixtures, decorators, sanitizers, resource naming, assertions, and cleanup before writing code; see [references\test-conventions.md](references/test-conventions.md).

## 2. Write complete, disabled new coverage

Use [templates\test-skeleton.py](templates/test-skeleton.py) and [templates\test-skeleton_async.py](templates/test-skeleton_async.py) only as structural starting points. Replace every placeholder and write the actual service calls, assertions, and cleanup.

- For a wholly new test file, put `@pytest.mark.skip(reason="TODO(<feature>): enable after Test Proxy recordings are added.")` on the class.
- When adding coverage to an active class, put that marker on each new test method only; never disable unrelated tests.
- Keep sync and async tests behaviorally equivalent when both APIs exist.
- Use `TestBase`, the narrowest existing preparer, `create_client`/`create_async_client`, and `recorded_by_proxy`/`recorded_by_proxy_async`.
- Add sanitized preparer values, function-scoped sanitizers, and files under `tests\test_data` only when the scenario requires them. Never place secrets or live resource identifiers in source.
- Use `RecordedTransport.HTTPX2` only when the path also calls an OpenAI/httpx client. Follow the package's passthrough-wrapper pattern when combining parametrization with recorded decorators.

Do not add recordings and do not modify `assets.json`. A skipped test must still import and collect successfully.

## 3. Validate without enabling new tests

```powershell
python -m compileall -q <edited-test-paths>
python -m black --check <edited-test-paths>
pytest --collect-only -q <edited-test-paths>
azpysdk pylint .
azpysdk mypy .
```

Run targeted playback for updated pre-existing tests only when recordings already exist. Do not run new service tests live and do not remove their skip markers.

## 4. Hand off

Report which new tests remain skipped and why, then run `azure-ai-projects-update-changelog`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# azure-ai-projects test conventions

Use the nearest feature tests as the final authority. Simple recorded anchors are [`test_deployments.py`](../../../../tests/deployments/test_deployments.py) and its [async mirror](../../../../tests/deployments/test_deployments_async.py); shared clients and preparers live in [`test_base.py`](../../../../tests/test_base.py).

## Test kind

- Keep deterministic model/customization tests as ordinary enabled unit tests when no service call or recording is needed.
- Service tests derive from `TestBase`, use `@servicePreparer()` (or a narrower existing preparer), and use `@recorded_by_proxy` or `@recorded_by_proxy_async`.
- Create clients with `self.create_client(**kwargs)` or `self.create_async_client(**kwargs)` so playback uses fake credentials and sanitized endpoints. Pass `allow_preview=True` only when the neighboring feature/API requires it.
- Mirror sync files/methods with `_async` naming and equivalent assertions/cleanup.

## New coverage stays disabled

A wholly new recorded file gets one class-level marker:

```python
@pytest.mark.skip(reason="TODO(<feature>): enable after Test Proxy recordings are added.")
```

In an active class, mark only each new recorded method. Write the complete operation flow before applying the marker; skipped placeholders are not coverage. Updates to pre-existing recorded tests retain their current state.

## Recording-safe design

Use stable inputs where possible. For generated names/IDs/timestamps, add the narrowest function-scoped sanitizer or reuse a sanitizer in `tests\conftest.py`. Add sanitized defaults to the narrowest `EnvironmentVariableLoader` preparer. Never commit credentials, tokens, account names, or raw endpoints.

Place reusable payloads under `tests\test_data`; preserve LF handling and `.gitattributes` patterns for uploaded text files. Use `try/finally` cleanup and assert returned IDs, names, states, paging behavior, and error contracts rather than only checking non-`None`.

Include `RecordedTransport.HTTPX2` when the test also uses an OpenAI/httpx client. When combining `pytest.mark.parametrize` with recorder decorators, copy the package's passthrough-wrapper pattern described in [`test_finetuning.py`](../../../../tests/finetuning/test_finetuning.py).

Do not edit `assets.json` or add recordings in this workflow. Collection must succeed while the new service tests remain skipped.
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# ------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------

import pytest

from devtools_testutils import recorded_by_proxy

from test_base import TestBase, servicePreparer


@pytest.mark.skip(reason="TODO(<feature>): enable after Test Proxy recordings are added.")
class TestFeature(TestBase):

@servicePreparer()
@recorded_by_proxy
def test_feature(self, **kwargs) -> None:
created_resource = None

with self.create_client(**kwargs) as project_client:
try:
# TODO(<feature>): exercise the public operation and assert behavior.
created_resource = project_client
assert created_resource is not None
finally:
if created_resource is not None:
# TODO(<feature>): delete any service resources created by the test.
pass
Loading
Loading