Release v0.3.3

.claude-plugin/marketplace.json

@@ -13,7 +13,7 @@
       "name": "agentops-accelerator",
       "source": "../../plugins/agentops",
       "description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Toolkit and Microsoft Foundry agents.",
-      "version": "0.3.2",
+      "version": "0.3.3",
       "keywords": [
         "agentops",
         "evaluation",

.github/plugin/marketplace.json

@@ -13,7 +13,7 @@
       "name": "agentops-accelerator",
       "source": "../../plugins/agentops",
       "description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Toolkit and Microsoft Foundry agents.",
-      "version": "0.3.2",
+      "version": "0.3.3",
       "keywords": [
         "agentops",
         "evaluation",

CHANGELOG.md

@@ -5,9 +5,73 @@ This format follows [Keep a Changelog](https://keepachangelog.com/) and adheres
 
 ## [Unreleased]
 
-## [0.3.2] - 2026-05-31
+## [0.3.3] - 2026-05-31
+
+### Changed
+- **Runtime dependencies now have upper bounds so a future SDK major release
+  cannot silently break installs.** `pyproject.toml` previously declared every
+  Azure-SDK dependency with only a lower bound (e.g. `azure-ai-projects>=2.0.1`),
+  so `pip install agentops-accelerator` could resolve `azure-ai-projects 3.x`
+  the day after that ships and break the agent-definition serialization (the
+  exact failure mode that produced the `invalid_payload — Required properties
+  ["kind"] are not present` regression below). Each Azure SDK dependency
+  (`azure-ai-projects`, `azure-ai-evaluation`, `azure-identity`, `azure-monitor-*`,
+  `azure-mgmt-*`) is now constrained to its current major. `pandas`, `fastapi`,
+  `uvicorn`, `httpx`, and `markdown` are similarly capped to their next major.
+  `cryptography` is intentionally left unbounded so security patches can flow
+  through without a coordinated AgentOps release. Lift any of these bounds via
+  an explicit PR that exercises the new SDK against `tests/`.
+
+- **`agentops workflow generate` now stamps the installed agentops version
+  into generated CI/CD templates instead of always installing from
+  `git+...@main`.** Every generated `agentops-pr.yml`, `agentops-deploy-*.yml`,
+  `agentops-watchdog.yml` (and their Azure DevOps pipeline equivalents) used to
+  contain `pip install "agentops-accelerator[...] @ git+https://github.com/Azure/agentops.git@main"`,
+  with no version pin and a stale "NOTE: pinned to GitHub main until the next
+  package release" comment. User CI runs were therefore non-reproducible: the
+  same workflow file pulled different agentops snapshots day to day, which is
+  how PO's recorded tutorial took a hard SDK regression mid-record. The
+  generator now writes a literal `==X.Y.Z` pin derived from the agentops version
+  currently installed on the machine running `agentops workflow generate` — so
+  a user who generates workflows against AgentOps `0.3.3` always installs
+  `agentops-accelerator==0.3.3` on every CI run, and `agentops-accelerator`
+  brings exact-major Azure SDKs along (per the upper bounds above). Editable
+  installs (versions carrying a local segment like `+gabcdef` or marked
+  `.devN`) keep the `@main` fallback so contributors testing template changes
+  still get a resolvable install. Existing user workflows are unaffected until
+  the user re-runs `agentops workflow generate --force` against a release of
+  AgentOps that ships this change.
 
 ### Fixed
+- **Doctor regression check no longer flags the previous PR run as "current"
+  in CI.** The results-history loader (`agent/sources/results_history.py`)
+  was reading the wrong fields from `results.json` and excluding
+  `.agentops/results/latest/` from the candidate list. Three coordinated
+  schema-alignment fixes restore correctness:
+  1. `_summarize` now reads top-level `aggregate_metrics` first (the field
+     the orchestrator actually writes, per `core/results.py`), then falls
+     back to legacy `metrics`/`run_metrics`. Previously the loader looked
+     only at the legacy fields, so every freshly-written local
+     `RunSummary` had `metrics = {}` and the regression check could never
+     see the current run's metrics.
+  2. `_summarize` now reads `summary.overall_passed` first when deriving
+     the `run_pass` flag, then falls back to the legacy `summary.run_pass`
+     / `metrics.run_pass` shapes.
+  3. `_summarize` now orders runs by `timestamp` → `finished_at` →
+     `started_at` → `created_at` → `summary.timestamp`. The previous list
+     omitted `finished_at`/`started_at`, which are the two fields
+     `results.json` actually contains, so every loaded run defaulted to
+     epoch-zero ordering.
+  4. `_collect_local_runs` now includes `.agentops/results/latest/` when it
+     is the only local results directory. In CI, generated workflows run
+     `agentops eval run --output .agentops/results/latest` and write
+     nowhere else; the old loader unconditionally skipped `latest/` for
+     dev-mode dedup, so in CI `local_runs` was always empty. With cloud
+     listing trailing behind by seconds (eventual consistency), the
+     regression check would then compute `latest = previous_run` and
+     blame the just-completed candidate's coherence/groundedness on the
+     prior PR. Dev-mode dedup is preserved: when a timestamped sibling
+     exists, `latest/` is still skipped.
 - **Prompt-agent deploy: `stage` no longer fails with `Required properties ["kind"] are not present` against `azure-ai-projects` 2.x.**
   `_copy_definition` previously called `.copy()` on the typed
   `PromptAgentDefinition` returned by `get_version`. In SDK 1.x that

plugins/agentops/package.json

@@ -2,7 +2,7 @@
   "name": "agentops-accelerator",
   "displayName": "AgentOps Accelerator — Skills for GitHub Copilot",
   "description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Accelerator and Microsoft Foundry agents.",
-  "version": "0.3.2",
+  "version": "0.3.3",
   "publisher": "AgentOpsAccelerator",
   "icon": "icon.png",
   "license": "MIT",

plugins/agentops/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "agentops-accelerator",
   "description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Accelerator and Microsoft Foundry agents.",
-  "version": "0.3.2",
+  "version": "0.3.3",
   "author": {
     "name": "AgentOps Accelerator",
     "url": "https://github.com/Azure/agentops"

pyproject.toml

@@ -12,29 +12,29 @@ dependencies = [
   "typer>=0.12,<1.0",
   "pydantic>=2,<3",
   "ruamel.yaml>=0.18,<1.0",
-  "azure-ai-projects>=2.0.1",
+  "azure-ai-projects>=2.0.1,<3.0",
 ]
 license = { file = "LICENSE" }
 
 [project.optional-dependencies]
 mcp = ["mcp>=1.0,<2"]
 foundry = [
-  "azure-ai-evaluation>=1.0",
-  "azure-identity>=1.17",
-  "azure-monitor-opentelemetry>=1.6",
-  "pandas>=2.0",
+  "azure-ai-evaluation>=1.0,<2.0",
+  "azure-identity>=1.17,<2.0",
+  "azure-monitor-opentelemetry>=1.6,<2.0",
+  "pandas>=2.0,<3.0",
 ]
 agent = [
-  "fastapi>=0.110",
-  "uvicorn[standard]>=0.30",
-  "httpx>=0.27",
+  "fastapi>=0.110,<1.0",
+  "uvicorn[standard]>=0.30,<1.0",
+  "httpx>=0.27,<1.0",
   "cryptography>=42",
-  "markdown>=3.6",
-  "azure-monitor-query>=1.3",
-  "azure-monitor-opentelemetry>=1.6",
-  "azure-identity>=1.17",
-  "azure-mgmt-cognitiveservices>=13.5",
-  "azure-mgmt-monitor>=6.0",
+  "markdown>=3.6,<4.0",
+  "azure-monitor-query>=1.3,<2.0",
+  "azure-monitor-opentelemetry>=1.6,<2.0",
+  "azure-identity>=1.17,<2.0",
+  "azure-mgmt-cognitiveservices>=13.5,<14.0",
+  "azure-mgmt-monitor>=6.0,<7.0",
 ]
 
 [project.scripts]
@@ -66,7 +66,7 @@ where = ["src"]
 
 [dependency-groups]
 dev = [
-    "azure-ai-evaluation>=1.0",
+    "azure-ai-evaluation>=1.0,<2.0",
     "mypy>=1.19.1",
     "pre-commit>=4.0",
     "pytest>=8.0",

src/agentops/agent/sources/results_history.py

@@ -74,7 +74,12 @@ def _summarize(path: Path) -> Optional[RunSummary]:
     if not isinstance(data, dict):
         return None
 
-    metrics_raw = data.get("metrics") or data.get("run_metrics") or {}
+    metrics_raw = (
+        data.get("aggregate_metrics")
+        or data.get("metrics")
+        or data.get("run_metrics")
+        or {}
+    )
     metrics: Dict[str, float] = {}
     if isinstance(metrics_raw, dict):
         for key, value in metrics_raw.items():
@@ -85,9 +90,11 @@ def _summarize(path: Path) -> Optional[RunSummary]:
 
     summary = data.get("summary") or {}
     run_pass: Optional[bool] = None
-    if isinstance(summary, dict) and "run_pass" in summary:
+    if isinstance(summary, dict) and "overall_passed" in summary:
+        run_pass = bool(summary["overall_passed"])
+    elif isinstance(summary, dict) and "run_pass" in summary:
         run_pass = bool(summary["run_pass"])
-    elif "run_pass" in metrics_raw:
+    elif isinstance(metrics_raw, dict) and "run_pass" in metrics_raw:
         try:
             run_pass = bool(float(metrics_raw["run_pass"]))
         except (TypeError, ValueError):
@@ -105,6 +112,8 @@ def _summarize(path: Path) -> Optional[RunSummary]:
 
     timestamp_raw = (
         data.get("timestamp")
+        or data.get("finished_at")
+        or data.get("started_at")
         or data.get("created_at")
         or (summary.get("timestamp") if isinstance(summary, dict) else None)
     )
@@ -184,14 +193,23 @@ def _collect_local_runs(
         return []
 
     candidates: List[Path] = []
+    latest_target: Optional[Path] = None
     for child in base.iterdir():
         if not child.is_dir():
             continue
+        target = child / "results.json"
+        if not target.is_file():
+            continue
         if child.name == "latest":
+            latest_target = target
             continue
-        target = child / "results.json"
-        if target.is_file():
-            candidates.append(target)
+        candidates.append(target)
+    # CI workflows write directly to `.agentops/results/latest/` with no
+    # timestamped sibling. Include the `latest/` entry only when it is the
+    # sole local result so dev-mode runs (which already have a timestamped
+    # sibling) are not double-counted.
+    if not candidates and latest_target is not None:
+        candidates.append(latest_target)
 
     summaries: List[RunSummary] = []
     for path in candidates:

src/agentops/cli/app.py

@@ -14,7 +14,7 @@
 from pathlib import Path
 from textwrap import wrap
 from collections.abc import Sequence
-from typing import Annotated, Optional
+from typing import Annotated, Any, Optional
 
 import typer
 
@@ -1554,7 +1554,8 @@ def _prompt(question: str, default: Optional[str]) -> str:
         }
 
         def _on_answer(field_name: str, value: str) -> None:
-            partial = WizardAnswers(**{field_name: value})
+            partial_kwargs: dict[str, Any] = {field_name: value}
+            partial = WizardAnswers(**partial_kwargs)
             try:
                 partial_result = apply_answers(
                     workspace,

src/agentops/services/cicd.py

@@ -27,6 +27,42 @@
 _CLOUD_EVAL_CONFIG_NAME = ".agentops.cloud.yaml"
 _CI_EVAL_OUTPUT = ".agentops/results/latest"
 
+# Git ref used by the dev/editable-install fallback when stamping the agentops
+# package version into generated CI/CD templates. Kept here so tests can target
+# the same constant without parsing template output.
+AGENTOPS_DEV_INSTALL_SPEC = " @ git+https://github.com/Azure/agentops.git@main"
+
+
+def _agentops_install_spec(version: str | None = None) -> str:
+    """Return the pip version-spec suffix used in generated workflows.
+
+    For a clean public release (e.g. ``0.3.2``, ``0.3.2.post1``, ``0.3.2rc1``)
+    we pin the exact version so user CI runs are reproducible regardless of
+    when they are triggered. For editable or dev installs (versions carrying
+    a local segment such as ``+gabcdef`` or marked as dev-releases) we fall
+    back to ``@main`` so contributors testing template changes still get a
+    resolvable install.
+    """
+
+    if version is None:
+        from agentops import __version__
+
+        resolved = __version__
+    else:
+        resolved = version
+    try:
+        from packaging.version import InvalidVersion, Version
+    except ImportError:  # pragma: no cover - packaging ships with pip/setuptools
+        return AGENTOPS_DEV_INSTALL_SPEC
+
+    try:
+        parsed = Version(resolved)
+    except InvalidVersion:
+        return AGENTOPS_DEV_INSTALL_SPEC
+
+    if parsed.local is not None or parsed.is_devrelease:
+        return AGENTOPS_DEV_INSTALL_SPEC
+    return f"=={resolved}"
 # CI/CD platforms supported by ``agentops workflow generate``.
 PLATFORMS: Tuple[str, ...] = ("github", "azure-devops")
 
@@ -648,7 +684,10 @@ def generate_cicd_workflows(
             continue
         seen.add(kind)
         result.kinds.append(kind)
-        substitutions: dict[str, str] = {"__DOCTOR_GATE__": doctor_gate}
+        substitutions: dict[str, str] = {
+            "__DOCTOR_GATE__": doctor_gate,
+            "__AGENTOPS_INSTALL_SPEC__": _agentops_install_spec(),
+        }
         eval_config = (
             "${{ inputs.config || 'agentops.yaml' }}"
             if platform == "github" and kind == "pr"

src/agentops/templates/pipelines/azuredevops/agentops-deploy-dev-azd.yml

@@ -74,7 +74,7 @@ __AILZ_PREFLIGHT_COMMAND__
               versionSpec: "3.11"
           - bash: |
               python -m pip install --upgrade pip
-              python -m pip install "agentops-accelerator[foundry] @ git+https://github.com/Azure/agentops.git@main"
+              python -m pip install "agentops-accelerator[foundry]__AGENTOPS_INSTALL_SPEC__"
             displayName: Install AgentOps Toolkit
 
 __EVAL_TASKS__

src/agentops/templates/pipelines/azuredevops/agentops-deploy-dev.yml

@@ -50,7 +50,7 @@ stages:
 
                 - bash: |
                     python -m pip install --upgrade pip
-                    python -m pip install "agentops-accelerator[foundry] @ git+https://github.com/Azure/agentops.git@main"
+                    python -m pip install "agentops-accelerator[foundry]__AGENTOPS_INSTALL_SPEC__"
                   displayName: Install AgentOps Toolkit
 
 __EVAL_TASKS__

src/agentops/templates/pipelines/azuredevops/agentops-deploy-prod-azd.yml

@@ -74,7 +74,7 @@ __AILZ_PREFLIGHT_COMMAND__
               versionSpec: "3.11"
           - bash: |
               python -m pip install --upgrade pip
-              python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/Azure/agentops.git@main"
+              python -m pip install "agentops-accelerator[foundry,agent]__AGENTOPS_INSTALL_SPEC__"
             displayName: Install AgentOps Toolkit
 
 __EVAL_TASKS__

src/agentops/templates/pipelines/azuredevops/agentops-deploy-prod.yml

@@ -48,7 +48,7 @@ stages:
 
                 - bash: |
                     python -m pip install --upgrade pip
-                    python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/Azure/agentops.git@main"
+                    python -m pip install "agentops-accelerator[foundry,agent]__AGENTOPS_INSTALL_SPEC__"
                   displayName: Install AgentOps Toolkit
 
 __EVAL_TASKS__

src/agentops/templates/pipelines/azuredevops/agentops-deploy-prompt-agent.yml

@@ -48,7 +48,7 @@ stages:
               scriptLocation: inlineScript
               inlineScript: |
                 python -m pip install --upgrade pip
-                python -m pip install "agentops-accelerator[foundry] @ git+https://github.com/Azure/agentops.git@main"
+                python -m pip install "agentops-accelerator[foundry]__AGENTOPS_INSTALL_SPEC__"
                 python -m agentops.pipeline.prompt_deploy stage \
                   --config "$(AGENTOPS_CONFIG)" \
                   --environment "$(TARGET_ENVIRONMENT)" \
@@ -93,7 +93,7 @@ stages:
 
                 - bash: |
                     python -m pip install --upgrade pip
-                    python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/Azure/agentops.git@main"
+                    python -m pip install "agentops-accelerator[foundry,agent]__AGENTOPS_INSTALL_SPEC__"
                   displayName: Install AgentOps Toolkit
 
 __EVAL_TASKS__
@@ -153,7 +153,7 @@ __EVAL_TASKS__
                     versionSpec: "3.11"
 
                 - bash: |
-                    python -m pip install "agentops-accelerator @ git+https://github.com/Azure/agentops.git@main"
+                    python -m pip install "agentops-accelerator__AGENTOPS_INSTALL_SPEC__"
                     python -m agentops.pipeline.prompt_deploy summarize \
                       --deployment ".agentops/deployments/foundry-agent.json" \
                       --environment "$(TARGET_ENVIRONMENT)"

src/agentops/templates/pipelines/azuredevops/agentops-deploy-qa-azd.yml

@@ -74,7 +74,7 @@ __AILZ_PREFLIGHT_COMMAND__
               versionSpec: "3.11"
           - bash: |
               python -m pip install --upgrade pip
-              python -m pip install "agentops-accelerator[foundry] @ git+https://github.com/Azure/agentops.git@main"
+              python -m pip install "agentops-accelerator[foundry]__AGENTOPS_INSTALL_SPEC__"
             displayName: Install AgentOps Toolkit
 
 __EVAL_TASKS__

src/agentops/templates/pipelines/azuredevops/agentops-deploy-qa.yml

@@ -46,7 +46,7 @@ stages:
 
                 - bash: |
                     python -m pip install --upgrade pip
-                    python -m pip install "agentops-accelerator[foundry] @ git+https://github.com/Azure/agentops.git@main"
+                    python -m pip install "agentops-accelerator[foundry]__AGENTOPS_INSTALL_SPEC__"
                   displayName: Install AgentOps Toolkit
 
 __EVAL_TASKS__