[codex] integrate extended-data package fabric

.github/workflows/ci.yml

@@ -26,7 +26,8 @@ jobs:
           version: "0.11.19"
           enable-cache: true
       - run: uv sync --python 3.13 --extra tests --extra typing
-      - run: uvx ruff check src tests
-      - run: uv run pytest tests/core tests/logging tests/inputs
-      - run: uv run pytest tests/connectors/test_cloud_params.py tests/connectors/test_connectors.py tests/connectors/test_secrets.py tests/connectors/test_cli.py tests/connectors/test_mcp.py tests/connectors/meshy/test_models.py
+      - run: uv run --with pip-audit==2.10.0 pip-audit --skip-editable
+      - run: uvx ruff check src tests examples README.md docs/package-surface.md
+      - run: uv run mypy src/extended_data
+      - run: uv run pytest tests
       - run: uv build

.github/workflows/release.yml

@@ -44,5 +44,10 @@ jobs:
         with:
           version: "0.11.19"
           enable-cache: true
+      - run: uv sync --python 3.13 --extra tests --extra typing
+      - run: uv run --with pip-audit==2.10.0 pip-audit --skip-editable
+      - run: uvx ruff check src tests examples README.md docs/package-surface.md
+      - run: uv run mypy src/extended_data
+      - run: uv run pytest tests
       - run: uv build
       - run: uv publish --trusted-publishing always

docs/PUBLISHING_CHECKLIST.md

@@ -0,0 +1,74 @@
+# Publishing Checklist
+
+`extended-data` releases are automated from `main` with release-please and
+PyPI trusted publishing. Do not hand-edit versions, changelog entries, release
+tags, or GitHub releases during the normal release path.
+
+## Release Model
+
+- `release-please` owns version detection, changelog updates, release PRs, and
+  Git tags.
+- The package name is `extended-data`; PyPI publication uses the tighter
+  `extended-data` distribution name.
+- The release workflow publishes only after release-please reports that a
+  release was created.
+- The PyPI job uses OIDC trusted publishing through `uv publish`; no PyPI token
+  should be stored in repository secrets for the normal path.
+
+## Maintainer Preflight
+
+Run these before merging a release PR or manually dispatching release workflow
+diagnostics:
+
+```bash
+uv sync --extra tests --extra typing
+uv run --with pip-audit==2.10.0 pip-audit --skip-editable
+uv run ruff check .
+uv run mypy src/extended_data
+uv run pytest
+uv build
+```
+
+## Workflow Hygiene
+
+- Keep `.github/workflows/*.yml` actions pinned to exact commit SHAs.
+- Update adjacent version comments when refreshing action SHAs.
+- Use `gh` to verify latest stable action releases before changing pins.
+- Keep top-level `permissions: {}` and grant only job-scoped permissions.
+
+Current workflow action pins:
+
+| Action | Stable version | Commit SHA |
+| --- | --- | --- |
+| `actions/checkout` | `v6.0.3` | `df4cb1c069e1874edd31b4311f1884172cec0e10` |
+| `actions/setup-python` | `v6.2.0` | `a309ff8b426b58ec0e2a45f0f869d46889d02405` |
+| `astral-sh/setup-uv` | `v8.2.0` | `fac544c07dec837d0ccb6301d7b5580bf5edae39` |
+| `googleapis/release-please-action` | `v5.0.0` | `45996ed1f6d02564a971a2fa1b5860e934307cf7` |
+
+## Publishing Flow
+
+1. Land normal feature, fix, docs, and maintenance commits using Conventional
+   Commit prefixes.
+2. Let the release workflow open or update the release-please PR.
+3. Review the release PR for the expected changelog and manifest updates.
+4. Merge the release PR.
+5. Confirm the release workflow created the GitHub release and published to
+   PyPI through trusted publishing.
+6. Verify the package can be installed from PyPI:
+
+```bash
+python -m pip install extended-data
+python -c "import extended_data; print(extended_data.__version__)"
+```
+
+## Manual Repairs
+
+Manual tags or PyPI uploads are repair paths, not the release process. If a
+release workflow fails after release-please creates a tag:
+
+1. Keep the failed tag intact while diagnosing unless the release is proven
+   unrecoverable.
+2. Prefer rerunning the failed workflow job.
+3. If a bad GitHub release was published, delete only the bad artifacts needed
+   for repair.
+4. Document the repair in the PR or release notes.

examples/connectors/README.md

@@ -1,24 +1,39 @@
-# Examples
+# Connector Examples
 
-This directory contains working examples demonstrating how to use extended-data.
+This directory contains working examples for `extended_data.connectors` and the
+registered integrations that hang off `ConnectorFabric`.
+
+Connector examples assume the major-version `extended-data` contract: external
+data payloads are promoted into Tier 2 containers at connector boundaries.
+Callers can use `ExtendedDict`, `ExtendedList`, and `ExtendedString` methods on
+decoded API, file, and SDK-shaped results, then call `to_builtin()` only when a
+plain Python payload is needed for serialization or SDK handoff.
+The direct AI-tool functions follow that same payload contract; only the
+framework factory helpers return plain framework tool objects.
 
 ## Quick Start
 
 Install extended-data with the extras you need:
 
 ```bash
 # Install with all connectors
-pip install extended-data[all]
+pip install "extended-data[all]"
 
 # Or install specific connectors
-pip install extended-data[aws,google,meshy]
+pip install "extended-data[aws,google,meshy]"
 
 # For AI framework integration
-pip install extended-data[langchain]
-pip install extended-data[crewai]
+pip install "extended-data[langchain]"
+
+# CrewAI adapters require a user-managed CrewAI install. extended-data does not
+# currently publish a CrewAI extra because current CrewAI releases pull
+# vulnerable chromadb versions transitively.
 
 # For the Meshy MCP server
-pip install extended-data[meshy,mcp]
+pip install "extended-data[meshy,mcp]"
+
+# For SecretSync pipeline inspection and dry-run syncs
+pip install "extended-data[secrets]"
 ```
 
 ## Examples
@@ -28,6 +43,7 @@ pip install extended-data[meshy,mcp]
 - [`basic_aws.py`](basic_aws.py) - AWS connector with Organizations and S3
 - [`basic_google.py`](basic_google.py) - Google Cloud connector with Workspace and Billing
 - [`basic_meshy.py`](basic_meshy.py) - Meshy AI 3D generation
+- [`basic_secrets.py`](basic_secrets.py) - SecretSync pipeline config inspection and dry-run execution
 
 ### AI Agent Integration
 
@@ -49,6 +65,10 @@ export GOOGLE_SERVICE_ACCOUNT='{"type": "service_account", ...}'
 # Meshy AI
 export MESHY_API_KEY="msy_your_key"
 
+# SecretSync
+export VAULT_ADDR="https://vault.example.com"
+export AWS_REGION="us-east-1"
+
 # For LangChain examples
 export ANTHROPIC_API_KEY="sk-ant-..."
 ```
@@ -57,8 +77,11 @@ export ANTHROPIC_API_KEY="sk-ant-..."
 
 ```bash
 # Run any example
-python examples/basic_meshy.py
+uv run python examples/connectors/basic_meshy.py
 
 # Run with debug logging
-LOGLEVEL=DEBUG python examples/basic_meshy.py
+LOGLEVEL=DEBUG uv run python examples/connectors/basic_meshy.py
+
+# Run the SecretSync bridge against a pipeline config
+uv run python examples/connectors/basic_secrets.py pipeline.yaml
 ```

examples/connectors/basic_aws.py

@@ -27,27 +27,22 @@ def main() -> int:
         return 1
 
     try:
-        from extended_data.connectors import AWSConnector, AWSConnectorFull
+        from extended_data.connectors import AWSConnector
     except ImportError:
         print("Error: Could not import extended_data.connectors. Install with: pip install extended-data[aws]")
         return 1
 
-    # Basic connector - just session management
-    print("Creating basic AWS connector...")
-    AWSConnector()
-    print("Basic connector created successfully.")
-
-    # Full connector with all operations
-    print("\nCreating full AWS connector...")
-    full_connector = AWSConnectorFull()
-    print("Full connector created successfully.")
+    print("Creating AWS connector...")
+    connector = AWSConnector()
+    print("AWS connector created successfully.")
 
     # List S3 buckets
     print("\n--- S3 Buckets ---")
     try:
-        buckets = full_connector.list_buckets()
-        for bucket in buckets[:5]:  # Show first 5
-            print(f"  Bucket: {bucket}")
+        buckets = connector.list_s3_buckets()
+        for bucket_name, bucket in list(buckets.items())[:5]:  # Show first 5
+            created = bucket.get("creation_date") or bucket.get("CreationDate")
+            print(f"  Bucket: {bucket_name} ({created})")
         if len(buckets) > 5:
             print(f"  ... and {len(buckets) - 5} more buckets")
     except Exception as e:
@@ -56,9 +51,10 @@ def main() -> int:
     # List organization accounts (if using Organizations)
     print("\n--- Organization Accounts ---")
     try:
-        accounts = full_connector.get_accounts()
-        for account in accounts[:5]:
-            print(f"  Account: {account}")
+        accounts = connector.get_accounts()
+        for account_id, account in list(accounts.items())[:5]:
+            name = account.get("name") or account.get("Name") or account_id
+            print(f"  Account: {account_id} ({name})")
         if len(accounts) > 5:
             print(f"  ... and {len(accounts) - 5} more accounts")
     except Exception as e:

examples/connectors/basic_google.py

@@ -26,25 +26,19 @@ def main() -> int:
         return 1
 
     try:
-        from extended_data.connectors import GoogleConnector, GoogleConnectorFull
+        from extended_data.connectors import GoogleConnector
     except ImportError:
         print("Error: Could not import extended_data.connectors. Install with: pip install extended-data[google]")
         return 1
 
-    # Basic connector
-    print("Creating basic Google connector...")
-    GoogleConnector()
-    print("Basic connector created successfully.")
-
-    # Full connector with all operations
-    print("\nCreating full Google connector...")
-    full_connector = GoogleConnectorFull()
-    print("Full connector created successfully.")
+    print("Creating Google connector...")
+    connector = GoogleConnector()
+    print("Google connector created successfully.")
 
     # List projects
     print("\n--- Google Cloud Projects ---")
     try:
-        projects = full_connector.list_projects()
+        projects = connector.list_projects()
         for project in projects[:5]:
             print(f"  Project: {project}")
         if len(projects) > 5:
@@ -56,7 +50,7 @@ def main() -> int:
     if os.getenv("GOOGLE_DOMAIN"):
         print("\n--- Workspace Users ---")
         try:
-            users = full_connector.list_users()
+            users = connector.list_users()
             for user in users[:5]:
                 email = user.get("primaryEmail", "Unknown")
                 print(f"  User: {email}")

examples/connectors/basic_secrets.py

@@ -0,0 +1,70 @@
+#!/usr/bin/env python3
+"""Example: SecretSync connector usage.
+
+This example demonstrates the `extended-data[secrets]` bridge to the
+standalone `jbcom/secrets-sync` CLI.
+
+Requirements:
+    pip install extended-data[secrets]
+    secretsync must be installed on PATH
+
+Run:
+    uv run python examples/connectors/basic_secrets.py pipeline.yaml
+"""
+
+from __future__ import annotations
+
+import sys
+
+from pathlib import Path
+
+
+def main() -> int:
+    """Inspect a SecretSync pipeline config and run a dry-run through the CLI contract."""
+    try:
+        from extended_data import OutputFormat, SecretsConnector, SyncOptions
+    except ImportError:
+        print("Error: Could not import SecretSync support. Install with: pip install extended-data[secrets]")
+        return 1
+
+    config_path = Path(sys.argv[1] if len(sys.argv) > 1 else "pipeline.yaml")
+    connector = SecretsConnector()
+
+    print(f"Inspecting SecretSync config: {config_path}")
+    config_info = connector.get_config_info(str(config_path))
+    if not config_info["valid"]:
+        print(f"Error: {config_info['error_message']}")
+        return 1
+
+    print(
+        "Config summary: "
+        f"{config_info['source_count']} source(s), "
+        f"{config_info['target_count']} target(s), "
+        f"merge store={config_info['has_merge_store']}",
+    )
+
+    if not connector.cli_available:
+        print("Error: secretsync CLI not available on PATH.")
+        print("Install jbcom/secrets-sync and re-run this example to exercise the dry-run contract.")
+        return 1
+
+    result = connector.run_pipeline(
+        str(config_path),
+        SyncOptions(dry_run=True, compute_diff=True, output_format=OutputFormat.JSON),
+    )
+
+    if not result["success"]:
+        print("Error: secretsync dry run failed.")
+        print("Run secretsync directly in a secure terminal for full diagnostics.")
+        print("The CLI must emit the stable `secretsync pipeline --output json` result envelope.")
+        return 1
+
+    print("Dry run completed successfully.")
+    if result["diff_output"]:
+        print("Diff output was returned by secretsync and is not printed because it may contain secret values.")
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

examples/connectors/langchain_tools.py

@@ -6,7 +6,7 @@
 
 Requirements:
     pip install extended-data[meshy,langchain]
-    pip install langchain-anthropic  # For Claude as the LLM
+    pip install langchain-anthropic langgraph  # For Claude as the LLM and agent loop
 
 Environment Variables:
     MESHY_API_KEY: Your Meshy API key
@@ -39,7 +39,7 @@ def main() -> int:
         from extended_data.connectors.meshy.tools import get_tools
     except ImportError:
         print("Error: Could not import required packages.")
-        print("Install with: pip install extended-data[meshy,langchain] langchain-anthropic")
+        print("Install with: pip install extended-data[meshy,langchain] langchain-anthropic langgraph")
         return 1
 
     # Get Meshy tools for LangChain

examples/connectors/mcp_server.py

@@ -13,7 +13,7 @@
 
 Usage:
     # Run the server (connects via stdio)
-    python examples/mcp_server.py
+    python examples/connectors/mcp_server.py
 
     # Or use the installed command
     meshy-mcp