Skip to content

[codex] integrate extended-data package fabric#6

Open
jbdevprimary wants to merge 287 commits into
mainfrom
codex/extended-data-tiered-fabric
Open

[codex] integrate extended-data package fabric#6
jbdevprimary wants to merge 287 commits into
mainfrom
codex/extended-data-tiered-fabric

Conversation

@jbdevprimary

Copy link
Copy Markdown
Contributor

Summary

  • Consolidates extended-data into a tiered Python package with first-class primitives, containers, IO, inputs, logging, connectors, secrets, and workflow surface.
  • Adds connector catalog metadata, lazy optional SDK loading, response data decoding, recursive Extended* container promotion, and a full-package CI/release gate.
  • Aligns the secrets connector CLI wrapper with the standalone secrets-sync command contract.

Validation

  • uv run --extra tests pytest tests
  • uv run --extra typing mypy src/extended_data
  • uvx ruff check src tests examples README.md docs/package-surface.md
  • uv build

Notes

  • This is intentionally a clean major-version break; no old package compatibility shims are preserved.

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 223 files, which is 73 over the limit of 150.

To get a review, narrow the scope:
• coderabbit review --type committed # exclude uncommitted changes
• coderabbit review --dir # limit to a subdirectory
• coderabbit review --base # compare against a closer base

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5c0262e5-0e61-4752-8394-f577d8e8ff05

📥 Commits

Reviewing files that changed from the base of the PR and between 138ebf5 and 9aaadce.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (223)
  • .github/workflows/ci.yml
  • .github/workflows/release.yml
  • README.md
  • docs/PUBLISHING_CHECKLIST.md
  • docs/package-surface.md
  • examples/connectors/README.md
  • examples/connectors/basic_aws.py
  • examples/connectors/basic_google.py
  • examples/connectors/basic_secrets.py
  • examples/connectors/langchain_tools.py
  • examples/connectors/mcp_server.py
  • examples/core/README.md
  • examples/core/basic_usage.py
  • examples/core/composed_workflows.py
  • examples/core/file_operations.py
  • examples/core/serialization.py
  • examples/core/string_transformations.py
  • examples/inputs/README.md
  • examples/inputs/basic_usage.py
  • examples/inputs/decorator_api.py
  • examples/inputs/encoding_decoding.py
  • examples/logging/README.md
  • examples/logging/exit_run_formatting.py
  • examples/logging/markers_and_storage.py
  • pyproject.toml
  • src/extended_data/__init__.py
  • src/extended_data/cli.py
  • src/extended_data/connectors/__init__.py
  • src/extended_data/connectors/_optional.py
  • src/extended_data/connectors/ai_tools.py
  • src/extended_data/connectors/anthropic/__init__.py
  • src/extended_data/connectors/anthropic/tools.py
  • src/extended_data/connectors/aws/__init__.py
  • src/extended_data/connectors/aws/_diagnostics.py
  • src/extended_data/connectors/aws/codedeploy.py
  • src/extended_data/connectors/aws/organizations.py
  • src/extended_data/connectors/aws/s3.py
  • src/extended_data/connectors/aws/sso.py
  • src/extended_data/connectors/aws/tools.py
  • src/extended_data/connectors/base.py
  • src/extended_data/connectors/cli.py
  • src/extended_data/connectors/cloud_params.py
  • src/extended_data/connectors/connectors.py
  • src/extended_data/connectors/cursor/__init__.py
  • src/extended_data/connectors/cursor/tools.py
  • src/extended_data/connectors/github/__init__.py
  • src/extended_data/connectors/github/_diagnostics.py
  • src/extended_data/connectors/github/tools.py
  • src/extended_data/connectors/google/__init__.py
  • src/extended_data/connectors/google/_diagnostics.py
  • src/extended_data/connectors/google/billing.py
  • src/extended_data/connectors/google/cloud.py
  • src/extended_data/connectors/google/constants.py
  • src/extended_data/connectors/google/jules.py
  • src/extended_data/connectors/google/services.py
  • src/extended_data/connectors/google/tools.py
  • src/extended_data/connectors/google/workspace.py
  • src/extended_data/connectors/mcp.py
  • src/extended_data/connectors/meshy/README.md
  • src/extended_data/connectors/meshy/__init__.py
  • src/extended_data/connectors/meshy/animate.py
  • src/extended_data/connectors/meshy/animations.py
  • src/extended_data/connectors/meshy/base.py
  • src/extended_data/connectors/meshy/connector.py
  • src/extended_data/connectors/meshy/image3d.py
  • src/extended_data/connectors/meshy/jobs.py
  • src/extended_data/connectors/meshy/logging.py
  • src/extended_data/connectors/meshy/mcp.py
  • src/extended_data/connectors/meshy/persistence/repository.py
  • src/extended_data/connectors/meshy/persistence/vector_store.py
  • src/extended_data/connectors/meshy/retexture.py
  • src/extended_data/connectors/meshy/rigging.py
  • src/extended_data/connectors/meshy/text3d.py
  • src/extended_data/connectors/meshy/tools.py
  • src/extended_data/connectors/meshy/webhooks/handler.py
  • src/extended_data/connectors/meshy/webhooks/schemas.py
  • src/extended_data/connectors/registry.py
  • src/extended_data/connectors/secrets/__init__.py
  • src/extended_data/connectors/secrets/tools.py
  • src/extended_data/connectors/slack/__init__.py
  • src/extended_data/connectors/slack/tools.py
  • src/extended_data/connectors/surface.py
  • src/extended_data/connectors/vault/__init__.py
  • src/extended_data/connectors/vault/tools.py
  • src/extended_data/connectors/zoom/__init__.py
  • src/extended_data/connectors/zoom/tools.py
  • src/extended_data/containers/__init__.py
  • src/extended_data/containers/factory.py
  • src/extended_data/containers/mappings.py
  • src/extended_data/containers/sequences.py
  • src/extended_data/containers/strings.py
  • src/extended_data/import_utils.py
  • src/extended_data/inputs/__main__.py
  • src/extended_data/inputs/decorators.py
  • src/extended_data/io/__init__.py
  • src/extended_data/io/base64.py
  • src/extended_data/io/exporters.py
  • src/extended_data/io/files.py
  • src/extended_data/io/importers.py
  • src/extended_data/logging/logging.py
  • src/extended_data/logging/utils.py
  • src/extended_data/primitives/__init__.py
  • src/extended_data/primitives/formats/__init__.py
  • src/extended_data/primitives/formats/_normalization.py
  • src/extended_data/primitives/formats/errors.py
  • src/extended_data/primitives/formats/hcl.py
  • src/extended_data/primitives/formats/json.py
  • src/extended_data/primitives/formats/toml.py
  • src/extended_data/primitives/formats/yaml/__init__.py
  • src/extended_data/primitives/formats/yaml/constructors.py
  • src/extended_data/primitives/formats/yaml/dumpers.py
  • src/extended_data/primitives/formats/yaml/loaders.py
  • src/extended_data/primitives/formats/yaml/representers.py
  • src/extended_data/primitives/formats/yaml/tag_classes.py
  • src/extended_data/primitives/formats/yaml/utils.py
  • src/extended_data/primitives/introspection.py
  • src/extended_data/primitives/mappings.py
  • src/extended_data/primitives/matching.py
  • src/extended_data/primitives/numbers.py
  • src/extended_data/primitives/redaction.py
  • src/extended_data/primitives/sequences.py
  • src/extended_data/primitives/serialization.py
  • src/extended_data/primitives/splitting.py
  • src/extended_data/primitives/state.py
  • src/extended_data/primitives/string_transforms.py
  • src/extended_data/primitives/strings.py
  • src/extended_data/primitives/transformations/__init__.py
  • src/extended_data/primitives/transformations/numbers/__init__.py
  • src/extended_data/primitives/transformations/numbers/notation.py
  • src/extended_data/primitives/transformations/numbers/words.py
  • src/extended_data/primitives/transformations/strings/__init__.py
  • src/extended_data/primitives/transformations/strings/inflection.py
  • src/extended_data/primitives/types.py
  • src/extended_data/secrets/__init__.py
  • src/extended_data/secrets/tools.py
  • src/extended_data/workflows/__init__.py
  • tests/connectors/meshy/conftest.py
  • tests/connectors/meshy/test_animations.py
  • tests/connectors/meshy/test_jobs.py
  • tests/connectors/meshy/test_meshy_base.py
  • tests/connectors/meshy/test_meshy_logging.py
  • tests/connectors/meshy/test_meshy_mcp.py
  • tests/connectors/meshy/test_repository.py
  • tests/connectors/meshy/test_task_ids.py
  • tests/connectors/meshy/test_tools.py
  • tests/connectors/meshy/test_vector_store.py
  • tests/connectors/meshy/test_webhooks.py
  • tests/connectors/test_ai_tools.py
  • tests/connectors/test_anthropic.py
  • tests/connectors/test_anthropic_tools.py
  • tests/connectors/test_aws_codedeploy.py
  • tests/connectors/test_aws_connector.py
  • tests/connectors/test_aws_organizations.py
  • tests/connectors/test_aws_s3.py
  • tests/connectors/test_aws_sso.py
  • tests/connectors/test_aws_tools.py
  • tests/connectors/test_base.py
  • tests/connectors/test_cli.py
  • tests/connectors/test_cloud_params.py
  • tests/connectors/test_connector_payload_contracts.py
  • tests/connectors/test_connectors.py
  • tests/connectors/test_cursor.py
  • tests/connectors/test_cursor_tools.py
  • tests/connectors/test_github_connector.py
  • tests/connectors/test_github_payload_contract.py
  • tests/connectors/test_github_tools.py
  • tests/connectors/test_github_workflow_builder.py
  • tests/connectors/test_google_activity.py
  • tests/connectors/test_google_billing.py
  • tests/connectors/test_google_cloud.py
  • tests/connectors/test_google_connector.py
  • tests/connectors/test_google_jules.py
  • tests/connectors/test_google_services.py
  • tests/connectors/test_google_tools.py
  • tests/connectors/test_google_workspace.py
  • tests/connectors/test_mcp.py
  • tests/connectors/test_optional_dependencies.py
  • tests/connectors/test_secrets.py
  • tests/connectors/test_slack_connector.py
  • tests/connectors/test_slack_tools.py
  • tests/connectors/test_tool_framework_adapters.py
  • tests/connectors/test_tool_frameworks.py
  • tests/connectors/test_tool_payload_contracts.py
  • tests/connectors/test_vault_connector.py
  • tests/connectors/test_vault_tools.py
  • tests/connectors/test_zoom_connector.py
  • tests/connectors/test_zoom_tools.py
  • tests/core/test_base64_utils.py
  • tests/core/test_containers.py
  • tests/core/test_export_utils.py
  • tests/core/test_file_data_type.py
  • tests/core/test_hcl2_utils.py
  • tests/core/test_import_utils.py
  • tests/core/test_integration_workflows.py
  • tests/core/test_json_utils.py
  • tests/core/test_list_data_type.py
  • tests/core/test_map_data_type.py
  • tests/core/test_matcher_utils.py
  • tests/core/test_number_transformations.py
  • tests/core/test_package_cli.py
  • tests/core/test_package_surface.py
  • tests/core/test_redaction.py
  • tests/core/test_release_hygiene.py
  • tests/core/test_serialization_utils.py
  • tests/core/test_splitter_utils.py
  • tests/core/test_stack_utils.py
  • tests/core/test_state_utils.py
  • tests/core/test_string_data_type.py
  • tests/core/test_string_transformations.py
  • tests/core/test_toml_utils.py
  • tests/core/test_type_utils.py
  • tests/core/test_workflows.py
  • tests/core/test_yaml_utils.py
  • tests/core/transformations/numbers/test_notation.py
  • tests/core/transformations/numbers/test_words.py
  • tests/core/transformations/strings/test_inflection.py
  • tests/examples/test_safe_examples.py
  • tests/inputs/test_decorators.py
  • tests/inputs/test_main.py
  • tests/logging/integration/test_lifecycle_logging.py
  • tests/logging/test_exit_run.py
  • tests/logging/test_logging.py
  • tests/logging/test_properties.py

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/extended-data-tiered-fabric

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Comment thread src/extended_data/connectors/cli.py Fixed

@amazon-q-developer amazon-q-developer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

I've reviewed this major refactoring PR that consolidates extended-data into a tiered Python package. The changes are extensive (128 files, 2669 additions, 592 deletions) and represent a significant architectural improvement with first-class primitives, containers, IO, inputs, logging, connectors, secrets, and workflow surface.

Critical Issue Found

Logic Error in CLI (src/extended_data/connectors/cli.py lines 170-172):
The cmd_methods function computes a docstring but never uses or displays it. Lines 171-172 execute attr.__doc__.split("\n")[0].strip()[:50] without assigning the result to a variable or outputting it. This causes the "methods" command to silently iterate through methods but display nothing to the user.

Fix needed:

# Get first line of docstring
doc = attr.__doc__.split("\n")[0].strip()[:50] if attr.__doc__ else "No description"
_write_stdout(f"  {name:<30} {doc}")

Security & Architecture Assessment

Positive aspects:

  • The VendorConnectorBase class provides good security primitives including rate limiting, automatic retries, and proper error handling
  • The SecretsConnector properly handles both native and CLI fallback modes for the secrets sync functionality
  • Type annotations are comprehensive throughout the codebase
  • The connector registry pattern provides good separation of concerns with lazy loading of optional dependencies

No blocking security vulnerabilities found in the reviewed files, but the logic error in the CLI blocks functionality.

Recommendation

Please fix the critical logic error in cmd_methods before merging. The rest of the changes appear well-structured and properly implemented. The architectural consolidation improves maintainability and the tiered package structure provides clear separation of concerns.


You can now have the agent implement changes and create commits directly on your pull request's source branch. Simply comment with /q followed by your request in natural language to ask the agent to make changes.


⚠️ This PR contains more than 30 files. Amazon Q is better at reviewing smaller PRs, and may miss issues in larger changesets.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new 'Tier 2' container layer to the extended-data package, including ExtendedDict, ExtendedList, ExtendedSet, and ExtendedString classes that provide ergonomic, chainable methods for data manipulation. It also refactors the package structure to separate pure primitives (Tier 1) from these containers (Tier 2) and higher-order processors (Tier 3), while updating the ConnectorFabric and various vendor connectors to support these new containers. The review identified a critical issue in ExtendedDict where the UserDict.update method bypasses the overridden __setitem__, preventing proper promotion of nested data; an actionable fix was provided to override update to ensure all data is correctly processed through the extend_data factory.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/extended_data/containers/mappings.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants