fix: decouple oa-vm from the ML training stack via lazy package imports#263
Merged
Conversation
The oa-vm console script imports openadapt_evals.benchmarks.vm_cli, which triggered the package __init__ modules' eager `from openadapt_evals.agents import ...`. That cascaded into transformers/peft — dead weight for VM lifecycle management, and a hard crash under a NumPy 2 / stale-transformers environment (a module compiled against NumPy 1.x cannot run on NumPy 2.x), which broke `oa-vm` entirely. Resolve the re-exported public API of openadapt_evals and openadapt_evals.benchmarks lazily (PEP 562 module __getattr__), caching each name as a module global on first access. Importing either package is now light and touches no ML/vision dependency; agents/transformers load only when an agent-dependent name is actually used. Public API and object identity are unchanged. Adds a subprocess regression guard (TestVmCliStaysLight) asserting the oa-vm import path pulls in none of transformers/peft/torch/agents, plus a check that the lazy re-exports still resolve to the real objects. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
oa-vm(VM lifecycle CLI) crashed on startup:Root cause is an import-chain problem, not NumPy:
oa-vm's entry pointimports
openadapt_evals.benchmarks.vm_cli, which triggers the package__init__modules. Those eagerly didfrom openadapt_evals.agents import ...,cascading into
transformers/peft— the entire ML training stack — just tostart and stop VMs. Under a NumPy 2 / stale-transformers environment that
cascade hard-crashes, taking
oa-vmdown with it. Downgrading NumPy would be aworkaround; a VM CLI should not import the training stack at all.
Fix
Resolve the re-exported public API of
openadapt_evalsandopenadapt_evals.benchmarkslazily (PEP 562 module__getattr__), cachingeach name as a module global on first access. The repo already used this pattern
for
PolicyAgent/Azure; this extends it to the heavy re-exports.dependency.
agents/transformersload only when an agent-dependent name isactually accessed.
from openadapt_evals import BenchmarkAgentandoe.WAAAdapter is <direct import>both hold.Verification
oa-vm --helpandoa-vm pool-statusrun cleanly (both in the brokenanaconda env that reproduced the crash and in the
uvenv).TestVmCliStaysLightasserts theoa-vmimport path pulls in none of
transformers/peft/torch/agents, and thatlazy re-exports still resolve to the real objects.
(
test_demo_persistence.py::...test_demo_format_and_persistenceimports theoptional
openadapt_mldirectly without animportorskipguard — fails onmaintoo; flagged, not in scope here).🤖 Generated with Claude Code