diff --git a/AGENTS.md b/AGENTS.md index 2e5e771e..15d9d0ff 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,7 +17,7 @@ uv sync --group dev ```bash uv run pytest # run all tests uv run pytest tests/path/to/test_file.py::test_name # run a single test -uv run pytest --cov gems --cov-report xml # with coverage +uv run pytest --cov gems_craft --cov gems_runner --cov gems_craft_hybrid --cov-report xml # with coverage ``` **Lint & Format:** @@ -30,26 +30,21 @@ uv run mypy **Running:** ```bash # CLI entry point -gemspy \ - --model-libs path/to/lib1.yml path/to/lib2.yml \ - --components path/to/components.yml \ - --timeseries path/to/timeseries/ \ - --duration 8760 \ - --scenarios 1 +gemspy --study path/to/study_dir # Python API — directory-based study -from gems.study.folder import load_study -from gems.study.runner import run_study +from gems_runner.study.folder import load_study +from gems_runner.study.runner import run_study study = load_study(Path("path/to/study_dir")) # reads input/, model-libraries/, data-series/ run_study(Path("path/to/study_dir")) # loads study, solves, writes CSV to output/ # Python API — programmatic study -from gems.study import Study -from gems.simulation import build_problem, TimeBlock +from gems_runner.study import Study +from gems_runner.simulation import build_problem, TimeBlock study = Study(system=system, database=database) -problem = build_problem(study, TimeBlock(1, list(range(8760))), scenarios=1) +problem = build_problem(study, TimeBlock(1, list(range(8760))), scenario_ids=[0]) problem.solve(solver_name="highs") ``` @@ -59,13 +54,41 @@ The pipeline flows: **YAML input → parsing → model resolution → system ins An optional `optim-config.yml` activates decomposition: variables and constraints are split across a master problem and subproblems, with either sequential resolution or full Benders decomposition. -### Core Modules (`src/gems/`) +### Core Modules + +The source tree is split into three packages under `src/`: + +#### `gems_craft/` — I/O and data structures (no solver dependency) **`model/`** — Immutable model templates. - `Model`: defines component behavior (parameters, variables, constraints, ports) - `Library`: a collection of models, loaded from YAML - `Taxonomy` (`taxonomy.py`): categories naming the items a model must expose. Models opt in via `taxonomy-category`; `check_library_against_taxonomy` enforces conformance. +**`study/`** — Study schema and parsing. +- `SystemSchema` (`parsing.py`): Pydantic model for `system.yml` +- `ScenarioBuilder` (`scenario_builder.py`): maps MC scenarios to data-series column indices; + +**`optim_config/`** — Optional decomposition configuration. +- `OptimConfig` (`parsing.py`): top-level config loaded from `optim-config.yml` +- `ResolutionMode` (`parsing.py`): `FRONTAL` (default), `SEQUENTIAL_SUBPROBLEMS`, `PARALLEL_SUBPROBLEMS`, or `BENDERS_DECOMPOSITION` +- `ModelDecompositionConfig` (`parsing.py`): per-model assignment of variables/constraints/objective contributions to master or subproblems + +#### `gems_craft_hybrid/` — Hybrid study I/O (read/write only, no simulation) + +Extends `gems_craft` schemas for hybrid GEMS studies. **Hybrid studies cannot be simulated with GemsPy** — simulation support is not yet implemented. + +**`study/`** — `HybridSystemSchema(SystemSchema)`: adds `area-connections` and `thermal-capacity-connections`. +- Use `load_yaml_system(path, HybridSystemSchema)` and `write_yaml_system(system, path)` from `gems_craft` + +**`model/`** — `HybridPortTypeSchema(PortTypeSchema)`: adds `area-connection` and `thermal-capacity-connection`. `HybridLibrarySchema(LibrarySchema)`: overrides `port-types` with `HybridPortTypeSchema`. +- Use `load_yaml_library(path, HybridLibrarySchema)` and `write_yaml_library(library, path)` from `gems_craft` + +#### `gems_runner/` — Solver and execution (depends on `gems_craft`) + +**`model/`** — Resolves YAML schemas into runtime objects. +- `resolve_library`: validates and cross-links models across libraries + **`expression/`** — Mathematical expression language and AST. - `ExpressionNode`: base frozen dataclass for all expression tree nodes - Grammar is defined in `grammar/Expr.g4` and parsed via ANTLR4 (generated files live in `expression/parsing/antlr/` — do not edit directly) @@ -74,7 +97,7 @@ An optional `optim-config.yml` activates decomposition: variables and constraint **`study/`** — Study definition and instantiation. - `System` (`system.py`): resolved topology — graph of `Component`s, `PortRef`s, and `PortsConnection`s after library references are substituted - `Study` (`study.py`): dataclass pairing a `System` with a `DataBase`; validates that the database supplies every parameter required by the system -- `DataBase` (`data.py`): manages time-series and scenario data; +- `DataBase` (`data.py`): manages time-series and scenario data - `load_study` / `run_study` (`folder.py`): convenience functions for directory-based studies (`input/system.yml`, `input/model-libraries/`, `input/data-series/`) **`simulation/`** — Optimization problem construction and solving. @@ -85,11 +108,6 @@ An optional `optim-config.yml` activates decomposition: variables and constraint - `TimeBlock` (`time_block.py`): defines the temporal window for one solve - `SimulationTableBuilder` / `SimulationTableWriter` (`simulation_table.py`): result extraction as a flat pandas `DataFrame` -**`optim_config/`** — Optional decomposition configuration. -- `OptimConfig` (`parsing.py`): top-level config loaded from `optim-config.yml` -- `ResolutionMode` (`parsing.py`): `FRONTAL` (default), `SEQUENTIAL_SUBPROBLEMS`, `PARALLEL_SUBPROBLEMS`, or `BENDERS_DECOMPOSITION` -- `ModelDecompositionConfig` (`parsing.py`): per-model assignment of variables/constraints/objective contributions to master or subproblems - **`libs/`** — Resolves the path to bundled YAML model libraries shipped with the package. ### Key Design Patterns diff --git a/README.md b/README.md index 9bbfff30..08d03fc0 100644 --- a/README.md +++ b/README.md @@ -50,12 +50,12 @@ Given a study directory containing your `library.yml`, `system.yml` and timeseri ```python from pathlib import Path -from gems.study.folder import load_study -from gems.session import SimulationSession -from gems.optim_config import load_optim_config +from gems_runner.study.folder import load_study +from gems_runner.session import SimulationSession +from gems_craft.optim_config import load_yaml_optim_config study = load_study(Path("my_study")) -optim_config = load_optim_config(Path("my_study/input/optim-config.yml")) +optim_config = load_yaml_optim_config(Path("my_study/input/optim-config.yml")) session = SimulationSession(study=study, optim_config=optim_config) results = session.run() @@ -65,7 +65,7 @@ Or, in a single call: ```python from pathlib import Path -from gems.study.runner import run_study +from gems_runner.study.runner import run_study run_study(Path("my_study")) ``` diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index b901c7e6..2ccd7ce7 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -2,6 +2,31 @@ All notable changes to GemsPy are documented here. +## [2.0.0] - Unreleased + +### Changed + +- **Package split**: the `gems` package is replaced by three dedicated packages: + - `gems_craft`: I/O and data structures (YAML parsing/writing for libraries, systems, optim-config, scenario builder) — no solver dependency + - `gems_runner`: solver and execution logic (expression language, model resolution, simulation, session) + - `gems_craft_hybrid`: read/write support for hybrid GEMS studies (not simulable with GemsPy — use Antares Simulator) +- **Renamed functions**: all public YAML functions now follow `load_yaml_*` / `write_yaml_*` with `Path` inputs. Stream-based helpers are private (`_parse_yaml_*`). + - `parse_yaml_library` → `load_yaml_library` + - `parse_yaml_components` / `load_input_system` → `load_yaml_system` + - `load_optim_config` → `load_yaml_optim_config`, `write_optim_config` → `write_yaml_optim_config` + - `ScenarioBuilder.load` → `ScenarioBuilder.load_dat`, `ScenarioBuilder.dump` → `ScenarioBuilder.write_dat` +- **Generic schema support**: `load_yaml_system` and `load_yaml_library` accept an optional `schema` argument (TypeVar with default) to load subclass schemas without code duplication. + +### Added + +- **`gems_craft_hybrid`** package with: + - `HybridSystemSchema`: extends `SystemSchema` with `area-connections` and `thermal-capacity-connections` + - `HybridPortTypeSchema`: extends `PortTypeSchema` with `area-connection` and `thermal-capacity-connection` + - `HybridLibrarySchema`: extends `LibrarySchema` with `HybridPortTypeSchema` port-types +- **`version`** field on `LibrarySchema` + +--- + ## [0.1.2] - 2026-06-11 ### Added diff --git a/docs/agents/python-convention.md b/docs/agents/python-convention.md index 23919335..4efe19cb 100644 --- a/docs/agents/python-convention.md +++ b/docs/agents/python-convention.md @@ -9,8 +9,9 @@ is run with `disallow_untyped_defs = true` and `disallow_untyped_calls = true` (see `mypy.ini`). - **Dataclasses**: Prefer `@dataclass(frozen=True)` for value objects (expression nodes, model definitions). Mutability must be justified explicitly. -- **Pydantic**: Use `ConfigDict(alias_generator=to_camel)` or kebab-case alias generation for YAML - round-tripping; use Pydantic v2 APIs only. +- **Pydantic**: Use `ModifiedBaseModel` (from `gems_craft.utils`) which applies kebab-case alias + generation via `alias_generator=_to_kebab` and `extra="forbid"`. Use Pydantic v2 APIs only. + Serialize with `model_dump(by_alias=True, exclude_none=True, mode="json")` for YAML output. - **Naming**: - Classes: `PascalCase` - Functions / variables: `snake_case` — use descriptive names; avoid single-letter or two-letter @@ -28,7 +29,7 @@ Rules for automated agents (CI bots, AI coding assistants, Dependabot, etc.): 1. **Never auto-merge** to `main`; all changes require at least one human review. -2. **Do not edit generated files** under `src/gems/expression/parsing/antlr/`; regenerate them +2. **Do not edit generated files** under `src/gems_runner/expression/parsing/antlr/`; regenerate them from `grammar/Expr.g4` instead. 3. **Do not modify `pyproject.toml` version** manually; version bumps are handled via the `feat(release):` commit workflow. diff --git a/docs/agents/testing.md b/docs/agents/testing.md index d65ece6b..7d479422 100644 --- a/docs/agents/testing.md +++ b/docs/agents/testing.md @@ -14,6 +14,8 @@ in `tests/` sub-directories. No mocking of the solver—tests use real HiGHS cal | Unit — libraries | `tests/unittests/lib_parsing/` | Model library YAML parsing | | Unit — system | `tests/unittests/system/` | Model, network, and port object behaviour | | Unit — system parsing | `tests/unittests/system_parsing/` | System YAML parsing | +| Unit — gems_craft | `tests/unittests/gems_craft/` | Read/write roundtrips for system, library, optim-config, scenario builder | +| Unit — gems_craft_hybrid | `tests/unittests/gems_craft_hybrid/` | Hybrid system and library parsing (HybridSystemSchema, HybridLibrarySchema) | | Unit — scenario builder | `tests/unittests/scenario_builder/` | Scenario and time-series builder | | Integration | `tests/unittests/simulation/` | Full problem build + solve on small networks | | End-to-end — functional | `tests/e2e/functional/` | Cross-cutting tests: library/system combinations, stochastic, investment, scenario builder | diff --git a/docs/getting-started.md b/docs/getting-started.md index dcf86dae..8115e13e 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -86,11 +86,9 @@ The second category of input files mentioned above corresponds to system files. ~~~yaml system: model-libraries: basic - nodes: + components: - id: N model: basic.node - - components: - id: G1 model: basic.generator properties: @@ -168,12 +166,12 @@ If your inputs are organised in a study directory (see [Reading input files](use ~~~ python from pathlib import Path -from gems.study.folder import load_study -from gems.session import SimulationSession -from gems.optim_config import load_optim_config +from gems_runner.study.folder import load_study +from gems_runner.session import SimulationSession +from gems_craft.optim_config import load_yaml_optim_config study = load_study(Path("my_study")) -optim_config = load_optim_config(Path("my_study/input/optim-config.yml")) +optim_config = load_yaml_optim_config(Path("my_study/input/optim-config.yml")) session = SimulationSession(study=study, optim_config=optim_config) results = session.run() @@ -183,7 +181,7 @@ Or, in a single call: ~~~ python from pathlib import Path -from gems.study.runner import run_study +from gems_runner.study.runner import run_study run_study(Path("my_study")) ~~~ @@ -198,16 +196,13 @@ Here is the GemsPy syntax to read a test case described by ~~~ python from pathlib import Path -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import resolve_system, build_data_base - -with open("library.yml") as lib_file: - input_libraries = [parse_yaml_library(lib_file)] +from gems_craft.model.parsing import load_yaml_library +from gems_runner.model.resolve_library import resolve_library +from gems_craft.study.parsing import load_yaml_system +from gems_runner.study.resolve_components import resolve_system, build_data_base -with open("system.yml") as compo_file: - input_system = parse_yaml_components(compo_file) +input_libraries = [load_yaml_library(Path("library.yml"))] +input_system = load_yaml_system(Path("system.yml")) result_lib = resolve_library(input_libraries) system = resolve_system(input_system, result_lib) @@ -217,8 +212,8 @@ database = build_data_base(input_system, Path(series_dir)) ### Building the optimisation problem ~~~ python -from gems.study import Study -from gems.simulation import build_problem, TimeBlock +from gems_runner.study import Study +from gems_runner.simulation import build_problem, TimeBlock problem = build_problem( Study(system, database), diff --git a/docs/user-guide/building.md b/docs/user-guide/building.md index d20d2a0d..7646d873 100644 --- a/docs/user-guide/building.md +++ b/docs/user-guide/building.md @@ -10,7 +10,7 @@ The Pydantic schema classes used to describe systems programmatically follow the The syntax to build components with the GemsPy API is the following: ~~~ python -from gems.study.parsing import ComponentSchema, ComponentParameterSchema +from gems_craft.study.parsing import ComponentSchema, ComponentParameterSchema components = [] @@ -76,7 +76,7 @@ components.append( The syntax to build connections between components with the GemsPy API is the following: ~~~ python -from gems.study.parsing import PortConnectionsSchema +from gems_craft.study.parsing import PortConnectionsSchema connections = [] @@ -102,7 +102,7 @@ connections.append( ## Defining a SystemSchema ~~~ python -from gems.study.parsing import SystemSchema +from gems_craft.study.parsing import SystemSchema input_system = SystemSchema( components=components, @@ -110,4 +110,15 @@ input_system = SystemSchema( ) ~~~ -The `input_system` variable can then be used in the same way as when it was created using the [parse_yaml_components](inputs.md) method. +The `input_system` variable can then be used in the same way as when it was created using the [load_yaml_system](inputs.md) function. + +## Writing a SystemSchema to file + +Once you have built or modified a `SystemSchema`, you can save it as a `system.yml` file: + +~~~ python +from pathlib import Path +from gems_craft.study.parsing import write_yaml_system + +write_yaml_system(input_system, Path("my_study/input/system.yml")) +~~~ diff --git a/docs/user-guide/inputs.md b/docs/user-guide/inputs.md index 76514d87..c56778b1 100644 --- a/docs/user-guide/inputs.md +++ b/docs/user-guide/inputs.md @@ -23,7 +23,7 @@ my_study/ ~~~ python from pathlib import Path -from gems.study.folder import load_study +from gems_runner.study.folder import load_study study = load_study(Path("my_study")) ~~~ @@ -63,17 +63,14 @@ Duplicate ids for properties are rejected. ### Loading the library and the system ~~~ python -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import resolve_system, build_data_base +from gems_craft.model.parsing import load_yaml_library +from gems_runner.model.resolve_library import resolve_library +from gems_craft.study.parsing import load_yaml_system +from gems_runner.study.resolve_components import resolve_system, build_data_base from pathlib import Path -with open("simple_library.yml") as lib_file: - input_libraries = [parse_yaml_library(lib_file)] - -with open("system_example.yml") as compo_file: - input_system = parse_yaml_components(compo_file) +input_libraries = [load_yaml_library(Path("simple_library.yml"))] +input_system = load_yaml_system(Path("system_example.yml")) result_lib = resolve_library(input_libraries) system = resolve_system(input_system, result_lib) @@ -93,10 +90,100 @@ database = build_data_base(input_system, Path(series_dir)) Once you have `system` and `database`, wrap them in a `Study`: ~~~ python -from gems.study import Study +from gems_runner.study import Study study = Study(system=system, database=database) ~~~ This `study` object can then be passed directly to [`build_problem()`](optimisation.md) or `SimulationSession`. + +--- + +## Hybrid studies (`gems_craft_hybrid`) + +Hybrid GEMS studies extend the standard format with additional fields. + +> **Limitation:** hybrid studies cannot be simulated with GemsPy. The +> `gems_craft_hybrid` package only provides reading and writing of hybrid +> files. To simulate hybrid studies, use +> [Antares Simulator](https://antares-simulator.org/). + +### Hybrid system + +A hybrid `system.yml` extends the standard format with two optional sections: + +- `area-connections`: maps component ports to areas. +- `thermal-capacity-connections`: maps component ports to Antares thermal clusters, each identified by `area` and `cluster-id`. + +~~~ yaml +system: + components: + - id: G + model: basic.generator + parameters: + - id: cost + value: 30 + - id: p_max + value: 100 + connections: + - component1: N + port1: injection_port + component2: G + port2: injection_port + area-connections: + - component: G + port: injection_port + area: fr + thermal-capacity-connections: + - component: G + port: injection_port + thermal-component: + area: fr + cluster-id: nuclear1 +~~~ + +Load and write hybrid systems with `load_yaml_system` / `write_yaml_system` +passing `HybridSystemSchema` as the schema: + +~~~ python +from gems_craft.study.parsing import load_yaml_system, write_yaml_system +from gems_craft_hybrid.study.parsing import HybridSystemSchema + +system = load_yaml_system(Path("system.yml"), HybridSystemSchema) +write_yaml_system(system, Path("output/system.yml")) +~~~ + +### Hybrid library + +Each port-type of an hybrid library YAML may carry two additional sub-fields: + +- `area-connection`: maps port roles to port fields for area coupling (`injection-to-balance`, `spillage-bound`, `unsupplied-energy-bound`). +- `thermal-capacity-connection`: identifies the port field that carries thermal capacity (`capacity-field`). + +~~~ yaml +library: + id: my_lib + version: "1.0" + port-types: + - id: flow + fields: + - id: flow + area-connection: + injection-to-balance: flow + spillage-bound: flow + unsupplied-energy-bound: + thermal-capacity-connection: + capacity-field: flow +~~~ + +Load and write hybrid libraries with `load_yaml_library` / `write_yaml_library` +passing `HybridLibrarySchema` as the schema: + +~~~ python +from gems_craft.model.parsing import load_yaml_library, write_yaml_library +from gems_craft_hybrid.model.parsing import HybridLibrarySchema + +library = load_yaml_library(Path("lib.yml"), HybridLibrarySchema) +write_yaml_library(library, Path("output/lib.yml")) +~~~ diff --git a/docs/user-guide/optim-config.md b/docs/user-guide/optim-config.md index b6c7d708..e63b203e 100644 --- a/docs/user-guide/optim-config.md +++ b/docs/user-guide/optim-config.md @@ -10,7 +10,7 @@ By convention the file lives at: ``` my_study/ └── input/ - └── optim-config.yml ← read automatically by run_study() / load_optim_config() + └── optim-config.yml ← read automatically by run_study() / load_yaml_optim_config() ``` When the file is absent, `run_study()` and `SimulationSession` use the defaults @@ -158,7 +158,7 @@ scenario-scope: - "8-10" ~~~ -GemsPy reads and validates the playlist eagerly when `load_optim_config()` is +GemsPy reads and validates the playlist eagerly when `load_yaml_optim_config()` is called, so any I/O or format errors surface immediately at load time. **Rules:** @@ -331,8 +331,8 @@ You can load, inspect, and build the config programmatically: ~~~ python from pathlib import Path -from gems.optim_config import ( - load_optim_config, +from gems_craft.optim_config import ( + load_yaml_optim_config, OptimConfig, ResolutionConfig, ResolutionMode, @@ -342,7 +342,7 @@ from gems.optim_config import ( ) # Load from file (returns None if the file does not exist) -config = load_optim_config(Path("my_study/input/optim-config.yml")) +config = load_yaml_optim_config(Path("my_study/input/optim-config.yml")) # Build programmatically — inline form (scenarios 0–9) config = OptimConfig( @@ -362,8 +362,8 @@ config_pf = OptimConfig( ) # Pass to SimulationSession -from gems.session import SimulationSession -from gems.study.folder import load_study +from gems_runner.session import SimulationSession +from gems_runner.study.folder import load_study study = load_study(Path("my_study")) session = SimulationSession(study=study, optim_config=config) diff --git a/docs/user-guide/optimisation.md b/docs/user-guide/optimisation.md index 6f1cd029..e88ce8af 100644 --- a/docs/user-guide/optimisation.md +++ b/docs/user-guide/optimisation.md @@ -18,7 +18,7 @@ the problem, and writes results to `output//`. ~~~ python from pathlib import Path -from gems.study.runner import run_study +from gems_runner.study.runner import run_study run_study(Path("my_study")) ~~~ @@ -38,12 +38,12 @@ and lets you inspect the returned `SimulationTable` in memory. ~~~ python from pathlib import Path -from gems.study.folder import load_study -from gems.session import SimulationSession -from gems.optim_config import load_optim_config +from gems_runner.study.folder import load_study +from gems_runner.session import SimulationSession +from gems_craft.optim_config import load_yaml_optim_config study = load_study(Path("my_study")) -optim_config = load_optim_config(Path("my_study/input/optim-config.yml")) +optim_config = load_yaml_optim_config(Path("my_study/input/optim-config.yml")) session = SimulationSession(study=study, optim_config=optim_config) results = session.run() # returns a SimulationTable @@ -65,9 +65,9 @@ multiple solve calls. ~~~ python from pathlib import Path -from gems.study import Study -from gems.study.folder import load_study -from gems.simulation import build_problem, TimeBlock +from gems_runner.study import Study +from gems_runner.study.folder import load_study +from gems_runner.simulation import build_problem, TimeBlock study = load_study(Path("my_study")) diff --git a/docs/user-guide/outputs.md b/docs/user-guide/outputs.md index d82b943d..90ec5462 100644 --- a/docs/user-guide/outputs.md +++ b/docs/user-guide/outputs.md @@ -6,12 +6,12 @@ ~~~ python from pathlib import Path -from gems.study.folder import load_study -from gems.session import SimulationSession -from gems.optim_config import load_optim_config +from gems_runner.study.folder import load_study +from gems_runner.session import SimulationSession +from gems_craft.optim_config import load_yaml_optim_config study = load_study(Path("my_study")) -optim_config = load_optim_config(Path("my_study/input/optim-config.yml")) +optim_config = load_yaml_optim_config(Path("my_study/input/optim-config.yml")) session = SimulationSession(study=study, optim_config=optim_config) results = session.run() # SimulationTable @@ -22,7 +22,7 @@ results = session.run() # SimulationTable When using `build_problem()` directly, build the table from the solved problem: ~~~ python -from gems.simulation.simulation_table import SimulationTableBuilder +from gems_runner.simulation.simulation_table import SimulationTableBuilder results = SimulationTableBuilder().build(problem) ~~~ @@ -83,3 +83,20 @@ results.to_parquet(Path("output/")) # writes Parquet files results.to_netcdf(Path("output/")) # writes a NetCDF file ds = results.to_dataset() # returns an xarray Dataset ~~~ + +--- + +## Reloading previously saved results + +`SimulationTable` wraps a plain pandas DataFrame, so reloading a CSV is straightforward: + +~~~ python +import pandas as pd +from gems_craft.simulation_table.simulation_table import SimulationTable + +df = pd.read_csv(Path("output/20240101T1200/simulation_table_20240101T1200.csv")) +results = SimulationTable(df) + +# then use the normal accessor API +results.component("gen_de").output("generation").value(scenario_index=0) +~~~ diff --git a/docs/user-guide/scenario-builder.md b/docs/user-guide/scenario-builder.md index af7b1e98..8c30a451 100644 --- a/docs/user-guide/scenario-builder.md +++ b/docs/user-guide/scenario-builder.md @@ -103,7 +103,7 @@ attaches the resulting `ScenarioBuilder` to the returned `Study` object: ~~~ python from pathlib import Path -from gems.study.folder import load_study +from gems_runner.study.folder import load_study study = load_study(Path("my_study")) # study.scenario_builder is populated from modeler-scenariobuilder.dat @@ -119,10 +119,10 @@ that passes MC scenario indices through unchanged. ~~~ python import numpy as np from pathlib import Path -from gems.study.scenario_builder import ScenarioBuilder +from gems_craft.study.scenario_builder import ScenarioBuilder # Load from file -sb = ScenarioBuilder.load(Path("my_study/input/data-series/modeler-scenariobuilder.dat")) +sb = ScenarioBuilder.load_dat(Path("my_study/input/data-series/modeler-scenariobuilder.dat")) # Resolve a batch of MC scenarios for the "wind" group mc_scenarios = np.array([0, 1, 2, 3]) # 0-based internally @@ -137,4 +137,12 @@ col_indices = sb.resolve_vectorized(None, mc_scenarios) !!! note `resolve_vectorized` works with **0-based** indices internally, even though the `.dat` file uses 1-based numbering. The conversion is handled - automatically by `ScenarioBuilder.load()`. + automatically by `ScenarioBuilder.load_dat()`. + +## Writing a scenario builder + +Use `write_dat()` to write a `ScenarioBuilder` back to disk in the standard `.dat` format: + +~~~ python +sb.write_dat(Path("my_study/input/data-series/modeler-scenariobuilder.dat")) +~~~ diff --git a/pyproject.toml b/pyproject.toml index e988fa21..9e3d37e9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "gemspy" -version = "0.1.2" +version = "2.0.0" description = "Python interpreter for GEMS: modelling and simulation of complex energy systems under uncertainty" readme = "README.md" license = { file = "LICENSE" } @@ -60,7 +60,7 @@ doc = [ ] [project.scripts] -gemspy = "gems.main.main:main_cli" +gemspy = "gems_runner.main.main:main_cli" [project.urls] Repository = "https://github.com/AntaresSimulatorTeam/GemsPy" @@ -108,7 +108,7 @@ log_cli = true [tool.mypy] mypy_path = "src" -packages = ["gems"] +packages = ["gems_craft", "gems_runner", "gems_craft_hybrid"] disallow_untyped_defs = true disallow_untyped_calls = true @@ -117,5 +117,5 @@ module = ["anytree.*", "linopy.*", "xarray.*", "antlr4.*", "pytest.*", "pydantic ignore_missing_imports = true [[tool.mypy.overrides]] -module = ["gems.expression.parsing.antlr.*"] +module = ["gems_runner.expression.parsing.antlr.*"] ignore_errors = true diff --git a/src/gems/__init__.py b/src/gems_craft/__init__.py similarity index 100% rename from src/gems/__init__.py rename to src/gems_craft/__init__.py diff --git a/src/gems/model/parsing.py b/src/gems_craft/model/parsing.py similarity index 70% rename from src/gems/model/parsing.py rename to src/gems_craft/model/parsing.py index 7cf1cf23..1bad1c34 100644 --- a/src/gems/model/parsing.py +++ b/src/gems_craft/model/parsing.py @@ -11,20 +11,13 @@ # This file is part of the Antares project. import typing from dataclasses import dataclass -from typing import List, Optional +from pathlib import Path +from typing import List, Optional, Type, TypeVar, overload from pydantic import ConfigDict, Field, ValidationError -from yaml import safe_load +from yaml import safe_dump, safe_load -from gems.utils import ModifiedBaseModel - - -def parse_yaml_library(input: typing.TextIO) -> "LibrarySchema": - tree = safe_load(input) - try: - return LibrarySchema.model_validate(tree["library"]) - except ValidationError as e: - raise ValueError(f"An error occurred during parsing: {e}") +from gems_craft.utils import ModifiedBaseModel class ParameterSchema(ModifiedBaseModel): @@ -114,3 +107,41 @@ class LibrarySchema(ModifiedBaseModel): port_types: List[PortTypeSchema] = Field(default_factory=list) models: List[ModelSchema] = Field(default_factory=list) description: Optional[str] = None + version: Optional[str] = None + + +_L = TypeVar("_L", bound=LibrarySchema) + + +@overload +def _parse_yaml_library(input: typing.TextIO) -> LibrarySchema: ... +@overload +def _parse_yaml_library(input: typing.TextIO, schema: Type[_L]) -> _L: ... +def _parse_yaml_library( + input: typing.TextIO, schema: Type[LibrarySchema] = LibrarySchema +) -> LibrarySchema: + tree = safe_load(input) + try: + return schema.model_validate(tree["library"]) + except ValidationError as e: + raise ValueError(f"An error occurred during parsing: {e}") + + +@overload +def load_yaml_library(path: Path) -> LibrarySchema: ... +@overload +def load_yaml_library(path: Path, schema: Type[_L]) -> _L: ... +def load_yaml_library( + path: Path, schema: Type[LibrarySchema] = LibrarySchema +) -> LibrarySchema: + with path.open() as f: + return _parse_yaml_library(f, schema) + + +def write_yaml_library(library: LibrarySchema, path: Path) -> None: + data = { + "library": library.model_dump(by_alias=True, exclude_none=True, mode="json") + } + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w") as f: + safe_dump(data, f, allow_unicode=True, sort_keys=False) diff --git a/src/gems/model/taxonomy.py b/src/gems_craft/model/taxonomy.py similarity index 97% rename from src/gems/model/taxonomy.py rename to src/gems_craft/model/taxonomy.py index 569b1ff7..eba774b3 100644 --- a/src/gems/model/taxonomy.py +++ b/src/gems_craft/model/taxonomy.py @@ -17,8 +17,8 @@ import yaml from pydantic import Field -from gems.model.parsing import LibrarySchema, ModelSchema -from gems.utils import ModifiedBaseModel +from gems_craft.model.parsing import LibrarySchema, ModelSchema +from gems_craft.utils import ModifiedBaseModel class TaxonomyItem(ModifiedBaseModel): diff --git a/src/gems/optim_config/__init__.py b/src/gems_craft/optim_config/__init__.py similarity index 85% rename from src/gems/optim_config/__init__.py rename to src/gems_craft/optim_config/__init__.py index 8fb8a328..7346bcc5 100644 --- a/src/gems/optim_config/__init__.py +++ b/src/gems_craft/optim_config/__init__.py @@ -18,8 +18,8 @@ ScenarioScopeConfig, SolverOptionsConfig, TimeScopeConfig, - load_optim_config, - validate_optim_config, + load_yaml_optim_config, + write_yaml_optim_config, ) __all__ = [ @@ -30,6 +30,6 @@ "ScenarioScopeConfig", "SolverOptionsConfig", "TimeScopeConfig", - "load_optim_config", - "validate_optim_config", + "load_yaml_optim_config", + "write_yaml_optim_config", ] diff --git a/src/gems/optim_config/parsing.py b/src/gems_craft/optim_config/parsing.py similarity index 54% rename from src/gems/optim_config/parsing.py rename to src/gems_craft/optim_config/parsing.py index 27af9930..45f9e67a 100644 --- a/src/gems/optim_config/parsing.py +++ b/src/gems_craft/optim_config/parsing.py @@ -15,7 +15,7 @@ import warnings from enum import Enum from pathlib import Path -from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Union +from typing import Any, Dict, List, Optional, Set, Union from pydantic import ( Field, @@ -24,23 +24,9 @@ field_validator, model_validator, ) -from yaml import safe_load - -from gems.expression.expression import ( - AdditionNode, - BinaryOperatorNode, - ExpressionNode, - MaxNode, - MinNode, - UnaryOperatorNode, - VariableNode, -) -from gems.utils import ModifiedBaseModel +from yaml import safe_dump, safe_load -if TYPE_CHECKING: - from gems.model.model import Model - from gems.study.scenario_builder import ScenarioBuilder - from gems.study.system import System +from gems_craft.utils import ModifiedBaseModel class ElementLocation(str, Enum): @@ -180,7 +166,7 @@ class ScenarioScopeConfig(ModifiedBaseModel): and/or ``"a-b"`` range strings. - **File** (``playlist_file``): path to a flat JSON array of 0-based integers, resolved relative to ``optim-config.yml`` by - ``load_optim_config()``. + ``load_yaml_optim_config()``. ``exclude`` is optional and compatible with *both* forms. It subtracts a set of scenarios from the base set using the same entry format. Entries @@ -195,7 +181,7 @@ class ScenarioScopeConfig(ModifiedBaseModel): The resolved list is computed lazily on first access to ``scenario_ids`` and cached for the lifetime of the object. - ``load_optim_config()`` triggers eager resolution so that file I/O + ``load_yaml_optim_config()`` triggers eager resolution so that file I/O errors surface at load time. """ @@ -285,7 +271,14 @@ class OptimConfig(ModifiedBaseModel): models: List[ModelOptimConfig] = Field(default_factory=list) -def load_optim_config(config_path: Path) -> Optional[OptimConfig]: +def write_yaml_optim_config(config: OptimConfig, path: Path) -> None: + data = config.model_dump(by_alias=True, exclude_none=True, mode="json") + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w") as f: + safe_dump(data, f, allow_unicode=True, sort_keys=False) + + +def load_yaml_optim_config(config_path: Path) -> Optional[OptimConfig]: """Load and fully resolve an ``optim-config.yml`` file. Returns ``None`` if the file does not exist. @@ -314,199 +307,3 @@ def load_optim_config(config_path: Path) -> Optional[OptimConfig]: # exactly once and any I/O or format errors surface at load time. _ = config.scenario_scope.scenario_ids return config - - -_MASTER_LOCS: Set[ElementLocation] = { - ElementLocation.MASTER, - ElementLocation.MASTER_AND_SUBPROBLEMS, -} - - -def _collect_variable_names(expr: ExpressionNode) -> Set[str]: - """Recursively collect all variable names referenced in an expression.""" - if isinstance(expr, VariableNode): - return {expr.name} - if isinstance(expr, (AdditionNode, MaxNode, MinNode)): - result: Set[str] = set() - for operand in expr.operands: - result |= _collect_variable_names(operand) - return result - if isinstance(expr, UnaryOperatorNode): - return _collect_variable_names(expr.operand) - if isinstance(expr, BinaryOperatorNode): - return _collect_variable_names(expr.left) | _collect_variable_names(expr.right) - return set() - - -def _check_oob_constraint_ids( - oob_processing: OutOfBoundsProcessingConfig, - model: "Model", - model_config_id: str, - errors: List[str], -) -> None: - for constraint_config in oob_processing.constraints: - if ( - constraint_config.id not in model.constraints - and constraint_config.id not in model.binding_constraints - ): - errors.append( - f"Out-of-bounds constraint '{constraint_config.id}' not found in model '{model_config_id}'" - ) - - -def _check_id_existence( - decomposition: ModelDecompositionConfig, - model: "Model", - model_config_id: str, - errors: List[str], -) -> None: - for variable_config in decomposition.variables: - if variable_config.id not in model.variables: - errors.append( - f"Variable '{variable_config.id}' not found in model '{model_config_id}'" - ) - for constraint_config in decomposition.constraints: - if ( - constraint_config.id not in model.constraints - and constraint_config.id not in model.binding_constraints - ): - errors.append( - f"Constraint '{constraint_config.id}' not found in model '{model_config_id}'" - ) - obj_keys = set(model.objective_contributions or {}) - for obj_config in decomposition.objective_contributions: - if obj_config.id not in obj_keys: - errors.append( - f"Objective-contribution '{obj_config.id}' not found in model '{model_config_id}'" - ) - - -def _check_master_variables_not_time_dependent( - decomposition: ModelDecompositionConfig, - model: "Model", - model_config_id: str, - errors: List[str], -) -> None: - """Variables assigned to master or master-and-subproblems must not depend on time.""" - for variable_config in decomposition.variables: - if ( - variable_config.location in _MASTER_LOCS - and variable_config.id in model.variables - ): - if model.variables[variable_config.id].structure.time: - errors.append( - f"Variable '{variable_config.id}' in model '{model_config_id}' is time-dependent " - f"but is assigned to '{variable_config.location.value}'; " - "master variables must not depend on time" - ) - - -def _check_master_constraints_use_master_variables( - decomposition: ModelDecompositionConfig, - model: "Model", - model_config_id: str, - errors: List[str], -) -> None: - """Constraints in master must only reference variables in master or master-and-subproblems.""" - master_var_ids = { - variable_config.id - for variable_config in decomposition.variables - if variable_config.location in _MASTER_LOCS - and variable_config.id in model.variables - } - for constraint_config in decomposition.constraints: - if constraint_config.location == ElementLocation.MASTER: - constraint = model.constraints.get( - constraint_config.id - ) or model.binding_constraints.get(constraint_config.id) - if constraint is not None: - for var_name in sorted( - _collect_variable_names(constraint.expression) - master_var_ids - ): - errors.append( - f"Constraint '{constraint_config.id}' in model '{model_config_id}' references variable '{var_name}' " - "which is not assigned to master or master-and-subproblems" - ) - - -def _check_master_objectives_use_master_variables( - decomposition: ModelDecompositionConfig, - model: "Model", - model_config_id: str, - errors: List[str], -) -> None: - """Objective contributions in master must only reference variables in master or master-and-subproblems.""" - master_var_ids = { - variable_config.id - for variable_config in decomposition.variables - if variable_config.location in _MASTER_LOCS - and variable_config.id in model.variables - } - obj_contribs = model.objective_contributions or {} - for obj_config in decomposition.objective_contributions: - if obj_config.location == ElementLocation.MASTER: - expr = obj_contribs.get(obj_config.id) - if expr is not None: - for var_name in sorted(_collect_variable_names(expr) - master_var_ids): - errors.append( - f"Objective contribution '{obj_config.id}' in model '{model_config_id}' references variable '{var_name}' " - "which is not assigned to master or master-and-subproblems" - ) - - -def validate_optim_config( - config: OptimConfig, - system: "System", - scenario_builder: Optional["ScenarioBuilder"] = None, -) -> None: - """Cross-validate optim-config entries against the resolved system. - - Performs the following checks: - - - Every model ID referenced in ``config.models`` exists in the system. - - Master variables are time-independent. - - Master constraints and objective contributions only reference variables - assigned to ``master`` or ``master-and-subproblems``. - - If ``scenario_builder`` is provided, every scenario index in - ``config.scenario_scope.scenario_ids`` is defined for every scenario - group in the builder. - - Raises ``ValueError`` listing all violations found. - """ - models_in_system = {c.model.id: c.model for c in system.all_components} - errors: List[str] = [] - - if scenario_builder is not None: - errors.extend( - scenario_builder.validate_mc_scenarios(config.scenario_scope.scenario_ids) - ) - - for model_config in config.models: - model = models_in_system.get(model_config.id) - if model is None: - errors.append(f"Model '{model_config.id}' not found in system") - else: - if model_config.model_decomposition is not None: - decomposition = model_config.model_decomposition - _check_id_existence(decomposition, model, model_config.id, errors) - _check_master_variables_not_time_dependent( - decomposition, model, model_config.id, errors - ) - _check_master_constraints_use_master_variables( - decomposition, model, model_config.id, errors - ) - _check_master_objectives_use_master_variables( - decomposition, model, model_config.id, errors - ) - if model_config.out_of_bounds_processing is not None: - _check_oob_constraint_ids( - model_config.out_of_bounds_processing, - model, - model_config.id, - errors, - ) - - if errors: - raise ValueError( - f"Errors in optim config file:\n" + "\n".join(f" - {e}" for e in errors) - ) diff --git a/src/gems_craft/simulation_table/simulation_table.py b/src/gems_craft/simulation_table/simulation_table.py new file mode 100644 index 00000000..125b9e61 --- /dev/null +++ b/src/gems_craft/simulation_table/simulation_table.py @@ -0,0 +1,174 @@ +from enum import Enum +from pathlib import Path +from typing import Any, Optional, Union, cast + +import pandas as pd +import xarray as xr + + +class OutputView: + """A Time × Scenario pivot for one (component, output) combination. + + Obtain via ``SimulationTable.component(...).output(...)``. + """ + + def __init__(self, df: pd.DataFrame) -> None: + # df: index = absolute-time-index, columns = scenario-index + self._df = df + + @property + def data(self) -> pd.DataFrame: + """Return the underlying Time × Scenario DataFrame.""" + return self._df + + def value( + self, + time_index: Optional[int] = None, + scenario_index: Optional[int] = None, + ) -> Union[pd.DataFrame, "pd.Series[Any]", float]: + """Return results filtered by time and/or scenario index. + + Called with no arguments returns the full Time × Scenario DataFrame. + Called with one argument returns a ``pd.Series``: + - ``value(scenario_index=s)`` → Series indexed by absolute-time-index + - ``value(time_index=t)`` → Series indexed by scenario-index + Called with both arguments returns a scalar ``float``. + """ + if time_index is None and scenario_index is None: + return self._df + if time_index is not None and scenario_index is not None: + return float(cast(Any, self._df.loc[time_index, scenario_index])) + if time_index is not None: + return self._df.loc[time_index] # Series over scenarios + return self._df[scenario_index] # Series over time + + def __repr__(self) -> str: + return repr(self._df) + + +class ComponentView: + """Filtered view of simulation results for one component. + + Obtain via ``SimulationTable.component(...)``. + """ + + def __init__(self, df: pd.DataFrame) -> None: + self._df = df + + def output(self, output_id: str) -> OutputView: + """Return an OutputView for the given output name.""" + col_output = SimulationColumns.OUTPUT.value + col_time = SimulationColumns.ABSOLUTE_TIME_INDEX.value + col_scenario = SimulationColumns.SCENARIO_INDEX.value + col_value = SimulationColumns.VALUE.value + + filtered = self._df[self._df[col_output] == output_id].copy() + # Dimension-independent outputs store None for the missing index. + # Fill with 0 so the pivot is always well-formed and the accessor + # API (value(time_index=t, scenario_index=s)) keeps working. + filtered[col_time] = filtered[col_time].fillna(0) + filtered[col_scenario] = filtered[col_scenario].fillna(0) + pivot = filtered.pivot_table( + index=col_time, + columns=col_scenario, + values=col_value, + aggfunc="first", + ) + pivot.index.name = col_time + pivot.columns.name = col_scenario + return OutputView(pivot) + + +class SimulationTable: + """Wrapper around the raw simulation results DataFrame. + + Provides a fluent accessor API:: + + st = SimulationTableBuilder().build(problem) + + # Full Time × Scenario DataFrame + st.component("gen_1").output("p").value() + + # Scalar at a specific time and scenario + st.component("gen_1").output("p").value(time_index=0, scenario_index=0) + + # Time series for scenario 0 + st.component("gen_1").output("p").value(scenario_index=0) + + # Scenario distribution at time step 3 + st.component("gen_1").output("p").value(time_index=3) + + The underlying long-format DataFrame is accessible via the ``data`` property. + """ + + def __init__(self, df: pd.DataFrame, table_id: str = "") -> None: + self._df = df + self.table_id = table_id + + @property + def data(self) -> pd.DataFrame: + """Return the underlying long-format DataFrame.""" + return self._df + + def component(self, component_id: str) -> ComponentView: + """Return a ComponentView filtered to the given component ID.""" + mask = self._df[SimulationColumns.COMPONENT.value] == component_id + return ComponentView(self._df[mask]) + + def to_csv(self, output_dir: Path) -> Path: + output_dir = Path(output_dir) + output_dir.mkdir(parents=True, exist_ok=True) + path = output_dir / f"simulation_table_{self.table_id}.csv" + self._df.to_csv(path, index=False) + return path + + def to_parquet(self, output_dir: Path) -> Path: + output_dir = Path(output_dir) + output_dir.mkdir(parents=True, exist_ok=True) + path = output_dir / f"simulation_table_{self.table_id}.parquet" + self._df.to_parquet(path, index=False) + return path + + def to_netcdf(self, output_dir: Path) -> Path: + output_dir = Path(output_dir) + output_dir.mkdir(parents=True, exist_ok=True) + path = output_dir / f"simulation_table_{self.table_id}.nc" + self.to_dataset().to_netcdf(path) + return path + + def to_dataset(self) -> xr.Dataset: + """Return simulation results as an xr.Dataset. + + Each output variable becomes a DataArray with dimensions + (component, absolute-time-index, scenario-index). + Scalar rows without component/time/scenario (e.g. objective-value) + are stored as zero-dimensional variables. + """ + df = self._df + col_comp = SimulationColumns.COMPONENT.value + col_out = SimulationColumns.OUTPUT.value + col_time = SimulationColumns.ABSOLUTE_TIME_INDEX.value + col_scen = SimulationColumns.SCENARIO_INDEX.value + col_val = SimulationColumns.VALUE.value + + main = df.dropna(subset=[col_comp, col_time, col_scen]) + indexed = main.set_index([col_comp, col_time, col_scen, col_out])[col_val] + unstacked = indexed.unstack(col_out) + ds = xr.Dataset.from_dataframe(unstacked) + + scalars = df[df[col_comp].isna() & df[col_time].isna()] + for _, row in scalars.iterrows(): + ds[row[col_out]] = xr.DataArray(float(row[col_val])) + + return ds + + +class SimulationColumns(str, Enum): + BLOCK = "block" + COMPONENT = "component" + OUTPUT = "output" + ABSOLUTE_TIME_INDEX = "absolute-time-index" + BLOCK_TIME_INDEX = "block-time-index" + SCENARIO_INDEX = "scenario-index" + VALUE = "value" + BASIS_STATUS = "basis-status" diff --git a/src/gems/libs/__init__.py b/src/gems_craft/study/__init__.py similarity index 85% rename from src/gems/libs/__init__.py rename to src/gems_craft/study/__init__.py index b08d36bf..71cd06d8 100644 --- a/src/gems/libs/__init__.py +++ b/src/gems_craft/study/__init__.py @@ -10,6 +10,4 @@ # # This file is part of the Antares project. -from pathlib import Path - -lib_path = Path(__file__).parent +from .scenario_builder import ScenarioBuilder diff --git a/src/gems/study/parsing.py b/src/gems_craft/study/parsing.py similarity index 64% rename from src/gems/study/parsing.py rename to src/gems_craft/study/parsing.py index 9bc9ae71..f0028c62 100644 --- a/src/gems/study/parsing.py +++ b/src/gems_craft/study/parsing.py @@ -13,31 +13,12 @@ import argparse from dataclasses import dataclass from pathlib import Path -from typing import List, Optional, TextIO, Union +from typing import List, Optional, TextIO, Type, TypeVar, Union, overload from pydantic import Field, ValidationError -from yaml import safe_load +from yaml import safe_dump, safe_load -from gems.utils import ModifiedBaseModel - - -def load_input_system(input_study: Path) -> "SystemSchema": - try: - with input_study.open() as f: - return SystemSchema.model_validate(safe_load(f)) - except ValidationError as e: - raise ValueError(f"An error occurred during parsing: {e}") - - -def parse_yaml_components(input_study: TextIO) -> "SystemSchema": - tree = safe_load(input_study) - return SystemSchema.model_validate(tree["system"]) - - -class AreaConnectionsSchema(ModifiedBaseModel): - component: str - port: str - area: str +from gems_craft.utils import ModifiedBaseModel class PortConnectionsSchema(ModifiedBaseModel): @@ -73,7 +54,41 @@ class SystemSchema(ModifiedBaseModel): model_libraries: Optional[str] = None # Parsed but unused for now components: List[ComponentSchema] = Field(default_factory=list) connections: Optional[List[PortConnectionsSchema]] = None - area_connections: Optional[List[AreaConnectionsSchema]] = None + + +_S = TypeVar("_S", bound=SystemSchema) + + +@overload +def _parse_yaml_components(input_study: TextIO) -> SystemSchema: ... +@overload +def _parse_yaml_components(input_study: TextIO, schema: Type[_S]) -> _S: ... +def _parse_yaml_components( + input_study: TextIO, schema: Type[SystemSchema] = SystemSchema +) -> SystemSchema: + tree = safe_load(input_study) + return schema.model_validate(tree["system"]) + + +@overload +def load_yaml_system(path: Path) -> SystemSchema: ... +@overload +def load_yaml_system(path: Path, schema: Type[_S]) -> _S: ... +def load_yaml_system( + path: Path, schema: Type[SystemSchema] = SystemSchema +) -> SystemSchema: + try: + with path.open() as f: + return _parse_yaml_components(f, schema) + except (ValidationError, ValueError) as e: + raise ValueError(f"An error occurred during parsing: {e}") from e + + +def write_yaml_system(system: SystemSchema, path: Path) -> None: + data = {"system": system.model_dump(by_alias=True, exclude_none=True, mode="json")} + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w") as f: + safe_dump(data, f, allow_unicode=True, sort_keys=False) @dataclass(frozen=True) diff --git a/src/gems/study/scenario_builder.py b/src/gems_craft/study/scenario_builder.py similarity index 88% rename from src/gems/study/scenario_builder.py rename to src/gems_craft/study/scenario_builder.py index 7fd83a31..58dcd52f 100644 --- a/src/gems/study/scenario_builder.py +++ b/src/gems_craft/study/scenario_builder.py @@ -86,8 +86,23 @@ def validate_mc_scenarios(self, scenario_ids: List[int]) -> List[str]: ) return errors + def write_dat(self, path: Path) -> None: + """Write the scenario builder to a ``modeler-scenariobuilder.dat`` file. + + Each line follows the format:: + + group_name, mc_scenario = time_serie_number + + where ``time_serie_number`` is 1-based. + """ + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w") as f: + for group, arr in self._group_arrays.items(): + for mc_scenario, col_idx in enumerate(arr): + f.write(f"{group}, {mc_scenario} = {col_idx + 1}\n") + @classmethod - def load(cls, path: Path) -> "ScenarioBuilder": + def load_dat(cls, path: Path) -> "ScenarioBuilder": """Parse a ``modeler-scenariobuilder.dat`` file. Each non-blank, non-comment line must follow:: diff --git a/src/gems_craft/study/timeseries.py b/src/gems_craft/study/timeseries.py new file mode 100644 index 00000000..22028b56 --- /dev/null +++ b/src/gems_craft/study/timeseries.py @@ -0,0 +1,46 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. + +from pathlib import Path +from typing import Optional + +import pandas as pd + + +def load_ts_from_file( + timeseries_name: Optional[str], path_to_file: Optional[Path] +) -> pd.DataFrame: + if path_to_file is None or timeseries_name is None: + raise FileNotFoundError(f"File '{timeseries_name}' does not exist") + + base_path = path_to_file / timeseries_name + candidates = [base_path.with_suffix(".txt"), base_path.with_suffix(".tsv")] + + last_exc: Optional[Exception] = None + for candidate in candidates: + if not candidate.exists(): + continue + try: + sep = r"\s+" if candidate.suffix == ".txt" else "\t" + return pd.read_csv(candidate, header=None, sep=sep) + except Exception as e: + last_exc = e + break + + if last_exc is not None: + raise Exception( + f"An error has arrived when processing '{candidate}': {last_exc}" + ) + + raise FileNotFoundError( + f"File '{timeseries_name}.txt' or '{timeseries_name}.tsv' does not exist" + ) diff --git a/src/gems_craft/utils.py b/src/gems_craft/utils.py new file mode 100644 index 00000000..6a03a297 --- /dev/null +++ b/src/gems_craft/utils.py @@ -0,0 +1,32 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. + +""" +Module for technical utilities. +""" + +from pydantic import BaseModel, ConfigDict + +""" +Only _to_kebab adn ModifiedBaseModel are used in the project. +""" + + +# Design note: actual parsing and validation is delegated to pydantic models +def _to_kebab(snake: str) -> str: + return snake.replace("_", "-") + + +class ModifiedBaseModel(BaseModel): + model_config = ConfigDict( + alias_generator=_to_kebab, extra="forbid", populate_by_name=True + ) diff --git a/src/gems_craft_hybrid/__init__.py b/src/gems_craft_hybrid/__init__.py new file mode 100644 index 00000000..058c6b22 --- /dev/null +++ b/src/gems_craft_hybrid/__init__.py @@ -0,0 +1,11 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. diff --git a/src/gems_craft_hybrid/model/__init__.py b/src/gems_craft_hybrid/model/__init__.py new file mode 100644 index 00000000..058c6b22 --- /dev/null +++ b/src/gems_craft_hybrid/model/__init__.py @@ -0,0 +1,11 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. diff --git a/src/gems_craft_hybrid/model/parsing.py b/src/gems_craft_hybrid/model/parsing.py new file mode 100644 index 00000000..71154139 --- /dev/null +++ b/src/gems_craft_hybrid/model/parsing.py @@ -0,0 +1,37 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. + +from typing import List, Optional + +from pydantic import Field + +from gems_craft.model.parsing import LibrarySchema, PortTypeSchema, load_yaml_library +from gems_craft.utils import ModifiedBaseModel + + +class AreaConnectionSchema(ModifiedBaseModel): + injection_to_balance: Optional[str] = None + spillage_bound: Optional[str] = None + unsupplied_energy_bound: Optional[str] = None + + +class PortThermalCapacitySchema(ModifiedBaseModel): + capacity_field: str + + +class HybridPortTypeSchema(PortTypeSchema): + area_connection: Optional[AreaConnectionSchema] = None + thermal_capacity_connection: Optional[PortThermalCapacitySchema] = None + + +class HybridLibrarySchema(LibrarySchema): + port_types: List[HybridPortTypeSchema] = Field(default_factory=list) # type: ignore[assignment] diff --git a/src/gems_craft_hybrid/study/__init__.py b/src/gems_craft_hybrid/study/__init__.py new file mode 100644 index 00000000..058c6b22 --- /dev/null +++ b/src/gems_craft_hybrid/study/__init__.py @@ -0,0 +1,11 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. diff --git a/src/gems_craft_hybrid/study/parsing.py b/src/gems_craft_hybrid/study/parsing.py new file mode 100644 index 00000000..6437abd9 --- /dev/null +++ b/src/gems_craft_hybrid/study/parsing.py @@ -0,0 +1,38 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. + +from typing import List, Optional + +from gems_craft.study.parsing import SystemSchema +from gems_craft.utils import ModifiedBaseModel + + +class AreaConnectionsSchema(ModifiedBaseModel): + component: str + port: str + area: str + + +class ThermalComponentSchema(ModifiedBaseModel): + area: str + cluster_id: str + + +class ThermalCapacityConnectionSchema(ModifiedBaseModel): + component: str + port: str + thermal_component: ThermalComponentSchema + + +class HybridSystemSchema(SystemSchema): + area_connections: Optional[List[AreaConnectionsSchema]] = None + thermal_capacity_connections: Optional[List[ThermalCapacityConnectionSchema]] = None diff --git a/src/gems_runner/__init__.py b/src/gems_runner/__init__.py new file mode 100644 index 00000000..058c6b22 --- /dev/null +++ b/src/gems_runner/__init__.py @@ -0,0 +1,11 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. diff --git a/src/gems/expression/__init__.py b/src/gems_runner/expression/__init__.py similarity index 100% rename from src/gems/expression/__init__.py rename to src/gems_runner/expression/__init__.py diff --git a/src/gems/expression/copy.py b/src/gems_runner/expression/copy.py similarity index 100% rename from src/gems/expression/copy.py rename to src/gems_runner/expression/copy.py diff --git a/src/gems/expression/degree.py b/src/gems_runner/expression/degree.py similarity index 95% rename from src/gems/expression/degree.py rename to src/gems_runner/expression/degree.py index 1d68aba0..3ceb5d4a 100644 --- a/src/gems/expression/degree.py +++ b/src/gems_runner/expression/degree.py @@ -12,8 +12,8 @@ import math -import gems.expression.scenario_operator -from gems.expression.expression import ( +import gems_runner.expression.scenario_operator +from gems_runner.expression.expression import ( AbsNode, AdditionNode, AllTimeSumNode, @@ -92,7 +92,9 @@ def all_time_sum(self, node: AllTimeSumNode) -> int | float: return visit(node.operand, self) def scenario_operator(self, node: ScenarioOperatorNode) -> int | float: - scenario_operator_cls = getattr(gems.expression.scenario_operator, node.name) + scenario_operator_cls = getattr( + gems_runner.expression.scenario_operator, node.name + ) # TODO: Carefully check if this formula is correct return scenario_operator_cls.degree() * visit(node.operand, self) diff --git a/src/gems/expression/equality.py b/src/gems_runner/expression/equality.py similarity index 99% rename from src/gems/expression/equality.py rename to src/gems_runner/expression/equality.py index 9b61bea9..f2961b85 100644 --- a/src/gems/expression/equality.py +++ b/src/gems_runner/expression/equality.py @@ -14,7 +14,7 @@ from dataclasses import dataclass from typing import Optional -from gems.expression import ( +from gems_runner.expression import ( AdditionNode, ComparisonNode, DivisionNode, @@ -25,7 +25,7 @@ ParameterNode, VariableNode, ) -from gems.expression.expression import ( +from gems_runner.expression.expression import ( AbsNode, AllTimeSumNode, BinaryOperatorNode, diff --git a/src/gems/expression/evaluate.py b/src/gems_runner/expression/evaluate.py similarity index 98% rename from src/gems/expression/evaluate.py rename to src/gems_runner/expression/evaluate.py index 8373299d..e9e60a56 100644 --- a/src/gems/expression/evaluate.py +++ b/src/gems_runner/expression/evaluate.py @@ -15,7 +15,7 @@ from dataclasses import dataclass, field from typing import Dict -from gems.expression.expression import ( +from gems_runner.expression.expression import ( AbsNode, AllTimeSumNode, CeilNode, diff --git a/src/gems/expression/expression.py b/src/gems_runner/expression/expression.py similarity index 95% rename from src/gems/expression/expression.py rename to src/gems_runner/expression/expression.py index aafb1d8f..d90647fc 100644 --- a/src/gems/expression/expression.py +++ b/src/gems_runner/expression/expression.py @@ -19,8 +19,8 @@ from dataclasses import dataclass from typing import Any, Callable, List, Optional, Sequence, Union -import gems.expression.port_operator -import gems.expression.scenario_operator +import gems_runner.expression.port_operator +import gems_runner.expression.scenario_operator AnyExpression = Union[int, float, "ExpressionNode"] @@ -216,9 +216,9 @@ def __post_init__(self) -> None: valid_names = [ cls.__name__ for _, cls in inspect.getmembers( - gems.expression.port_operator, inspect.isclass + gems_runner.expression.port_operator, inspect.isclass ) - if issubclass(cls, gems.expression.port_operator.PortAggregator) + if issubclass(cls, gems_runner.expression.port_operator.PortAggregator) ] if self.aggregator not in valid_names: raise NotImplementedError( @@ -337,9 +337,11 @@ def __post_init__(self) -> None: valid_names = [ cls.__name__ for _, cls in inspect.getmembers( - gems.expression.scenario_operator, inspect.isclass + gems_runner.expression.scenario_operator, inspect.isclass + ) + if issubclass( + cls, gems_runner.expression.scenario_operator.ScenarioOperator ) - if issubclass(cls, gems.expression.scenario_operator.ScenarioOperator) ] if self.name not in valid_names: raise ValueError( diff --git a/src/gems/expression/indexing.py b/src/gems_runner/expression/indexing.py similarity index 98% rename from src/gems/expression/indexing.py rename to src/gems_runner/expression/indexing.py index a71b8484..75f7ac26 100644 --- a/src/gems/expression/indexing.py +++ b/src/gems_runner/expression/indexing.py @@ -14,7 +14,7 @@ from dataclasses import dataclass from typing import List -from gems.expression.indexing_structure import IndexingStructure +from gems_runner.expression.indexing_structure import IndexingStructure from .expression import ( AbsNode, diff --git a/src/gems/expression/indexing_structure.py b/src/gems_runner/expression/indexing_structure.py similarity index 100% rename from src/gems/expression/indexing_structure.py rename to src/gems_runner/expression/indexing_structure.py diff --git a/src/gems/expression/parsing/__init__.py b/src/gems_runner/expression/parsing/__init__.py similarity index 100% rename from src/gems/expression/parsing/__init__.py rename to src/gems_runner/expression/parsing/__init__.py diff --git a/src/gems/expression/parsing/antlr/Expr.interp b/src/gems_runner/expression/parsing/antlr/Expr.interp similarity index 100% rename from src/gems/expression/parsing/antlr/Expr.interp rename to src/gems_runner/expression/parsing/antlr/Expr.interp diff --git a/src/gems/expression/parsing/antlr/Expr.tokens b/src/gems_runner/expression/parsing/antlr/Expr.tokens similarity index 100% rename from src/gems/expression/parsing/antlr/Expr.tokens rename to src/gems_runner/expression/parsing/antlr/Expr.tokens diff --git a/src/gems/expression/parsing/antlr/ExprLexer.interp b/src/gems_runner/expression/parsing/antlr/ExprLexer.interp similarity index 100% rename from src/gems/expression/parsing/antlr/ExprLexer.interp rename to src/gems_runner/expression/parsing/antlr/ExprLexer.interp diff --git a/src/gems/expression/parsing/antlr/ExprLexer.py b/src/gems_runner/expression/parsing/antlr/ExprLexer.py similarity index 100% rename from src/gems/expression/parsing/antlr/ExprLexer.py rename to src/gems_runner/expression/parsing/antlr/ExprLexer.py diff --git a/src/gems/expression/parsing/antlr/ExprLexer.tokens b/src/gems_runner/expression/parsing/antlr/ExprLexer.tokens similarity index 100% rename from src/gems/expression/parsing/antlr/ExprLexer.tokens rename to src/gems_runner/expression/parsing/antlr/ExprLexer.tokens diff --git a/src/gems/expression/parsing/antlr/ExprParser.py b/src/gems_runner/expression/parsing/antlr/ExprParser.py similarity index 100% rename from src/gems/expression/parsing/antlr/ExprParser.py rename to src/gems_runner/expression/parsing/antlr/ExprParser.py diff --git a/src/gems/expression/parsing/antlr/ExprVisitor.py b/src/gems_runner/expression/parsing/antlr/ExprVisitor.py similarity index 100% rename from src/gems/expression/parsing/antlr/ExprVisitor.py rename to src/gems_runner/expression/parsing/antlr/ExprVisitor.py diff --git a/src/gems/expression/parsing/parse_expression.py b/src/gems_runner/expression/parsing/parse_expression.py similarity index 96% rename from src/gems/expression/parsing/parse_expression.py rename to src/gems_runner/expression/parsing/parse_expression.py index 63970a49..69187fbb 100644 --- a/src/gems/expression/parsing/parse_expression.py +++ b/src/gems_runner/expression/parsing/parse_expression.py @@ -15,9 +15,9 @@ from antlr4 import CommonTokenStream, InputStream from antlr4.error.ErrorStrategy import BailErrorStrategy -from gems.expression import ExpressionNode, literal, param, var -from gems.expression.equality import expressions_equal -from gems.expression.expression import ( +from gems_runner.expression import ExpressionNode, literal, param, var +from gems_runner.expression.equality import expressions_equal +from gems_runner.expression.expression import ( Comparator, ComparisonNode, DualNode, @@ -27,9 +27,9 @@ maximum, minimum, ) -from gems.expression.parsing.antlr.ExprLexer import ExprLexer -from gems.expression.parsing.antlr.ExprParser import ExprParser -from gems.expression.parsing.antlr.ExprVisitor import ExprVisitor +from gems_runner.expression.parsing.antlr.ExprLexer import ExprLexer +from gems_runner.expression.parsing.antlr.ExprParser import ExprParser +from gems_runner.expression.parsing.antlr.ExprVisitor import ExprVisitor @dataclass(frozen=True) diff --git a/src/gems/expression/port_operator.py b/src/gems_runner/expression/port_operator.py similarity index 100% rename from src/gems/expression/port_operator.py rename to src/gems_runner/expression/port_operator.py diff --git a/src/gems/expression/print.py b/src/gems_runner/expression/print.py similarity index 98% rename from src/gems/expression/print.py rename to src/gems_runner/expression/print.py index 1e873877..10f45342 100644 --- a/src/gems/expression/print.py +++ b/src/gems_runner/expression/print.py @@ -13,7 +13,7 @@ from dataclasses import dataclass from typing import Dict -from gems.expression.expression import ( +from gems_runner.expression.expression import ( AbsNode, AllTimeSumNode, CeilNode, diff --git a/src/gems/expression/scenario_operator.py b/src/gems_runner/expression/scenario_operator.py similarity index 100% rename from src/gems/expression/scenario_operator.py rename to src/gems_runner/expression/scenario_operator.py diff --git a/src/gems/expression/uses_sum_connections_on.py b/src/gems_runner/expression/uses_sum_connections_on.py similarity index 98% rename from src/gems/expression/uses_sum_connections_on.py rename to src/gems_runner/expression/uses_sum_connections_on.py index bc89dcea..69976cd9 100644 --- a/src/gems/expression/uses_sum_connections_on.py +++ b/src/gems_runner/expression/uses_sum_connections_on.py @@ -10,7 +10,7 @@ # # This file is part of the Antares project. -from gems.expression.expression import ( +from gems_runner.expression.expression import ( AbsNode, AdditionNode, AllTimeSumNode, diff --git a/src/gems/expression/visitor.py b/src/gems_runner/expression/visitor.py similarity index 99% rename from src/gems/expression/visitor.py rename to src/gems_runner/expression/visitor.py index bd96e512..0edabf7a 100644 --- a/src/gems/expression/visitor.py +++ b/src/gems_runner/expression/visitor.py @@ -18,7 +18,7 @@ from abc import ABC, abstractmethod from typing import Generic, Protocol, TypeVar -from gems.expression.expression import ( +from gems_runner.expression.expression import ( AbsNode, AdditionNode, AllTimeSumNode, diff --git a/src/gems/main/main.py b/src/gems_runner/main/main.py similarity index 56% rename from src/gems/main/main.py rename to src/gems_runner/main/main.py index 59e21578..18a4d86b 100644 --- a/src/gems/main/main.py +++ b/src/gems_runner/main/main.py @@ -13,40 +13,37 @@ from pathlib import Path from typing import Dict, List, Optional -from gems.model.library import Library -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.optim_config.parsing import OptimConfig -from gems.simulation import DecomposedProblems, build_couplings, dump_couplings -from gems.study import Study -from gems.study.data import DataBase -from gems.study.parsing import parse_cli, parse_yaml_components -from gems.study.resolve_components import build_data_base, resolve_system -from gems.study.runner import run_study -from gems.study.system import System +from gems_craft.model.parsing import LibrarySchema, load_yaml_library +from gems_craft.optim_config.parsing import OptimConfig +from gems_craft.study.parsing import load_yaml_system, parse_cli +from gems_runner.model.library import Library +from gems_runner.model.resolve_library import resolve_library +from gems_runner.simulation import DecomposedProblems, build_couplings, dump_couplings +from gems_runner.study import Study +from gems_runner.study.data import DataBase +from gems_runner.study.resolve_components import build_data_base, resolve_system +from gems_runner.study.runner import run_study +from gems_runner.study.system import System def input_libs(yaml_lib_paths: List[Path]) -> Dict[str, Library]: yaml_libraries = [] yaml_library_ids = set() for path in yaml_lib_paths: - with path.open("r") as file: - yaml_lib = parse_yaml_library(file) - if yaml_lib.id in yaml_library_ids: - raise ValueError(f"The identifier '{yaml_lib.id}' is defined twice") - yaml_libraries.append(yaml_lib) - yaml_library_ids.add(yaml_lib.id) + yaml_lib: LibrarySchema = load_yaml_library(path) + if yaml_lib.id in yaml_library_ids: + raise ValueError(f"The identifier '{yaml_lib.id}' is defined twice") + yaml_libraries.append(yaml_lib) + yaml_library_ids.add(yaml_lib.id) return resolve_library(yaml_libraries) def input_database(study_path: Path, timeseries_path: Optional[Path]) -> DataBase: - with study_path.open() as comp: - return build_data_base(parse_yaml_components(comp), timeseries_path) + return build_data_base(load_yaml_system(study_path), timeseries_path) def input_system(study_path: Path, libraries: Dict[str, Library]) -> System: - with study_path.open() as comp: - return resolve_system(parse_yaml_components(comp), libraries) + return resolve_system(load_yaml_system(study_path), libraries) def _write_structure_txt( diff --git a/src/gems/model/__init__.py b/src/gems_runner/model/__init__.py similarity index 100% rename from src/gems/model/__init__.py rename to src/gems_runner/model/__init__.py diff --git a/src/gems/model/common.py b/src/gems_runner/model/common.py similarity index 100% rename from src/gems/model/common.py rename to src/gems_runner/model/common.py diff --git a/src/gems/model/constraint.py b/src/gems_runner/model/constraint.py similarity index 92% rename from src/gems/model/constraint.py rename to src/gems_runner/model/constraint.py index 1f53a924..62a219c1 100644 --- a/src/gems/model/constraint.py +++ b/src/gems_runner/model/constraint.py @@ -13,9 +13,12 @@ from dataclasses import dataclass, field, replace from typing import Any -from gems.expression.degree import is_constant -from gems.expression.equality import expressions_equal, expressions_equal_if_present -from gems.expression.expression import ( +from gems_runner.expression.degree import is_constant +from gems_runner.expression.equality import ( + expressions_equal, + expressions_equal_if_present, +) +from gems_runner.expression.expression import ( Comparator, ComparisonNode, ExpressionNode, @@ -23,7 +26,7 @@ is_unbounded, literal, ) -from gems.expression.print import print_expr +from gems_runner.expression.print import print_expr @dataclass diff --git a/src/gems/model/library.py b/src/gems_runner/model/library.py similarity index 94% rename from src/gems/model/library.py rename to src/gems_runner/model/library.py index 5d29d1bd..49f9989d 100644 --- a/src/gems/model/library.py +++ b/src/gems_runner/model/library.py @@ -12,7 +12,7 @@ from dataclasses import dataclass from typing import Dict, Iterable -from gems.model import Model, PortType +from gems_runner.model import Model, PortType @dataclass(frozen=True) diff --git a/src/gems/model/model.py b/src/gems_runner/model/model.py similarity index 95% rename from src/gems/model/model.py rename to src/gems_runner/model/model.py index 3870edce..a1a64654 100644 --- a/src/gems/model/model.py +++ b/src/gems_runner/model/model.py @@ -21,14 +21,17 @@ from dataclasses import dataclass, field, replace from typing import Any, Dict, Iterable, List, Optional -from gems.expression import ExpressionNode -from gems.expression.degree import is_linear -from gems.expression.indexing import IndexingStructureProvider, compute_indexation -from gems.expression.indexing_structure import IndexingStructure -from gems.model.constraint import Constraint -from gems.model.parameter import Parameter -from gems.model.port import PortFieldDefinition, PortFieldId, PortType -from gems.model.variable import Variable +from gems_runner.expression import ExpressionNode +from gems_runner.expression.degree import is_linear +from gems_runner.expression.indexing import ( + IndexingStructureProvider, + compute_indexation, +) +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model.constraint import Constraint +from gems_runner.model.parameter import Parameter +from gems_runner.model.port import PortFieldDefinition, PortFieldId, PortType +from gems_runner.model.variable import Variable # TODO: Introduce bool_variable ? diff --git a/src/gems/model/parameter.py b/src/gems_runner/model/parameter.py similarity index 91% rename from src/gems/model/parameter.py rename to src/gems_runner/model/parameter.py index e9f60bca..59ed07d2 100644 --- a/src/gems/model/parameter.py +++ b/src/gems_runner/model/parameter.py @@ -13,8 +13,8 @@ from dataclasses import dataclass, replace from typing import Any -from gems.expression.indexing_structure import IndexingStructure -from gems.model.common import ValueType +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model.common import ValueType @dataclass(frozen=True) diff --git a/src/gems/model/port.py b/src/gems_runner/model/port.py similarity index 97% rename from src/gems/model/port.py rename to src/gems_runner/model/port.py index cc474a22..8ac0d93a 100644 --- a/src/gems/model/port.py +++ b/src/gems_runner/model/port.py @@ -13,7 +13,7 @@ from dataclasses import dataclass, replace from typing import Any, List -from gems.expression import ( +from gems_runner.expression import ( AdditionNode, ComparisonNode, DivisionNode, @@ -25,7 +25,7 @@ ParameterNode, VariableNode, ) -from gems.expression.expression import ( +from gems_runner.expression.expression import ( AbsNode, AllTimeSumNode, BinaryOperatorNode, @@ -43,7 +43,7 @@ TimeShiftNode, TimeSumNode, ) -from gems.expression.visitor import visit +from gems_runner.expression.visitor import visit @dataclass(frozen=True) diff --git a/src/gems/model/resolve_library.py b/src/gems_runner/model/resolve_library.py similarity index 94% rename from src/gems/model/resolve_library.py rename to src/gems_runner/model/resolve_library.py index e36d55c5..ed39b9ad 100644 --- a/src/gems/model/resolve_library.py +++ b/src/gems_runner/model/resolve_library.py @@ -11,9 +11,20 @@ # This file is part of the Antares project. from typing import Dict, List, Optional, Set -from gems.expression import ExpressionNode, literal -from gems.expression.degree import is_linear -from gems.expression.expression import ( +from gems_craft.model.parsing import ( + ConstraintSchema, + FieldSchema, + LibrarySchema, + ModelPortSchema, + ModelSchema, + ParameterSchema, + PortFieldDefinitionSchema, + PortTypeSchema, + VariableSchema, +) +from gems_runner.expression import ExpressionNode, literal +from gems_runner.expression.degree import is_linear +from gems_runner.expression.expression import ( AbsNode, AdditionNode, AllTimeSumNode, @@ -38,11 +49,14 @@ TimeSumNode, VariableNode, ) -from gems.expression.indexing_structure import IndexingStructure -from gems.expression.parsing.parse_expression import ModelIdentifiers, parse_expression -from gems.expression.uses_sum_connections_on import uses_sum_connections_on -from gems.expression.visitor import ExpressionVisitor, visit -from gems.model import ( +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.expression.parsing.parse_expression import ( + ModelIdentifiers, + parse_expression, +) +from gems_runner.expression.uses_sum_connections_on import uses_sum_connections_on +from gems_runner.expression.visitor import ExpressionVisitor, visit +from gems_runner.model import ( Constraint, Model, ModelPort, @@ -53,20 +67,9 @@ Variable, model, ) -from gems.model.library import Library -from gems.model.model import model -from gems.model.parsing import ( - ConstraintSchema, - FieldSchema, - LibrarySchema, - ModelPortSchema, - ModelSchema, - ParameterSchema, - PortFieldDefinitionSchema, - PortTypeSchema, - VariableSchema, -) -from gems.model.port import PortFieldDefinition, port_field_def +from gems_runner.model.library import Library +from gems_runner.model.model import model +from gems_runner.model.port import PortFieldDefinition, port_field_def def resolve_library( diff --git a/src/gems/model/variable.py b/src/gems_runner/model/variable.py similarity index 88% rename from src/gems/model/variable.py rename to src/gems_runner/model/variable.py index 5899ca9e..725f1373 100644 --- a/src/gems/model/variable.py +++ b/src/gems_runner/model/variable.py @@ -13,11 +13,11 @@ from dataclasses import dataclass, replace from typing import Any, Optional -from gems.expression import ExpressionNode, literal -from gems.expression.degree import is_constant -from gems.expression.equality import expressions_equal_if_present -from gems.expression.indexing_structure import IndexingStructure -from gems.model.common import ValueType +from gems_runner.expression import ExpressionNode, literal +from gems_runner.expression.degree import is_constant +from gems_runner.expression.equality import expressions_equal_if_present +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model.common import ValueType @dataclass diff --git a/src/gems_runner/optim_config/__init__.py b/src/gems_runner/optim_config/__init__.py new file mode 100644 index 00000000..397179c3 --- /dev/null +++ b/src/gems_runner/optim_config/__init__.py @@ -0,0 +1,19 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. + +from .optim_config import ( + validate_optim_config, +) + +__all__ = [ + "validate_optim_config", +] diff --git a/src/gems_runner/optim_config/optim_config.py b/src/gems_runner/optim_config/optim_config.py new file mode 100644 index 00000000..be53e768 --- /dev/null +++ b/src/gems_runner/optim_config/optim_config.py @@ -0,0 +1,229 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. + +from typing import TYPE_CHECKING, List, Optional, Set + +from gems_craft.optim_config.parsing import ( + ElementLocation, + ModelDecompositionConfig, + OptimConfig, + OutOfBoundsProcessingConfig, +) +from gems_runner.expression.expression import ( + AdditionNode, + BinaryOperatorNode, + ExpressionNode, + MaxNode, + MinNode, + UnaryOperatorNode, + VariableNode, +) + +if TYPE_CHECKING: + from gems_craft.study.scenario_builder import ScenarioBuilder + from gems_runner.model.model import Model + from gems_runner.study.system import System + +_MASTER_LOCS: Set[ElementLocation] = { + ElementLocation.MASTER, + ElementLocation.MASTER_AND_SUBPROBLEMS, +} + + +def _collect_variable_names(expr: ExpressionNode) -> Set[str]: + """Recursively collect all variable names referenced in an expression.""" + if isinstance(expr, VariableNode): + return {expr.name} + if isinstance(expr, (AdditionNode, MaxNode, MinNode)): + result: Set[str] = set() + for operand in expr.operands: + result |= _collect_variable_names(operand) + return result + if isinstance(expr, UnaryOperatorNode): + return _collect_variable_names(expr.operand) + if isinstance(expr, BinaryOperatorNode): + return _collect_variable_names(expr.left) | _collect_variable_names(expr.right) + return set() + + +def _check_oob_constraint_ids( + oob_processing: OutOfBoundsProcessingConfig, + model: "Model", + model_config_id: str, + errors: List[str], +) -> None: + for constraint_config in oob_processing.constraints: + if ( + constraint_config.id not in model.constraints + and constraint_config.id not in model.binding_constraints + ): + errors.append( + f"Out-of-bounds constraint '{constraint_config.id}' not found in model '{model_config_id}'" + ) + + +def _check_id_existence( + decomposition: ModelDecompositionConfig, + model: "Model", + model_config_id: str, + errors: List[str], +) -> None: + for variable_config in decomposition.variables: + if variable_config.id not in model.variables: + errors.append( + f"Variable '{variable_config.id}' not found in model '{model_config_id}'" + ) + for constraint_config in decomposition.constraints: + if ( + constraint_config.id not in model.constraints + and constraint_config.id not in model.binding_constraints + ): + errors.append( + f"Constraint '{constraint_config.id}' not found in model '{model_config_id}'" + ) + obj_keys = set(model.objective_contributions or {}) + for obj_config in decomposition.objective_contributions: + if obj_config.id not in obj_keys: + errors.append( + f"Objective-contribution '{obj_config.id}' not found in model '{model_config_id}'" + ) + + +def _check_master_variables_not_time_dependent( + decomposition: ModelDecompositionConfig, + model: "Model", + model_config_id: str, + errors: List[str], +) -> None: + """Variables assigned to master or master-and-subproblems must not depend on time.""" + for variable_config in decomposition.variables: + if ( + variable_config.location in _MASTER_LOCS + and variable_config.id in model.variables + ): + if model.variables[variable_config.id].structure.time: + errors.append( + f"Variable '{variable_config.id}' in model '{model_config_id}' is time-dependent " + f"but is assigned to '{variable_config.location.value}'; " + "master variables must not depend on time" + ) + + +def _check_master_constraints_use_master_variables( + decomposition: ModelDecompositionConfig, + model: "Model", + model_config_id: str, + errors: List[str], +) -> None: + """Constraints in master must only reference variables in master or master-and-subproblems.""" + master_var_ids = { + variable_config.id + for variable_config in decomposition.variables + if variable_config.location in _MASTER_LOCS + and variable_config.id in model.variables + } + for constraint_config in decomposition.constraints: + if constraint_config.location == ElementLocation.MASTER: + constraint = model.constraints.get( + constraint_config.id + ) or model.binding_constraints.get(constraint_config.id) + if constraint is not None: + for var_name in sorted( + _collect_variable_names(constraint.expression) - master_var_ids + ): + errors.append( + f"Constraint '{constraint_config.id}' in model '{model_config_id}' references variable '{var_name}' " + "which is not assigned to master or master-and-subproblems" + ) + + +def _check_master_objectives_use_master_variables( + decomposition: ModelDecompositionConfig, + model: "Model", + model_config_id: str, + errors: List[str], +) -> None: + """Objective contributions in master must only reference variables in master or master-and-subproblems.""" + master_var_ids = { + variable_config.id + for variable_config in decomposition.variables + if variable_config.location in _MASTER_LOCS + and variable_config.id in model.variables + } + obj_contribs = model.objective_contributions or {} + for obj_config in decomposition.objective_contributions: + if obj_config.location == ElementLocation.MASTER: + expr = obj_contribs.get(obj_config.id) + if expr is not None: + for var_name in sorted(_collect_variable_names(expr) - master_var_ids): + errors.append( + f"Objective contribution '{obj_config.id}' in model '{model_config_id}' references variable '{var_name}' " + "which is not assigned to master or master-and-subproblems" + ) + + +def validate_optim_config( + config: OptimConfig, + system: "System", + scenario_builder: Optional["ScenarioBuilder"] = None, +) -> None: + """Cross-validate optim-config entries against the resolved system. + + Performs the following checks: + + - Every model ID referenced in ``config.models`` exists in the system. + - Master variables are time-independent. + - Master constraints and objective contributions only reference variables + assigned to ``master`` or ``master-and-subproblems``. + - If ``scenario_builder`` is provided, every scenario index in + ``config.scenario_scope.scenario_ids`` is defined for every scenario + group in the builder. + + Raises ``ValueError`` listing all violations found. + """ + models_in_system = {c.model.id: c.model for c in system.all_components} + errors: List[str] = [] + + if scenario_builder is not None: + errors.extend( + scenario_builder.validate_mc_scenarios(config.scenario_scope.scenario_ids) + ) + + for model_config in config.models: + model = models_in_system.get(model_config.id) + if model is None: + errors.append(f"Model '{model_config.id}' not found in system") + else: + if model_config.model_decomposition is not None: + decomposition = model_config.model_decomposition + _check_id_existence(decomposition, model, model_config.id, errors) + _check_master_variables_not_time_dependent( + decomposition, model, model_config.id, errors + ) + _check_master_constraints_use_master_variables( + decomposition, model, model_config.id, errors + ) + _check_master_objectives_use_master_variables( + decomposition, model, model_config.id, errors + ) + if model_config.out_of_bounds_processing is not None: + _check_oob_constraint_ids( + model_config.out_of_bounds_processing, + model, + model_config.id, + errors, + ) + + if errors: + raise ValueError( + f"Errors in optim config file:\n" + "\n".join(f" - {e}" for e in errors) + ) diff --git a/src/gems/session/__init__.py b/src/gems_runner/session/__init__.py similarity index 100% rename from src/gems/session/__init__.py rename to src/gems_runner/session/__init__.py diff --git a/src/gems/session/session.py b/src/gems_runner/session/session.py similarity index 95% rename from src/gems/session/session.py rename to src/gems_runner/session/session.py index 0d4f0ad8..26068a70 100644 --- a/src/gems/session/session.py +++ b/src/gems_runner/session/session.py @@ -17,16 +17,16 @@ import xarray as xr -from gems.optim_config.parsing import OptimConfig, ResolutionMode, load_optim_config -from gems.simulation.optimization import OptimizationProblem, build_problem -from gems.simulation.simulation_table import ( +from gems_craft.optim_config.parsing import OptimConfig, ResolutionMode +from gems_runner.simulation.optimization import OptimizationProblem, build_problem +from gems_runner.simulation.simulation_table import ( SimulationTable, SimulationTableBuilder, merge_simulation_tables, ) -from gems.simulation.time_block import TimeBlock -from gems.study.folder import load_study -from gems.study.study import Study +from gems_runner.simulation.time_block import TimeBlock +from gems_runner.study.folder import load_study +from gems_runner.study.study import Study @dataclass @@ -137,7 +137,7 @@ def _run_parallel(self) -> SimulationTable: def _run_benders(self) -> SimulationTable: import pandas as pd - from gems.simulation import ( + from gems_runner.simulation import ( BendersRunner, build_couplings, build_decomposed_problems, diff --git a/src/gems/simulation/__init__.py b/src/gems_runner/simulation/__init__.py similarity index 100% rename from src/gems/simulation/__init__.py rename to src/gems_runner/simulation/__init__.py diff --git a/src/gems/simulation/couplings.py b/src/gems_runner/simulation/couplings.py similarity index 93% rename from src/gems/simulation/couplings.py rename to src/gems_runner/simulation/couplings.py index ac586edc..703883cb 100644 --- a/src/gems/simulation/couplings.py +++ b/src/gems_runner/simulation/couplings.py @@ -15,8 +15,8 @@ from typing import TYPE_CHECKING, List, Optional if TYPE_CHECKING: - from gems.optim_config.parsing import OptimConfig - from gems.simulation.optimization import DecomposedProblems + from gems_craft.optim_config.parsing import OptimConfig + from gems_runner.simulation.optimization import DecomposedProblems @dataclass(frozen=True) @@ -86,7 +86,7 @@ def build_couplings( decomposed: "DecomposedProblems", optim_config: "OptimConfig", ) -> List[CouplingRow]: - from gems.optim_config.parsing import ElementLocation + from gems_craft.optim_config.parsing import ElementLocation rows: List[CouplingRow] = [] for mc in optim_config.models: diff --git a/src/gems/simulation/extra_output.py b/src/gems_runner/simulation/extra_output.py similarity index 95% rename from src/gems/simulation/extra_output.py rename to src/gems_runner/simulation/extra_output.py index bd5f1d78..eb3513fe 100644 --- a/src/gems/simulation/extra_output.py +++ b/src/gems_runner/simulation/extra_output.py @@ -31,10 +31,10 @@ import numpy as np import xarray as xr -from gems.expression.expression import DualNode, ReducedCostNode, VariableNode -from gems.model.port import PortFieldId -from gems.simulation.vectorized_builder import VectorizedBuilderBase -from gems.study.system import Component +from gems_runner.expression.expression import DualNode, ReducedCostNode, VariableNode +from gems_runner.model.port import PortFieldId +from gems_runner.simulation.vectorized_builder import VectorizedBuilderBase +from gems_runner.study.system import Component @dataclass diff --git a/src/gems/simulation/linearize.py b/src/gems_runner/simulation/linearize.py similarity index 97% rename from src/gems/simulation/linearize.py rename to src/gems_runner/simulation/linearize.py index e551d535..fc06a584 100644 --- a/src/gems/simulation/linearize.py +++ b/src/gems_runner/simulation/linearize.py @@ -30,7 +30,7 @@ import numpy as np import xarray as xr -from gems.expression.expression import ( +from gems_runner.expression.expression import ( AbsNode, AdditionNode, CeilNode, @@ -40,9 +40,9 @@ RoundNode, VariableNode, ) -from gems.expression.visitor import visit -from gems.model.port import PortFieldId -from gems.simulation.vectorized_builder import ( +from gems_runner.expression.visitor import visit +from gems_runner.model.port import PortFieldId +from gems_runner.simulation.vectorized_builder import ( VectorizedBuilderBase, VectorizedExpr, _linopy_add, diff --git a/src/gems/simulation/optimization.py b/src/gems_runner/simulation/optimization.py similarity index 97% rename from src/gems/simulation/optimization.py rename to src/gems_runner/simulation/optimization.py index a73bcf72..1463ecd3 100644 --- a/src/gems/simulation/optimization.py +++ b/src/gems_runner/simulation/optimization.py @@ -38,24 +38,28 @@ import numpy as np import xarray as xr -from gems.expression.degree import is_linear -from gems.expression.expression import is_unbounded -from gems.expression.visitor import visit -from gems.model.common import ValueType -from gems.model.model import Model -from gems.model.port import PortField, PortFieldId -from gems.simulation.linearize import ( +from gems_runner.expression.degree import is_linear +from gems_runner.expression.expression import is_unbounded +from gems_runner.expression.visitor import visit +from gems_runner.model.common import ValueType +from gems_runner.model.model import Model +from gems_runner.model.port import PortField, PortFieldId +from gems_runner.simulation.linearize import ( VectorizedExpr, VectorizedLinearExprBuilder, _linopy_add, ) -from gems.simulation.time_block import TimeBlock -from gems.simulation.vectorized_builder import ShiftValidityVisitor -from gems.study.study import Study -from gems.study.system import Component +from gems_runner.simulation.time_block import TimeBlock +from gems_runner.simulation.vectorized_builder import ShiftValidityVisitor +from gems_runner.study.study import Study +from gems_runner.study.system import Component if TYPE_CHECKING: - from gems.optim_config.parsing import ElementLocation, OptimConfig, OutOfBoundsMode + from gems_craft.optim_config.parsing import ( + ElementLocation, + OptimConfig, + OutOfBoundsMode, + ) # --------------------------------------------------------------------------- # Public types @@ -86,7 +90,7 @@ class DecompositionFilter: def __init__( self, config: "OptimConfig", target_locations: "Set[ElementLocation]" ) -> None: - from gems.optim_config.parsing import ElementLocation as EL + from gems_craft.optim_config.parsing import ElementLocation as EL self._target = target_locations self._default = EL.SUBPROBLEMS @@ -684,7 +688,7 @@ def _create_constraints_for_model( # Compute a per-(component, time) validity mask for drop mode. validity_mask: Optional[xr.DataArray] = None if self._oob_filter is not None: - from gems.optim_config.parsing import OutOfBoundsMode + from gems_craft.optim_config.parsing import OutOfBoundsMode mode = self._oob_filter.get_mode(model.id, constraint.name) if mode == OutOfBoundsMode.DROP: @@ -907,7 +911,7 @@ def build_decomposed_problems( subproblem_name, master_name: Labels used for the underlying linopy models. """ - from gems.optim_config.parsing import ElementLocation + from gems_craft.optim_config.parsing import ElementLocation study.check_consistency() @@ -949,7 +953,7 @@ def build_decomposed_problems( def _has_any_master_element(config: "OptimConfig") -> bool: """Return True if *config* declares at least one master-side element.""" - from gems.optim_config.parsing import ElementLocation + from gems_craft.optim_config.parsing import ElementLocation master_locs = {ElementLocation.MASTER, ElementLocation.MASTER_AND_SUBPROBLEMS} for mc in config.models: diff --git a/src/gems/simulation/runner.py b/src/gems_runner/simulation/runner.py similarity index 100% rename from src/gems/simulation/runner.py rename to src/gems_runner/simulation/runner.py diff --git a/src/gems/simulation/simulation_table.py b/src/gems_runner/simulation/simulation_table.py similarity index 67% rename from src/gems/simulation/simulation_table.py rename to src/gems_runner/simulation/simulation_table.py index e662079a..1a9ca48c 100644 --- a/src/gems/simulation/simulation_table.py +++ b/src/gems_runner/simulation/simulation_table.py @@ -7,178 +7,13 @@ import pandas as pd import xarray as xr - -class OutputView: - """A Time × Scenario pivot for one (component, output) combination. - - Obtain via ``SimulationTable.component(...).output(...)``. - """ - - def __init__(self, df: pd.DataFrame) -> None: - # df: index = absolute-time-index, columns = scenario-index - self._df = df - - @property - def data(self) -> pd.DataFrame: - """Return the underlying Time × Scenario DataFrame.""" - return self._df - - def value( - self, - time_index: Optional[int] = None, - scenario_index: Optional[int] = None, - ) -> Union[pd.DataFrame, "pd.Series[Any]", float]: - """Return results filtered by time and/or scenario index. - - Called with no arguments returns the full Time × Scenario DataFrame. - Called with one argument returns a ``pd.Series``: - - ``value(scenario_index=s)`` → Series indexed by absolute-time-index - - ``value(time_index=t)`` → Series indexed by scenario-index - Called with both arguments returns a scalar ``float``. - """ - if time_index is None and scenario_index is None: - return self._df - if time_index is not None and scenario_index is not None: - return float(cast(Any, self._df.loc[time_index, scenario_index])) - if time_index is not None: - return self._df.loc[time_index] # Series over scenarios - return self._df[scenario_index] # Series over time - - def __repr__(self) -> str: - return repr(self._df) - - -class ComponentView: - """Filtered view of simulation results for one component. - - Obtain via ``SimulationTable.component(...)``. - """ - - def __init__(self, df: pd.DataFrame) -> None: - self._df = df - - def output(self, output_id: str) -> OutputView: - """Return an OutputView for the given output name.""" - col_output = SimulationColumns.OUTPUT.value - col_time = SimulationColumns.ABSOLUTE_TIME_INDEX.value - col_scenario = SimulationColumns.SCENARIO_INDEX.value - col_value = SimulationColumns.VALUE.value - - filtered = self._df[self._df[col_output] == output_id].copy() - # Dimension-independent outputs store None for the missing index. - # Fill with 0 so the pivot is always well-formed and the accessor - # API (value(time_index=t, scenario_index=s)) keeps working. - filtered[col_time] = filtered[col_time].fillna(0) - filtered[col_scenario] = filtered[col_scenario].fillna(0) - pivot = filtered.pivot_table( - index=col_time, - columns=col_scenario, - values=col_value, - aggfunc="first", - ) - pivot.index.name = col_time - pivot.columns.name = col_scenario - return OutputView(pivot) - - -class SimulationTable: - """Wrapper around the raw simulation results DataFrame. - - Provides a fluent accessor API:: - - st = SimulationTableBuilder().build(problem) - - # Full Time × Scenario DataFrame - st.component("gen_1").output("p").value() - - # Scalar at a specific time and scenario - st.component("gen_1").output("p").value(time_index=0, scenario_index=0) - - # Time series for scenario 0 - st.component("gen_1").output("p").value(scenario_index=0) - - # Scenario distribution at time step 3 - st.component("gen_1").output("p").value(time_index=3) - - The underlying long-format DataFrame is accessible via the ``data`` property. - """ - - def __init__(self, df: pd.DataFrame, table_id: str = "") -> None: - self._df = df - self.table_id = table_id - - @property - def data(self) -> pd.DataFrame: - """Return the underlying long-format DataFrame.""" - return self._df - - def component(self, component_id: str) -> ComponentView: - """Return a ComponentView filtered to the given component ID.""" - mask = self._df[SimulationColumns.COMPONENT.value] == component_id - return ComponentView(self._df[mask]) - - def to_csv(self, output_dir: Path) -> Path: - output_dir = Path(output_dir) - output_dir.mkdir(parents=True, exist_ok=True) - path = output_dir / f"simulation_table_{self.table_id}.csv" - self._df.to_csv(path, index=False) - return path - - def to_parquet(self, output_dir: Path) -> Path: - output_dir = Path(output_dir) - output_dir.mkdir(parents=True, exist_ok=True) - path = output_dir / f"simulation_table_{self.table_id}.parquet" - self._df.to_parquet(path, index=False) - return path - - def to_netcdf(self, output_dir: Path) -> Path: - output_dir = Path(output_dir) - output_dir.mkdir(parents=True, exist_ok=True) - path = output_dir / f"simulation_table_{self.table_id}.nc" - self.to_dataset().to_netcdf(path) - return path - - def to_dataset(self) -> xr.Dataset: - """Return simulation results as an xr.Dataset. - - Each output variable becomes a DataArray with dimensions - (component, absolute-time-index, scenario-index). - Scalar rows without component/time/scenario (e.g. objective-value) - are stored as zero-dimensional variables. - """ - df = self._df - col_comp = SimulationColumns.COMPONENT.value - col_out = SimulationColumns.OUTPUT.value - col_time = SimulationColumns.ABSOLUTE_TIME_INDEX.value - col_scen = SimulationColumns.SCENARIO_INDEX.value - col_val = SimulationColumns.VALUE.value - - main = df.dropna(subset=[col_comp, col_time, col_scen]) - indexed = main.set_index([col_comp, col_time, col_scen, col_out])[col_val] - unstacked = indexed.unstack(col_out) - ds = xr.Dataset.from_dataframe(unstacked) - - scalars = df[df[col_comp].isna() & df[col_time].isna()] - for _, row in scalars.iterrows(): - ds[row[col_out]] = xr.DataArray(float(row[col_val])) - - return ds - - -from gems.expression.visitor import visit -from gems.simulation.extra_output import VectorizedExtraOutputBuilder -from gems.simulation.optimization import OptimizationProblem, build_port_arrays - - -class SimulationColumns(str, Enum): - BLOCK = "block" - COMPONENT = "component" - OUTPUT = "output" - ABSOLUTE_TIME_INDEX = "absolute-time-index" - BLOCK_TIME_INDEX = "block-time-index" - SCENARIO_INDEX = "scenario-index" - VALUE = "value" - BASIS_STATUS = "basis-status" +from gems_craft.simulation_table.simulation_table import ( + SimulationColumns, + SimulationTable, +) +from gems_runner.expression.visitor import visit +from gems_runner.simulation.extra_output import VectorizedExtraOutputBuilder +from gems_runner.simulation.optimization import OptimizationProblem, build_port_arrays class SimulationTableBuilder: diff --git a/src/gems/simulation/time_block.py b/src/gems_runner/simulation/time_block.py similarity index 100% rename from src/gems/simulation/time_block.py rename to src/gems_runner/simulation/time_block.py diff --git a/src/gems/simulation/vectorized_builder.py b/src/gems_runner/simulation/vectorized_builder.py similarity index 98% rename from src/gems/simulation/vectorized_builder.py rename to src/gems_runner/simulation/vectorized_builder.py index 52e085cc..dc8ece9a 100644 --- a/src/gems/simulation/vectorized_builder.py +++ b/src/gems_runner/simulation/vectorized_builder.py @@ -25,7 +25,7 @@ - Pre-solve: returns a ``linopy.Variable`` (symbolic decision variable). - Post-solve: returns an ``xr.DataArray`` of optimal solution values. -All 18 other :class:`~gems.expression.visitor.ExpressionVisitor` methods are +All 18 other :class:`~gems_runner.expression.visitor.ExpressionVisitor` methods are implemented here once, with DataArray-friendly semantics as their default. ``VectorizedLinearExprBuilder`` overrides a small subset to add linopy-specific behaviour (operand-swap in addition, type guards in nonlinear functions). @@ -40,8 +40,8 @@ import numpy as np import xarray as xr -from gems.expression.evaluate import EvaluationContext, EvaluationVisitor -from gems.expression.expression import ( +from gems_runner.expression.evaluate import EvaluationContext, EvaluationVisitor +from gems_runner.expression.expression import ( AbsNode, AdditionNode, AllTimeSumNode, @@ -67,12 +67,12 @@ TimeSumNode, VariableNode, ) -from gems.expression.visitor import ( +from gems_runner.expression.visitor import ( ExpressionVisitor, ExpressionVisitorOperations, visit, ) -from gems.model.port import PortFieldId +from gems_runner.model.port import PortFieldId # --------------------------------------------------------------------------- # Public types @@ -116,7 +116,7 @@ class VectorizedBuilderBase(ExpressionVisitor[VectorizedExpr], Generic[T_expr]): """ Abstract base for vectorized expression builders. - Implements all :class:`~gems.expression.visitor.ExpressionVisitor` methods + Implements all :class:`~gems_runner.expression.visitor.ExpressionVisitor` methods with DataArray-friendly defaults. The sole abstract method is :meth:`variable` — subclasses differ only in how they resolve a ``VariableNode`` (symbolic linopy variable vs. solved DataArray value). diff --git a/src/gems/study/__init__.py b/src/gems_runner/study/__init__.py similarity index 93% rename from src/gems/study/__init__.py rename to src/gems_runner/study/__init__.py index f95d2c23..886dced4 100644 --- a/src/gems/study/__init__.py +++ b/src/gems_runner/study/__init__.py @@ -20,6 +20,5 @@ TimeScenarioSeriesData, TimeSeriesData, ) -from .scenario_builder import ScenarioBuilder from .study import Study from .system import Component, PortRef, PortsConnection, System, create_component diff --git a/src/gems/study/data.py b/src/gems_runner/study/data.py similarity index 88% rename from src/gems/study/data.py rename to src/gems_runner/study/data.py index a941ad7a..7dc9258f 100644 --- a/src/gems/study/data.py +++ b/src/gems_runner/study/data.py @@ -18,7 +18,7 @@ import pandas as pd if TYPE_CHECKING: - from gems.study.scenario_builder import ScenarioBuilder + from gems_craft.study.scenario_builder import ScenarioBuilder @dataclass(frozen=True) @@ -148,36 +148,6 @@ def check_requirement(self, time: bool, scenario: bool) -> bool: return time and scenario -def load_ts_from_file( - timeseries_name: Optional[str], path_to_file: Optional[Path] -) -> pd.DataFrame: - if path_to_file is None or timeseries_name is None: - raise FileNotFoundError(f"File '{timeseries_name}' does not exist") - - base_path = path_to_file / timeseries_name - candidates = [base_path.with_suffix(".txt"), base_path.with_suffix(".tsv")] - - last_exc: Optional[Exception] = None - for candidate in candidates: - if not candidate.exists(): - continue - try: - sep = r"\s+" if candidate.suffix == ".txt" else "\t" - return pd.read_csv(candidate, header=None, sep=sep) - except Exception as e: - last_exc = e - break - - if last_exc is not None: - raise Exception( - f"An error has arrived when processing '{candidate}': {last_exc}" - ) - - raise FileNotFoundError( - f"File '{timeseries_name}.txt' or '{timeseries_name}.tsv' does not exist" - ) - - def dataframe_to_time_series(ts_dataframe: pd.DataFrame) -> pd.Series: if ts_dataframe.shape[1] != 1: raise ValueError( diff --git a/src/gems/study/folder.py b/src/gems_runner/study/folder.py similarity index 73% rename from src/gems/study/folder.py rename to src/gems_runner/study/folder.py index 482c25a9..3a58241c 100644 --- a/src/gems/study/folder.py +++ b/src/gems_runner/study/folder.py @@ -9,17 +9,17 @@ from pathlib import Path -from gems.model.model import Model -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import ( +from gems_craft.model.parsing import LibrarySchema, load_yaml_library +from gems_craft.study.parsing import SystemSchema, load_yaml_system +from gems_craft.study.scenario_builder import ScenarioBuilder +from gems_runner.model.model import Model +from gems_runner.model.resolve_library import resolve_library +from gems_runner.study.resolve_components import ( build_data_base, consistency_check, resolve_system, ) -from gems.study.scenario_builder import ScenarioBuilder -from gems.study.study import Study +from gems_runner.study.study import Study def load_study(study_dir: Path) -> Study: @@ -40,13 +40,11 @@ def load_study(study_dir: Path) -> Study: lib_folder = study_dir / "input" / "model-libraries" series_dir = study_dir / "input" / "data-series" - input_libraries = [] + input_libraries: list[LibrarySchema] = [] for lib_file in lib_folder.glob("*.yml"): - with lib_file.open() as lib: - input_libraries.append(parse_yaml_library(lib)) + input_libraries.append(load_yaml_library(lib_file)) - with system_file.open() as c: - input_study = parse_yaml_components(c) + input_study: SystemSchema = load_yaml_system(system_file) lib_dict = resolve_library(input_libraries) system = resolve_system(input_study, lib_dict) model_dict: dict[str, Model] = {} @@ -58,7 +56,7 @@ def load_study(study_dir: Path) -> Study: study_dir / "input" / "data-series" / "modeler-scenariobuilder.dat" ) scenario_builder = ( - ScenarioBuilder.load(scenario_builder_path) + ScenarioBuilder.load_dat(scenario_builder_path) if scenario_builder_path.exists() else ScenarioBuilder() ) diff --git a/src/gems/study/resolve_components.py b/src/gems_runner/study/resolve_components.py similarity index 95% rename from src/gems/study/resolve_components.py rename to src/gems_runner/study/resolve_components.py index 2e41b23d..7f9aa5e3 100644 --- a/src/gems/study/resolve_components.py +++ b/src/gems_runner/study/resolve_components.py @@ -12,31 +12,31 @@ from pathlib import Path from typing import Dict, List, Optional, Tuple, Union -from gems.model import Model -from gems.model.library import Library -from gems.study import ( +from gems_craft.study.parsing import ( + ComponentPropertySchema, + ComponentSchema, + PortConnectionsSchema, + SystemSchema, +) +from gems_craft.study.scenario_builder import ScenarioBuilder +from gems_craft.study.timeseries import load_ts_from_file +from gems_runner.model import Model +from gems_runner.model.library import Library +from gems_runner.study import ( Component, ConstantData, DataBase, PortRef, System, ) -from gems.study.data import ( +from gems_runner.study.data import ( AbstractDataStructure, ScenarioSeriesData, TimeScenarioSeriesData, TimeSeriesData, dataframe_to_scenario_series, dataframe_to_time_series, - load_ts_from_file, -) -from gems.study.parsing import ( - ComponentPropertySchema, - ComponentSchema, - PortConnectionsSchema, - SystemSchema, ) -from gems.study.scenario_builder import ScenarioBuilder def _resolve_properties_raw_to_dict( diff --git a/src/gems/study/runner.py b/src/gems_runner/study/runner.py similarity index 79% rename from src/gems/study/runner.py rename to src/gems_runner/study/runner.py index 887bdda1..d9b5d806 100644 --- a/src/gems/study/runner.py +++ b/src/gems_runner/study/runner.py @@ -2,13 +2,13 @@ from pathlib import Path from typing import Optional -from gems.optim_config.parsing import ( +from gems_craft.optim_config.parsing import ( OptimConfig, - load_optim_config, - validate_optim_config, + load_yaml_optim_config, ) -from gems.session.session import SimulationSession -from gems.study.folder import load_study +from gems_runner.optim_config import validate_optim_config +from gems_runner.session.session import SimulationSession +from gems_runner.study.folder import load_study def run_study( @@ -32,7 +32,7 @@ def run_study( resolved_config_path = optim_config_path or ( study_dir / "input" / "optim-config.yml" ) - optim_config = load_optim_config(resolved_config_path) or OptimConfig() + optim_config = load_yaml_optim_config(resolved_config_path) or OptimConfig() validate_optim_config(optim_config, study.system, study.scenario_builder) run_id = datetime.now().strftime("%Y%m%dT%H%M") diff --git a/src/gems/study/study.py b/src/gems_runner/study/study.py similarity index 92% rename from src/gems/study/study.py rename to src/gems_runner/study/study.py index 07caa2fb..64ac5cb5 100644 --- a/src/gems/study/study.py +++ b/src/gems_runner/study/study.py @@ -15,10 +15,10 @@ from functools import cached_property from typing import Dict, List -from gems.model.model import Model -from gems.study.data import DataBase -from gems.study.scenario_builder import ScenarioBuilder -from gems.study.system import Component, System +from gems_craft.study.scenario_builder import ScenarioBuilder +from gems_runner.model.model import Model +from gems_runner.study.data import DataBase +from gems_runner.study.system import Component, System @dataclass diff --git a/src/gems/study/system.py b/src/gems_runner/study/system.py similarity index 95% rename from src/gems/study/system.py rename to src/gems_runner/study/system.py index 85577d27..308f906a 100644 --- a/src/gems/study/system.py +++ b/src/gems_runner/study/system.py @@ -18,12 +18,12 @@ from dataclasses import dataclass, field, replace from typing import Any, Dict, Iterable, List, Optional -from gems.expression.degree import is_linear -from gems.expression.uses_sum_connections_on import uses_sum_connections_on -from gems.model import PortField, PortType -from gems.model.model import Model -from gems.model.port import PortFieldId -from gems.utils import require_not_none +from gems_runner.expression.degree import is_linear +from gems_runner.expression.uses_sum_connections_on import uses_sum_connections_on +from gems_runner.model import PortField, PortType +from gems_runner.model.model import Model +from gems_runner.model.port import PortFieldId +from gems_runner.utils import require_not_none @dataclass(frozen=True) diff --git a/src/gems/utils.py b/src/gems_runner/utils.py similarity index 81% rename from src/gems/utils.py rename to src/gems_runner/utils.py index fb7eff94..c5aeb6f2 100644 --- a/src/gems/utils.py +++ b/src/gems_runner/utils.py @@ -18,8 +18,6 @@ import pathlib from typing import Any, Callable, Dict, Optional, TypeVar -from pydantic import BaseModel, ConfigDict - T = TypeVar("T") K = TypeVar("K") V = TypeVar("V") @@ -71,19 +69,3 @@ def read_json(filename: str, path: pathlib.Path) -> Dict[str, Any]: with (path / filename).open() as file: data = json.load(file) return data - - -""" -Only _to_kebab adn ModifiedBaseModel are used in the project. -""" - - -# Design note: actual parsing and validation is delegated to pydantic models -def _to_kebab(snake: str) -> str: - return snake.replace("_", "-") - - -class ModifiedBaseModel(BaseModel): - model_config = ConfigDict( - alias_generator=_to_kebab, extra="forbid", populate_by_name=True - ) diff --git a/tests/e2e/functional/conftest.py b/tests/e2e/functional/conftest.py index 057e63a7..6e53e789 100644 --- a/tests/e2e/functional/conftest.py +++ b/tests/e2e/functional/conftest.py @@ -13,9 +13,9 @@ import pytest -from gems.model.parsing import LibrarySchema, parse_yaml_library -from gems.model.resolve_library import Library, resolve_library -from gems.study.parsing import SystemSchema, parse_yaml_components +from gems_craft.model.parsing import LibrarySchema, load_yaml_library +from gems_craft.study.parsing import SystemSchema, load_yaml_system +from gems_runner.model.resolve_library import Library, resolve_library @pytest.fixture(scope="session") @@ -35,36 +35,21 @@ def series_dir() -> Path: @pytest.fixture def input_system(systems_dir: Path) -> SystemSchema: - compo_file = systems_dir / "system.yml" - - with compo_file.open() as c: - return parse_yaml_components(c) + return load_yaml_system(systems_dir / "system.yml") @pytest.fixture def input_library(libs_dir: Path) -> LibrarySchema: - library = libs_dir / "lib_unittest.yml" - - with library.open() as lib: - return parse_yaml_library(lib) + return load_yaml_library(libs_dir / "lib_unittest.yml") @pytest.fixture(scope="session") def lib_dict(libs_dir: Path) -> dict[str, Library]: - lib_file = libs_dir / "lib.yml" - - with lib_file.open() as f: - input_lib = parse_yaml_library(f) - - lib_dict = resolve_library([input_lib]) - return lib_dict + input_lib = load_yaml_library(libs_dir / "lib.yml") + return resolve_library([input_lib]) @pytest.fixture(scope="session") def lib_dict_unittest(libs_dir: Path) -> dict[str, Library]: - lib_file = libs_dir / "lib_unittest.yml" - - with lib_file.open() as f: - input_lib = parse_yaml_library(f) - + input_lib = load_yaml_library(libs_dir / "lib_unittest.yml") return resolve_library([input_lib]) diff --git a/tests/e2e/functional/libs/standard.py b/tests/e2e/functional/libs/standard.py index ca6a93a1..c18e3336 100644 --- a/tests/e2e/functional/libs/standard.py +++ b/tests/e2e/functional/libs/standard.py @@ -14,14 +14,14 @@ The standard module contains the definition of standard models. """ -from gems.expression import literal, param, var -from gems.expression.expression import port_field -from gems.expression.indexing_structure import IndexingStructure -from gems.model.constraint import Constraint -from gems.model.model import ModelPort, model -from gems.model.parameter import float_parameter, int_parameter -from gems.model.port import PortField, PortFieldDefinition, PortFieldId, PortType -from gems.model.variable import float_variable, int_variable +from gems_runner.expression import literal, param, var +from gems_runner.expression.expression import port_field +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model.constraint import Constraint +from gems_runner.model.model import ModelPort, model +from gems_runner.model.parameter import float_parameter, int_parameter +from gems_runner.model.port import PortField, PortFieldDefinition, PortFieldId, PortType +from gems_runner.model.variable import float_variable, int_variable CONSTANT = IndexingStructure(False, False) TIME_AND_SCENARIO_FREE = IndexingStructure(True, True) diff --git a/tests/e2e/functional/perf_pypsa.py b/tests/e2e/functional/perf_pypsa.py index e20934dd..18ef8cb3 100644 --- a/tests/e2e/functional/perf_pypsa.py +++ b/tests/e2e/functional/perf_pypsa.py @@ -5,29 +5,26 @@ import numpy as np import pandas as pd -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.simulation import TimeBlock, build_problem -from gems.study import Study -from gems.study.data import DataBase -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import ( +from gems_craft.model.parsing import load_yaml_library +from gems_craft.study.parsing import load_yaml_system +from gems_runner.model.resolve_library import resolve_library +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study import Study +from gems_runner.study.data import DataBase +from gems_runner.study.resolve_components import ( build_data_base, consistency_check, resolve_system, ) -from gems.study.system import System +from gems_runner.study.system import System def setup_data(pypsa_dir: Path) -> Tuple[System, DataBase]: study_file = pypsa_dir / "input" / "system.yml" lib_file = pypsa_dir / "input" / "model-libraries" / "pypsa_models.yml" series_dir = pypsa_dir / "input" / "data-series" - with lib_file.open() as lib: - input_library = parse_yaml_library(lib) - - with study_file.open() as c: - input_study = parse_yaml_components(c) + input_library = load_yaml_library(lib_file) + input_study = load_yaml_system(study_file) lib_dict = resolve_library([input_library]) system = resolve_system(input_study, lib_dict) consistency_check(system, lib_dict["pypsa_models"].models) diff --git a/tests/e2e/functional/test_build_decomposed_problem.py b/tests/e2e/functional/test_build_decomposed_problem.py index 0ecca041..98a165b9 100644 --- a/tests/e2e/functional/test_build_decomposed_problem.py +++ b/tests/e2e/functional/test_build_decomposed_problem.py @@ -30,15 +30,16 @@ import pytest -from gems.main.main import ( +from gems_craft.optim_config.parsing import load_yaml_optim_config +from gems_runner.main.main import ( _write_structure_txt, input_database, input_libs, input_system, ) -from gems.optim_config.parsing import load_optim_config, validate_optim_config -from gems.simulation import TimeBlock, build_decomposed_problems -from gems.study import Study +from gems_runner.optim_config import validate_optim_config +from gems_runner.simulation import TimeBlock, build_decomposed_problems +from gems_runner.study import Study STUDIES_DIR = Path(__file__).parent / "studies" STUDY_IDS = ["13_1", "13_2"] @@ -61,7 +62,7 @@ def test_study_mps_matches_expected(study_id: str, tmp_path: Path) -> None: # --- Load and validate optim-config --- config_path = input_dir / "optim-config.yml" - optim_config = load_optim_config(config_path) + optim_config = load_yaml_optim_config(config_path) assert optim_config is not None, f"optim-config.yml not found in {input_dir}" validate_optim_config(optim_config, system) diff --git a/tests/e2e/functional/test_component_dependent_time_shift.py b/tests/e2e/functional/test_component_dependent_time_shift.py index 17735656..04714ed0 100644 --- a/tests/e2e/functional/test_component_dependent_time_shift.py +++ b/tests/e2e/functional/test_component_dependent_time_shift.py @@ -80,15 +80,16 @@ import pandas as pd import pytest -from gems.expression import literal, param, var -from gems.expression.indexing_structure import IndexingStructure -from gems.model import ModelPort, float_parameter, float_variable, model -from gems.model.constraint import Constraint -from gems.model.parsing import parse_yaml_library -from gems.model.port import PortFieldDefinition, PortFieldId -from gems.model.resolve_library import resolve_library -from gems.simulation import TimeBlock, build_problem -from gems.study import ( +from gems_craft.model.parsing import load_yaml_library +from gems_craft.study.parsing import load_yaml_system +from gems_runner.expression import literal, param, var +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model import ModelPort, float_parameter, float_variable, model +from gems_runner.model.constraint import Constraint +from gems_runner.model.port import PortFieldDefinition, PortFieldId +from gems_runner.model.resolve_library import resolve_library +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study import ( Component, ConstantData, DataBase, @@ -98,8 +99,7 @@ TimeScenarioSeriesData, create_component, ) -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import ( +from gems_runner.study.resolve_components import ( build_data_base, consistency_check, resolve_system, @@ -406,10 +406,8 @@ def test_two_components_different_lags_yaml( lib_file = _libs_dir / "lib_time_shift.yml" system_file = _systems_dir / "system_time_shift_per_component.yml" - with lib_file.open() as f: - input_library = parse_yaml_library(f) - with system_file.open() as f: - input_system = parse_yaml_components(f) + input_library = load_yaml_library(lib_file) + input_system = load_yaml_system(system_file) lib_dict = resolve_library([input_library]) system = resolve_system(input_system, lib_dict) diff --git a/tests/e2e/functional/test_dual_reduced_cost.py b/tests/e2e/functional/test_dual_reduced_cost.py index 91929624..13418409 100644 --- a/tests/e2e/functional/test_dual_reduced_cost.py +++ b/tests/e2e/functional/test_dual_reduced_cost.py @@ -25,9 +25,9 @@ import pytest -from gems.simulation import TimeBlock, build_problem -from gems.simulation.simulation_table import SimulationTableBuilder -from gems.study.folder import load_study +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.simulation.simulation_table import SimulationTableBuilder +from gems_runner.study.folder import load_study STUDIES_DIR = Path(__file__).parent / "studies" diff --git a/tests/e2e/functional/test_investment.py b/tests/e2e/functional/test_investment.py index 03c4d2ef..87d0d6e3 100644 --- a/tests/e2e/functional/test_investment.py +++ b/tests/e2e/functional/test_investment.py @@ -17,9 +17,16 @@ import pandas as pd import pytest -from gems.expression.expression import literal, param, var -from gems.expression.indexing_structure import IndexingStructure -from gems.model import ( +from gems_craft.optim_config.parsing import ( + ElementLocation, + ElementLocationConfig, + ModelDecompositionConfig, + ModelOptimConfig, + OptimConfig, +) +from gems_runner.expression.expression import literal, param, var +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model import ( Constraint, Model, ModelPort, @@ -28,18 +35,11 @@ int_variable, model, ) -from gems.model.port import PortFieldDefinition, PortFieldId -from gems.optim_config.parsing import ( - ElementLocation, - ElementLocationConfig, - ModelDecompositionConfig, - ModelOptimConfig, - OptimConfig, - validate_optim_config, -) -from gems.simulation import TimeBlock, build_problem -from gems.simulation.simulation_table import SimulationTableBuilder -from gems.study import ( +from gems_runner.model.port import PortFieldDefinition, PortFieldId +from gems_runner.optim_config import validate_optim_config +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.simulation.simulation_table import SimulationTableBuilder +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/functional/test_libs_python_system_python.py b/tests/e2e/functional/test_libs_python_system_python.py index df6a1bf7..591ec103 100644 --- a/tests/e2e/functional/test_libs_python_system_python.py +++ b/tests/e2e/functional/test_libs_python_system_python.py @@ -37,12 +37,12 @@ import pandas as pd import pytest -from gems.expression import literal, param, var -from gems.expression.indexing_structure import IndexingStructure -from gems.model import Model, ModelPort, float_parameter, float_variable, model -from gems.model.port import PortFieldDefinition, PortFieldId -from gems.simulation import TimeBlock, build_problem -from gems.study import ( +from gems_runner.expression import literal, param, var +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model import Model, ModelPort, float_parameter, float_variable, model +from gems_runner.model.port import PortFieldDefinition, PortFieldId +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/functional/test_libs_yaml_system_python.py b/tests/e2e/functional/test_libs_yaml_system_python.py index 1407f73f..15b1eba5 100644 --- a/tests/e2e/functional/test_libs_yaml_system_python.py +++ b/tests/e2e/functional/test_libs_yaml_system_python.py @@ -33,9 +33,9 @@ import pandas as pd import pytest -from gems.model.library import Library -from gems.simulation import TimeBlock, build_problem -from gems.study import ( +from gems_runner.model.library import Library +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/functional/test_libs_yaml_system_yaml.py b/tests/e2e/functional/test_libs_yaml_system_yaml.py index 635d9f89..a9f42452 100644 --- a/tests/e2e/functional/test_libs_yaml_system_yaml.py +++ b/tests/e2e/functional/test_libs_yaml_system_yaml.py @@ -41,18 +41,18 @@ import pytest -from gems.model.parsing import LibrarySchema, parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.simulation import TimeBlock, build_problem -from gems.study.data import DataBase -from gems.study.parsing import SystemSchema, parse_yaml_components -from gems.study.resolve_components import ( +from gems_craft.model.parsing import LibrarySchema, load_yaml_library +from gems_craft.study.parsing import SystemSchema, load_yaml_system +from gems_runner.model.resolve_library import resolve_library +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study.data import DataBase +from gems_runner.study.resolve_components import ( build_data_base, consistency_check, resolve_system, ) -from gems.study.study import Study -from gems.study.system import System +from gems_runner.study.study import Study +from gems_runner.study.system import System def test_basic_balance_using_yaml( @@ -80,11 +80,8 @@ def setup_test( def _setup_test(study_file_name: str) -> Study: study_file = systems_dir / study_file_name lib_file = libs_dir / "lib_unittest.yml" - with lib_file.open() as lib: - input_library = parse_yaml_library(lib) - - with study_file.open() as c: - input_system = parse_yaml_components(c) + input_library = load_yaml_library(lib_file) + input_system = load_yaml_system(study_file) lib_dict = resolve_library([input_library]) system = resolve_system(input_system, lib_dict) consistency_check(system, lib_dict["basic"].models) diff --git a/tests/e2e/functional/test_optim_modes.py b/tests/e2e/functional/test_optim_modes.py index 9f314e11..569a22e4 100644 --- a/tests/e2e/functional/test_optim_modes.py +++ b/tests/e2e/functional/test_optim_modes.py @@ -28,7 +28,7 @@ import pandas as pd import pytest -from gems.study.runner import run_study +from gems_runner.study.runner import run_study _STUDY_SRC = Path(__file__).parent / "studies" / "dsr_3_blocks" diff --git a/tests/e2e/functional/test_out_of_bounds_processing.py b/tests/e2e/functional/test_out_of_bounds_processing.py index fad8cebe..fb0d4612 100644 --- a/tests/e2e/functional/test_out_of_bounds_processing.py +++ b/tests/e2e/functional/test_out_of_bounds_processing.py @@ -70,10 +70,10 @@ import linopy import pytest -from gems.optim_config.parsing import load_optim_config -from gems.simulation import TimeBlock, build_decomposed_problems -from gems.simulation.simulation_table import SimulationTableBuilder -from gems.study.folder import load_study +from gems_craft.optim_config.parsing import load_yaml_optim_config +from gems_runner.simulation import TimeBlock, build_decomposed_problems +from gems_runner.simulation.simulation_table import SimulationTableBuilder +from gems_runner.study.folder import load_study STUDIES_DIR = Path(__file__).parent / "studies" @@ -96,7 +96,7 @@ def _count_active(model: linopy.Model, name: str) -> int: def test_out_of_bounds_processing(study_id: str, expected_objective: float) -> None: study = load_study(STUDIES_DIR / study_id) config_path = STUDIES_DIR / study_id / "input" / "optim-config.yml" - optim_config = load_optim_config(config_path) + optim_config = load_yaml_optim_config(config_path) # 3-step block matching the study parameters (first-time-step: 0, last-time-step: 2) time_block = TimeBlock(1, [0, 1, 2]) @@ -169,7 +169,7 @@ def test_constraint_instantiation(study_id: str) -> None: instantiated at exactly the expected (component, time) pairs.""" study = load_study(STUDIES_DIR / study_id) config_path = STUDIES_DIR / study_id / "input" / "optim-config.yml" - optim_config = load_optim_config(config_path) + optim_config = load_yaml_optim_config(config_path) time_block = TimeBlock(1, [0, 1, 2]) decomposed = build_decomposed_problems(study, time_block, [0], optim_config) linopy_model = decomposed.subproblem.linopy_model diff --git a/tests/e2e/functional/test_performance.py b/tests/e2e/functional/test_performance.py index ef162f9b..337cab8b 100644 --- a/tests/e2e/functional/test_performance.py +++ b/tests/e2e/functional/test_performance.py @@ -16,11 +16,11 @@ import pandas as pd -from gems.expression.expression import ExpressionNode, literal, param, var -from gems.expression.indexing_structure import IndexingStructure -from gems.model import float_parameter, float_variable, model -from gems.simulation import TimeBlock, build_problem -from gems.study import ( +from gems_runner.expression.expression import ExpressionNode, literal, param, var +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model import float_parameter, float_variable, model +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study import ( Component, ConstantData, DataBase, @@ -29,7 +29,7 @@ System, create_component, ) -from gems.study.data import TimeScenarioSeriesData +from gems_runner.study.data import TimeScenarioSeriesData from tests.e2e.functional.libs.standard import ( DEMAND_MODEL, GENERATOR_MODEL, diff --git a/tests/e2e/functional/test_rolling_horizon_suboptimality.py b/tests/e2e/functional/test_rolling_horizon_suboptimality.py index 316d6a18..840c563e 100644 --- a/tests/e2e/functional/test_rolling_horizon_suboptimality.py +++ b/tests/e2e/functional/test_rolling_horizon_suboptimality.py @@ -64,7 +64,7 @@ import pytest -from gems.study.runner import run_study +from gems_runner.study.runner import run_study _STUDY_SRC = Path(__file__).parent / "studies" / "rolling_horizon_suboptimality" diff --git a/tests/e2e/functional/test_scalability.py b/tests/e2e/functional/test_scalability.py index 40585a4f..96367ba1 100644 --- a/tests/e2e/functional/test_scalability.py +++ b/tests/e2e/functional/test_scalability.py @@ -3,8 +3,8 @@ import numpy as np import pandas as pd -from gems.simulation import TimeBlock, build_problem -from gems.study import ( +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/functional/test_scenario_builder.py b/tests/e2e/functional/test_scenario_builder.py index b91a4bf8..32c399bc 100644 --- a/tests/e2e/functional/test_scenario_builder.py +++ b/tests/e2e/functional/test_scenario_builder.py @@ -14,24 +14,24 @@ import pytest -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.simulation import build_problem -from gems.simulation.time_block import TimeBlock -from gems.study import Study -from gems.study.data import DataBase -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import ( +from gems_craft.model.parsing import load_yaml_library +from gems_craft.study.parsing import load_yaml_system +from gems_craft.study.scenario_builder import ScenarioBuilder +from gems_runner.model.resolve_library import resolve_library +from gems_runner.simulation import build_problem +from gems_runner.simulation.time_block import TimeBlock +from gems_runner.study import Study +from gems_runner.study.data import DataBase +from gems_runner.study.resolve_components import ( build_data_base, consistency_check, resolve_system, ) -from gems.study.scenario_builder import ScenarioBuilder @pytest.fixture def scenario_builder(series_dir: Path) -> ScenarioBuilder: - return ScenarioBuilder.load(series_dir / "modeler-scenariobuilder.dat") + return ScenarioBuilder.load_dat(series_dir / "modeler-scenariobuilder.dat") @pytest.fixture @@ -39,24 +39,19 @@ def database( series_dir: Path, systems_dir: Path, scenario_builder: ScenarioBuilder ) -> DataBase: system_path = systems_dir / "with_scenarization.yml" - with system_path.open() as components: - return build_data_base( - parse_yaml_components(components), series_dir, scenario_builder - ) + return build_data_base(load_yaml_system(system_path), series_dir, scenario_builder) def test_system_with_scenarization( libs_dir: Path, systems_dir: Path, database: DataBase ) -> None: library_path = libs_dir / "lib_unittest.yml" - with library_path.open("r") as file: - yaml_lib = parse_yaml_library(file) - lib_dict = resolve_library([yaml_lib]) + yaml_lib = load_yaml_library(library_path) + lib_dict = resolve_library([yaml_lib]) components_path = systems_dir / "with_scenarization.yml" - with components_path.open("r") as file: - yaml_comp = parse_yaml_components(file) - components = resolve_system(yaml_comp, lib_dict) + yaml_comp = load_yaml_system(components_path) + components = resolve_system(yaml_comp, lib_dict) consistency_check(components, lib_dict["basic"].models) diff --git a/tests/e2e/functional/test_simtable_timeblock.py b/tests/e2e/functional/test_simtable_timeblock.py index c4a4822c..f88c99e8 100644 --- a/tests/e2e/functional/test_simtable_timeblock.py +++ b/tests/e2e/functional/test_simtable_timeblock.py @@ -25,10 +25,13 @@ import pandas as pd import pytest -from gems.model.library import Library -from gems.simulation import TimeBlock, build_problem -from gems.simulation.simulation_table import SimulationColumns, SimulationTableBuilder -from gems.study import ( +from gems_runner.model.library import Library +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.simulation.simulation_table import ( + SimulationColumns, + SimulationTableBuilder, +) +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/functional/test_stochastic.py b/tests/e2e/functional/test_stochastic.py index f4c993ab..cbab009d 100644 --- a/tests/e2e/functional/test_stochastic.py +++ b/tests/e2e/functional/test_stochastic.py @@ -15,8 +15,8 @@ import pandas as pd import pytest -from gems.simulation import TimeBlock, build_problem -from gems.study import ( +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study import ( Component, ConstantData, DataBase, @@ -25,7 +25,7 @@ System, create_component, ) -from gems.study.data import TimeScenarioSeriesData +from gems_runner.study.data import TimeScenarioSeriesData from tests.e2e.functional.libs.standard import ( DEMAND_MODEL, NODE_BALANCE_MODEL, diff --git a/tests/e2e/functional/test_study_from_folder.py b/tests/e2e/functional/test_study_from_folder.py index 73f34445..654f3472 100644 --- a/tests/e2e/functional/test_study_from_folder.py +++ b/tests/e2e/functional/test_study_from_folder.py @@ -3,8 +3,8 @@ import pandas as pd -from gems.study.folder import load_study -from gems.study.runner import run_study +from gems_runner.study.folder import load_study +from gems_runner.study.runner import run_study def test_load_study(): diff --git a/tests/e2e/models/andromede-v1/test_andromede_v1_models.py b/tests/e2e/models/andromede-v1/test_andromede_v1_models.py index 53af1ea3..4d692568 100644 --- a/tests/e2e/models/andromede-v1/test_andromede_v1_models.py +++ b/tests/e2e/models/andromede-v1/test_andromede_v1_models.py @@ -17,13 +17,13 @@ import pandas as pd import pytest -from gems.model.parsing import LibrarySchema, parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.simulation import build_problem -from gems.simulation.time_block import TimeBlock -from gems.study import Study -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import build_data_base, resolve_system +from gems_craft.model.parsing import LibrarySchema, load_yaml_library +from gems_craft.study.parsing import load_yaml_system +from gems_runner.model.resolve_library import resolve_library +from gems_runner.simulation import build_problem +from gems_runner.simulation.time_block import TimeBlock +from gems_runner.study import Study +from gems_runner.study.resolve_components import build_data_base, resolve_system @pytest.fixture @@ -49,10 +49,8 @@ def series_dir(data_dir: Path) -> Path: @pytest.fixture def input_libraries(data_dir: Path) -> List[LibrarySchema]: libs_dir = data_dir / "libs" - with open(libs_dir / "antares_historic.yml") as lib_file: - lib_historic = parse_yaml_library(lib_file) - with open(libs_dir / "andromede_v1_models.yml") as lib_file: - lib_v1 = parse_yaml_library(lib_file) + lib_historic = load_yaml_library(libs_dir / "antares_historic.yml") + lib_v1 = load_yaml_library(libs_dir / "andromede_v1_models.yml") return [lib_historic, lib_v1] @@ -122,8 +120,7 @@ def test_model_behaviour( series_dir: Path, ) -> None: scenarios = 1 - with open(systems_dir / system_file) as compo_file: - input_component = parse_yaml_components(compo_file) + input_component = load_yaml_system(systems_dir / system_file) result_lib = resolve_library(input_libraries) system_input = resolve_system(input_component, result_lib) database = build_data_base(input_component, Path(series_dir)) diff --git a/tests/e2e/models/operators/test_operators_v1.py b/tests/e2e/models/operators/test_operators_v1.py index 6e32e3ed..1b13d116 100644 --- a/tests/e2e/models/operators/test_operators_v1.py +++ b/tests/e2e/models/operators/test_operators_v1.py @@ -17,14 +17,14 @@ import pandas as pd import pytest -from gems.model.parsing import LibrarySchema, parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.simulation import build_problem -from gems.simulation.simulation_table import SimulationTableBuilder -from gems.simulation.time_block import TimeBlock -from gems.study import Study -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import build_data_base, resolve_system +from gems_craft.model.parsing import LibrarySchema, load_yaml_library +from gems_craft.study.parsing import load_yaml_system +from gems_runner.model.resolve_library import resolve_library +from gems_runner.simulation import build_problem +from gems_runner.simulation.simulation_table import SimulationTableBuilder +from gems_runner.simulation.time_block import TimeBlock +from gems_runner.study import Study +from gems_runner.study.resolve_components import build_data_base, resolve_system @pytest.fixture @@ -70,8 +70,7 @@ def relative_accuracy() -> float: @pytest.fixture def input_libraries(input_dir: Path) -> List[LibrarySchema]: libs_dir = input_dir / "model-libraries" - with open(libs_dir / "test_lib.yml") as lib_file: - lib_new = parse_yaml_library(lib_file) + lib_new = load_yaml_library(libs_dir / "test_lib.yml") return [lib_new] @@ -101,8 +100,7 @@ def test_model_behaviour( last_timestep = 167 timesteps = list(range(first_timestep, last_timestep + 1)) - with open(system_file) as compo_file: - input_component = parse_yaml_components(compo_file) + input_component = load_yaml_system(system_file) result_lib = resolve_library(input_libraries) system_input = resolve_system(input_component, result_lib) diff --git a/tests/e2e/models/poc-various-models/conftest.py b/tests/e2e/models/poc-various-models/conftest.py index 8c69fb0c..6965b11b 100644 --- a/tests/e2e/models/poc-various-models/conftest.py +++ b/tests/e2e/models/poc-various-models/conftest.py @@ -13,8 +13,8 @@ import pytest -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import Library, resolve_library +from gems_craft.model.parsing import load_yaml_library +from gems_runner.model.resolve_library import Library, resolve_library @pytest.fixture(scope="session") @@ -29,21 +29,11 @@ def data_dir() -> Path: @pytest.fixture(scope="session") def lib_dict(libs_dir: Path) -> dict[str, Library]: - lib_file = libs_dir / "lib_unittest.yml" - - with lib_file.open() as f: - input_lib = parse_yaml_library(f) - - lib_dict = resolve_library([input_lib]) - return lib_dict + input_lib = load_yaml_library(libs_dir / "lib_unittest.yml") + return resolve_library([input_lib]) @pytest.fixture(scope="session") def lib_dict_sc(libs_dir: Path) -> dict[str, Library]: - lib_sc_file = libs_dir / "standard_sc.yml" - - with lib_sc_file.open() as f: - input_lib_sc = parse_yaml_library(f) - - lib_dict_sc = resolve_library([input_lib_sc]) - return lib_dict_sc + input_lib_sc = load_yaml_library(libs_dir / "standard_sc.yml") + return resolve_library([input_lib_sc]) diff --git a/tests/e2e/models/poc-various-models/libs/standard.py b/tests/e2e/models/poc-various-models/libs/standard.py index ca6a93a1..c18e3336 100644 --- a/tests/e2e/models/poc-various-models/libs/standard.py +++ b/tests/e2e/models/poc-various-models/libs/standard.py @@ -14,14 +14,14 @@ The standard module contains the definition of standard models. """ -from gems.expression import literal, param, var -from gems.expression.expression import port_field -from gems.expression.indexing_structure import IndexingStructure -from gems.model.constraint import Constraint -from gems.model.model import ModelPort, model -from gems.model.parameter import float_parameter, int_parameter -from gems.model.port import PortField, PortFieldDefinition, PortFieldId, PortType -from gems.model.variable import float_variable, int_variable +from gems_runner.expression import literal, param, var +from gems_runner.expression.expression import port_field +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model.constraint import Constraint +from gems_runner.model.model import ModelPort, model +from gems_runner.model.parameter import float_parameter, int_parameter +from gems_runner.model.port import PortField, PortFieldDefinition, PortFieldId, PortType +from gems_runner.model.variable import float_variable, int_variable CONSTANT = IndexingStructure(False, False) TIME_AND_SCENARIO_FREE = IndexingStructure(True, True) diff --git a/tests/e2e/models/poc-various-models/libs/standard_sc.py b/tests/e2e/models/poc-various-models/libs/standard_sc.py index 64665071..b478af26 100644 --- a/tests/e2e/models/poc-various-models/libs/standard_sc.py +++ b/tests/e2e/models/poc-various-models/libs/standard_sc.py @@ -12,9 +12,9 @@ from libs.standard import BALANCE_PORT_TYPE, CONSTANT -from gems.expression import literal, param, var -from gems.expression.expression import port_field -from gems.model import ( +from gems_runner.expression import literal, param, var +from gems_runner.expression.expression import port_field +from gems_runner.model import ( Constraint, ModelPort, PortField, @@ -23,7 +23,7 @@ float_variable, model, ) -from gems.model.port import PortFieldDefinition, PortFieldId +from gems_runner.model.port import PortFieldDefinition, PortFieldId """ Simple Convertor model. diff --git a/tests/e2e/models/poc-various-models/test_ac_link.py b/tests/e2e/models/poc-various-models/test_ac_link.py index 15adc9e7..24f9cf3c 100644 --- a/tests/e2e/models/poc-various-models/test_ac_link.py +++ b/tests/e2e/models/poc-various-models/test_ac_link.py @@ -14,12 +14,12 @@ import pytest from libs.standard import BALANCE_PORT_TYPE, DEMAND_MODEL, GENERATOR_MODEL -from gems.model.library import Library, library -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.simulation import TimeBlock, build_problem -from gems.simulation.simulation_table import SimulationTableBuilder -from gems.study import ( +from gems_craft.model.parsing import load_yaml_library +from gems_runner.model.library import Library, library +from gems_runner.model.resolve_library import resolve_library +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.simulation.simulation_table import SimulationTableBuilder +from gems_runner.study import ( Component, ConstantData, DataBase, @@ -39,10 +39,8 @@ def std_lib() -> Library: @pytest.fixture def ac_lib(libs_dir: Path, std_lib: Library) -> dict[str, Library]: - lib_file = libs_dir / "ac.yml" - with lib_file.open() as f: - input_lib = parse_yaml_library(f) - return resolve_library([input_lib], preloaded_libs=[std_lib]) + input_lib = load_yaml_library(libs_dir / "ac.yml") + return resolve_library([input_lib], preloaded_libs=[std_lib]) def test_ac_network_no_links(ac_lib: dict[str, Library]) -> None: diff --git a/tests/e2e/models/poc-various-models/test_electrolyzer.py b/tests/e2e/models/poc-various-models/test_electrolyzer.py index eb1c2179..0956fc74 100644 --- a/tests/e2e/models/poc-various-models/test_electrolyzer.py +++ b/tests/e2e/models/poc-various-models/test_electrolyzer.py @@ -12,9 +12,9 @@ from libs.standard import CONSTANT, TIME_AND_SCENARIO_FREE -from gems.expression import literal, param, var -from gems.expression.expression import port_field -from gems.model import ( +from gems_runner.expression import literal, param, var +from gems_runner.expression.expression import port_field +from gems_runner.model import ( Constraint, ModelPort, PortField, @@ -23,9 +23,9 @@ float_variable, model, ) -from gems.model.port import PortFieldDefinition, PortFieldId -from gems.simulation import TimeBlock, build_problem -from gems.study import ( +from gems_runner.model.port import PortFieldDefinition, PortFieldId +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/models/poc-various-models/test_electrolyzer_n_inputs.py b/tests/e2e/models/poc-various-models/test_electrolyzer_n_inputs.py index cf5eacfc..bf79f63f 100644 --- a/tests/e2e/models/poc-various-models/test_electrolyzer_n_inputs.py +++ b/tests/e2e/models/poc-various-models/test_electrolyzer_n_inputs.py @@ -21,9 +21,9 @@ TWO_INPUTS_CONVERTOR_MODEL, ) -from gems.simulation import TimeBlock, build_problem -from gems.simulation.simulation_table import SimulationTableBuilder -from gems.study import ( +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.simulation.simulation_table import SimulationTableBuilder +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/models/poc-various-models/test_electrolyzer_n_inputs_yaml.py b/tests/e2e/models/poc-various-models/test_electrolyzer_n_inputs_yaml.py index f838ef49..07c3785d 100644 --- a/tests/e2e/models/poc-various-models/test_electrolyzer_n_inputs_yaml.py +++ b/tests/e2e/models/poc-various-models/test_electrolyzer_n_inputs_yaml.py @@ -12,10 +12,10 @@ import math -from gems.model.library import Library -from gems.simulation import TimeBlock, build_problem -from gems.simulation.simulation_table import SimulationTableBuilder -from gems.study import ( +from gems_runner.model.library import Library +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.simulation.simulation_table import SimulationTableBuilder +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/models/poc-various-models/test_quota_co2.py b/tests/e2e/models/poc-various-models/test_quota_co2.py index 3f721988..20a8fe03 100644 --- a/tests/e2e/models/poc-various-models/test_quota_co2.py +++ b/tests/e2e/models/poc-various-models/test_quota_co2.py @@ -19,9 +19,9 @@ from libs.standard import DEMAND_MODEL, LINK_MODEL, NODE_BALANCE_MODEL from libs.standard_sc import C02_POWER_MODEL, QUOTA_CO2_MODEL -from gems.simulation import TimeBlock, build_problem -from gems.simulation.simulation_table import SimulationTableBuilder -from gems.study import ( +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.simulation.simulation_table import SimulationTableBuilder +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/models/poc-various-models/test_quota_co2_yaml.py b/tests/e2e/models/poc-various-models/test_quota_co2_yaml.py index 0cafcc47..5cb588c1 100644 --- a/tests/e2e/models/poc-various-models/test_quota_co2_yaml.py +++ b/tests/e2e/models/poc-various-models/test_quota_co2_yaml.py @@ -16,10 +16,10 @@ import math -from gems.model.library import Library -from gems.simulation import TimeBlock, build_problem -from gems.simulation.simulation_table import SimulationTableBuilder -from gems.study import ( +from gems_runner.model.library import Library +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.simulation.simulation_table import SimulationTableBuilder +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/e2e/models/poc-various-models/test_short_term_storage_complex.py b/tests/e2e/models/poc-various-models/test_short_term_storage_complex.py index 156daff3..fab10cb2 100644 --- a/tests/e2e/models/poc-various-models/test_short_term_storage_complex.py +++ b/tests/e2e/models/poc-various-models/test_short_term_storage_complex.py @@ -9,8 +9,8 @@ ) from libs.standard_sc import SHORT_TERM_STORAGE_COMPLEX -from gems.simulation import TimeBlock, build_problem -from gems.study import ( +from gems_runner.simulation import TimeBlock, build_problem +from gems_runner.study import ( Component, ConstantData, DataBase, diff --git a/tests/unittests/data/test_data.py b/tests/unittests/data/test_data.py index 6bbbcbae..4ebae45d 100644 --- a/tests/unittests/data/test_data.py +++ b/tests/unittests/data/test_data.py @@ -19,7 +19,10 @@ import pandas as pd import pytest -from gems.study.data import ( +from gems_craft.study.parsing import _parse_yaml_components +from gems_craft.study.scenario_builder import ScenarioBuilder +from gems_craft.study.timeseries import load_ts_from_file +from gems_runner.study.data import ( ConstantData, DataBase, ScenarioSeriesData, @@ -27,11 +30,8 @@ TimeSeriesData, dataframe_to_scenario_series, dataframe_to_time_series, - load_ts_from_file, ) -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import _build_data, build_data_base -from gems.study.scenario_builder import ScenarioBuilder +from gems_runner.study.resolve_components import _build_data, build_data_base # --------------------------------------------------------------------------- # load_ts_from_file @@ -206,11 +206,11 @@ def test_build_data_base_param_group_overrides_component_group( (tmp_path / "series.txt").write_text("100 200\n") sb = ScenarioBuilder() db = build_data_base( - parse_yaml_components(io.StringIO(_SYSTEM_WITH_PARAM_GROUP)), + _parse_yaml_components(io.StringIO(_SYSTEM_WITH_PARAM_GROUP)), tmp_path, scenario_builder=sb, ) - from gems.study.data import ComponentParameterIndex + from gems_runner.study.data import ComponentParameterIndex # p_max has param-group; cost has component-group (constant so group is irrelevant) idx_pmax = ComponentParameterIndex("G", "p_max") @@ -250,7 +250,7 @@ def test_database_get_value_constant_returns_scalar() -> None: """get_value on ConstantData returns a plain float, not a numpy array.""" db = DataBase() db.add_data("C", "val", ConstantData(7.5)) - from gems.study.data import ComponentParameterIndex + from gems_runner.study.data import ComponentParameterIndex result = db.get_value(ComponentParameterIndex("C", "val"), 0, 0) assert result == 7.5 diff --git a/tests/unittests/expressions/parsing/test_expression_parsing.py b/tests/unittests/expressions/parsing/test_expression_parsing.py index bc20c899..666f1c95 100644 --- a/tests/unittests/expressions/parsing/test_expression_parsing.py +++ b/tests/unittests/expressions/parsing/test_expression_parsing.py @@ -13,16 +13,16 @@ import pytest -from gems.expression import ExpressionNode, literal, param, print_expr, var -from gems.expression.equality import expressions_equal -from gems.expression.expression import ( +from gems_runner.expression import ExpressionNode, literal, param, print_expr, var +from gems_runner.expression.equality import expressions_equal +from gems_runner.expression.expression import ( DualNode, ReducedCostNode, maximum, minimum, port_field, ) -from gems.expression.parsing.parse_expression import ( +from gems_runner.expression.parsing.parse_expression import ( ModelIdentifiers, ParsingException, parse_expression, diff --git a/tests/unittests/expressions/visitor/test_copy.py b/tests/unittests/expressions/visitor/test_copy.py index b4a1aead..4d7a9a79 100644 --- a/tests/unittests/expressions/visitor/test_copy.py +++ b/tests/unittests/expressions/visitor/test_copy.py @@ -13,16 +13,16 @@ import time -from gems.expression import ( +from gems_runner.expression import ( AdditionNode, DivisionNode, LiteralNode, ParameterNode, VariableNode, ) -from gems.expression.copy import copy_expression -from gems.expression.equality import expressions_equal -from gems.expression.expression import ( +from gems_runner.expression.copy import copy_expression +from gems_runner.expression.equality import expressions_equal +from gems_runner.expression.expression import ( AllTimeSumNode, MultiplicationNode, TimeEvalNode, diff --git a/tests/unittests/expressions/visitor/test_degree.py b/tests/unittests/expressions/visitor/test_degree.py index 7f640f8f..c22d2d7f 100644 --- a/tests/unittests/expressions/visitor/test_degree.py +++ b/tests/unittests/expressions/visitor/test_degree.py @@ -14,7 +14,7 @@ import pytest -from gems.expression import ( +from gems_runner.expression import ( ExpressionDegreeVisitor, LiteralNode, maximum, @@ -23,7 +23,7 @@ var, visit, ) -from gems.expression.expression import ( +from gems_runner.expression.expression import ( AbsNode, CeilNode, DualNode, diff --git a/tests/unittests/expressions/visitor/test_equality.py b/tests/unittests/expressions/visitor/test_equality.py index a64a8fba..b02273f6 100644 --- a/tests/unittests/expressions/visitor/test_equality.py +++ b/tests/unittests/expressions/visitor/test_equality.py @@ -12,9 +12,14 @@ import pytest -from gems.expression import ExpressionNode, copy_expression, literal, param, var -from gems.expression.equality import expressions_equal -from gems.expression.expression import DualNode, ReducedCostNode, maximum, minimum +from gems_runner.expression import ExpressionNode, copy_expression, literal, param, var +from gems_runner.expression.equality import expressions_equal +from gems_runner.expression.expression import ( + DualNode, + ReducedCostNode, + maximum, + minimum, +) @pytest.mark.parametrize( diff --git a/tests/unittests/expressions/visitor/test_evaluation.py b/tests/unittests/expressions/visitor/test_evaluation.py index 6e7c2b29..4017f7ab 100644 --- a/tests/unittests/expressions/visitor/test_evaluation.py +++ b/tests/unittests/expressions/visitor/test_evaluation.py @@ -12,7 +12,7 @@ import pytest -from gems.expression import ( +from gems_runner.expression import ( AdditionNode, DivisionNode, EvaluationContext, @@ -29,8 +29,8 @@ var, visit, ) -from gems.expression.equality import expressions_equal -from gems.expression.expression import DualNode, ReducedCostNode +from gems_runner.expression.equality import expressions_equal +from gems_runner.expression.expression import DualNode, ReducedCostNode def test_ast() -> None: @@ -66,7 +66,7 @@ def test_sum_expressions() -> None: def test_floor_ceil_max_min() -> None: - from gems.expression.expression import maximum, minimum + from gems_runner.expression.expression import maximum, minimum context = EvaluationContext(parameters={"p": 2.7, "q": 1.3}) diff --git a/tests/unittests/expressions/visitor/test_indexing.py b/tests/unittests/expressions/visitor/test_indexing.py index c55250de..aa21de5b 100644 --- a/tests/unittests/expressions/visitor/test_indexing.py +++ b/tests/unittests/expressions/visitor/test_indexing.py @@ -11,10 +11,13 @@ # This file is part of the Antares project. -from gems.expression import param, var -from gems.expression.expression import DualNode, ReducedCostNode -from gems.expression.indexing import IndexingStructureProvider, compute_indexation -from gems.expression.indexing_structure import IndexingStructure +from gems_runner.expression import param, var +from gems_runner.expression.expression import DualNode, ReducedCostNode +from gems_runner.expression.indexing import ( + IndexingStructureProvider, + compute_indexation, +) +from gems_runner.expression.indexing_structure import IndexingStructure class StructureProvider(IndexingStructureProvider): diff --git a/tests/unittests/expressions/visitor/test_printer.py b/tests/unittests/expressions/visitor/test_printer.py index 439e23d3..c64835ec 100644 --- a/tests/unittests/expressions/visitor/test_printer.py +++ b/tests/unittests/expressions/visitor/test_printer.py @@ -10,8 +10,8 @@ # # This file is part of the Antares project. -from gems.expression import ExpressionNode, PrinterVisitor, param, var, visit -from gems.expression.expression import DualNode, ReducedCostNode +from gems_runner.expression import ExpressionNode, PrinterVisitor, param, var, visit +from gems_runner.expression.expression import DualNode, ReducedCostNode def test_comparison() -> None: @@ -23,7 +23,7 @@ def test_comparison() -> None: def test_floor_ceil_max_min_printer() -> None: - from gems.expression.expression import maximum, minimum + from gems_runner.expression.expression import maximum, minimum p = param("p") q = param("q") diff --git a/tests/unittests/gems_craft/__init__.py b/tests/unittests/gems_craft/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/unittests/gems_craft/test_writers.py b/tests/unittests/gems_craft/test_writers.py new file mode 100644 index 00000000..47352a28 --- /dev/null +++ b/tests/unittests/gems_craft/test_writers.py @@ -0,0 +1,246 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. + +"""Tests for gems_craft write functions: system, library, optim-config, scenario builder.""" + +from pathlib import Path + +import numpy as np +import pytest + +from gems_craft.model.parsing import ( + LibrarySchema, + load_yaml_library, + write_yaml_library, +) +from gems_craft.optim_config import load_yaml_optim_config, write_yaml_optim_config +from gems_craft.optim_config.parsing import ( + OptimConfig, + ResolutionConfig, + ResolutionMode, + ScenarioScopeConfig, + SolverOptionsConfig, + TimeScopeConfig, +) +from gems_craft.study.parsing import SystemSchema, load_yaml_system, write_yaml_system +from gems_craft.study.scenario_builder import ScenarioBuilder + +# --------------------------------------------------------------------------- +# write_yaml_system +# --------------------------------------------------------------------------- + + +def test_write_yaml_system_roundtrip(tmp_path: Path) -> None: + """A SystemSchema written then re-parsed must equal the original.""" + original_path = ( + Path(__file__).parent.parent / "system_parsing" / "systems" / "system.yml" + ) + original = load_yaml_system(original_path) + + out = tmp_path / "system.yml" + write_yaml_system(original, out) + + reloaded = load_yaml_system(out) + assert reloaded == original + + +def test_write_yaml_system_creates_parent_dirs(tmp_path: Path) -> None: + """write_yaml_system creates missing parent directories.""" + system = SystemSchema(components=[], connections=None) + out = tmp_path / "sub" / "dir" / "system.yml" + write_yaml_system(system, out) + assert out.exists() + + +def test_write_yaml_system_uses_kebab_keys(tmp_path: Path) -> None: + """Output YAML must use kebab-case keys (e.g. 'time-dependent', not 'time_dependent').""" + from gems_craft.study.parsing import ComponentParameterSchema, ComponentSchema + + system = SystemSchema( + components=[ + ComponentSchema( + id="gen", + model="lib.generator", + parameters=[ + ComponentParameterSchema( + id="cost", + time_dependent=True, + scenario_dependent=False, + value=30.0, + ) + ], + ) + ] + ) + out = tmp_path / "system.yml" + write_yaml_system(system, out) + content = out.read_text() + assert "time-dependent" in content + assert "time_dependent" not in content + + +# --------------------------------------------------------------------------- +# write_yaml_library +# --------------------------------------------------------------------------- + + +def test_write_yaml_library_roundtrip(tmp_path: Path) -> None: + """A LibrarySchema written then re-parsed must equal the original.""" + original_path = ( + Path(__file__).parent.parent / "lib_parsing" / "libs" / "basic_lib.yml" + ) + original = load_yaml_library(original_path) + + out = tmp_path / "lib.yml" + write_yaml_library(original, out) + + reloaded = load_yaml_library(out) + assert reloaded == original + + +def test_write_yaml_library_creates_parent_dirs(tmp_path: Path) -> None: + """write_yaml_library creates missing parent directories.""" + library = LibrarySchema(id="empty") + out = tmp_path / "sub" / "lib.yml" + write_yaml_library(library, out) + assert out.exists() + + +def test_write_yaml_library_uses_kebab_keys(tmp_path: Path) -> None: + """Output YAML must use kebab-case keys (e.g. 'port-types', not 'port_types').""" + from gems_craft.model.parsing import FieldSchema, PortTypeSchema + + library = LibrarySchema( + id="mylib", + port_types=[PortTypeSchema(id="flow", fields=[FieldSchema(id="flow")])], + ) + out = tmp_path / "lib.yml" + write_yaml_library(library, out) + content = out.read_text() + assert "port-types" in content + assert "port_types" not in content + + +# --------------------------------------------------------------------------- +# write_yaml_optim_config +# --------------------------------------------------------------------------- + + +def test_write_yaml_optim_config_roundtrip(tmp_path: Path) -> None: + """An OptimConfig written then re-loaded must equal the original.""" + config = OptimConfig( + time_scope=TimeScopeConfig(first_time_step=0, last_time_step=8759), + solver_options=SolverOptionsConfig(name="highs", logs=False), + scenario_scope=ScenarioScopeConfig(include=["0-2"]), + resolution=ResolutionConfig( + mode=ResolutionMode.SEQUENTIAL_SUBPROBLEMS, block_length=168 + ), + ) + out = tmp_path / "optim-config.yml" + write_yaml_optim_config(config, out) + + reloaded = load_yaml_optim_config(out) + assert reloaded is not None + assert reloaded.time_scope.last_time_step == 8759 + assert reloaded.resolution.mode == ResolutionMode.SEQUENTIAL_SUBPROBLEMS + assert reloaded.resolution.block_length == 168 + assert reloaded.scenario_scope.scenario_ids == [0, 1, 2] + + +def test_write_yaml_optim_config_creates_parent_dirs(tmp_path: Path) -> None: + """write_yaml_optim_config creates missing parent directories.""" + config = OptimConfig() + out = tmp_path / "sub" / "optim-config.yml" + write_yaml_optim_config(config, out) + assert out.exists() + + +def test_write_yaml_optim_config_uses_kebab_keys(tmp_path: Path) -> None: + """Output YAML must use kebab-case keys.""" + config = OptimConfig( + time_scope=TimeScopeConfig(first_time_step=10, last_time_step=100), + ) + out = tmp_path / "optim-config.yml" + write_yaml_optim_config(config, out) + content = out.read_text() + assert "time-scope" in content + assert "time_scope" not in content + assert "first-time-step" in content + + +# --------------------------------------------------------------------------- +# ScenarioBuilder.write_dat / load_dat +# --------------------------------------------------------------------------- + + +def test_scenario_builder_write_dat_roundtrip(tmp_path: Path) -> None: + """A ScenarioBuilder dumped then re-loaded must produce identical mappings.""" + sb = ScenarioBuilder( + _group_arrays={ + "load": np.array([0, 1, 0, 1]), + "cost-group": np.array([0, 0, 1, 1]), + } + ) + out = tmp_path / "modeler-scenariobuilder.dat" + sb.write_dat(out) + + reloaded = ScenarioBuilder.load_dat(out) + + mc = np.array([0, 1, 2, 3]) + np.testing.assert_array_equal( + sb.resolve_vectorized("load", mc), + reloaded.resolve_vectorized("load", mc), + ) + np.testing.assert_array_equal( + sb.resolve_vectorized("cost-group", mc), + reloaded.resolve_vectorized("cost-group", mc), + ) + + +def test_scenario_builder_write_dat_format(tmp_path: Path) -> None: + """Written file uses 1-based column indices and correct line format.""" + sb = ScenarioBuilder(_group_arrays={"wind": np.array([0, 2])}) + out = tmp_path / "sb.dat" + sb.write_dat(out) + + lines = [l for l in out.read_text().splitlines() if l.strip()] + assert lines[0] == "wind, 0 = 1" # col_idx 0 → 1-based = 1 + assert lines[1] == "wind, 1 = 3" # col_idx 2 → 1-based = 3 + + +def test_scenario_builder_write_dat_creates_parent_dirs(tmp_path: Path) -> None: + """ScenarioBuilder.write_dat creates missing parent directories.""" + sb = ScenarioBuilder() + out = tmp_path / "sub" / "modeler-scenariobuilder.dat" + sb.write_dat(out) + assert out.exists() + + +def test_scenario_builder_write_dat_load_existing_fixture(tmp_path: Path) -> None: + """Load an existing fixture, write it, reload it, verify identity.""" + fixture = ( + Path(__file__).parent.parent + / "scenario_builder" + / "series" + / "modeler-scenariobuilder.dat" + ) + original = ScenarioBuilder.load_dat(fixture) + out = tmp_path / "modeler-scenariobuilder.dat" + original.write_dat(out) + reloaded = ScenarioBuilder.load_dat(out) + + mc = np.array([0, 1, 2, 3]) + for group in ("load", "cost-group"): + np.testing.assert_array_equal( + original.resolve_vectorized(group, mc), + reloaded.resolve_vectorized(group, mc), + ) diff --git a/tests/unittests/gems_craft_hybrid/__init__.py b/tests/unittests/gems_craft_hybrid/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/unittests/gems_craft_hybrid/fixtures/hybrid_lib.yml b/tests/unittests/gems_craft_hybrid/fixtures/hybrid_lib.yml new file mode 100644 index 00000000..2afbd2b6 --- /dev/null +++ b/tests/unittests/gems_craft_hybrid/fixtures/hybrid_lib.yml @@ -0,0 +1,50 @@ +library: + id: hybrid_basic + version: "1.0" + port-types: + - id: flow + description: A port which transfers power flow + fields: + - id: flow + area-connection: + injection-to-balance: flow + spillage-bound: flow + unsupplied-energy-bound: + thermal-capacity-connection: + capacity-field: flow + - id: signal + description: A signal port without area connection + fields: + - id: value + models: + - id: node + description: A basic balancing node model + ports: + - id: injection_port + type: flow + binding-constraints: + - id: balance + expression: sum_connections(injection_port.flow) = 0 + - id: generator + description: A basic generator model + parameters: + - id: cost + time-dependent: false + scenario-dependent: false + - id: p_max + time-dependent: false + scenario-dependent: false + variables: + - id: generation + lower-bound: 0 + upper-bound: p_max + ports: + - id: injection_port + type: flow + port-field-definitions: + - port: injection_port + field: flow + definition: generation + objective-contributions: + - id: generation_cost + expression: cost * generation \ No newline at end of file diff --git a/tests/unittests/gems_craft_hybrid/fixtures/hybrid_system.yml b/tests/unittests/gems_craft_hybrid/fixtures/hybrid_system.yml new file mode 100644 index 00000000..ebe326ad --- /dev/null +++ b/tests/unittests/gems_craft_hybrid/fixtures/hybrid_system.yml @@ -0,0 +1,27 @@ +system: + model-libraries: hybrid_basic + components: + - id: N + model: hybrid_basic.node + - id: G + model: hybrid_basic.generator + parameters: + - id: cost + value: 30 + - id: p_max + value: 100 + connections: + - component1: N + port1: injection_port + component2: G + port2: injection_port + area-connections: + - component: G + port: injection_port + area: fr + thermal-capacity-connections: + - component: G + port: injection_port + thermal-component: + area: fr + cluster-id: nuclear1 diff --git a/tests/unittests/gems_craft_hybrid/test_hybrid_parsing.py b/tests/unittests/gems_craft_hybrid/test_hybrid_parsing.py new file mode 100644 index 00000000..44cf717f --- /dev/null +++ b/tests/unittests/gems_craft_hybrid/test_hybrid_parsing.py @@ -0,0 +1,157 @@ +# Copyright (c) 2024, RTE (https://www.rte-france.com) +# +# See AUTHORS.txt +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 +# +# This file is part of the Antares project. + +"""Tests for gems_craft_hybrid parsing: HybridSystemSchema and HybridLibrarySchema.""" + +from pathlib import Path + +import pytest + +from gems_craft.model.parsing import load_yaml_library, write_yaml_library +from gems_craft.study.parsing import load_yaml_system, write_yaml_system +from gems_craft_hybrid.model.parsing import ( + AreaConnectionSchema, + HybridLibrarySchema, + HybridPortTypeSchema, +) +from gems_craft_hybrid.study.parsing import ( + AreaConnectionsSchema, + HybridSystemSchema, + ThermalCapacityConnectionSchema, + ThermalComponentSchema, +) + +FIXTURES = Path(__file__).parent / "fixtures" + + +# --------------------------------------------------------------------------- +# HybridSystemSchema — parsing +# --------------------------------------------------------------------------- + + +def test_load_yaml_hybrid_system_parses_area_connections() -> None: + system = load_yaml_system(FIXTURES / "hybrid_system.yml", HybridSystemSchema) + assert isinstance(system, HybridSystemSchema) + assert system.area_connections is not None + assert len(system.area_connections) == 1 + assert system.area_connections[0] == AreaConnectionsSchema( + component="G", port="injection_port", area="fr" + ) + + +def test_load_yaml_hybrid_system_parses_standard_fields() -> None: + system = load_yaml_system(FIXTURES / "hybrid_system.yml", HybridSystemSchema) + assert len(system.components) == 2 + assert system.connections is not None + assert len(system.connections) == 1 + + +def test_load_yaml_system_on_hybrid_file_raises() -> None: + """load_yaml_system (standard) rejects hybrid-only fields due to extra='forbid'.""" + with pytest.raises(ValueError, match="An error occurred during parsing"): + load_yaml_system(FIXTURES / "hybrid_system.yml") + + +# --------------------------------------------------------------------------- +# HybridSystemSchema — roundtrip via write_yaml_system +# --------------------------------------------------------------------------- + + +def test_write_yaml_system_roundtrip_hybrid(tmp_path: Path) -> None: + original = load_yaml_system(FIXTURES / "hybrid_system.yml", HybridSystemSchema) + out = tmp_path / "system.yml" + write_yaml_system(original, out) + reloaded = load_yaml_system(out, HybridSystemSchema) + assert reloaded == original + + +# --------------------------------------------------------------------------- +# HybridLibrarySchema — parsing +# --------------------------------------------------------------------------- + + +def test_load_yaml_hybrid_library_parses_area_connection() -> None: + lib = load_yaml_library(FIXTURES / "hybrid_lib.yml", HybridLibrarySchema) + flow_port = next(pt for pt in lib.port_types if pt.id == "flow") + assert isinstance(flow_port, HybridPortTypeSchema) + assert flow_port.area_connection == AreaConnectionSchema( + injection_to_balance="flow", + spillage_bound="flow", + unsupplied_energy_bound=None, + ) + + +def test_load_yaml_hybrid_library_port_type_without_area_connection() -> None: + lib = load_yaml_library(FIXTURES / "hybrid_lib.yml", HybridLibrarySchema) + signal_port = next(pt for pt in lib.port_types if pt.id == "signal") + assert signal_port.area_connection is None + + +def test_load_yaml_library_on_hybrid_file_raises() -> None: + """load_yaml_library (standard) rejects hybrid-only fields due to extra='forbid'.""" + with pytest.raises(ValueError, match="An error occurred during parsing"): + load_yaml_library(FIXTURES / "hybrid_lib.yml") + + +# --------------------------------------------------------------------------- +# HybridLibrarySchema — roundtrip via write_yaml_library +# --------------------------------------------------------------------------- + + +def test_write_yaml_library_roundtrip_hybrid(tmp_path: Path) -> None: + original = load_yaml_library(FIXTURES / "hybrid_lib.yml", HybridLibrarySchema) + out = tmp_path / "lib.yml" + write_yaml_library(original, out) + reloaded = load_yaml_library(out, HybridLibrarySchema) + assert reloaded == original + + +# --------------------------------------------------------------------------- +# HybridLibrarySchema — thermal-capacity-connections +# --------------------------------------------------------------------------- + + +def test_load_yaml_hybrid_library_parses_thermal_capacity_connection() -> None: + from gems_craft_hybrid.model.parsing import PortThermalCapacitySchema + + lib = load_yaml_library(FIXTURES / "hybrid_lib.yml", HybridLibrarySchema) + flow_port = next(pt for pt in lib.port_types if pt.id == "flow") + assert isinstance(flow_port, HybridPortTypeSchema) + assert flow_port.thermal_capacity_connection == PortThermalCapacitySchema( + capacity_field="flow" + ) + + +def test_load_yaml_hybrid_library_port_type_without_thermal_capacity_connection() -> ( + None +): + lib = load_yaml_library(FIXTURES / "hybrid_lib.yml", HybridLibrarySchema) + signal_port = next(pt for pt in lib.port_types if pt.id == "signal") + assert signal_port.thermal_capacity_connection is None + + +# --------------------------------------------------------------------------- +# HybridSystemSchema — thermal-capacity-connections +# --------------------------------------------------------------------------- + + +def test_load_yaml_hybrid_system_parses_thermal_capacity_connections() -> None: + system = load_yaml_system(FIXTURES / "hybrid_system.yml", HybridSystemSchema) + assert system.thermal_capacity_connections is not None + assert len(system.thermal_capacity_connections) == 1 + conn = system.thermal_capacity_connections[0] + assert isinstance(conn, ThermalCapacityConnectionSchema) + assert conn.component == "G" + assert conn.port == "injection_port" + assert conn.thermal_component == ThermalComponentSchema( + area="fr", cluster_id="nuclear1" + ) diff --git a/tests/unittests/lib_parsing/test_lib_parsing.py b/tests/unittests/lib_parsing/test_lib_parsing.py index 5977111b..fd88a6ab 100644 --- a/tests/unittests/lib_parsing/test_lib_parsing.py +++ b/tests/unittests/lib_parsing/test_lib_parsing.py @@ -14,18 +14,22 @@ import pytest -from gems.expression import literal, param, var -from gems.expression.equality import expressions_equal, expressions_equal_if_present -from gems.expression.expression import ( +from gems_craft.model.parsing import _parse_yaml_library, load_yaml_library +from gems_runner.expression import literal, param, var +from gems_runner.expression.equality import ( + expressions_equal, + expressions_equal_if_present, +) +from gems_runner.expression.expression import ( DualNode, ReducedCostNode, maximum, minimum, port_field, ) -from gems.expression.indexing_structure import IndexingStructure -from gems.expression.parsing.parse_expression import ParsingException -from gems.model import ( +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.expression.parsing.parse_expression import ParsingException +from gems_runner.model import ( Constraint, ModelPort, PortField, @@ -35,9 +39,8 @@ float_variable, model, ) -from gems.model.model import PortFieldDefinition, PortFieldId -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library +from gems_runner.model.model import PortFieldDefinition, PortFieldId +from gems_runner.model.resolve_library import resolve_library CONSTANT = IndexingStructure(False, False) @@ -45,8 +48,7 @@ def test_library_parsing(libs_dir: Path) -> None: lib_file = libs_dir / "lib_unittest.yml" - with lib_file.open() as f: - input_lib = parse_yaml_library(f) + input_lib = load_yaml_library(lib_file) assert input_lib.id == "basic" assert len(input_lib.models) == 7 assert len(input_lib.port_types) == 1 @@ -138,7 +140,7 @@ def test_binary_variable_parsing() -> None: - id: on_off variable-type: binary """ - input_lib = parse_yaml_library(io.StringIO(yaml_content)) + input_lib = _parse_yaml_library(io.StringIO(yaml_content)) lib = resolve_library([input_lib]) on_off = lib["test"].models["test.binary_model"].variables["on_off"] assert on_off.data_type == ValueType.BINARY @@ -156,7 +158,7 @@ def test_model_taxonomy_category_parses() -> None: time-dependent: false scenario-dependent: false """ - input_lib = parse_yaml_library(io.StringIO(yaml_content)) + input_lib = _parse_yaml_library(io.StringIO(yaml_content)) assert input_lib.models[0].taxonomy_category == "balance" @@ -171,12 +173,12 @@ def test_model_taxonomy_category_parses() -> None: def test_parse_yaml_library_model_properties() -> None: - lib = parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) + lib = _parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) assert [p.id for p in lib.models[0].properties] == ["technology"] def test_resolve_library_exposes_model_properties() -> None: - lib = parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) + lib = _parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) lib_dict = resolve_library([lib]) assert lib_dict["basic"].models["basic.generator"].properties == ["technology"] @@ -184,8 +186,7 @@ def test_resolve_library_exposes_model_properties() -> None: def test_library_error_parsing(libs_dir: Path) -> None: lib_file = libs_dir / "model_port_definition_ko.yml" - with lib_file.open() as f: - input_lib = parse_yaml_library(f) + input_lib = load_yaml_library(lib_file) assert input_lib.id == "basic" with pytest.raises( ParsingException, @@ -197,8 +198,7 @@ def test_library_error_parsing(libs_dir: Path) -> None: def test_library_port_model_ok_parsing(libs_dir: Path) -> None: lib_file = libs_dir / "model_port_definition_ok.yml" - with lib_file.open() as f: - input_lib = parse_yaml_library(f) + input_lib = load_yaml_library(lib_file) assert input_lib.id == "basic" lib = resolve_library([input_lib]) @@ -253,7 +253,7 @@ def test_dual_in_constraint_is_rejected() -> None: - id: bad expression: dual(balance) + x = 0 """) - input_lib = parse_yaml_library(lib_yaml) + input_lib = _parse_yaml_library(lib_yaml) with pytest.raises(ValueError, match="Non-linear expression is not allowed"): resolve_library([input_lib]) @@ -274,7 +274,7 @@ def test_reduced_cost_in_objective_is_rejected() -> None: - id: bad-obj expression: reduced_cost(x) """) - input_lib = parse_yaml_library(lib_yaml) + input_lib = _parse_yaml_library(lib_yaml) with pytest.raises(ValueError, match="Non-linear expression is not allowed"): resolve_library([input_lib]) @@ -405,7 +405,7 @@ def _no_port_lib_yaml( _UNRESTRICTED_EXPRS + _CONSTANT_EXPRS + _LINEAR_EXPRS, ) def test_expr_in_port_field_definition(yaml_expr: str, expected_expr: object) -> None: - input_lib = parse_yaml_library(io.StringIO(_base_lib_yaml(yaml_expr))) + input_lib = _parse_yaml_library(io.StringIO(_base_lib_yaml(yaml_expr))) lib = resolve_library([input_lib]) pfd = lib["test"].models["test.test_model"].port_fields_definitions[_PFIELD_ID] assert expressions_equal(pfd.definition, expected_expr) # type: ignore[arg-type] @@ -420,7 +420,7 @@ def test_expr_in_port_field_definition(yaml_expr: str, expected_expr: object) -> def test_constant_expr_in_variable_upper_bound( yaml_expr: str, expected_expr: object ) -> None: - input_lib = parse_yaml_library(io.StringIO(_no_port_lib_yaml(var_bound=yaml_expr))) + input_lib = _parse_yaml_library(io.StringIO(_no_port_lib_yaml(var_bound=yaml_expr))) lib = resolve_library([input_lib]) variable = lib["test"].models["test.test_model"].variables["x"] assert expressions_equal_if_present(variable.upper_bound, expected_expr) # type: ignore[arg-type] @@ -433,7 +433,7 @@ def test_constant_expr_in_variable_upper_bound( @pytest.mark.parametrize("yaml_expr,expected_expr", _CONSTANT_EXPRS) def test_constant_expr_in_objective(yaml_expr: str, expected_expr: object) -> None: - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO(_no_port_lib_yaml(objective_expr=yaml_expr)) ) lib = resolve_library([input_lib]) @@ -444,7 +444,7 @@ def test_constant_expr_in_objective(yaml_expr: str, expected_expr: object) -> No @pytest.mark.parametrize("yaml_expr,expected_expr", _LINEAR_EXPRS) def test_linear_expr_in_objective(yaml_expr: str, expected_expr: object) -> None: - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO(_no_port_lib_yaml(objective_expr=f"expec(sum({yaml_expr}))")) ) lib = resolve_library([input_lib]) @@ -462,7 +462,7 @@ def test_linear_expr_in_objective(yaml_expr: str, expected_expr: object) -> None def test_constant_expr_in_binding_constraint( yaml_expr: str, expected_expr: object ) -> None: - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO(_no_port_lib_yaml(constraint_expr=f"{yaml_expr} = 0")) ) lib = resolve_library([input_lib]) @@ -474,7 +474,7 @@ def test_constant_expr_in_binding_constraint( def test_linear_expr_in_binding_constraint( yaml_expr: str, expected_expr: object ) -> None: - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO(_no_port_lib_yaml(constraint_expr=f"{yaml_expr} >= 0")) ) lib = resolve_library([input_lib]) @@ -491,7 +491,7 @@ def test_linear_expr_in_binding_constraint( def test_unrestricted_expr_rejected_in_binding_constraint( yaml_expr: str, expected_expr: object ) -> None: - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO(_no_port_lib_yaml(constraint_expr=f"{yaml_expr} = 0")) ) with pytest.raises(ValueError, match="Non-linear expression is not allowed"): @@ -507,7 +507,7 @@ def test_unrestricted_expr_rejected_in_binding_constraint( def test_expr_rejected_in_variable_upper_bound( yaml_expr: str, expected_expr: object ) -> None: - input_lib = parse_yaml_library(io.StringIO(_no_port_lib_yaml(var_bound=yaml_expr))) + input_lib = _parse_yaml_library(io.StringIO(_no_port_lib_yaml(var_bound=yaml_expr))) with pytest.raises(ValueError, match="bounds of variables must be constant"): resolve_library([input_lib]) @@ -521,7 +521,7 @@ def test_expr_rejected_in_variable_upper_bound( def test_unrestricted_expr_rejected_in_objective( yaml_expr: str, expected_expr: object ) -> None: - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO(_no_port_lib_yaml(objective_expr=yaml_expr)) ) with pytest.raises(ValueError): @@ -581,7 +581,7 @@ def _port_model_yaml(*, constraint_expr: str = "", extra_output_expr: str = "") def test_sum_connections_on_own_port_in_binding_constraint_raises() -> None: """sum_connections(balance_port.flow) in a BC is invalid: flow is defined here.""" - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO( _port_model_yaml(constraint_expr="sum_connections(balance_port.flow) >= 0") ) @@ -592,7 +592,7 @@ def test_sum_connections_on_own_port_in_binding_constraint_raises() -> None: def test_sum_connections_on_own_port_in_extra_output_raises() -> None: """sum_connections(balance_port.flow) in an extra-output is invalid: flow is defined here.""" - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO( _port_model_yaml( extra_output_expr="sum_connections(balance_port.flow) * generation" @@ -610,7 +610,7 @@ def test_sum_connections_on_own_port_in_extra_output_raises() -> None: def test_bare_defined_port_field_in_binding_constraint_raises() -> None: """balance_port.flow bare (no sum_connections) in a BC is invalid.""" - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO(_port_model_yaml(constraint_expr="balance_port.flow >= 0")) ) with pytest.raises(ValueError, match="Bare port field"): @@ -619,7 +619,7 @@ def test_bare_defined_port_field_in_binding_constraint_raises() -> None: def test_bare_undefined_port_field_in_binding_constraint_raises() -> None: """balance_port.price bare (not defined in this model either) in a BC is invalid.""" - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO(_port_model_yaml(constraint_expr="balance_port.price >= 0")) ) with pytest.raises(ValueError, match="Bare port field"): @@ -633,7 +633,7 @@ def test_bare_undefined_port_field_in_binding_constraint_raises() -> None: def test_sum_connections_on_non_own_port_accepted() -> None: """sum_connections(balance_port.price) is valid: price is not defined in this model.""" - input_lib = parse_yaml_library( + input_lib = _parse_yaml_library( io.StringIO( _port_model_yaml(constraint_expr="sum_connections(balance_port.price) >= 0") ) diff --git a/tests/unittests/lib_parsing/test_multiple_libs_parsing.py b/tests/unittests/lib_parsing/test_multiple_libs_parsing.py index 05475ee0..39f20302 100644 --- a/tests/unittests/lib_parsing/test_multiple_libs_parsing.py +++ b/tests/unittests/lib_parsing/test_multiple_libs_parsing.py @@ -14,8 +14,8 @@ import pytest -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library +from gems_craft.model.parsing import load_yaml_library +from gems_runner.model.resolve_library import resolve_library # in following tests "lib_A -> lib_B" means lib_A must be resolved before lib_B @@ -34,8 +34,7 @@ def test_simple_dependency_tree(libs_dir: Path) -> None: input_libs = [] for lib_file in lib_files: - with lib_file.open() as f: - input_libs.append(parse_yaml_library(f)) + input_libs.append(load_yaml_library(lib_file)) lib_dict = resolve_library(input_libs) assert len(lib_dict) == 3 @@ -57,8 +56,7 @@ def test_simple_dependency_tree(libs_dir: Path) -> None: input_libs = [] for lib_file in lib_files: - with lib_file.open() as f: - input_libs.append(parse_yaml_library(f)) + input_libs.append(load_yaml_library(lib_file)) lib_dict = resolve_library(input_libs) assert len(lib_dict["basic"].models) == 1 @@ -85,8 +83,7 @@ def test_multiple_dependencies_tree(libs_dir: Path) -> None: input_libs = [] for lib_file in lib_files: - with lib_file.open() as f: - input_libs.append(parse_yaml_library(f)) + input_libs.append(load_yaml_library(lib_file)) lib_dict = resolve_library(input_libs) assert len(lib_dict["basic"].models) == 1 @@ -111,8 +108,7 @@ def test_looping_dependency(libs_dir: Path) -> None: input_libs = [] for lib_file in lib_files: - with lib_file.open() as f: - input_libs.append(parse_yaml_library(f)) + input_libs.append(load_yaml_library(lib_file)) with pytest.raises(Exception, match=r"Circular import in yaml libraries"): resolve_library(input_libs) @@ -133,8 +129,7 @@ def test_model_with_same_name_in_different_lib_ok(libs_dir: Path) -> None: input_libs = [] for lib_file in lib_files: - with lib_file.open() as f: - input_libs.append(parse_yaml_library(f)) + input_libs.append(load_yaml_library(lib_file)) lib_dict = resolve_library(input_libs) assert len(lib_dict["basic"].models) == 1 @@ -156,8 +151,7 @@ def test_model_redefinition_in_same_lib(libs_dir: Path) -> None: input_libs = [] for lib_file in lib_files: - with lib_file.open() as f: - input_libs.append(parse_yaml_library(f)) + input_libs.append(load_yaml_library(lib_file)) with pytest.raises(Exception, match=re.escape("Model generator is defined twice")): resolve_library(input_libs) @@ -172,8 +166,7 @@ def test_port_redefinition(libs_dir: Path) -> None: input_libs = [] for lib_file in lib_files: - with lib_file.open() as f: - input_libs.append(parse_yaml_library(f)) + input_libs.append(load_yaml_library(lib_file)) with pytest.raises( Exception, match=re.escape("Port(s) : {'flow'} is(are) defined twice") diff --git a/tests/unittests/lib_parsing/test_taxonomy_check.py b/tests/unittests/lib_parsing/test_taxonomy_check.py index 61d2ab73..056da6a7 100644 --- a/tests/unittests/lib_parsing/test_taxonomy_check.py +++ b/tests/unittests/lib_parsing/test_taxonomy_check.py @@ -15,8 +15,8 @@ import pytest -from gems.model.parsing import parse_yaml_library -from gems.model.taxonomy import ( +from gems_craft.model.parsing import _parse_yaml_library +from gems_craft.model.taxonomy import ( Taxonomy, TaxonomyCategory, TaxonomyItem, @@ -34,7 +34,7 @@ def _make_category(cat_id: str, port_ids: list[str]) -> TaxonomyCategory: def _parse_lib(yaml_content: str): - return parse_yaml_library(io.StringIO(yaml_content)) + return _parse_yaml_library(io.StringIO(yaml_content)) # --- valid cases --- diff --git a/tests/unittests/optim_config/test_scenario_scope_config.py b/tests/unittests/optim_config/test_scenario_scope_config.py index f99ad4b1..0a6c37f4 100644 --- a/tests/unittests/optim_config/test_scenario_scope_config.py +++ b/tests/unittests/optim_config/test_scenario_scope_config.py @@ -16,7 +16,7 @@ import pytest -from gems.optim_config.parsing import ScenarioScopeConfig +from gems_craft.optim_config.parsing import ScenarioScopeConfig # --------------------------------------------------------------------------- # Inline form — include only @@ -271,14 +271,14 @@ def test_playlist_file_boolean_values_rejected(tmp_path: Path) -> None: def test_yaml_inline_include_only() -> None: - from gems.optim_config.parsing import OptimConfig + from gems_craft.optim_config.parsing import OptimConfig cfg = OptimConfig.model_validate({"scenario-scope": {"include": ["0-2", 4]}}) assert cfg.scenario_scope.scenario_ids == [0, 1, 2, 4] def test_yaml_inline_include_exclude() -> None: - from gems.optim_config.parsing import OptimConfig + from gems_craft.optim_config.parsing import OptimConfig cfg = OptimConfig.model_validate( {"scenario-scope": {"include": ["0-4"], "exclude": [2]}} @@ -289,7 +289,9 @@ def test_yaml_inline_include_exclude() -> None: def test_yaml_playlist_file_relative_resolved_by_load_optim_config( tmp_path: Path, ) -> None: - from gems.optim_config.parsing import load_optim_config + from gems_craft.optim_config.parsing import ( + load_yaml_optim_config as load_optim_config, + ) playlist = tmp_path / "playlist.json" playlist.write_text(json.dumps([0, 1, 2])) @@ -303,7 +305,7 @@ def test_yaml_playlist_file_relative_resolved_by_load_optim_config( def test_yaml_nb_scenarios_rejected() -> None: - from gems.optim_config.parsing import OptimConfig + from gems_craft.optim_config.parsing import OptimConfig with pytest.raises(ValueError): OptimConfig.model_validate({"scenario-scope": {"nb-scenarios": 1}}) @@ -364,7 +366,9 @@ def test_scenario_ids_cached_inline() -> None: def test_scenario_ids_cached_playlist_file_via_load_optim_config( tmp_path: Path, ) -> None: - from gems.optim_config.parsing import load_optim_config + from gems_craft.optim_config.parsing import ( + load_yaml_optim_config as load_optim_config, + ) playlist = tmp_path / "playlist.json" playlist.write_text(json.dumps([0, 1, 2])) @@ -386,9 +390,10 @@ def test_scenario_ids_cached_playlist_file_via_load_optim_config( def test_validate_optim_config_scenario_builder_rejects_out_of_bounds() -> None: import numpy as np - from gems.optim_config.parsing import OptimConfig, validate_optim_config - from gems.study.scenario_builder import ScenarioBuilder - from gems.study.system import System + from gems_craft.optim_config.parsing import OptimConfig + from gems_craft.study.scenario_builder import ScenarioBuilder + from gems_runner.optim_config import validate_optim_config + from gems_runner.study.system import System config = OptimConfig.model_validate( {"scenario-scope": {"include": ["0-4"]}} # scenarios 0,1,2,3,4 @@ -404,9 +409,10 @@ def test_validate_optim_config_scenario_builder_rejects_out_of_bounds() -> None: def test_validate_optim_config_scenario_builder_accepts_valid_playlist() -> None: import numpy as np - from gems.optim_config.parsing import OptimConfig, validate_optim_config - from gems.study.scenario_builder import ScenarioBuilder - from gems.study.system import System + from gems_craft.optim_config.parsing import OptimConfig + from gems_craft.study.scenario_builder import ScenarioBuilder + from gems_runner.optim_config import validate_optim_config + from gems_runner.study.system import System config = OptimConfig.model_validate( {"scenario-scope": {"include": ["0-2"]}} # scenarios 0,1,2 diff --git a/tests/unittests/scenario_builder/test_scenario_builder.py b/tests/unittests/scenario_builder/test_scenario_builder.py index 71669569..19ed637b 100644 --- a/tests/unittests/scenario_builder/test_scenario_builder.py +++ b/tests/unittests/scenario_builder/test_scenario_builder.py @@ -15,11 +15,11 @@ import numpy as np import pytest -from gems.study import DataBase -from gems.study.data import ComponentParameterIndex -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import build_data_base -from gems.study.scenario_builder import ScenarioBuilder +from gems_craft.study.parsing import load_yaml_system +from gems_craft.study.scenario_builder import ScenarioBuilder +from gems_runner.study import DataBase +from gems_runner.study.data import ComponentParameterIndex +from gems_runner.study.resolve_components import build_data_base @pytest.fixture(scope="session") @@ -29,16 +29,13 @@ def series_dir() -> Path: @pytest.fixture(scope="session") def scenario_builder(series_dir: Path) -> ScenarioBuilder: - return ScenarioBuilder.load(series_dir / "modeler-scenariobuilder.dat") + return ScenarioBuilder.load_dat(series_dir / "modeler-scenariobuilder.dat") @pytest.fixture def database(series_dir: Path, scenario_builder: ScenarioBuilder) -> DataBase: system_path = Path(__file__).parent / "systems/with_scenarization.yml" - with system_path.open() as components: - return build_data_base( - parse_yaml_components(components), series_dir, scenario_builder - ) + return build_data_base(load_yaml_system(system_path), series_dir, scenario_builder) def test_scenario_builder_load(scenario_builder: ScenarioBuilder) -> None: diff --git a/tests/unittests/scenario_builder/test_scenario_builder_dispatch.py b/tests/unittests/scenario_builder/test_scenario_builder_dispatch.py index 8cb2f61f..065801f9 100644 --- a/tests/unittests/scenario_builder/test_scenario_builder_dispatch.py +++ b/tests/unittests/scenario_builder/test_scenario_builder_dispatch.py @@ -15,9 +15,9 @@ import numpy as np import pytest -from gems.study.parsing import parse_yaml_components -from gems.study.resolve_components import build_data_base -from gems.study.scenario_builder import ScenarioBuilder +from gems_craft.study.parsing import _parse_yaml_components +from gems_craft.study.scenario_builder import ScenarioBuilder +from gems_runner.study.resolve_components import build_data_base @pytest.fixture(scope="session") @@ -52,8 +52,8 @@ def dispatch_system_yml() -> str: def test_scenario_builder_load(dispatch_series_dir: Path) -> None: - """ScenarioBuilder.load() parses the .dat file into correct 0-based col_idx arrays.""" - sb = ScenarioBuilder.load(dispatch_series_dir / "modeler-scenariobuilder.dat") + """ScenarioBuilder.load_dat() parses the .dat file into correct 0-based col_idx arrays.""" + sb = ScenarioBuilder.load_dat(dispatch_series_dir / "modeler-scenariobuilder.dat") mc = np.array([0, 1, 2], dtype=int) cols = sb.resolve_vectorized("load", mc) assert list(cols) == [2, 0, 1] @@ -63,12 +63,12 @@ def test_dispatch_mc_scenarios_to_columns( dispatch_series_dir: Path, dispatch_system_yml: str ) -> None: """DataBase.get_values() dispatches each MC scenario to the correct data column.""" - sb = ScenarioBuilder.load(dispatch_series_dir / "modeler-scenariobuilder.dat") + sb = ScenarioBuilder.load_dat(dispatch_series_dir / "modeler-scenariobuilder.dat") import io db = build_data_base( - parse_yaml_components(io.StringIO(dispatch_system_yml)), + _parse_yaml_components(io.StringIO(dispatch_system_yml)), dispatch_series_dir, scenario_builder=sb, ) diff --git a/tests/unittests/simulation/test_shift_validity_visitor.py b/tests/unittests/simulation/test_shift_validity_visitor.py index bdb1f1b5..1037e6f1 100644 --- a/tests/unittests/simulation/test_shift_validity_visitor.py +++ b/tests/unittests/simulation/test_shift_validity_visitor.py @@ -16,9 +16,9 @@ import pytest import xarray as xr -from gems.expression.expression import literal, param, var -from gems.expression.visitor import visit -from gems.simulation.vectorized_builder import ShiftValidityVisitor +from gems_runner.expression.expression import literal, param, var +from gems_runner.expression.visitor import visit +from gems_runner.simulation.vectorized_builder import ShiftValidityVisitor def _visitor(param_arrays=None, block_length=4): diff --git a/tests/unittests/simulation/test_simulation_table_accessor.py b/tests/unittests/simulation/test_simulation_table_accessor.py index ce0d953e..442bab4c 100644 --- a/tests/unittests/simulation/test_simulation_table_accessor.py +++ b/tests/unittests/simulation/test_simulation_table_accessor.py @@ -11,9 +11,8 @@ import pytest import xarray as xr -from gems.simulation.simulation_table import ( - ComponentView, - OutputView, +from gems_craft.simulation_table.simulation_table import ComponentView, OutputView +from gems_runner.simulation.simulation_table import ( SimulationTable, SimulationTableBuilder, ) diff --git a/tests/unittests/simulation/test_simulation_table_export.py b/tests/unittests/simulation/test_simulation_table_export.py index 75c3459d..3e9e8eaa 100644 --- a/tests/unittests/simulation/test_simulation_table_export.py +++ b/tests/unittests/simulation/test_simulation_table_export.py @@ -12,7 +12,7 @@ import pytest import xarray as xr -from gems.simulation.simulation_table import ( +from gems_runner.simulation.simulation_table import ( SimulationColumns, SimulationTable, SimulationTableBuilder, diff --git a/tests/unittests/simulation/test_simulation_table_extra_outputs.py b/tests/unittests/simulation/test_simulation_table_extra_outputs.py index 7e71d980..208d5abd 100644 --- a/tests/unittests/simulation/test_simulation_table_extra_outputs.py +++ b/tests/unittests/simulation/test_simulation_table_extra_outputs.py @@ -3,7 +3,7 @@ import pytest -from gems.simulation.simulation_table import SimulationTableBuilder +from gems_runner.simulation.simulation_table import SimulationTableBuilder def test_extra_output_with_sum_connections() -> None: @@ -16,13 +16,25 @@ def test_extra_output_with_sum_connections() -> None: GEN model defines balance_port.flow = var("gen"). Expected: total_flow at node_1 == 5.0. """ - from gems.expression import var - from gems.expression.expression import literal, port_field - from gems.model.model import ModelPort, model - from gems.model.port import PortField, PortFieldDefinition, PortFieldId, PortType - from gems.model.variable import float_variable - from gems.simulation import TimeBlock, build_problem - from gems.study import Component, DataBase, PortRef, Study, System, create_component + from gems_runner.expression import var + from gems_runner.expression.expression import literal, port_field + from gems_runner.model.model import ModelPort, model + from gems_runner.model.port import ( + PortField, + PortFieldDefinition, + PortFieldId, + PortType, + ) + from gems_runner.model.variable import float_variable + from gems_runner.simulation import TimeBlock, build_problem + from gems_runner.study import ( + Component, + DataBase, + PortRef, + Study, + System, + create_component, + ) BALANCE_PORT_TYPE = PortType(id="balance", fields=[PortField("flow")]) @@ -87,12 +99,12 @@ def test_extra_output_nonlinear() -> None: Setup: one component with variable a=3 (fixed). Extra output squared = a*a. Expected: squared = 9.0. """ - from gems.expression import var - from gems.expression.expression import literal - from gems.model.model import model - from gems.model.variable import float_variable - from gems.simulation import TimeBlock, build_problem - from gems.study import DataBase, Study, System, create_component + from gems_runner.expression import var + from gems_runner.expression.expression import literal + from gems_runner.model.model import model + from gems_runner.model.variable import float_variable + from gems_runner.simulation import TimeBlock, build_problem + from gems_runner.study import DataBase, Study, System, create_component SIMPLE_MODEL = model( id="SIMPLE_NL", @@ -124,12 +136,12 @@ def test_extra_output_abs_round_on_variable() -> None: outputs (post-solve evaluation), even though they would be rejected as nonlinear inside a constraint or bound. """ - from gems.expression import var - from gems.expression.expression import literal - from gems.model.model import model - from gems.model.variable import float_variable - from gems.simulation import TimeBlock, build_problem - from gems.study import DataBase, Study, System, create_component + from gems_runner.expression import var + from gems_runner.expression.expression import literal + from gems_runner.model.model import model + from gems_runner.model.variable import float_variable + from gems_runner.simulation import TimeBlock, build_problem + from gems_runner.study import DataBase, Study, System, create_component SIMPLE_MODEL = model( id="SIMPLE_ABS_ROUND", diff --git a/tests/unittests/simulation/test_simulation_table_mock.py b/tests/unittests/simulation/test_simulation_table_mock.py index f1886a64..25d3795a 100644 --- a/tests/unittests/simulation/test_simulation_table_mock.py +++ b/tests/unittests/simulation/test_simulation_table_mock.py @@ -10,7 +10,10 @@ import pytest import xarray as xr -from gems.simulation.simulation_table import SimulationColumns, SimulationTableBuilder +from gems_runner.simulation.simulation_table import ( + SimulationColumns, + SimulationTableBuilder, +) @dataclass(frozen=True) diff --git a/tests/unittests/simulation/test_vectorized_linear_expr_builder.py b/tests/unittests/simulation/test_vectorized_linear_expr_builder.py index 684691c9..b376b1a3 100644 --- a/tests/unittests/simulation/test_vectorized_linear_expr_builder.py +++ b/tests/unittests/simulation/test_vectorized_linear_expr_builder.py @@ -23,7 +23,7 @@ import pytest import xarray as xr -from gems.expression.expression import ( +from gems_runner.expression.expression import ( AllTimeSumNode, LiteralNode, MaxNode, @@ -37,10 +37,10 @@ param, var, ) -from gems.expression.visitor import visit -from gems.model.port import PortFieldId -from gems.simulation.linearize import VectorizedLinearExprBuilder -from gems.simulation.vectorized_builder import VectorizedBuilderBase, _linopy_add +from gems_runner.expression.visitor import visit +from gems_runner.model.port import PortFieldId +from gems_runner.simulation.linearize import VectorizedLinearExprBuilder +from gems_runner.simulation.vectorized_builder import VectorizedBuilderBase, _linopy_add # --------------------------------------------------------------------------- # Fixtures @@ -757,7 +757,7 @@ def test_port_field_found() -> None: port_arrays={key: da}, block_length=1, ) - from gems.expression.expression import port_field + from gems_runner.expression.expression import port_field result = visit(port_field("port_a", "flow"), b) assert float(result) == pytest.approx(99.0) @@ -766,7 +766,7 @@ def test_port_field_found() -> None: def test_port_field_missing_raises_key_error( empty_builder: VectorizedLinearExprBuilder, ) -> None: - from gems.expression.expression import port_field + from gems_runner.expression.expression import port_field with pytest.raises(KeyError): visit(port_field("no_such_port", "flow"), empty_builder) @@ -787,7 +787,7 @@ def test_port_sum_with_connection_returns_expression() -> None: port_arrays={key: da}, block_length=1, ) - from gems.expression.expression import port_field + from gems_runner.expression.expression import port_field expr = port_field("port_a", "flow").sum_connections() result = visit(expr, b) @@ -798,7 +798,7 @@ def test_port_sum_no_connection_returns_zero( empty_builder: VectorizedLinearExprBuilder, ) -> None: """PortFieldAggregatorNode with no matching port returns DataArray(0.0).""" - from gems.expression.expression import port_field + from gems_runner.expression.expression import port_field expr = port_field("absent_port", "flow").sum_connections() result = visit(expr, empty_builder) @@ -808,7 +808,7 @@ def test_port_sum_no_connection_returns_zero( def test_unsupported_port_aggregator_raises_at_node_construction() -> None: """Only 'PortSum' is valid; other aggregator names raise at node construction.""" - from gems.expression.expression import PortFieldAggregatorNode, port_field + from gems_runner.expression.expression import PortFieldAggregatorNode, port_field with pytest.raises(NotImplementedError): PortFieldAggregatorNode(operand=port_field("p", "f"), aggregator="PortMax") diff --git a/tests/unittests/system/libs/standard.py b/tests/unittests/system/libs/standard.py index ca6a93a1..c18e3336 100644 --- a/tests/unittests/system/libs/standard.py +++ b/tests/unittests/system/libs/standard.py @@ -14,14 +14,14 @@ The standard module contains the definition of standard models. """ -from gems.expression import literal, param, var -from gems.expression.expression import port_field -from gems.expression.indexing_structure import IndexingStructure -from gems.model.constraint import Constraint -from gems.model.model import ModelPort, model -from gems.model.parameter import float_parameter, int_parameter -from gems.model.port import PortField, PortFieldDefinition, PortFieldId, PortType -from gems.model.variable import float_variable, int_variable +from gems_runner.expression import literal, param, var +from gems_runner.expression.expression import port_field +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model.constraint import Constraint +from gems_runner.model.model import ModelPort, model +from gems_runner.model.parameter import float_parameter, int_parameter +from gems_runner.model.port import PortField, PortFieldDefinition, PortFieldId, PortType +from gems_runner.model.variable import float_variable, int_variable CONSTANT = IndexingStructure(False, False) TIME_AND_SCENARIO_FREE = IndexingStructure(True, True) diff --git a/tests/unittests/system/test_data_consistency.py b/tests/unittests/system/test_data_consistency.py index 5886ffa0..349c45c5 100644 --- a/tests/unittests/system/test_data_consistency.py +++ b/tests/unittests/system/test_data_consistency.py @@ -16,9 +16,10 @@ import pandas as pd import pytest -from gems.expression import param, var -from gems.expression.indexing_structure import IndexingStructure -from gems.model import ( +from gems_craft.study.timeseries import load_ts_from_file +from gems_runner.expression import param, var +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model import ( Constraint, Model, ModelPort, @@ -26,8 +27,8 @@ float_variable, model, ) -from gems.model.port import PortFieldDefinition, PortFieldId -from gems.study import ( +from gems_runner.model.port import PortFieldDefinition, PortFieldId +from gems_runner.study import ( Component, ConstantData, DataBase, @@ -40,7 +41,6 @@ TimeSeriesData, create_component, ) -from gems.study.data import load_ts_from_file from tests.unittests.system.libs.standard import ( BALANCE_PORT_TYPE, CONSTANT, diff --git a/tests/unittests/system/test_model.py b/tests/unittests/system/test_model.py index 93cdd77e..e6e16660 100644 --- a/tests/unittests/system/test_model.py +++ b/tests/unittests/system/test_model.py @@ -14,7 +14,7 @@ import pytest -from gems.expression.expression import ( +from gems_runner.expression.expression import ( ExpressionNode, ScenarioOperatorNode, literal, @@ -22,11 +22,11 @@ port_field, var, ) -from gems.expression.indexing_structure import IndexingStructure -from gems.model import Constraint, float_variable, model -from gems.model.common import ValueType -from gems.model.port import port_field_def -from gems.model.variable import bool_var, int_variable +from gems_runner.expression.indexing_structure import IndexingStructure +from gems_runner.model import Constraint, float_variable, model +from gems_runner.model.common import ValueType +from gems_runner.model.port import port_field_def +from gems_runner.model.variable import bool_var, int_variable @pytest.mark.parametrize( diff --git a/tests/unittests/system/test_port.py b/tests/unittests/system/test_port.py index 8350400d..78e16bbe 100644 --- a/tests/unittests/system/test_port.py +++ b/tests/unittests/system/test_port.py @@ -12,10 +12,10 @@ import pytest -from gems.expression import literal -from gems.expression.expression import port_field -from gems.model import Constraint, ModelPort, PortType, model -from gems.study import Component, PortRef, PortsConnection, create_component +from gems_runner.expression import literal +from gems_runner.expression.expression import port_field +from gems_runner.model import Constraint, ModelPort, PortType, model +from gems_runner.study import Component, PortRef, PortsConnection, create_component from tests.unittests.system.libs.standard import DEMAND_MODEL diff --git a/tests/unittests/system/test_system.py b/tests/unittests/system/test_system.py index 0cca5697..f045a146 100644 --- a/tests/unittests/system/test_system.py +++ b/tests/unittests/system/test_system.py @@ -14,10 +14,10 @@ import pytest -from gems.model.library import Library -from gems.model.parsing import parse_yaml_library -from gems.model.resolve_library import resolve_library -from gems.study.system import Component, System +from gems_craft.model.parsing import load_yaml_library +from gems_runner.model.library import Library +from gems_runner.model.resolve_library import resolve_library +from gems_runner.study.system import Component, System @pytest.fixture(scope="session") @@ -29,8 +29,7 @@ def libs_dir() -> Path: def lib_dict(libs_dir: Path) -> dict[str, Library]: lib_file = libs_dir / "lib.yml" - with lib_file.open() as f: - input_lib = parse_yaml_library(f) + input_lib = load_yaml_library(lib_file) lib_dict = resolve_library([input_lib]) return lib_dict diff --git a/tests/unittests/system_parsing/test_components_parsing.py b/tests/unittests/system_parsing/test_components_parsing.py index fbc66b14..b56c8484 100644 --- a/tests/unittests/system_parsing/test_components_parsing.py +++ b/tests/unittests/system_parsing/test_components_parsing.py @@ -5,32 +5,37 @@ from pydantic import ValidationError from yaml import dump, safe_load -from gems.expression import literal, maximum, var -from gems.expression.expression import port_field -from gems.model import Constraint, ModelPort, PortType, model -from gems.model.parsing import LibrarySchema, parse_yaml_library -from gems.model.port import PortField, PortFieldDefinition, PortFieldId -from gems.model.resolve_library import resolve_library -from gems.model.variable import float_variable -from gems.study import Component, PortRef, PortsConnection -from gems.study.parsing import SystemSchema, load_input_system, parse_yaml_components -from gems.study.resolve_components import consistency_check, resolve_system +from gems_craft.model.parsing import ( + LibrarySchema, + _parse_yaml_library, + load_yaml_library, +) +from gems_craft.study.parsing import ( + SystemSchema, + _parse_yaml_components, + load_yaml_system, +) +from gems_runner.expression import literal, maximum, var +from gems_runner.expression.expression import port_field +from gems_runner.model import Constraint, ModelPort, PortType, model +from gems_runner.model.port import PortField, PortFieldDefinition, PortFieldId +from gems_runner.model.resolve_library import resolve_library +from gems_runner.model.variable import float_variable +from gems_runner.study import Component, PortRef, PortsConnection +from gems_runner.study.resolve_components import consistency_check, resolve_system COMPO_FILE = Path(__file__).parent / "systems/system.yml" @pytest.fixture def input_system() -> SystemSchema: - with COMPO_FILE.open() as c: - return parse_yaml_components(c) + return load_yaml_system(COMPO_FILE) @pytest.fixture def input_library() -> LibrarySchema: library = Path(__file__).parent / "libs/lib_unittest.yml" - - with library.open() as lib: - return parse_yaml_library(lib) + return load_yaml_library(library) def test_parsing_components_ok( @@ -54,39 +59,22 @@ def test_consistency_check_ok( consistency_check(result_system, result_lib["basic"].models) -def test_load_input_system_ok(tmp_path: Path) -> None: - data = safe_load(COMPO_FILE.read_text()) - system_only = data["system"] - file_for_load = tmp_path / "system.yml" - file_for_load.write_text(dump(system_only)) - - result = load_input_system(file_for_load) - - assert isinstance(result, SystemSchema) - assert len(result.components) == 3 - assert result.components[0].id == "N" - assert result.components[1].id == "G" - assert result.components[2].id == "D" - assert result.connections is not None - assert len(result.connections) == 2 - - def test_load_input_system_invalid_yaml_raises_value_error(tmp_path: Path) -> None: data = safe_load(COMPO_FILE.read_text()) system_only = data["system"].copy() system_only["unknown_field"] = "not_allowed" bad_file = tmp_path / "system.yml" - bad_file.write_text(dump(system_only)) + bad_file.write_text(dump({"system": system_only})) with pytest.raises(ValueError, match="An error occurred during parsing"): - load_input_system(bad_file) + load_yaml_system(bad_file) def test_load_input_system_missing_file_raises_error() -> None: missing = Path(__file__).parent / "systems/does_not_exist.yml" with pytest.raises(FileNotFoundError): - load_input_system(missing) + load_yaml_system(missing) def test_consistency_check_ko( @@ -188,7 +176,7 @@ def test_sum_connections_with_linear_port_field_ok() -> None: def test_parse_yaml_components_properties_optional_and_normalized() -> None: - system = parse_yaml_components(io.StringIO(_SYSTEM_WITH_COMPONENT_PROPERTIES)) + system = _parse_yaml_components(io.StringIO(_SYSTEM_WITH_COMPONENT_PROPERTIES)) props_by_id = {c.id: c.properties for c in system.components} assert props_by_id["load"] is None raw = props_by_id["nuclear_1"] @@ -202,7 +190,7 @@ def test_parse_yaml_components_properties_optional_and_normalized() -> None: def test_resolve_system_normalizes_list_properties_to_dict( input_library: LibrarySchema, ) -> None: - system = parse_yaml_components(io.StringIO(_SYSTEM_WITH_COMPONENT_PROPERTIES)) + system = _parse_yaml_components(io.StringIO(_SYSTEM_WITH_COMPONENT_PROPERTIES)) lib_dict = resolve_library([input_library]) resolved = resolve_system(system, lib_dict) assert resolved.get_component("nuclear_1").properties == { @@ -223,7 +211,7 @@ def test_resolve_system_normalizes_list_properties_to_dict( def test_parse_yaml_components_properties_missing_key_raises() -> None: with pytest.raises(ValidationError): - parse_yaml_components(io.StringIO(_SYSTEM_WITH_PROPERTIES_MISSING_KEY)) + _parse_yaml_components(io.StringIO(_SYSTEM_WITH_PROPERTIES_MISSING_KEY)) _SYSTEM_WITH_PROPERTIES_DUPLICATE_KEYS = """\ @@ -242,7 +230,7 @@ def test_parse_yaml_components_properties_missing_key_raises() -> None: def test_resolve_component_properties_duplicate_keys_raises( input_library: LibrarySchema, ) -> None: - system = parse_yaml_components(io.StringIO(_SYSTEM_WITH_PROPERTIES_DUPLICATE_KEYS)) + system = _parse_yaml_components(io.StringIO(_SYSTEM_WITH_PROPERTIES_DUPLICATE_KEYS)) lib_dict = resolve_library([input_library]) with pytest.raises(ValueError, match="duplicate properties id"): resolve_system(system, lib_dict) @@ -260,7 +248,7 @@ def test_resolve_component_properties_duplicate_keys_raises( def test_parse_yaml_components_system_level_properties_rejected() -> None: with pytest.raises(ValidationError): - parse_yaml_components(io.StringIO(_SYSTEM_WITH_SYSTEM_LEVEL_PROPERTIES)) + _parse_yaml_components(io.StringIO(_SYSTEM_WITH_SYSTEM_LEVEL_PROPERTIES)) # --- model-declared properties --- @@ -276,8 +264,8 @@ def test_parse_yaml_components_system_level_properties_rejected() -> None: def test_resolve_component_with_declared_property_ok() -> None: - lib = parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) - system = parse_yaml_components(io.StringIO("""\ + lib = _parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) + system = _parse_yaml_components(io.StringIO("""\ system: components: - id: G @@ -291,8 +279,8 @@ def test_resolve_component_with_declared_property_ok() -> None: def test_resolve_component_missing_declared_property_raises() -> None: - lib = parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) - system = parse_yaml_components(io.StringIO("""\ + lib = _parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) + system = _parse_yaml_components(io.StringIO("""\ system: components: - id: G @@ -320,8 +308,8 @@ def test_resolve_component_missing_declared_property_raises() -> None: def test_resolve_component_with_declared_parameters_ok() -> None: - lib = parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PARAMETERS)) - system = parse_yaml_components(io.StringIO("""\ + lib = _parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PARAMETERS)) + system = _parse_yaml_components(io.StringIO("""\ system: components: - id: G @@ -337,8 +325,8 @@ def test_resolve_component_with_declared_parameters_ok() -> None: def test_resolve_component_missing_declared_parameter_raises() -> None: - lib = parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PARAMETERS)) - system = parse_yaml_components(io.StringIO("""\ + lib = _parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PARAMETERS)) + system = _parse_yaml_components(io.StringIO("""\ system: components: - id: G @@ -352,8 +340,8 @@ def test_resolve_component_missing_declared_parameter_raises() -> None: def test_resolve_component_extra_undeclared_property_allowed() -> None: - lib = parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) - system = parse_yaml_components(io.StringIO("""\ + lib = _parse_yaml_library(io.StringIO(_LIB_WITH_MODEL_PROPERTIES)) + system = _parse_yaml_components(io.StringIO("""\ system: components: - id: G diff --git a/tests/unittests/test_utils.py b/tests/unittests/test_utils.py index 2e0849f1..bfe03cbb 100644 --- a/tests/unittests/test_utils.py +++ b/tests/unittests/test_utils.py @@ -13,7 +13,7 @@ import pytest -from gems.utils import get_or_add +from gems_runner.utils import get_or_add def test_get_or_add_should_evaluate_lazily() -> None: diff --git a/uv.lock b/uv.lock index f6f6edd8..64ea532d 100644 --- a/uv.lock +++ b/uv.lock @@ -678,7 +678,7 @@ wheels = [ [[package]] name = "gemspy" -version = "0.1.2" +version = "2.0.0" source = { editable = "." } dependencies = [ { name = "antlr4-python3-runtime" },