Skip to content

feat(iterate2): support per-metric direction (minimize/maximize)#316

Merged
romeokienzler merged 2 commits into
mainfrom
add_metric_direction
Jun 29, 2026
Merged

feat(iterate2): support per-metric direction (minimize/maximize)#316
romeokienzler merged 2 commits into
mainfrom
add_metric_direction

Conversation

@romeokienzler

Copy link
Copy Markdown
Member

Summary

Adds support for specifying whether each metric should be minimized or maximized in iterate2. Previously all metrics were unconditionally maximized.

Changes

terratorch_iterate/iterate2/_iterate2.py

  • _default_direction(metric_name) — new heuristic helper: metrics whose name contains loss, error, err, mse, mae, or rmse default to minimize; all others default to maximize.
  • load_metrics() — now returns (names, directions) tuple. The metrics: YAML key now accepts two forms:
    • Simple string — direction inferred by name heuristic
    • Extended dict {name: ..., direction: minimize|maximize} — explicit per-metric control
    • A mix of both is allowed; invalid direction values raise ValueError.
  • main() — unpacks the tuple and passes the real directions to optuna.create_study() instead of the hardcoded ["maximize"] * len(metrics).

docs/iterate2.md

  • Added "Metric directions (minimize / maximize)" subsection under "Metric extraction" documenting the heuristic, both YAML forms, and a mixed-form example.
  • Removed the stale "All objectives are maximised" statement.

YAML usage

Simple form (direction inferred from name):

metrics:
  - val_loss      # → minimize  (contains "loss")
  - accuracy      # → maximize

Extended form (explicit per-metric):

metrics:
  - name: val_loss
    direction: minimize
  - name: accuracy
    direction: maximize

Mixed form is also supported.

- Add _default_direction() heuristic: metrics containing loss/error/err/mse/mae/rmse default to minimize, all others to maximize
- Extend load_metrics() to return (names, directions) tuple; accept both simple string form and extended {name, direction} dict form in the YAML metrics: section
- Remove hardcoded ['maximize'] * len(metrics) in main(); use directions from load_metrics() instead
- Log directions at startup
- Update docs/iterate2.md with new Metric directions subsection

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
@romeokienzler romeokienzler force-pushed the add_metric_direction branch from 693c2ee to 344eeac Compare June 29, 2026 12:40
- Replace broken 'python ./run_tests.py' (LSF/bsub job submitter, not
  runnable in GitHub Actions) with 'pytest tests/unit/test_iterate2.py'
- Install pytest, pyyaml, optuna and the package itself (--no-deps) so
  the test module can be imported without a ModuleNotFoundError for yaml

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
@romeokienzler romeokienzler merged commit f0dc755 into main Jun 29, 2026
10 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants