add_piecewise_formulation: support a partial `active` gate (treat missing/null entries as always-active)

[FabianHofmann]

We need partial active triggers in pw formulations.

Note

AI written issue

Model.add_piecewise_formulation(..., active=<var>) gates a piecewise function on a binary active variable: when active=0, the auxiliary variables (and thus the output) are pushed to zero. The gating enters through delta_var <= active and the bp0 * active intercept term in _add_incremental (and the analogous disjunctive path).

This works when active is defined over the entire coordinate of the formulation. It breaks down when active is only defined over a subset of the indexed dimension.

Problem

When the active expression is null/missing for part of the formulation's dimension, those entries get gated as if active=0 and their output is forced to zero — rather than being treated as ungated (always on).

This surfaced downstream in PyPSA (PyPSA/PyPSA#1755). PyPSA builds one piecewise formulation across a mix of committable and non-committable components and passes the unit-commitment status variable as active. The status variable only exists for committable components; for the rest it is null. The result: non-committable components — which should always dispatch — were incorrectly forced to zero by the _active piecewise constraint.

Proposed feature

Let add_piecewise_formulation accept an active expression/variable that is defined only over a subset of the formulation's dimension (i.e. null/missing elsewhere). Entries where active is missing should be treated as always active (gate = 1, ungated), and entries where it is present should be gated as today.

Effectively:

• where active.isnull() → no gating
• where ~active.isnull() → gated on active

Notes / open questions

• The active semantics are already flagged as an evolving API (EvolvingAPIWarning), so refining them here fits.
• Decide whether the trigger is "NaN entries in a full-coordinate active" vs. "active indexed over a strict subset of the dimension" — both should resolve to the same behaviour, ideally without the caller pre-aligning.
• Applies to both the incremental/SOS2 (_add_incremental) and disjunctive (_add_disjunctive) paths; the LP path already rejects active.

[FBumann]

Strongly support feature request. Not sure about the api/ergonomics!
Should hold v1 misalignment concerns. So passing a subset should not be incentivised. I would rather see an easy way to align/padd.

[FBumann]

@FabianHofmann I investigated the implementation options, especially how the best approach might differ between legacy and v1 #717.

Overall, id like to not introduce new surface that accepts misaligned coords. It rather push the fix to the user.

The intend is to treat missing variables "correctly". BUT CORRECTLY IS AMBIGUOUS**. Is missing always on or always off?

The user should be able to choose how to treat missing variables. The idiomatic linopy/xarray answer to this is fillna(fill_value), cobined with reindex if misaligned.

Under v1, its pretty straight forward: reindex to full coords, fill missing values with 1--> active.reindex(full).fillna(1).
Unfortunately, this breaks under legacy when using a plain variable, or masked variables (see below)

So with the upcoming v1, we would not need a helper in my opinion. Just a good Error message.

With legacy, a smal helper is pretty much needed. We can simply Deprecate it as soon as legacy is dropped...

This kind of shows that we should move onto v1 as soon as possible #717
.

Note

The following content was generated by AI.

Two shapes of "partial active"

It's worth separating two things, because they behave differently and only one is the bug:

• Missing coordinate labels along a shared dim — this is add_piecewise_formulation: support a partial active gate (treat missing/null entries as always-active) #796. Two sub-shapes:
  • (a) strict subset — active is indexed over fewer labels than the formulation (gen=["a"] vs gen=["a","b"]).
  • (b) full coord, masked — active spans the full dim but is masked/null at some entries (add_variables(..., mask=...), label -1).
• Fewer dimensions — e.g. active over gen when the formulation is gen × snapshot. This is not partial; it just broadcasts to every entry along the absent dim and is fine. (Verified.)

What actually works, per shape and regime

const value at the missing entry (1 = always-on ✅, 0 = silently always-off ❌):

spelling	(a) subset · legacy	(a) subset · v1	(b) masked · legacy	(b) masked · v1
active.fillna(1)	can't add labels	—	0 ❌	1 ✅
(1*active).reindex(full).fillna(1)	1 ✅	1 ✅	0 ❌	1 ✅
active.where(active.labels>=0, 1)	—	—	1 ✅	1 ✅

Reading the table:

• Under v1, active.reindex(coords).fillna(1) is universal — correct for both shapes, with or without the 1* cast. One spelling, no footgun. This matches @FBumann's point above.
• Under legacy there is no single correct one-liner: fillna can't add absent labels (subset needs reindex); bare Variable.fillna silently yields always-off (needs the 1* cast); and reindex().fillna(1) is right for the subset shape but silently wrong for the masked shape (which needs .where instead).

So the footgun is entirely a legacy artifact — v1 collapses the whole table into one clean line.

Why legacy is shape-dependent (mechanism)

reindex on a strict-subset LinearExpression introduces a real NaN const that fillna then fills. A masked entry instead carries coeff=1, var=-1 (missing), const=0 — there is no NaN to fill — so legacy fillna no-ops (the fill lands on the coefficient of a non-existent variable and is multiplied to zero), leaving const 0 = always-off. v1 fixed fillna to route the fill into the constant for masked/missing entries, which is why every v1 cell is 1.

Implications for the fix

1. Validate, don't auto-fix — and make the check catch both shapes: after aligning active to the formulation coords, reject any entry that is missing or masked (label < 0 / absent). One check, shape- and regime-agnostic, turns today's silent-infeasible (legacy) / opaque alignment ValueError (v1) into an actionable message.
2. Document the pad — under v1 it's just active.reindex(coords).fillna(1); the only caveat worth printing is the legacy spelling, which dies when v1 becomes the default.
3. Helper value is legacy-lifespan only — it would exist purely to hide the legacy shape/cast footguns above. Under v1 it's redundant sugar over fillna.

The three gate sites that assume full coverage, for reference: delta_var <= active and the bp0 * active intercept in _add_incremental, and rhs = active (the Σλ == active / Σbinary == active convexity constraint) in the SOS2 and disjunctive paths.

Reproduction + fix (HiGHS, legacy)

Two generators; active defined only over the committable subset {a}; force a off and require b to dispatch.

import pandas as pd, linopy
from linopy import Model

m = Model()
gens = pd.Index(["a", "b"], name="gen")
x = m.add_variables(lower=0, upper=100, coords=[gens], name="x")
y = m.add_variables(lower=0, coords=[gens], name="y")
u = m.add_variables(binary=True, coords=[pd.Index(["a"], name="gen")], name="u")

# BROKEN: subset active -> "b" wrongly gated to zero -> infeasible
m.add_piecewise_formulation((x, [0, 50, 100]), (y, [0, 10, 50]),
                            active=u, method="incremental")
# ... force u<=0, x[b]>=50  =>  Status: infeasible

# FIXED: pad to full coverage (missing -> always-on)
gate = (1 * u).reindex(gen=gens).fillna(1)
m.add_piecewise_formulation((x, [0, 50, 100]), (y, [0, 10, 50]),
                            active=gate, method="incremental")
# => a off (0, 0), b on (50, 10), Status: ok