Skip to content

fix(spp_import_match): conditional rows act as pure gates, hide UI columns (#991)#197

Merged
emjay0921 merged 4 commits into
19.0from
fix/991-import-match-condition-field
May 13, 2026
Merged

fix(spp_import_match): conditional rows act as pure gates, hide UI columns (#991)#197
emjay0921 merged 4 commits into
19.0from
fix/991-import-match-condition-field

Conversation

@emjay0921
Copy link
Copy Markdown
Contributor

Why is this change needed?

Per OP#991 ("Condition Field" is missing for spp_import_match): is_conditional=True rows on spp.import.match.fields were being used both as the gate (CSV value check) and as a search predicate (field_id injected into the DB domain). When the CSV had a metadata column that didn't exist on the registrant model — e.g. data_source — the generated search query either failed or silently returned zero matches, defeating the whole conditional-rule mechanism.

How was the change implemented?

Two commits, one fix and one polish:

da3f563e — fix(matching): add Condition Field, treat conditional rows as pure gates

  • Add a condition_field_id Many2one column on spp.import.match.fields, separate from the now-existing field_id.
  • Rewrite the matching loop in _match_find so conditional rows are evaluated as pure gates (CSV value compared to condition_value) and never added to the DB search domain.
  • Rename the IMPORTED VALUE column heading to Condition Value for clarity.
  • Backwards-compatible: existing rules without condition_field_id fall back to field_id.name for the gate check.

7d7c8796 — chore(views): hide conditional-gate columns until a use case lands
Reviewers asked to keep the new UI surface dormant until a real import flow needs the gate. Set column_invisible=\"1\" on Is Conditional, Condition Field, and Condition Value in the match-rule fields list. The model schema, the _match_find gate semantics, and the backwards-compat fallback to field_id.name all stay in place — only the column rendering is suppressed.

Module version bumped:

  • spp_import_match 19.0.2.0.0 → 19.0.2.0.2 (two HISTORY entries: one per commit)

New unit tests

None — the matching logic change is exercised by existing import-match scenarios that QA already validated. The column-hide commit is presentation-only.

Unit tests executed by the author

pre-commit run --files <scoped paths> passed locally in the CI-matched container.

How to test manually

QA already passed this branch (status Test pass on OP#991). For re-verification:

  1. Install / upgrade spp_import_match.
  2. Open Imports → Match Rules → Fields tab — the Is Conditional, Condition Field, Condition Value columns should NOT be visible.
  3. Existing rules with is_conditional=True and a condition_value set should still correctly gate imports without injecting non-model columns into the search domain.

Related links

emjay0921 added 2 commits May 5, 2026 16:32
… pure gates

`is_conditional=True` rows on `spp.import.match.fields` were being used
as both the gate (CSV value check) and a search predicate (`field_id`
added to the DB domain). When the CSV had a metadata column that
didn't exist on the registrant model — e.g. `data_source` — the
generated search query would either fail or silently return zero
matches, defeating the whole conditional-rule mechanism.

Two-part fix:

- Add a separate `condition_field_id` Many2one on
  `spp.import.match.fields` so a rule can gate on a different field
  than the one it searches on. The new field is shown in the form's
  Fields list as **Condition Field**, readonly until `Is Conditional`
  is ticked.
- Rewrite the matching loop in `spp.import.match._match_find`:
  conditional rows are now pure gates and never contribute to the
  search domain. The gate column is `condition_field_id.name` when
  set, falling back to `field_id.name` for backwards compatibility
  with rules created before this change.
- Rename the existing `imported_value` column heading to
  **Condition Value** to match the documentation.

Refs OP#991.
…se lands

The Condition Field / Condition Value / Is Conditional columns added
under OP#991 introduced UI surface for a feature that no current import
flow uses. Reviewers asked for them to be hidden until a real use case
shows up.

Switch all three columns on the match-rule fields list to
`column_invisible="1"`. The model schema, the `_match_find` gate
semantics, and the backwards-compat fallback to `field_id.name` when
`condition_field_id` is empty all stay in place — the data path is
unchanged. Only the form's column rendering is suppressed, so existing
records keep their values and the engine keeps honoring them; new rules
just don't expose the controls in the UI.

Bumps version 19.0.2.0.1 → 19.0.2.0.2 and fills in the missing
HISTORY entry for the OP#991 round-1 work.

Refs OP#991.
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a "conditional gate" mechanism to the import matching engine, allowing specific CSV columns to act as filters for matching rules without being included in the database search domain. Key changes include the addition of a condition_field_id to the matching fields and an update to the matching loop. The review highlights several critical issues: a regression in legacy rule handling due to incorrect field name usage, potential comparison errors from missing type normalization, and a design contradiction where using a Many2one to ir.model.fields prevents the selection of CSV-only metadata fields as originally intended. Furthermore, the change to exclude conditional fields from the search domain is noted as a significant breaking change for existing configurations.

Comment thread spp_import_match/models/import_match.py Outdated
Comment on lines +80 to +88
gate_field_name = (
field.condition_field_id.name
if field.condition_field_id
else field.field_id.name
)
if imported_row.get(gate_field_name) != field.imported_value:
combination_valid = False
break
continue
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

There are several issues in this logic block:

  1. Regression for Legacy Rules: The old code used field.name (line 71 in the previous version), which is a computed field that correctly handles sub-fields (e.g., parent_id/child_id). By switching to field.field_id.name (line 83), legacy conditional rules using sub-fields will now look for the parent field name in the CSV instead of the full path, likely causing them to fail.
  2. Comparison Robustness: Comparing imported_row.get() directly with field.imported_value (a Char field) is risky. If the CSV value is an integer or boolean, or if the field is empty (None vs False), the comparison might fail unexpectedly. It is safer to normalize both to strings.
  3. Breaking Change: The addition of continue (line 88) means that conditional fields are now excluded from the database search domain. Previously, they acted as both a gate and a search predicate. This is a significant change in matching logic that will affect existing rules. If this is intentional, it should be clearly documented as a breaking change.
Suggested change
gate_field_name = (
field.condition_field_id.name
if field.condition_field_id
else field.field_id.name
)
if imported_row.get(gate_field_name) != field.imported_value:
combination_valid = False
break
continue
gate_field_name = (
field.condition_field_id.name
if field.condition_field_id
else field.name
)
# Normalize to string to handle None/False/Int types correctly
csv_value = str(imported_row.get(gate_field_name) or "")
cond_value = field.imported_value or ""
if csv_value != cond_value:
combination_valid = False
break
continue

Comment on lines +160 to +175
condition_field_id = fields.Many2one(
"ir.model.fields",
string="Condition Field",
ondelete="cascade",
domain="[('model_id', '=', model_id)]",
help=(
"When `Is Conditional` is set, the rule only fires for CSV rows "
"whose value in this field equals `Condition Value`. The "
"condition field is used purely as a gate — it is **not** added "
"to the database search domain, so it can safely be a CSV-only "
"metadata column (e.g. `data_source`) that doesn't have data on "
"the registrant. Leave empty to fall back to the legacy "
"behaviour where `Field` is used as both the gate and the "
"search predicate."
),
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The implementation of condition_field_id as a Many2one to ir.model.fields with a domain restricted to the current model (model_id) contradicts the stated requirement in the PR description.

The description mentions that the gate column may be a "CSV-only metadata field (e.g. data_source) that doesn't exist on the registrant model". If the field does not exist on the model, it will not be present in ir.model.fields for that model, making it impossible to select in the UI. To support arbitrary CSV columns as gates, this field should likely be a Char field or allow for non-model field selection.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.12%. Comparing base (0627ab1) to head (d44adb3).
⚠️ Report is 37 commits behind head on 19.0.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             19.0     #197      +/-   ##
==========================================
- Coverage   71.63%   66.12%   -5.51%     
==========================================
  Files         933       86     -847     
  Lines       55370     7514   -47856     
==========================================
- Hits        39664     4969   -34695     
+ Misses      15706     2545   -13161     
Flag Coverage Δ
endpoint_route_handler ?
fastapi ?
spp_aggregation ?
spp_alerts ?
spp_analytics ?
spp_api_v2 ?
spp_api_v2_change_request ?
spp_api_v2_cycles ?
spp_api_v2_data ?
spp_api_v2_entitlements ?
spp_api_v2_gis ?
spp_api_v2_products ?
spp_api_v2_service_points ?
spp_api_v2_simulation ?
spp_api_v2_vocabulary ?
spp_approval ?
spp_area ?
spp_area_hdx ?
spp_attachment_av_scan ?
spp_audit ?
spp_banking ?
spp_base_common 90.26% <ø> (ø)
spp_base_setting ?
spp_case_base ?
spp_case_cel ?
spp_case_demo ?
spp_case_entitlements ?
spp_case_graduation ?
spp_case_programs ?
spp_case_registry ?
spp_case_session ?
spp_cel_domain ?
spp_cel_event ?
spp_cel_registry_search ?
spp_cel_vocabulary ?
spp_change_request_v2 ?
spp_claim_169 ?
spp_cr_types_advanced ?
spp_cr_types_base ?
spp_dci_client_dr ?
spp_dci_client_ibr ?
spp_dci_demo ?
spp_dci_server ?
spp_demo ?
spp_disability_registry ?
spp_drims ?
spp_drims_sl_demo ?
spp_encryption ?
spp_farmer_registry ?
spp_farmer_registry_cr ?
spp_farmer_registry_demo ?
spp_farmer_registry_vocabularies ?
spp_gis ?
spp_gis_indicators ?
spp_gis_report ?
spp_graduation ?
spp_grm ?
spp_grm_case_link ?
spp_grm_demo ?
spp_hazard ?
spp_hazard_programs ?
spp_hxl_area ?
spp_import_match 83.91% <100.00%> (+0.14%) ⬆️
spp_indicator ?
spp_irrigation ?
spp_land_record ?
spp_metric ?
spp_metric_service ?
spp_metrics_core ?
spp_metrics_services ?
spp_mis_demo_v2 ?
spp_oauth ?
spp_programs 64.87% <ø> (+0.38%) ⬆️
spp_registry_group_hierarchy ?
spp_scoring ?
spp_scoring_programs ?
spp_security 66.66% <ø> (ø)
spp_simulation ?
spp_starter_social_registry ?
spp_starter_sp_mis ?
spp_statistic ?
spp_storage_backend ?
spp_studio_change_requests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
spp_import_match/__manifest__.py 0.00% <ø> (ø)
spp_import_match/models/import_match.py 88.18% <100.00%> (+0.28%) ⬆️

... and 862 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

emjay0921 added 2 commits May 12, 2026 15:09
…te semantics (OP#991)

CI surfaced a regression in test_match_find_conditional_match. The
test was written under the pre-OP#991 semantics where an
`is_conditional=True` row contributed BOTH a gate (CSV value check)
AND a search predicate (`field_id` added to the search domain). The
OP#991 fix (commit da3f563) intentionally removes that dual role —
conditional rows are now pure gates, never injected into the domain.

Under the new semantics, a rule with only one conditional row produces
an empty search domain and is skipped (the existing `if not domain:
continue` guard at line 109). The test's assertion that the partner
would be found is therefore obsolete.

Rewrite the test to demonstrate the new gate + non-conditional-search
shape: a conditional row gates on `name`, and a non-conditional row
provides the actual `email` search predicate. Verifies the gate-
passing path while respecting the OP#991 semantics. Sibling test
test_match_find_conditional_skip (gate-failing path) still passes
unchanged.

44 tests, 0 failed locally.
@emjay0921
Copy link
Copy Markdown
Contributor Author

@gonzalesedwin1123 — ready for review. QA on OP#991 already passed; CI is fully green on d44adb38 (18 pass).

@emjay0921 emjay0921 merged commit b3510d8 into 19.0 May 13, 2026
19 checks passed
@emjay0921 emjay0921 deleted the fix/991-import-match-condition-field branch May 13, 2026 01:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants