Skip to content

Adjust handling of scalar-like values with new NumPy.#6331

Open
mzient wants to merge 3 commits intoNVIDIA:mainfrom
mzient:scalar_constant_numpy_fix
Open

Adjust handling of scalar-like values with new NumPy.#6331
mzient wants to merge 3 commits intoNVIDIA:mainfrom
mzient:scalar_constant_numpy_fix

Conversation

@mzient
Copy link
Copy Markdown
Contributor

@mzient mzient commented May 4, 2026

Category:

Bug fix (non-breaking change which fixes an issue)

Description:

This PR fixes handling of scalar-like NumPy tensors in Numpy 2.4, where converting a one-element (but non-0D) tensor to a number is illegal.

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: N/A

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [50214972]: BUILD STARTED

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 4, 2026

Greptile Summary

This PR fixes a NumPy 2.4 compatibility break where converting a single-element (but non-0-D) numpy array to a Python scalar via int(), float(), or bool() is no longer allowed. The fix adds .item() calls in ScalarConstant.__init__ and a duck-typed item() unwrap in _type_convert_value to extract the Python scalar before any further conversion.

  • ScalarConstant.__init__: replaces bare int(value) / float(value) / bool(value) with int(value.item()) etc., and adds an else branch to handle dtypes that fall outside the known int/float/bool groups.
  • _type_convert_value: adds a pre-conversion step that calls .item() on any object exposing that method, using a walrus-operator duck-type check.
  • New test_pseudoscalar test covers both fn.rotate(angle=np.array([90])) and ScalarConstant(np.array([90])) scenarios.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to numpy scalar unwrapping and is guarded by the existing isinstance check downstream.

Both changes are small and targeted: they insert a .item() call to extract a Python scalar before any further type conversion, leaving all other code paths untouched. The existing isinstance guard in ScalarConstant.__init__ provides a safety net for unexpected return values from .item(). The new test exercises the exact scenarios the PR is fixing.

The duck-typed item() check in _type_convert_value (line 128) is slightly broader than numpy-specific; worth a second look but not a blocker.

Important Files Changed

Filename Overview
dali/python/nvidia/dali/types.py Adds .item() calls to extract Python scalars from numpy arrays before type conversion, fixing NumPy 2.4 breakage; introduces a walrus-operator duck-typing check in _type_convert_value that is slightly broader than strictly needed.
dali/test/python/test_functional_api.py Adds test_pseudoscalar to cover 1-D single-element numpy arrays passed as angle and ScalarConstant arguments; copyright year updated to 2026.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User passes value] --> B{Has .dtype attribute?}
    B -- No --> E[Use value as-is]
    B -- Yes --> C[Resolve dali_type]
    C --> D{dali_type category}
    D -- int --> F["int(value.item())"]
    D -- float --> G["float(value.item())"]
    D -- bool --> H["bool(value.item())"]
    D -- other --> I["value.item()"]
    F & G & H & I --> J{"isinstance check: bool/int/float"}
    J -- pass --> K[Store scalar + dtype]
    J -- fail --> L[raise TypeError]

    subgraph _type_convert_value
        M[val arrives] --> N{Has item method?}
        N -- Yes --> O["val = val.item()"]
        N -- No --> P[val unchanged]
        O & P --> Q[Apply known type converter]
    end
Loading

Reviews (3): Last reviewed commit: "Add tests for pseudoscalar handling." | Re-trigger Greptile

Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [50217953]: BUILD STARTED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [50217953]: BUILD FAILED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [50217953]: BUILD PASSED

@JanuszL JanuszL self-assigned this May 4, 2026
@JanuszL JanuszL self-requested a review May 4, 2026 16:53
Comment on lines 250 to +258
dali_type = to_dali_type(value.dtype)
if dali_type in _int_types:
value = int(value)
value = int(value.item())
elif dali_type in _float_types:
value = float(value)
value = float(value.item())
elif dali_type in _bool_types:
value = bool(value)
value = bool(value.item())
else:
value = value.item()
Copy link
Copy Markdown
Contributor

@JanuszL JanuszL May 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dali_type = to_dali_type(value.dtype)
if dali_type in _int_types:
value = int(value)
value = int(value.item())
elif dali_type in _float_types:
value = float(value)
value = float(value.item())
elif dali_type in _bool_types:
value = bool(value)
value = bool(value.item())
else:
value = value.item()
dali_type = to_dali_type(value.dtype)
if item := getattr(value, "item", None):
value = item()
if dali_type in _int_types:
value = int(value)
elif dali_type in _float_types:
value = float(value)
elif dali_type in _bool_types:
value = bool(value)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already know that the type has dtype, so we assume it also has item - it holds for all types we claim to be compatible with (numpy, torch, cupy).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we can call value = item() at top unconditionally then.

value = bool(value)
value = bool(value.item())
else:
value = value.item()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor / dead code] This else branch looks unreachable. to_dali_type(value.dtype) resolves through _type_name_to_dali_type (lines 458–482), whose values are exclusively BOOL / INT{8,16,32,64} / UINT{8,16,32,64} / FLOAT{16,,64} — i.e. exactly the union of _int_types ∪ _float_types ∪ _bool_types. Any dtype that doesn't map there causes to_dali_type to raise before we reach this if/elif. So dali_type will always hit one of the three branches above and value.item() here will never run.

If this was intended as a defensive fallback for a future DALI type, a brief comment explaining that would help. Otherwise I'd just drop the else: (and Janusz's suggestion above — pulling the .item() call out once before the if/elif chain — would naturally remove this branch too).

dali_type = to_dali_type(value.dtype)
if dali_type in _int_types:
value = int(value)
value = int(value.item())
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit / tests] PR description marks "Existing tests apply" but the bug being fixed is specifically non-0D, single-element arrays under NumPy ≥ 2.4 (e.g. np.array([5])). Existing coverage I could find exercises 0D arrays / numpy scalars, where int(value) worked even on 2.4. A 2-line test instantiating ScalarConstant(np.array([5])) / ScalarConstant(np.array([1.5])) / ScalarConstant(np.array([True])) and asserting .value / .dtype would lock in the regression — otherwise it'll silently re-break the next time someone refactors this block.

Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [50319318]: BUILD STARTED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [50319318]: BUILD PASSED

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants