Adjust handling of scalar-like values with new NumPy.#6331
Adjust handling of scalar-like values with new NumPy.#6331mzient wants to merge 3 commits intoNVIDIA:mainfrom
Conversation
|
CI MESSAGE: [50214972]: BUILD STARTED |
|
| Filename | Overview |
|---|---|
| dali/python/nvidia/dali/types.py | Adds .item() calls to extract Python scalars from numpy arrays before type conversion, fixing NumPy 2.4 breakage; introduces a walrus-operator duck-typing check in _type_convert_value that is slightly broader than strictly needed. |
| dali/test/python/test_functional_api.py | Adds test_pseudoscalar to cover 1-D single-element numpy arrays passed as angle and ScalarConstant arguments; copyright year updated to 2026. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[User passes value] --> B{Has .dtype attribute?}
B -- No --> E[Use value as-is]
B -- Yes --> C[Resolve dali_type]
C --> D{dali_type category}
D -- int --> F["int(value.item())"]
D -- float --> G["float(value.item())"]
D -- bool --> H["bool(value.item())"]
D -- other --> I["value.item()"]
F & G & H & I --> J{"isinstance check: bool/int/float"}
J -- pass --> K[Store scalar + dtype]
J -- fail --> L[raise TypeError]
subgraph _type_convert_value
M[val arrives] --> N{Has item method?}
N -- Yes --> O["val = val.item()"]
N -- No --> P[val unchanged]
O & P --> Q[Apply known type converter]
end
Reviews (3): Last reviewed commit: "Add tests for pseudoscalar handling." | Re-trigger Greptile
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
|
CI MESSAGE: [50217953]: BUILD STARTED |
|
CI MESSAGE: [50217953]: BUILD FAILED |
|
CI MESSAGE: [50217953]: BUILD PASSED |
| dali_type = to_dali_type(value.dtype) | ||
| if dali_type in _int_types: | ||
| value = int(value) | ||
| value = int(value.item()) | ||
| elif dali_type in _float_types: | ||
| value = float(value) | ||
| value = float(value.item()) | ||
| elif dali_type in _bool_types: | ||
| value = bool(value) | ||
| value = bool(value.item()) | ||
| else: | ||
| value = value.item() |
There was a problem hiding this comment.
| dali_type = to_dali_type(value.dtype) | |
| if dali_type in _int_types: | |
| value = int(value) | |
| value = int(value.item()) | |
| elif dali_type in _float_types: | |
| value = float(value) | |
| value = float(value.item()) | |
| elif dali_type in _bool_types: | |
| value = bool(value) | |
| value = bool(value.item()) | |
| else: | |
| value = value.item() | |
| dali_type = to_dali_type(value.dtype) | |
| if item := getattr(value, "item", None): | |
| value = item() | |
| if dali_type in _int_types: | |
| value = int(value) | |
| elif dali_type in _float_types: | |
| value = float(value) | |
| elif dali_type in _bool_types: | |
| value = bool(value) |
There was a problem hiding this comment.
We already know that the type has dtype, so we assume it also has item - it holds for all types we claim to be compatible with (numpy, torch, cupy).
There was a problem hiding this comment.
So we can call value = item() at top unconditionally then.
| value = bool(value) | ||
| value = bool(value.item()) | ||
| else: | ||
| value = value.item() |
There was a problem hiding this comment.
[Minor / dead code] This else branch looks unreachable. to_dali_type(value.dtype) resolves through _type_name_to_dali_type (lines 458–482), whose values are exclusively BOOL / INT{8,16,32,64} / UINT{8,16,32,64} / FLOAT{16,,64} — i.e. exactly the union of _int_types ∪ _float_types ∪ _bool_types. Any dtype that doesn't map there causes to_dali_type to raise before we reach this if/elif. So dali_type will always hit one of the three branches above and value.item() here will never run.
If this was intended as a defensive fallback for a future DALI type, a brief comment explaining that would help. Otherwise I'd just drop the else: (and Janusz's suggestion above — pulling the .item() call out once before the if/elif chain — would naturally remove this branch too).
| dali_type = to_dali_type(value.dtype) | ||
| if dali_type in _int_types: | ||
| value = int(value) | ||
| value = int(value.item()) |
There was a problem hiding this comment.
[Nit / tests] PR description marks "Existing tests apply" but the bug being fixed is specifically non-0D, single-element arrays under NumPy ≥ 2.4 (e.g. np.array([5])). Existing coverage I could find exercises 0D arrays / numpy scalars, where int(value) worked even on 2.4. A 2-line test instantiating ScalarConstant(np.array([5])) / ScalarConstant(np.array([1.5])) / ScalarConstant(np.array([True])) and asserting .value / .dtype would lock in the regression — otherwise it'll silently re-break the next time someone refactors this block.
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
|
CI MESSAGE: [50319318]: BUILD STARTED |
|
CI MESSAGE: [50319318]: BUILD PASSED |
Category:
Bug fix (non-breaking change which fixes an issue)
Description:
This PR fixes handling of scalar-like NumPy tensors in Numpy 2.4, where converting a one-element (but non-0D) tensor to a number is illegal.
Additional information:
Affected modules and functionalities:
Key points relevant for the review:
Tests:
Checklist
Documentation
DALI team only
Requirements
REQ IDs: N/A
JIRA TASK: N/A