Skip to content

Add invisible metadata in front of the units#98

Open
Ants-Aare wants to merge 5 commits into
Mc-Zen:mainfrom
Ants-Aare:metadata
Open

Add invisible metadata in front of the units#98
Ants-Aare wants to merge 5 commits into
Mc-Zen:mainfrom
Ants-Aare:metadata

Conversation

@Ants-Aare

Copy link
Copy Markdown

this allows for querying of units & numbers, and allows packages to reason about any units or numbers they receive, without changing anything for the user.

I had to move some things around, because I want all of the metadata to be outside of context and basically just be a parsed representation of the underlying data. I also added a method to utility so that it's easier to retrieve the data from a content we know is a unit/number.
With the current implementation units will have a unit metadata in front of them, as well as the number metadata inside of it. I personally don’t think this is necessary, and would prefer to use show-num inside of the qty function, but because we are calling the num function we inherit the attached metadata from that as well.

I wasn't sure if adding the named and positional arguments would be necessary, since digits and round:precision/figures are kind of important and relevant information. But for now I left it out.

The main reason for this PR is because I would like to use this in a package.

@Mc-Zen

Mc-Zen commented Jul 16, 2026

Copy link
Copy Markdown
Owner

Hi @Ants-Aare ,

Numbers and units will of course be queryable (is this a word?) when all of them become proper types. This is a high priority to me once Typst introduces them.

Until then I'm open to exposing the internals via metadata like you did, as long as it does not hurt performance (especially in tables and for Lilaq tick labels, this is critical).

Could you tell me more about your use case for this and which info you exactly need? We need to keep in mind that not all fields might be accessible when num etc become types because a lot of the internal parts are "synthesized" from the arguments (e.g. the input usually differs quite a lot from the value that is displayed ). Just to be careful not to rely on info that is available now through metadata and that might not be available later when this is replaced with types because eventually I don't want metadata to be hidden in num etc after they are types.

Comment thread src/align-column.typ

return numbers.map(components => components
.zip(max-widths, (right, left, left, left))
.zip(max-widths, (right, right, left, left, left))

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a mistake? There are only four components and only the first one should be right-aligned

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not a mistake, but a workaround for the metadata counting as a part of the component array. so to keep the first visible component right aligned, I need to pass in something for the metadata. A different workaround would be to do components.first() + components.slice(1, components.len()).map(x=> box(.......)) thus not doing all of the manual width fixes for the metadata. let me know what you prefer.

@Ants-Aare

Copy link
Copy Markdown
Author

I basically just expose the info dictionary that is synthesised right now. the reason for this is that it's easy to modify it and pass it back into num(). let's say I want to remove the sign, or manipulate the unit in some way. Additionally I expose the number as a float value with all the "e3" stuff applied, so that the number always exists as a processable value, even if it may loose precision from floating point, same for the uncertainty. keeping the raw value in it's unmodified form may be useful for some people, but I don't personally need it.

@Ants-Aare

Ants-Aare commented Jul 16, 2026

Copy link
Copy Markdown
Author

What I'm working on right now is a refactor/ ground up re-implementation of the pariman package. I like the Pariman package, but I don't like the API. Also it's currently doing a lot of work that zero already does and there are some weird edge cases around uncertainty/places where it does things on top of zero. I basically want to ideally make a package that's "zero-calc" which allows for all/most of the features in the typst calc namespace to be used with zero units. This would also include some displaying functions to display the calculation steps that were used.
I already made a PR for the original Pariman package to add error propagation and I intend to add it to "zero-calc" as well, so that uncertainties are kept up to date.

@Ants-Aare

Ants-Aare commented Jul 16, 2026

Copy link
Copy Markdown
Author

the reason I'm refactoring Pariman is because I am currently working on a package for the typsium chemistry rendering project I'm part of, where I want to automate the calculations involved in mixing solutions, dilutions, etc. Which is similar to the zero API where I declare a "unit" i.e. a molecule such as water and then I can use #water(ml(50)) to display "50 ml of water" and then use set rules etc to change if it should be displayed as grams or moles or if the formula/structure should be used. And since the least amount of work is done with raw substances it's important to be able to make mixtures, so mix(water(ml(50)), nacl(g(58))) will then display as "1molar NaCl solution". and then obviously you could continue to use that variable to calculate further. I want things to mesh neatly together with zero, even if it's probably heavier doing it this way than just using regular good ol floats.

@Ants-Aare

Ants-Aare commented Jul 16, 2026

Copy link
Copy Markdown
Author

I really like what you did with your approach to accessibility in the sense that the names are decoupled from the declarations in zi. This makes it a lot easier to synthesise units along the way and have accessibility still work perfectly.

I'm waiting for the types API too... 😔
I fiddled around with elembic a lot and it's great, but I don't think it's quite the right approach. I hope types get support for functions as well. Since I have different representations of variables, either the result of the calculation or the displaying calculation method, it would be nice to return the type and the user can decide if they want to use .display or .display-error or .display-calculation

@Mc-Zen

Mc-Zen commented Jul 19, 2026

Copy link
Copy Markdown
Owner

That sounds indeed very exciting! I'm stoked to see the result.

I have a few remarks but the performance seems fine.

Comment thread src/units.typ
..args,
) = context {
) = {
let info = process-input(value)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could really be left as parse-numeral because dictionary input is not even supported here and this is all that the process input function does on top.

Comment thread src/utility.typ
}

#let info-to-float(value) = {
let f = float(value.sign + value.int + if value.frac != ""{"." + value.frac} + if value.e != none and not value.e.contains("."){"e" + value.e})

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file needs to be formatted a bit better (for example spaces before and after braces)

Comment thread src/utility.typ
Comment on lines +89 to +90
float: if type(raw) != float and type(raw) != int {info-to-float(info)} else{raw},
uncertainty: info-to-uncertainty(info),

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these fields should be left out for performance and instead the conversion functions should be exported as utility functions so that a user can construct these fields themselves.

Comment thread src/utility.typ
))<zero-num>]

#let create-qty-metadata(info, raw, unit, args) = [#metadata((
float: if type(raw) != float and type(raw) != int {info-to-float(info)} else{raw},

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

Comment thread tests/metadata/ref/1.png

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if this test would be ephemeral, i.e., add a reference file ref.typ that produces the same output. Then, no reference test images are necessary. Check out the other tests for examples

Comment thread README.md

Furthermore, `num()` also allows `array` arguments for `number` which allows for more efficient batch-processing of numbers with the same setup. In this case, the caller of the function needs to provide `context`.

If you want to accept `num()` or `qty()` as arguments for your functions you can access the underlying raw data we use for drawing by calling the `impl.utility.retrieve.metadata` function. Manipulating this data will not change what content is drawn, but allows you to reason about the values. You can query for numbers and quantities by using `query(selector(metadata).and(<zero-num>))` and `query(selector(metadata).and(<zero-qty>))` respecticely.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a typo here: there should be a - between retrieve and metadata

Comment thread src/num.typ
)

if type(number) == array {
if (type(number) == array) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These parentheses are not needed

Comment thread src/num.typ
let info = process-input(number)
let metadata = utility.create-num-metadata(info, number, args)

if (state != auto) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same here

Comment thread src/num.typ
show-num(it)
if align == "components"{
(metadata,) + show-num(it)
}else {

@Mc-Zen Mc-Zen Jul 19, 2026

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

formatting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants