Skip to content

fix(lua): hand callbacks Lua.t regardless of entry point#379

Merged
davydog187 merged 1 commit into
mainfrom
fix/callback-state-asymmetry
Jul 3, 2026
Merged

fix(lua): hand callbacks Lua.t regardless of entry point#379
davydog187 merged 1 commit into
mainfrom
fix/callback-state-asymmetry

Conversation

@davydog187

Copy link
Copy Markdown
Contributor

Fixes #377.

Problem

A two-arity Elixir callback (fn args, state -> {results, state} end) received a different state — and had to return a different shape — depending on how it entered the VM:

Entry point state handed to the callback
Lua.set!(lua, [:f], fun) — fn at the path Lua.t (%Lua{}) ✅
deflua + Lua.load_api/3 Lua.t (%Lua{}) ✅
Lua.set!(lua, [:t], %{"f" => fun}) — fn nested in a value raw Lua.VM.State
Lua.encode!(lua, fun) — closure encoded at runtime raw Lua.VM.State

Lua.set!/3's is_function clauses wrap the raw %Lua.VM.State{} into the public %Lua{}, call the callback, unwrap the returned %Lua{} back to its .state, and validate the return is encoded. But functions reaching the VM as an encoded value — via Lua.encode!/2 or nested inside a value passed to Lua.set!/3 — go through Lua.VM.Value.encode/2, which stored them bare ({:native_func, fun}) with no wrapping. The VM then invoked them with the raw internal state, which the public API (Lua.decode!/2, Lua.get_private!/2, …) rejects with no function clause matching. So the same closure worked at a path and crashed as an encoded value.

Fix

Lua.VM.Value is deliberately ignorant of %Lua{}, so the wrapping stays in the Lua layer and is now shared by every entry point:

  • lib/lua.ex — extract the wrap/unwrap/validate closure builder into a shared wrap_callback/3 (+ validate_encoded!/3). Both set!-at-path clauses use it, and it is injected into the two Value.encode call sites (encode!/2 and the nested-value set! clause).
  • lib/lua/vm/value.exencode/3 takes an optional fun_wrapper threaded through the map/list recursion. The default preserves the prior raw behavior for low-level callers; the public Lua entry points supply the %Lua{} wrapper.

Behavior changes (clean break, intended)

  • 2-arity callbacks entered via encode!/2 or nested in a set! value now receive %Lua{} and return {results, %Lua{}} (or bare results), matching the documented convention. Code relying on the rc.3 raw-state behavior or the %Lua{state: raw} workaround should drop the workaround.
  • arity-1 callbacks entered via those paths now get the same return-encoding validation as the path entry point.

Verification

  • Added test/lua/callback_state_asymmetry_test.exs (the issue's test module). Its 3 encoded-value cases failed before the fix and pass after.
  • mix test --include lua53 — 2592 passed, mix format --check-formatted, mix dialyzer — 0 errors.

A two-arity Elixir callback (`fn args, state -> {results, state} end`)
received a different `state` depending on how it entered the VM. Functions
set directly at a path via `Lua.set!/3` and `deflua` functions were wrapped
so the callback saw the public `Lua.t`, but functions reaching the VM as an
encoded value — via `Lua.encode!/2` or nested inside a value passed to
`Lua.set!/3` — were stored bare and invoked with the raw `Lua.VM.State`.
The raw state is unusable with the public API, so the same closure worked
at a path and crashed as an encoded value.

Extract the `%Lua{}` wrap/unwrap/validate closure builder into a shared
`wrap_callback/3` and inject it into the `Value.encode/3` walk, so every
entry point speaks the documented `Lua.t` convention. Also unifies arity-1
and arity-2 under one builder, giving arity-1 the same return-encoding
validation regardless of entry point.
@davydog187 davydog187 merged commit 5ae213e into main Jul 3, 2026
5 checks passed
@davydog187 davydog187 deleted the fix/callback-state-asymmetry branch July 3, 2026 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Two-arity callbacks receive raw Lua.VM.State (not Lua.t) when entered via Lua.encode!/2 or nested in a set! value

1 participant