Bump clarity to 4.0.0 and cache FieldPath lookups (~18% faster)#84
Bump clarity to 4.0.0 and cache FieldPath lookups (~18% faster)#84spheenik wants to merge 2 commits intoodota:masterfrom
Conversation
Memoize per-DTClass FieldPath lookups so each (DTClass, property name) pair walks the field tree only once. Caller-side hits are 99.97% on a representative match; ~10% wall-clock parse-time reduction on a 195MB S2 replay. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bumps the clarity dependency from 3.1.3 to 4.0.0. Source-level adjustments: - DOTA_COMBATLOG_TYPES moved out of DOTAUserMessages into its own DOTACombatLog wrapper class (clarity-protobuf 5.4 -> 6.1). Imports updated in Parse, TrackVisitor, GreevilsGreedVisitor, Wards. - Custom event annotations (OnWardKilled / OnWardExpired / OnWardPlaced) ported to the typed-event pattern: nested Listener and Event interfaces, @GenerateEvent, parameterClasses dropped from @UsagePointMarker. Wards now holds typed Event fields and uses the single-arg createEvent with a cast (cast goes away once clarity ships the next release with a generic-return overload). - maven-shade plugin: ClassIndexTransformer (atteo) replaced by an AppendingTransformer for META-INF/clarity/providers.txt, the per-module file written by clarity 4.0.0's EventAnnotationProcessor to declare local @provides processors. Without this transformer the shaded jar would only carry clarity's own providers list and the Wards processor would not be discovered at runtime. Verified end-to-end on dota/s2/340/8168882574_1198277651.dem (195 MB): the Wards processor is discovered, the parse runs to completion, and median wall-clock drops from 4.384s on 3.1.3 to 3.609s on 4.0.0 (combined with the FieldPath cache from the previous commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Quick follow-up for the curious: I also pushed a preview branch
Same bench protocol as in this PR (2 warmup + 10 timed runs, 195 MB modern Dota S2 replay):
Worth noting: clarity 5.0 also drastically reduces allocations (a new flat entity-state representation cuts GC pressure by roughly 65% on this replay family vs 4.0). For a server-style workload like odota's parser that handles many replays back-to-back, that's likely an even bigger win than the wall-clock number above suggests. One more worth-knowing: clarity 5.0 raises the runtime baseline to Java 21 (the sealed-type refactor uses exhaustive Sharing FYI — the diff is right there for whenever 5.0.0 lands on Central proper. That said, if you're feeling adventurous, the snapshot is reasonably well-tested already (clarity-analyzer and the in-tree examples track it), so consuming it ahead of the GA isn't unsafe — just understand it's a moving target until release. |
Summary
Two independent commits:
Cache
FieldPathlookups ingetEntityProperty— clarity'sgetFieldPathForNamewalks the field tree and allocates a freshS2ModifiableFieldPathon every call, but the (DTClass, property-name) → FieldPath mapping is stable for the lifetime of a parse. Memoizing it eliminates ~5.6M redundant tree walks per Dota replay.Bump clarity 3.1.3 → 4.0.0. Source-level changes:
DOTA_COMBATLOG_TYPESmoved fromDOTAUserMessagesinto a dedicatedDOTACombatLogwrapper (clarity-protobuf 5.4 → 6.1) — import swap in 4 files.OnWardKilled / OnWardExpired / OnWardPlacedannotations ported to clarity 4.0's typed-event pattern (nestedListener+Eventinterfaces,@GenerateEvent,parameterClassesdropped from@UsagePointMarker).Wardsprocessor uses the new single-argcreateEventand typedEventfield types.maven-shadeplugin:ClassIndexTransformer(atteo) →AppendingTransformerforMETA-INF/clarity/providers.txt. Clarity 4.0 ships its ownEventAnnotationProcessorthat writes per-module provider lists to that path; without the new transformer the shaded jar would only carry clarity's own list and theWardsprocessor wouldn't be discovered at runtime.Performance
Bench setup: 195 MB modern Dota S2 replay (
dota/s2/340/8168882574_1198277651.dem), OpenJDK 21, 2 warmup + 10 timed runs, ByteArrayInputStream →OutputStream.nullOutputStream().Spreads under 4% on all configurations; ranges non-overlapping. Cache hit rate 99.97% over 5.6M
getEntityPropertycalls per parse.Cosmetic caveat — event casts
Wards.javanow contains lines like:```java
evKilled = (OnWardKilled.Event) ctx.createEvent(OnWardKilled.class);
```
That cast is unfortunate. Clarity 4.0.0's
Context.createEventreturnsEvent<A>(the base class), but the runtime instance is already the typed subclass. The next clarity release widens the signature to<E extends Event<A>> E createEvent(Class<A>), so the cast disappears with no source change here — just a clarity version bump.Looking ahead
The next clarity release brings further improvements that are expected to trim parse time on this workload again. Happy to open a follow-up PR once it ships.
Test plan
🤖 Generated with Claude Code