fix(build): link bundled-embeddings on Windows MSVC by dropping tokenizers' esaxx_fast

[tcconnally]

Closes #222.

Problem

bundled-embeddings compiled but failed to link on Windows MSVC:

mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match
value 'MT_StaticRelease' ...  (LNK2038 / LNK1169: LIBCMT vs MSVCP140)

esaxx-rs's build.rs compiles its C++ with cc::Build::new().cpp(true).static_crt(true) (/MT), while ort's downloaded prebuilt binaries use the dynamic CRT (/MD). The two CRTs can't be linked into one executable.

Root cause (more precise than the issue)

esaxx-rs is a non-optional dependency of tokenizers, but its C++ build is gated:

// esaxx-rs/build.rs
#[cfg(feature = "cpp")]
fn build() { cc::Build::new().cpp(true).static_crt(true).file("src/esaxx.cpp")... }

# tokenizers/Cargo.toml
default = ["progressbar", "onig", "esaxx_fast"]
esaxx_fast = ["esaxx-rs/cpp"]   # <-- the ONLY thing enabling the /MT C++ build

esaxx_fast only accelerates Unigram training. mimir uses tokenizers purely for inference (Tokenizer::from_file + encode), so the C++ path is dead weight for us.

Fix

Disable tokenizers' default features and re-list the other two defaults (onig, progressbar), dropping only esaxx_fast:

tokenizers = { version = "0.23", optional = true, default-features = false, features = ["onig", "progressbar"] }

esaxx-rs stays in the tree (it's non-optional) but now builds as pure Rust — no /MT C++ objects, so nothing clashes with ort's /MD.

• Cross-platform: no [patch]/fork, ort untouched, Linux unaffected.
• No functional change: esaxx_fast is training-only; mimir never trains tokenizers.
• Cargo.lock: esaxx-rs simply loses its cc build-dep edge.

Verification (x86_64-pc-windows-msvc)

• Before: cargo build --features bundled-embeddings → LNK1169: multiply defined symbols (LIBCMT vs MSVCP140).
• After: same command links cleanly and produces a working mimir.exe (Finished in ~12s).

🤖 Generated with Claude Code