Skip to content

Reduce allocation overhead in file type detection#9

Closed
Copilot wants to merge 2 commits into
masterfrom
copilot/optimize-memory-and-heap-allocation
Closed

Reduce allocation overhead in file type detection#9
Copilot wants to merge 2 commits into
masterfrom
copilot/optimize-memory-and-heap-allocation

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 28, 2026

File type detection was allocating more than necessary on hot paths, especially for stream inputs and text/UTF-8 validation. The result was avoidable heap pressure during repeated inspections.

  • Stream buffering

    • Rent buffers from ArrayPool<byte> instead of allocating a full byte array per stream detect call.
    • Preserve seek position while reading and return pooled buffers promptly.
  • Matching path

    • Remove the temporary List<int> used for subheader scans.
    • Scan subheader candidates in-place against the current span.
  • Text detection

    • Cache UTF-8 encodings and BOM bytes once.
    • Remove per-call encoding construction and LINQ-based BOM checks.
    • Keep validation span-based to avoid extra intermediate allocations.
ReadOnlySpan<byte> input = fileContent.AsSpan(0, length);
if (IsMatchingType(input, fileTypeInfo))
    return fileTypeInfo;

@ghost1face
Copy link
Copy Markdown
Owner

Thanks for the feedback, will pull into #10 and add benchmarks and minor changes

@ghost1face ghost1face closed this May 31, 2026
@ghost1face ghost1face deleted the copilot/optimize-memory-and-heap-allocation branch May 31, 2026 22:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants