Skip to content

Change the 'run rule' functions to special case empty arrays, in orde…#851

Draft
Numpsy wants to merge 1 commit intofsprojects:masterfrom
Numpsy:reduce_empty_arrays
Draft

Change the 'run rule' functions to special case empty arrays, in orde…#851
Numpsy wants to merge 1 commit intofsprojects:masterfrom
Numpsy:reduce_empty_arrays

Conversation

@Numpsy
Copy link
Copy Markdown
Contributor

@Numpsy Numpsy commented Apr 26, 2026

…r to reduce the number of allocated zero length arrays.

As it stands, running the rules allocates a massive number of zero length arrays, which has quite a large effect on the amount of memory allocated. Many of these are from FSharp.Core functions which always create a new array, but which can be avoided with small local wrappers.


Just trying to get back to looking at some perf tests I started some time back, and found this one.

Running the Visual Studio memory profiler on the functions in the benchmarks project shows the allocations of massive numbers of zero length arrays (over 4 million in total). e.g.

image image

If I run the benchmarks project with memory diagnosis enabled, I get this with the current code

| Method         | Mean     | Error   | StdDev  | Gen0       | Gen1      | Gen2      | Allocated |
|--------------- |---------:|--------:|--------:|-----------:|----------:|----------:|----------:|
| LintParsedFile | 858.6 ms | 7.15 ms | 6.34 ms | 25000.0000 | 8000.0000 | 1000.0000 | 397.55 MB |

Down to this with this change

| Method         | Mean     | Error   | StdDev  | Gen0       | Gen1      | Gen2      | Allocated |
|--------------- |---------:|--------:|--------:|-----------:|----------:|----------:|----------:|
| LintParsedFile | 847.8 ms | 9.65 ms | 8.55 ms | 20000.0000 | 7000.0000 | 1000.0000 | 322.41 MB |

If I enable all of the extra checks that are enabled in the 'run self checks' leg of the CI then the differences are even larger.
Before

| Method         | Mean    | Error    | StdDev   | Gen0       | Gen1       | Gen2      | Allocated |
|--------------- |--------:|---------:|---------:|-----------:|-----------:|----------:|----------:|
| LintParsedFile | 1.402 s | 0.0164 s | 0.0146 s | 43000.0000 | 14000.0000 | 3000.0000 | 726.16 MB |

After

| Method         | Mean    | Error    | StdDev   | Gen0       | Gen1       | Gen2      | Allocated |
|--------------- |--------:|---------:|---------:|-----------:|-----------:|----------:|----------:|
| LintParsedFile | 1.380 s | 0.0141 s | 0.0125 s | 36000.0000 | 11000.0000 | 3000.0000 | 622.94 MB |

These allocations are caused by FSHarp.Core liking to always create new arrays in various Array functions, but to me it looks like it's a big enough gain to special case it locally?...

Note: There are other places that could maybe be changed as well, and a similar situation with Array.collect, but I left it just doing one thing to get comments.

@Numpsy Numpsy changed the title Change the 'run ruie' functions to special case empty arrays, in orde… Change the 'run rule' functions to special case empty arrays, in orde… Apr 26, 2026
…r to reduce the number of allocated zero length arrays.

As it stands, running the rules allocates a massive number of zero length arrays, which has quite a large effect on the amount of memory allocated.
Many of these are from FSharp.Core functions which always create a new array, but which can be avoided with small local wrappers.
@Numpsy Numpsy force-pushed the reduce_empty_arrays branch from 749e33e to 093f1c2 Compare April 26, 2026 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant