BenchmarkDotNet
Some work projects use BenchmarkDotNet as the .NET library for benchmarking.
Getting familiar with it should pay dividends.
To run the benchmarks in the Day13ClawContraption class:
dotnet run -c Release -- -f '*Day13ClawContraption*'
A job describes how to run your benchmark, e.g, ID, environment, run settings.
BenchmarkDotNet has a smart algorithm for choosing values like
IterationCount, so you typically don’t need to specify those. Sample measurements for the default job:
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Foo | 3.845 s | 0.0747 s | 0.0800 s |
Memory Diagnoser
The Common Language Runtime continually balances two priorities when it comes to garbage collection (GC):
- Not letting an application’s working set get too large by delaying GC
- Not letting the GC run too frequently (during GC, all other managed threads are suspended)
The managed heap is divided into 3 generations, 0, 1, and 2, so it can handle long-lived and short-lived objects separately:
- Generation 0: The youngest and contains short-lived objects. New objects are stored here, unless they’re large in which case they go to the large object heap (LOH / generation 3).
- Generation 1: After GC collects
Gen 0, it compacts the memory for the reachable objects and promotes them toGen 1. Objects that survive collections tend to have longer lifetimes, and so the promotion makes sense. If a GC ofGen 0doesn’t reclaim enough memory, then the GC can collectGen 1and if need be,Gen 2, but in most cases,Gen 0collection is sufficient. Objects inGen 1that survive GC are promoted toGen 2. - Generation 2: Contains long-lived objects, e.g., static data in a
server application. Objects in
Gen 2that survive GC remain inGen 2. Objects on the large object heap are also collected inGen 2.
MemoryDiagnoser allows measuring the number of allocated bytes and garbage
collection frequency, e.g.,
| Method | Gen0 | Gen1 | Gen2 | Allocated |
|---|---|---|---|---|
| Foo | 596 | 193 | 48 | 3.49 MB |
- Allocated contains the size of allocated managed memory (not
stackallocor native heap allocations). It’s per single invocation, inclusive. - Gen X contains the number of
Gen Xcollections scaled to per 1,000 operations, e.g., GC collects memory 596 times per 1,000 benchmark invocations in generation 0. -and0are synonymous, e.g.,-inGen Xmeans no garbage collection was performed for generationX.
Perf Profiles
BenchmarkDotNet.Diagnostics.Windows allows collecting ETL traces that show
where most of the time is spent. However, this package is only available on
Windows because it internally uses Event Tracing for Windows (ETW) to capture
stack traces and important .NET Runtime events.
BenchmarkDotNet.Diagnostics.dotTrace can also capture traces using the
dotTrace command-line profiler. However, dotTrace is not available for free;
JetBrains charges for it.
Trying to use dotnet-trace as it’s free. For some reason,
dotnet-trace collect --format speedscope -- dotnet run -c Release
… doesn’t give me much. -- <command> has a disclaimer for commands that
launch multiple apps, but I don’t think it applies here. Running
dotnet run -c Release -- -f '*Day13ClawContraption*' in one terminal and then:
dotnet-trace collect --name "AoC2024.Benchmarks" --format speedscope --output advent-of-code/2024/AoC2024.Benchmarks/BenchmarkDotNet.Artifacts/perf-profile.nettrace
… in another seems like a step forward because dotnet-trace exits after the
benchmark exits.
Still no dice though; seeing a lot UNMANAGED_CODE_TIME without any actionable
insights. Creating a separate program that benchmarks the code directly instead
of going through BenchmarkDotNet works! In hindsight, I think I was profiling
BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args) instead of
my user code. One more .csproj it is!
References
- Home | BenchmarkDotNet. benchmarkdotnet.org . Accessed Jan 2, 2026.
- Jobs | BenchmarkDotNet. benchmarkdotnet.org . Accessed Jan 2, 2026.
- The new MemoryDiagnoser is now better than ever! – Adam Sitnik – .NET Performance and Reliability. Adam Sitnik. adamsitnik.com . Accessed Jan 2, 2026.
- Fundamentals of garbage collection - .NET | Microsoft Learn. learn.microsoft.com . Accessed Jan 2, 2026.
- Profiling .NET Code with BenchmarkDotNet – Adam Sitnik – .NET Performance and Reliability. Adam Sitnik. adamsitnik.com . Accessed Jan 2, 2026.
- NuGet Gallery | BenchmarkDotNet.Diagnostics.dotTrace 0.15.8. www.nuget.org . benchmarkdotnet.org . Accessed Jan 2, 2026.
- dotnet-trace diagnostic tool - .NET CLI - .NET | Microsoft Learn. learn.microsoft.com . Accessed Jan 2, 2026.