Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create parse micro-bench for JsonNode and fix other JsonNode bench namespace #4391

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

andre-ss6
Copy link

@andre-ss6 andre-ss6 commented Aug 12, 2024

Hey, I wanted to compare the current performance difference between JsonNode and JsonDocument for parsing-only. The docs say JsonDocument should be faster, but there doesn't seem to be much, if any at all, data on the subject out there. Given that the JsonNode API is friendlier and easier to work with overall, I wanted to see if the perf difference really justifies the hassle of using JsonDocument.

I came to the benchmarks here, but noticed there isn't one for just parsing with JsonNode. So I just duplicated the DocumentParse benchmark and changed it to test JsonNode instead.

Interestingly, after doing this I noticed some wonky results from JsonDocument on the 400KB payload benchmark, at : it was consistently slower than JsonNode, independently of whether I was testing random access, and when testing random access specifically the StdDev was through the roof, an order of magnitude higher than with JsonNode.

// * Summary *

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22631.3958/23H2/2023Update/SunValley3)
AMD Ryzen 7 5800X3D, 1 CPU, 16 logical and 8 physical cores
.NET SDK 9.0.100-preview.6.24328.19
  [Host]     : .NET 9.0.0 (9.0.24.32707), X64 RyuJIT AVX2
  Job-GKTPJR : .NET 9.0.0 (9.0.24.32707), X64 RyuJIT AVX2

OutlierMode=DontRemove  PowerPlanMode=00000000-0000-0000-0000-000000000000  IterationTime=250ms
MaxIterationCount=20  MemoryRandomization=True  MinIterationCount=15
WarmupCount=1

JsonNode

Method IsDataIndented TestRandomAccess TestCase Mean Error StdDev Median Min Max Gen0 Gen1 Gen2 Allocated
Parse False False HelloWorld 97.02 ns 2.140 ns 2.465 ns 96.25 ns 94.04 ns 103.0 ns 0.0050 - - 264 B
Parse False False BasicJson 359.44 ns 7.137 ns 6.676 ns 358.41 ns 350.69 ns 371.4 ns 0.0114 - - 608 B
Parse False False Json400B 522.44 ns 10.426 ns 11.589 ns 516.76 ns 510.13 ns 550.1 ns 0.0165 - - 864 B
Parse False False Json400KB 365,844.98 ns 2,866.525 ns 2,681.349 ns 365,716.62 ns 363,048.53 ns 373,740.9 ns 164.7059 164.7059 164.7059 587946 B
Parse False True HelloWorld 244.77 ns 5.033 ns 5.796 ns 244.35 ns 236.35 ns 254.0 ns 0.0125 - - 640 B
Parse False True BasicJson 1,235.86 ns 24.582 ns 28.309 ns 1,227.97 ns 1,199.52 ns 1,327.0 ns 0.0490 - - 2592 B
Parse False True Json400B 1,860.95 ns 22.247 ns 20.810 ns 1,859.07 ns 1,835.23 ns 1,900.3 ns 0.0737 - - 3744 B
Parse False True Json400KB 2,537,976.18 ns 19,155.600 ns 17,918.160 ns 2,539,268.75 ns 2,502,348.96 ns 2,563,884.4 ns 208.3333 197.9167 156.2500 3203769 B
Parse True False HelloWorld 100.98 ns 1.587 ns 1.484 ns 100.43 ns 99.22 ns 103.8 ns 0.0049 - - 264 B
Parse True False BasicJson 467.57 ns 7.670 ns 7.174 ns 465.56 ns 459.64 ns 483.1 ns 0.0130 - - 696 B
Parse True False Json400B 698.48 ns 13.009 ns 12.168 ns 700.77 ns 678.41 ns 725.4 ns 0.0167 - - 960 B
Parse True False Json400KB 547,934.78 ns 4,853.342 ns 4,539.819 ns 549,302.85 ns 538,140.13 ns 554,127.9 ns 164.4737 164.4737 164.4737 692451 B
Parse True True HelloWorld 248.84 ns 4.311 ns 4.032 ns 248.87 ns 243.32 ns 258.2 ns 0.0118 - - 640 B
Parse True True BasicJson 1,378.24 ns 20.488 ns 19.164 ns 1,374.21 ns 1,356.89 ns 1,413.0 ns 0.0492 - - 2680 B
Parse True True Json400B 1,965.36 ns 36.440 ns 35.789 ns 1,955.43 ns 1,932.88 ns 2,065.1 ns 0.0707 - - 3840 B
Parse True True Json400KB 3,037,103.64 ns 16,059.519 ns 15,022.084 ns 3,039,665.12 ns 3,001,490.70 ns 3,058,404.7 ns 232.5581 209.3023 186.0465 3306994 B

JsonDocument

Method IsDataIndented TestRandomAccess TestCase Mean Error StdDev Median Min Max Gen0 Allocated
Parse False False HelloWorld 85.81 ns 1.362 ns 1.274 ns 85.73 ns 83.39 ns 88.04 ns 0.0014 72 B
Parse False False BasicJson 336.00 ns 2.942 ns 2.752 ns 335.35 ns 332.69 ns 343.99 ns 0.0013 72 B
Parse False False Json400B 474.93 ns 6.101 ns 5.707 ns 473.52 ns 467.97 ns 488.17 ns - 72 B
Parse False False Json400KB 317,439.44 ns 4,305.240 ns 4,027.124 ns 315,846.98 ns 313,530.15 ns 328,462.69 ns - 72 B
Parse False True HelloWorld 127.96 ns 1.601 ns 1.497 ns 128.55 ns 125.86 ns 130.41 ns 0.0021 120 B
Parse False True BasicJson 749.59 ns 7.256 ns 6.788 ns 747.77 ns 742.22 ns 761.83 ns 0.0060 328 B
Parse False True Json400B 1,353.59 ns 26.735 ns 28.606 ns 1,345.58 ns 1,324.24 ns 1,449.76 ns 0.0107 592 B
Parse False True Json400KB 4,505,171.25 ns 1,167,072.166 ns 1,344,002.042 ns 3,598,700.00 ns 3,402,300.00 ns 6,700,600.00 ns - 611848 B
Parse True False HelloWorld 93.36 ns 1.876 ns 1.755 ns 93.38 ns 89.55 ns 97.05 ns 0.0011 72 B
Parse True False BasicJson 431.18 ns 8.182 ns 8.036 ns 428.31 ns 423.01 ns 452.02 ns - 72 B
Parse True False Json400B 632.65 ns 10.544 ns 9.862 ns 628.20 ns 623.12 ns 655.57 ns - 72 B
Parse True False Json400KB 483,170.78 ns 4,728.549 ns 4,423.087 ns 481,765.38 ns 477,670.38 ns 493,015.96 ns - 72 B
Parse True True HelloWorld 146.76 ns 2.846 ns 2.923 ns 145.43 ns 142.70 ns 155.69 ns 0.0023 120 B
Parse True True BasicJson 875.30 ns 14.357 ns 13.430 ns 872.13 ns 861.78 ns 918.00 ns 0.0035 328 B
Parse True True Json400B 1,542.27 ns 18.541 ns 17.343 ns 1,538.34 ns 1,521.32 ns 1,590.34 ns 0.0062 592 B
Parse True True Json400KB 5,580,596.25 ns 2,032,080.702 ns 2,340,147.159 ns 3,886,662.50 ns 3,644,600.00 ns 8,899,975.00 ns - 611848 B

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant