Mia Heidenstedt she/her
Developer, Knowledge Management Advocate
My Projects
Releasing: GoQueueBench

Releasing: GoQueueBench

As i coded on OuroborosDB i noticed that i need a very fast queue for a rather unique architectural design decision.
I try to build the network module in such a way that i can test the behavior completely deterministic while “simulating” entire clusters in a single process.

So i build a test prototype of my global network queue with go’s channels and noticed that it was a major performance bottleneck, after writing 2 different ring buffer queue implementations it became clear that some queues behave completely different under different congestion levels and core counts - some so unpredictable that i just did not wanted to use them in my project.

This prompted me to take a relatively large chunk out of my free time and write a suite to benchmark different queue implementations i build under different conditions and score them based on their performance and predictability.

The result of this work is GoQueueBench

These are the results of the benchmark suite:

Implementation Overall Score Throughput Light Load Throughput Heavy Load Throughput Average Stability Ratio Homogeneity Factor Uncertainty Total Tests
VortexQueue 11341466 6926449 5502925 8776309 1.15 0.87 0.25 681
LightningQueue 9631771 6638213 4627690 6036728 0.99 0.95 0.31 681
FastMPMCQueue 9384067 6870924 4598620 6070151 0.96 0.93 0.28 681
OptimizedMPMCQueue 9105262 6436385 4379823 5838555 0.97 0.94 0.32 681
OptimizedMPMCQueueSharded 8130197 6369891 3834140 6781865 0.84 0.88 0.39 681
MultiHeadQueue 7391203 4363332 3492068 5558849 1.12 0.91 0.36 681
BasicMPMCQueue 5599252 4370889 2669612 3667715 0.89 0.93 0.30 681
Golang Buffered Channel 5312485 6667828 2760985 4312720 0.54 0.82 0.66 681
FastMPMCQueueTicket 3229780 7705164 1203924 5803821 0.21 0.64 1.19 681

Please note that i build the package so that all queue adhere to the same interface and can be swapped out easily.

Technology Mia's Projects FLOSS Open Source Golang OuroborosDB Benchmark Queue Performance Optimization
Loading...