[Ur] Drop of several orders of magnitude in Techempower benchmarks

[Oisín Mac Fhearaí]

I've noticed that Ur/web's performance benchmarks on Techempower have
changed significantly between round 16 and 17.

For example, in round 16, Urweb measured 323,430 responses per second to
the "Fortunes" benchmark.
In round 17 (and beyond), it achieved 4,024 RPS with MySQL and 2,544 RPS
with Postgres.

What could explain such a drastic drop in performance? The blog entry for
round 17 mentioned query pipelining as an explanation for some of the
frameworks getting much faster, but I don't see why Urweb's RPS would drop
by a factor of 100x, unless perhaps previous rounds had query caching
enabled and then round 17 disabled them.

Can anyone here shed light on this? I built a simplified version of the
"sql" demo with the 2016 tarball version of Ur (used by the round 16
benchmarks) and a recent snapshot, and they both perform at similar speeds
on my laptop.

Oddly, the load testing tool I used (a Go program called "hey") seems to
have one request that takes 5 seconds if I set it to use more concurrent
threads than the number of threads available to the Ur/web program.
Otherwise, the longest request takes about 0.02 seconds. This seems
unrelated to the performance drop on the Techempower benchmarks, since the
max latency is quite low there.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.impredicative.com/pipermail/ur/attachments/20190801/1e5e7318/attachment.html>