[Ur] TechEmpower benchmarks Round 9

[Adam Chlipala]

They're out, and they include a serious Ur/Web configuration error for 
one of the platforms (and I hope this error is fixed in amended 
results), but there is already some good info here to support claims of 
high performance for Ur/Web.  Here's the text I added to the FAQ:


    Can you be more specific about run-time performance?

The TechEmpower Web Framework Benchmarks 
<http://www.techempower.com/benchmarks/> provide a performance 
comparison managed by a third party. Ur/Web does pretty well, and you 
can check that site for details, but a few caveats are important. First, 
the Ur/Web programming model is unusually oriented toward security and 
concurrency simplicity. For instance, Ur/Web's standard random number 
generation function generates cryptographically secure numbers, which 
imposes an extra run-time cost in the several benchmarks based on random 
numbers; and Ur/Web's concurrency model allows the programmer to think 
of every piece of code as running inside a transaction, which imposes 
extra run-time cost in the several benchmarks that use databases, 
necessarily within a transaction per request, unlike in almost all other 
frameworks' entries in the benchmarks. Also, by the way, the current i7 
numbers were captured under a serious configuration error and should be 
ignored.

OK, having said all that, Ur/Web is still doing pretty darn well! 
Consider the results for the highest-capacity machine in the benchmarks, 
which provides 48 hardware threads. In rough numbers, here's how Ur/Web 
is doing.

Test 	Requests/sec. 	Latency
Hello world in JSON 	400k 	0.6 ms
1 SQL query 	100k 	2 ms
20 SQL queries 	10k 	24 ms
20 SQL query/update pairs 	500 	2 s

Yeah, Ur/Web is really falling over in the many-updates test, with 
optimistic SQL concurrency thrashing to provide the transactional 
semantics that most benchmark entrants don't bother to shoot for. Still, 
very few web sites process as much as hundreds of requests per second!

Despite Ur/Web's performance handicaps in service of a pleasant 
programming model, on the Fortunes test, which is closest to the 
scenario Ur/Web was designed for, Ur/Web has the best latency and 
4th-best throughput, out of about 50 frameworks passing that test's 
basic sanity check. In the spirit of sibling rivalry, I'll also point 
out that the Haskell frameworks consistently achieve less than half the 
throughput of Ur/Web.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.impredicative.com/pipermail/ur/attachments/20140502/5b81c539/attachment.html>