[Ur] TechEmpower Benchmarks

[Adam Chlipala]

I've pushed a changeset to avoid all database operations for page 
handlers that don't need the database.  It does seem to have the 
performance effect I expected running the benchmark program.

However, now I see I was looking at the wrong part of the results table 
before, when I said I was seeing peak Ur/Web HTTP performance eerily 
close to the best reported for TechEmpower.  I was looking at the 
results for JSON serialization, when I meant to be looking at the 
results for "hello world," which are substantially zippier. ;)  So, now 
I don't want to guess that there aren't further opportunities for 
constant-factor improvement, though they probably wouldn't account for 
much more than a factor of 5X at this point, if my tests on my 
workstation are triggering sufficiently similar performance to the 
official benchmark runs.  This is good progress from a 100X performance 
gap. :)

Does anyone know how to replicate the official benchmark execution 
sufficiently well to do fine performance measurement?  This sounds 
easier for the EC2 runs than for the "hardware we have in our offices" runs.

I'm curious how involving Nginx could improve performance, talking to 
Ur/Web programs speaking either HTTP or FastCGI.  It would also be nice 
to see what is the optimal number of OS threads in the Ur/Web process in 
the real environment.  I wonder if the lame lock-based queue data 
structure in the Ur/Web HTTP code might even be a bottleneck, such that 
corresponding code in Nginx could do a much better job.

On 12/11/2013 11:16 AM, Adam Chlipala wrote:
> On 12/11/2013 10:47 AM, escalier at riseup.net wrote:
>> The final preview results have been posted:
>>
>> http://www.techempower.com/benchmarks/previews/round8/
>
>>    - Create an alternate configuration that places Ur/Web behind a 
>> frontend
>> like Nginx
>
> I'm now thinking this isn't necessary and probably would reduce 
> performance, but I'd still be interested to see the results of an 
> experiment.
>
> I believe I've pinpointed the reason for Ur/Web's poor showing on the 
> no-database benchmarks: we build a single executable, which includes 
> page handlers that do and do not touch the database.  However, the 
> runtime system is creating a new database transaction for every 
> request, even those that don't need the database!  Running a simple 
> application by itself brings Ur/Web eerily close to the performance 
> showing up in the benchmark results (within 1,000 requests per 
> second!), which makes me optimistic that this is actually the 
> "theoretical maximum" given the hardware, OS, etc.
>
> I'm going to work on not creating transactions for requests that don't 
> need them.