[Ur] Ur/Web in production

[Adam Chlipala]

Thanks, Vladimir, for sharing these details with us!  I had suggested a 
while ago that folks here might be interested in learning more about the 
details of the project, and this is just the sort of summary I was 
hoping to see.

On 01/16/2014 03:30 PM, Vladimir Shabanov wrote:
> Then I've switched from Postgres to Riak (needed to scale writes since 
> RSS reader is very write heavy thing and to simplify cluster 
> operations) and started to need more data processing and 3rd-party 
> integration in frontend. So my Ur/Web app now links with Haskell and 
> calls its functions via FFI.

One planned Ur/Web feature for the near future (which is a mix between 
research and practicality) is support for automated in-memory caching of 
results computed using the SQL database, which might bring performance 
to a point sufficient for your purposes.  Probably the "automated" part 
wouldn't work completely if the updates take place in Haskell code, though.

> Few major problems with Ur/Web:
>
> Slow compilation speed.

I'm hoping this will be fixed with a longer-time-horizon project we have 
going, to build better tools for building future Ur/Web versions and more!

> Exponential code bloat. This is actually the main reason for long 
> compilation. Ur/Web inlines too much.

It occurs to me it that it would be easy enough to add a 'neverInline' 
.urp directive, which would be dual to 'alwaysInline' that's already 
there.  Do you think that would solve the problem easily enough?

> Compiler bugs. As you can expect from a new and complex tool there 
> were several bugs in compiler and runtime library. Most of them are in 
> the past now. However I'm still not sure about one nasty bug when some 
> effectful computation were optimized out (click on button and part of 
> event handler is not executed at all, whoops). It's the most annoying 
> bug I've met in Ur/Web. Usually solved by some code reordering or by 
> calling it via JavaScript FFI function (look for 'forceImpure' in the 
> sources). Hope it's fixed since I haven't seen it for a while.

I believe we left that issue at the point where I was hoping for some 
example code that demonstrates the problem.  Now that the full code is 
freely available, I could poke at it directly if the issue reoccurs.

> Error messages can be huge. Full types of big records when you've just 
> mistyped field name. AST of a big chunk of code with all type 
> annotations when you have small type error in the middle and so on. It 
> usually helps to add some type annotations when you see that the error 
> "is somewhere here". But I would like to see more readable error 
> messages. Haskell had the same problem and solved it. Hope Ur/Web will 
> solve it too.

We're hoping to solve this in a generic way using the 
tool-for-building-compilers that's under development.

> Weak pattern matching. There are no named patterns, no pattern guards, 
> no pattern matching in "do"-notation (impossible to write (a,b,c) <- 
> someCode), no view patterns.

I'd love to accept a patch from someone to add such things.  I haven't 
felt enough personal motivation to go there yet. ;)

> FFI requires too much friction. It would be great to see Fay-like FFI. 
> Or at least have a possibility to define foreign functions inside 
> .ur-file (where I can put any local datatype as an argument) instead 
> of interface file.

I think an FFI here is harder than for Haskell, which has more uniform 
run-time representations, plus the attendant performance cost.

> Ur/Web speed? Wasn't an issue at all. Some early load testing have 
> shown 2K requests/sec for a dynamic page that makes a few requests to 
> database. That's about 100 times more than Hacker News have so it's 
> more than enough. There were performance issues with database and 
> feeds fetcher but not with Ur/Web itself.

With localhost-to-localhost testing on my 8-core Linux workstation, I 
can easily get 600k requests/sec for simple dynamic pages without 
database access.  Adding simple SQL queries brings the rate down into 
tens of thousands instead of hundreds of thousands.  When the next round 
of the TechEmpower framework benchmarks comes out, I'm hoping it will 
show these sorts of numbers in context with competitors.