[Ur] Ur/Web in production

[Vladimir Shabanov]

2014/1/17 Adam Chlipala <adamc at csail.mit.edu>

> Thanks, Vladimir, for sharing these details with us!  I had suggested a
> while ago that folks here might be interested in learning more about the
> details of the project, and this is just the sort of summary I was hoping
> to see.
>
>
> On 01/16/2014 03:30 PM, Vladimir Shabanov wrote:
>
>> Then I've switched from Postgres to Riak (needed to scale writes since
>> RSS reader is very write heavy thing and to simplify cluster operations)
>> and started to need more data processing and 3rd-party integration in
>> frontend. So my Ur/Web app now links with Haskell and calls its functions
>> via FFI.
>>
>
> One planned Ur/Web feature for the near future (which is a mix between
> research and practicality) is support for automated in-memory caching of
> results computed using the SQL database, which might bring performance to a
> point sufficient for your purposes.  Probably the "automated" part wouldn't
> work completely if the updates take place in Haskell code, though.


My performance problems were with heavy writing not reading. RSS reader
constantly updates feeds and users constantly mark items as read. So
caching won't help in these cases. Only faster drives or sharding database
to multiple servers would help.

But most web apps has something like 5% of write operations (not my
30-40%). So read caching could help there.


>  Few major problems with Ur/Web:
>>
>> Slow compilation speed.
>>
>
> I'm hoping this will be fixed with a longer-time-horizon project we have
> going, to build better tools for building future Ur/Web versions and more!


Do you have any plans when this project will be available?


>  Exponential code bloat. This is actually the main reason for long
>> compilation. Ur/Web inlines too much.
>>
>
> It occurs to me it that it would be easy enough to add a 'neverInline'
> .urp directive, which would be dual to 'alwaysInline' that's already there.
>  Do you think that would solve the problem easily enough?


I think yes. Although reloadFeed function (you could see commented calls to
it in sources) is local one. So it should be possible to somehow specify
local functions to neverInline pragma.


>  Compiler bugs. As you can expect from a new and complex tool there were
>> several bugs in compiler and runtime library. Most of them are in the past
>> now. However I'm still not sure about one nasty bug when some effectful
>> computation were optimized out (click on button and part of event handler
>> is not executed at all, whoops). It's the most annoying bug I've met in
>> Ur/Web. Usually solved by some code reordering or by calling it via
>> JavaScript FFI function (look for 'forceImpure' in the sources). Hope it's
>> fixed since I haven't seen it for a while.
>>
>
> I believe we left that issue at the point where I was hoping for some
> example code that demonstrates the problem.  Now that the full code is
> freely available, I could poke at it directly if the issue reoccurs.


It was hard to make little example. Next time I'll see it (hope I will
never see it ;) I'll update sources and point to the code that is optimized
out.


>  Error messages can be huge. Full types of big records when you've just
>> mistyped field name. AST of a big chunk of code with all type annotations
>> when you have small type error in the middle and so on. It usually helps to
>> add some type annotations when you see that the error "is somewhere here".
>> But I would like to see more readable error messages. Haskell had the same
>> problem and solved it. Hope Ur/Web will solve it too.
>>
>
> We're hoping to solve this in a generic way using the
> tool-for-building-compilers that's under development.


Probably it could be solved by a few modifications. When two record types
mismatch it would be nice to show only mismatched fields first instead of
the full type.

As for pretty-printing code, probably it'll be easier in the new tool.


>  Weak pattern matching. There are no named patterns, no pattern guards, no
>> pattern matching in "do"-notation (impossible to write (a,b,c) <-
>> someCode), no view patterns.
>>
>
> I'd love to accept a patch from someone to add such things.  I haven't
> felt enough personal motivation to go there yet. ;)





>  FFI requires too much friction. It would be great to see Fay-like FFI. Or
>> at least have a possibility to define foreign functions inside .ur-file
>> (where I can put any local datatype as an argument) instead of interface
>> file.
>>
>
> I think an FFI here is harder than for Haskell, which has more uniform
> run-time representations, plus the attendant performance cost.


I'm mostly speaking about JavaScript FFI where you need to put one line in
.urs and another one or two in .urp. And if you need to pass some type
defined in other module (not in .urs) you need to make polymorphic
functions (and make curried functions in JavaScript).
It was discussed here:
http://www.impredicative.com/mantis/view.php?id=157

Server-side FFI is very easy for me. Look at the bottom of
https://github.com/bazqux/bazqux-urweb/blob/master/crawler/Gen.hs
one line and I'm calling Haskell function with any argument/result type I
need.

But JavaScript FFI is more complex at the moment.

 Ur/Web speed? Wasn't an issue at all. Some early load testing have shown
>> 2K requests/sec for a dynamic page that makes a few requests to database.
>> That's about 100 times more than Hacker News have so it's more than enough.
>> There were performance issues with database and feeds fetcher but not with
>> Ur/Web itself.
>>
>
> With localhost-to-localhost testing on my 8-core Linux workstation, I can
> easily get 600k requests/sec for simple dynamic pages without database
> access.  Adding simple SQL queries brings the rate down into tens of
> thousands instead of hundreds of thousands.  When the next round of the
> TechEmpower framework benchmarks comes out, I'm hoping it will show these
> sorts of numbers in context with competitors.


That's great. It could help to get more attention to Ur/Web. Although as
I've said even 1K requests/sec is more than adequate for most sites. It's
1krps = 80M page-views per day. With such scale you probably can afford to
just by one more server to run Ur/Web ;)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.impredicative.com/pipermail/ur/attachments/20140118/27aa9eee/attachment.html>