[Ur] interfacing with C or other languages - example system

[Adam Chlipala]

Marc Weber wrote:
> Excerpts from Adam Chlipala's message of Sun Nov 28 21:58:18 +0100 2010:
>    
>> That's right.  The runtime system will also restart if an optimistic SQL
>> concurrency violation is detected, when certain errors are signaled, etc..
>>      
> Don't know yet whether that's smart behaviour.
>    

It's hard to do otherwise for SQL concurrency violations without adding 
performance hits to just about everything.  (Or requiring manual error 
handling at every SQL update)

> How exactly is determined how much heap will be allocated for a request?
>    

I haven't spent much time on the CGI protocol, since someone who cared 
about performance wouldn't use it.  There can be a ridiculous amount of 
restarting with CGI.

HTTP and FastCGI use pools of worker threads that maintain persistent 
per-thread buffers.  The buffers never shrink, so the number of heap 
resizings over the life of a thread is bounded by the logarithm of the 
maximum heap size (since I always at least double the buffer size if 
possible).  In other words, a long-running server will have a negligible 
amount of restarting due to this issue.

> There are some cases where you don't want to retry;
> Some payment systems redirect customers using a "return" url
> which tells the web system that the customer has entered all the data.
>
> Then the web system is supposed to verify and confirm the data - because
> the return url parameters can be manipulated by the user.
>
> However you can't run that confirmation query - which is typically a
> http request - twice.
>
> The second time they'll return an error telling you "that transaction id
> has been used previously".
>
> Worse: Because urweb rolls back everything db related you don't even
> have traces about it. So the custmer has to pay - and you don't have an
> order.
>    

That's an interesting case that I hadn't thought about.

> I can see 3 solutions:
> a) write a layer between urweb and the payment service which is caching
>    the confirmation results. Bad: You can't use urweb for writing it.
>    You could do this caching in C as well - but I'd like to use a higher
>    order language to get my jobs done..
>    

This seems like the best option to me.  I can't see any need to write 
this sort of code very often at all, so it seems like a mistake to use 
it to guide the whole language implementation.  There might be a role 
for another domain-specific language for handling this kind of coding.

> b) tell urweb to use a huge heap size so that its very unlikely that the
>     issue appears - is there a project configuration option specifying a
>     min heap size?
>    

No minimum heap setting yet.  This also wouldn't get around SQL 
concurrency failures.

> c) never fail for heap reasons. Use a linked list of heap blocks.
>     If one is full allocate another one which is large enough and
>     continue.
>    

This would be extra complexity and a new performance hit with no 
observable benefit for almost all cases, so I'm inclined against it.