[libxml-devel] Maintainer

[Ross Bamford]

On Sun, 22 Jul 2007 22:06:32 +0100, Charlie Savage <cfis at savagexi.com>  
wrote:

>> there are still a number
>> of fundamental problems with libxml-ruby - mostly to do with the fact  
>> that
>> libxml and ruby are constantly fighting over memory management and the
>> like.
>
> Ross - Could you summarize these issues, and exactly what is going wrong
>   where?  It would be extremely helpful as a guide to anyone who has
> time to dig into the issue.
>
> I did a lot of work on SWIG's ruby support, so have a good feeling for
> the various memory issues that can arise.  But I know nothing about
> libxml's api or memory management, or how the ruby bindings interact
> with it.
>

Basically, the problem is with the way we're handling libxml's structs.  
The best example is with the nodes: When freeing a node (or a document)  
libxml can free all the children, or not. With the first option we get  
problems when Ruby-side XML::Nodes have been created, since they free  
their node when garbage collected. With the second option, we'd have to  
create Node objects for every node, or otherwise ensure they get freed at  
the appropriate time - which doesn't work too well with the speed and  
efficiency thing (esp. with large documents).

Currently this is worked around with a sort of pointer-node system - when  
GCing a node, the libxml struct is only freed if it's a pointer node. IIRC  
new nodes are created as pointers, and only made into full nodes when  
they're copied outside of the document, or otherwise made independent of  
their containing document.

This kinda works (with nodes, at least) but is a bit messy. It gets a bit  
inconsistent when it comes to, for example, attributes though - there are  
still certain cases that cause things to get freed twice, or not at all.  
This is fixable, but difficult owing to the way the code's evolved - a  
case of having the tools, and the time, I guess. There's a fair few  
comments dotted around the code relating to these issues, though...

As an aside, would SWIG make any of this easier? I remember reading a  
comment (from Sam Ruby I think) that we should junk the current code and  
rewrite with SWIG. I disagreed at the time, but maybe he had a point?

> Also, does the patch from open street map fix the issue?

I honestly don't know.

>
>> The Windows issue matters, too, but since I've been lucky enough to  
>> avoid
>> Windows almost entirely in my professional life for several years now I
>> don't really know the platform well enough to handle this, and frankly  
>> I'm
>> unwilling to take the time to learn for the sake of compatibility on one
>> project. Another thing that needs to be done.
>
> I can solve the windows issues.  I have a patch to fix up memory
> allocation on windows, and with my experience with ruby-prof, I can
> easily create a win32 gem.
>

Cool - that would definitely be a big step forward for the project.

To be honest, this was one of the major things that pissed me off before -  
a lot of people were bemoaning our lack of Win32 gems, but no-one wanted  
to help with it. I ended up spending far more time that I wanted to  
messing about with a QEMU windows installation, and in the end admitted  
defeat since every way I turned seemed to require spending (more) money on  
Microsoftware I personally don't need.

Cheers,
Ross

-- 
Ross Bamford - rosco at roscopeco.co.uk