[libxml-devel] memory consumption when finding inside of large document never goes away

[Charlie Savage]

Hi Matt,

> I am making the parsed ruby objects available to a Rails application and 
> I find that if I call GC.start when using the library with Rails that it 
> takes several seconds to garbage collect and sometimes crashes.  If I 
> call GC.start in the loop when the program is running as a standalone 
> process then GC.start returns in a few dozen milliseconds.

What platform are you using?  Can you run a debug version and get a 
stack trace so we can see what is going on?  Are you using XPath?  If 
so, make sure to free pointers to your XPath result objects and call 
GC.start before the associated documents get freed (see the rdocs for 
more info, document#find I think it is).

> I wrote a SAX style parser using libxml-ruby that does not suffer from 
> the memory growth but it is about 30 times slower than the document 
> based parser so I am really trying to make the document based approach work.

Why do you suppose SAX is so much slower.  It should be a lot faster 
since it doesn't build an in-memory tree.

Any chance the XMLReader would work for you?

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://rubyforge.org/pipermail/libxml-devel/attachments/20080816/124c8c3b/attachment.bin>