From transfire at gmail.com  Sun Aug  5 09:08:51 2007
From: transfire at gmail.com (TRANS)
Date: Sun, 5 Aug 2007 06:08:51 -0700
Subject: [libxml-devel] A New LibXml
Message-ID: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>

Hi All,

I'm considering forking the libxml project. Once again I can't get a
hold of Sean Chittenden. So we can't get any new developers on the
project with commit rights. No one, as far as I know, is actively
patching. And no one is at the helm.

I am not a C coder, so really I'm not the best man for the job. But I
am willing to coordinate and do everything I am able for the project.
I really need a good, fast xml library for Ruby, and AFAIK bindings to
libxml are the best choice. Unfortunately there are some serious
issues with the current bindings --memory leaks and speed issues (I've
seen libxml run slower than rexml, which really threw me for a loop).
So I want to do something...

So if you all agree, that a fork is a good idea, the question arises:
would we be better off starting from scratch? (We can use some of the
old code piecemeal as proves useful, of course.)  If so, who wants to
take up that challenge?

T.

From cfis at savagexi.com  Sun Aug  5 17:33:17 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Sun, 05 Aug 2007 15:33:17 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
Message-ID: <46B6421D.4050006@savagexi.com>



TRANS wrote:
> So if you all agree, that a fork is a good idea, the question arises:
> would we be better off starting from scratch? (We can use some of the
> old code piecemeal as proves useful, of course.)  If so, who wants to
> take up that challenge?

Sorry, but I strongly disagree with a fork and starting over.  I think a 
much better approach is to create test cases that show the memory leaks, 
and then fix them.  Otherwise, I fear you'll just spend a lot of time 
duplicating work that has been done and end up exactly at the same 
place.  My quick glance showed nothing particularly wrong with the 
style/code of the bindings.  Instead, its some misusue of the libxml 
bindings or returning multiple ruby objects that point to the same C 
object (which is very difficult thing to avoid in some cases).

So before abandoning what's there, let's first fix it.  If we then prove 
to ourselves that is not fixable, then start over.  But I seriously 
doubt that would be the case.

Thus the first step - let's see some test cases checked in that show the 
leaks.  Then we can verify whatever fixes we come up with.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070805/238fc75a/attachment-0001.bin 

From has.sox at gmail.com  Sun Aug  5 18:57:35 2007
From: has.sox at gmail.com (Daniel N)
Date: Mon, 6 Aug 2007 08:57:35 +1000
Subject: [libxml-devel] A New LibXml
In-Reply-To: <46B6421D.4050006@savagexi.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
Message-ID: <2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>

On 8/6/07, Charlie Savage <cfis at savagexi.com> wrote:
>
>
>
> TRANS wrote:
> > So if you all agree, that a fork is a good idea, the question arises:
> > would we be better off starting from scratch? (We can use some of the
> > old code piecemeal as proves useful, of course.)  If so, who wants to
> > take up that challenge?
>
> Sorry, but I strongly disagree with a fork and starting over.  I think a
> much better approach is to create test cases that show the memory leaks,
> and then fix them.  Otherwise, I fear you'll just spend a lot of time
> duplicating work that has been done and end up exactly at the same
> place.  My quick glance showed nothing particularly wrong with the
> style/code of the bindings.  Instead, its some misusue of the libxml
> bindings or returning multiple ruby objects that point to the same C
> object (which is very difficult thing to avoid in some cases).
>
> So before abandoning what's there, let's first fix it.  If we then prove
> to ourselves that is not fixable, then start over.  But I seriously
> doubt that would be the case.
>
> Thus the first step - let's see some test cases checked in that show the
> leaks.  Then we can verify whatever fixes we come up with.
>
> Charlie


Thats the whole reason for a fork...  No one can check in...

I think it's a good idea.  I've been wanting to use libxml-ruby for a while
for speed, but the memory leaks when modifying XML read from a source was
unacceptable.

Unfortunately I don't know C, so I can't assist in that area.  I still think
it's a good idea though.

Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/libxml-devel/attachments/20070806/6ab12afe/attachment.html 

From cfis at savagexi.com  Sun Aug  5 21:19:55 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Sun, 05 Aug 2007 19:19:55 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
Message-ID: <46B6773B.4000704@savagexi.com>

> Thats the whole reason for a fork...  No one can check in...

Can't a ruby-forge administrator fix that?  Imagine the confusion having 
two libxml ruby bindings.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070805/e9b084f4/attachment.bin 

From ast at atownley.org  Sun Aug  5 20:45:19 2007
From: ast at atownley.org (Andrew S. Townley)
Date: Mon, 06 Aug 2007 01:45:19 +0100
Subject: [libxml-devel] A New LibXml
In-Reply-To: <2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
Message-ID: <1186361119.5083.27.camel@macross>

On Sun, 2007-08-05 at 23:57, Daniel N wrote:
> 
> On 8/6/07, Charlie Savage <cfis at savagexi.com> wrote:
>         
>         TRANS wrote:

> Thats the whole reason for a fork...  No one can check in... 

How about using something like bazaar (http://www.bazaar-vcs.org) to
start a branch off the anonymous CVS/SVN head and then whoever has
patches can put them on a website (hosting the whole branch).  If Trans
wants to coordinate pulling in the patches from the other branches and
hosting them, then maybe that'd work.

Once someone gets check-in privs for the project or the original
maintainer is back on the scene, a diff can easily be generated from the
bzr or whatever distributed VCS branch(es) as necessary and pulled in. 
It's a bit of a different development model, but I think it would
certainly work well in this case.  I've been using bazaar lightly on
some projects for the last year or so, and found it quite good.  There
are other alternatives, but I haven't used them myself.

I have to agree with Charlie.  I think forking and starting from scratch
is a bad idea if the code's relatively clean--learn from the
Netscape/Mozilla thing rather than make the same mistake (see Joel on
Software if necessary).  In the meantime, as long as one or more people
can make their branches available to the rest of the interested
developers, I think the existing problems could be addressed if there
were a few people available that could look at it.

Unfortunately, my C is quite rusty, and I'm not in the position to hack
on the code myself at the moment, but I'm sure there are those on the
list who are.  Once they have a way to track their changes, it seems
like that's all that's standing in the way of moving the project
forward.

ast
-- 
Andrew S. Townley <ast at atownley.org>
http://atownley.org


From cfis at savagexi.com  Sun Aug  5 22:48:29 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Sun, 05 Aug 2007 20:48:29 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <1186361119.5083.27.camel@macross>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>	<46B6421D.4050006@savagexi.com>	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<1186361119.5083.27.camel@macross>
Message-ID: <46B68BFD.10209@savagexi.com>

> How about using something like bazaar (http://www.bazaar-vcs.org) to
> start a branch off the anonymous CVS/SVN head and then whoever has
> patches can put them on a website (hosting the whole branch).  If Trans
> wants to coordinate pulling in the patches from the other branches and
> hosting them, then maybe that'd work.

I think people should use whatever SCM they like, but RubyForge is 
critical for this reason:

gem install libxml

So in my opinion, RubyForge has to be the hub of the project (in 
addition, its where people would expect to find the official source code).

> Unfortunately, my C is quite rusty, and I'm not in the position to hack
> on the code myself at the moment, but I'm sure there are those on the
> list who are.  Once they have a way to track their changes, it seems
> like that's all that's standing in the way of moving the project
> forward.

Yup, a source repository is the main thing.

Failing test cases are also very important...can anyone come up with some?

On the bright side, there is a patch from OpenStreetMap that reportedly 
fixes the memory issues, so this might turn out to be easy.  And even if 
the patch isn't perfect, hopefully it will show the way forward.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070805/7c0a566a/attachment.bin 

From pat.eyler at gmail.com  Mon Aug  6 09:31:41 2007
From: pat.eyler at gmail.com (pat eyler)
Date: Mon, 6 Aug 2007 07:31:41 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <46B6773B.4000704@savagexi.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
Message-ID: <6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>

On 8/5/07, Charlie Savage <cfis at savagexi.com> wrote:
> > Thats the whole reason for a fork...  No one can check in...
>
> Can't a ruby-forge administrator fix that?  Imagine the confusion having
> two libxml ruby bindings.

There are legal reasons that prevent the admins from 'fixing' it.
A fork is probably the only way to move forward.   Whether or not
the fork starts with a new code base or not is a question better
left for the folks with the C/libXML chops that will enable them to
make the decision.


>
> Charlie
>
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel
>
>


-- 
thanks,
-pate
-------------------------
   Duty makes us do things, Love make us do things well.
http://on-ruby.blogspot.com       http://on-erlang.blogspot.com
          http://on-soccer.blogspot.com

From transfire at gmail.com  Mon Aug  6 09:46:37 2007
From: transfire at gmail.com (TRANS)
Date: Mon, 6 Aug 2007 06:46:37 -0700
Subject: [libxml-devel] A New LibXml
In-Reply-To: <1186361119.5083.27.camel@macross>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<1186361119.5083.27.camel@macross>
Message-ID: <4b6f054f0708060646o4a92b313pc45d9271c2eab91@mail.gmail.com>

On 8/5/07, Andrew S. Townley <ast at atownley.org> wrote:
> On Sun, 2007-08-05 at 23:57, Daniel N wrote:
> >
> > On 8/6/07, Charlie Savage <cfis at savagexi.com> wrote:
> >
> >         TRANS wrote:
>
> > Thats the whole reason for a fork...  No one can check in...
>
> How about using something like bazaar (http://www.bazaar-vcs.org) to
> start a branch off the anonymous CVS/SVN head and then whoever has
> patches can put them on a website (hosting the whole branch).  If Trans
> wants to coordinate pulling in the patches from the other branches and
> hosting them, then maybe that'd work.
>
> Once someone gets check-in privs for the project or the original
> maintainer is back on the scene, a diff can easily be generated from the
> bzr or whatever distributed VCS branch(es) as necessary and pulled in.
> It's a bit of a different development model, but I think it would
> certainly work well in this case.  I've been using bazaar lightly on
> some projects for the last year or so, and found it quite good.  There
> are other alternatives, but I haven't used them myself.

it's an interesting idea. i've been feeling an urge to use Git myself.
but i think rubyforge is too important a resource for us to move away
from, and I fear we may never git adim privs to the current project.

T.

From transfire at gmail.com  Mon Aug  6 09:47:59 2007
From: transfire at gmail.com (TRANS)
Date: Mon, 6 Aug 2007 06:47:59 -0700
Subject: [libxml-devel] A New LibXml
In-Reply-To: <6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
Message-ID: <4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>

On 8/6/07, pat eyler <pat.eyler at gmail.com> wrote:
> On 8/5/07, Charlie Savage <cfis at savagexi.com> wrote:
> > > Thats the whole reason for a fork...  No one can check in...
> >
> > Can't a ruby-forge administrator fix that?  Imagine the confusion having
> > two libxml ruby bindings.
>
> There are legal reasons that prevent the admins from 'fixing' it.
> A fork is probably the only way to move forward.   Whether or not
> the fork starts with a new code base or not is a question better
> left for the folks with the C/libXML chops that will enable them to
> make the decision.

I think pat's right. rather than waste any more time, lets just fork
the dang thing. Charlie makes a good argument for sticking to the
current code base though. But I'm concerned about the license. Are we
good there for a fork? If so I will go ahead and do it. I suppose
libxml2 is a fitting name, yea? Or anyone have something better in
mind?

T.

From nmkolev at uni-bonn.de  Mon Aug  6 10:34:02 2007
From: nmkolev at uni-bonn.de (Nickolay Kolev)
Date: Mon, 6 Aug 2007 16:34:02 +0200
Subject: [libxml-devel] (no subject)
Message-ID: <E0A255EB-290E-4EEF-90D3-26837CE96550@uni-bonn.de>

Hi all,

I want to write a preprocessor in Ruby to hande input of the form:

<math xmlns="http://www.w3.org/1998/Math/MathML" id="1.1.5.1.1.3">
     <mtext id="1.1.5.1.1.3">
         <neg id="1.1.5.1.1.3#0"/>
         <exists id="1.1.5.1.1.3#1"/>x x<in id="1.1.5.1.1.3#4"/>
         <emptyset id="1.1.5.1.1.3#5"/>
     </mtext>
</math>

I would like to remove the namespace declaration and convert the  
elements within mtext to the corresponding Unicode characters. As the  
elements are *not* actually MathML, will probably need to construct  
the translation table myself, like so:

tr = {
  "exists" => "2203",
  "in" => "2208",
  ...
}

I think I can use the definitions given here (and change the names  
where necessary): http://www.w3.org/TR/REC-html40/sgml/ 
entities.html#h-24.3.1

Is there a way to read the input file into a DOM tree and modify it  
in place, or do I have to traverse it an build a new tree along the way?

Many thanks in advance for any pointers!

Nickolay

From cfis at savagexi.com  Mon Aug  6 12:07:36 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Mon, 06 Aug 2007 10:07:36 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>	<46B6421D.4050006@savagexi.com>	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
Message-ID: <46B74748.509@savagexi.com>

Hi Pat,

> There are legal reasons that prevent the admins from 'fixing' it.
> A fork is probably the only way to move forward.

Can you explain the legal reasons?  And can they be worked around by 
adding another administrator to the project, say Trans, as opposed to 
changing anyone's existing passwords?

Forking just to get checkin privileges seems like a really, really ugly 
thing to to.  Its just going to confuse everyone - the libxml gem gets 
downloaded a fair bit and how exactly are we going to explain to 
everyone that it is no longer valid?

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070806/b226bd39/attachment.bin 

From cfis at savagexi.com  Mon Aug  6 12:09:00 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Mon, 06 Aug 2007 10:09:00 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>	<46B6421D.4050006@savagexi.com>	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>	<46B6773B.4000704@savagexi.com>	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
Message-ID: <46B7479C.2010304@savagexi.com>

> I think pat's right. rather than waste any more time, lets just fork
> the dang thing. 

I agree its would be good to get going.  But do we really need to fork 
just to get checkin privileges?  Its just such an awful way of trying to 
reestablish checkin privileges.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070806/53617f4e/attachment.bin 

From pat.eyler at gmail.com  Mon Aug  6 12:16:50 2007
From: pat.eyler at gmail.com (pat eyler)
Date: Mon, 6 Aug 2007 10:16:50 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <46B74748.509@savagexi.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<46B74748.509@savagexi.com>
Message-ID: <6fd0654b0708060916n5912c586q346cb957fc2d64d6@mail.gmail.com>

On 8/6/07, Charlie Savage <cfis at savagexi.com> wrote:
> Hi Pat,
>
> > There are legal reasons that prevent the admins from 'fixing' it.
> > A fork is probably the only way to move forward.
>
> Can you explain the legal reasons?  And can they be worked around by
> adding another administrator to the project, say Trans, as opposed to
> changing anyone's existing passwords?


The problem is that the current administrator of the project is the
legal owner of the code under copyright, and adding new
administrators without consent violates his/her rights as the owner.
I've tried to work out an 'advance directive' statement that would
allow project owners/admins to protect against this (and tried
to engage some serious legal folks), but have been unable
to come up with anything satisfactory.  :(

>
> Forking just to get checkin privileges seems like a really, really ugly
> thing to to.  Its just going to confuse everyone - the libxml gem gets
> downloaded a fair bit and how exactly are we going to explain to
> everyone that it is no longer valid?
>

The same way that apache supplanted ncsa httpd.  Through
track record of producing and supporting good stuff.


> Charlie
>
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel
>
>


-- 
thanks,
-pate
-------------------------
   Duty makes us do things, Love make us do things well.
http://on-ruby.blogspot.com       http://on-erlang.blogspot.com
          http://on-soccer.blogspot.com

From cfis at savagexi.com  Mon Aug  6 12:32:02 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Mon, 06 Aug 2007 10:32:02 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <6fd0654b0708060916n5912c586q346cb957fc2d64d6@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>	<46B6421D.4050006@savagexi.com>	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>	<46B6773B.4000704@savagexi.com>	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>	<46B74748.509@savagexi.com>
	<6fd0654b0708060916n5912c586q346cb957fc2d64d6@mail.gmail.com>
Message-ID: <46B74D02.7080604@savagexi.com>

Hi Pat,

> The problem is that the current administrator of the project is the
> legal owner of the code under copyright, and adding new
> administrators without consent violates his/her rights as the owner.
> I've tried to work out an 'advance directive' statement that would
> allow project owners/admins to protect against this (and tried
> to engage some serious legal folks), but have been unable
> to come up with anything satisfactory.  :(

Ah - thanks for the explanation.  Looking at the copyright in license.txt:

  Copyright (c) 2002-2006 Sean Chittenden <sean at chittenden.org> and 
contributors
  Copyright (c) 2001 Wai-Sun "Squidster" Chia <waisun.chia at compaq.com>

Does "and contributors" leave any wiggle room?  Say someone currently on 
the project with non-admin rights?  Or someone who helped write it, like 
Ross?  Also, its an MIT license, does that help?

Just trying to avoid the nuclear option of a fork since in fact we 
aren't forking the code at all (well, at least not yet).

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070806/cd2f496e/attachment-0001.bin 

From bryant.doug at gmail.com  Mon Aug  6 12:32:32 2007
From: bryant.doug at gmail.com (Doug Bryant)
Date: Mon, 6 Aug 2007 12:32:32 -0400
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
Message-ID: <177c686f0708060932h69f70267k393b1c1a02f8020a@mail.gmail.com>

How have y'all tried to get ahold of Sam.

I found samc at rubyforge and freebsd.org email addresses.  Also found his
linkedin page at http://www.linkedin.com/in/seanc

I think a fork would just confuse things and a re-write would just drag this
project out that much longer.  I (and it appears alot of others)
unfortunately have no C skills worth mentioning which would help.

The ruby libxml bindings are critical to my project and I certainly want to
see this library succeed.  If I had the funds to hire someone to work on the
bindings, I would.

Doug

On 8/5/07, TRANS <transfire at gmail.com> wrote:
>
> Hi All,
>
> I'm considering forking the libxml project. Once again I can't get a
> hold of Sean Chittenden. So we can't get any new developers on the
> project with commit rights. No one, as far as I know, is actively
> patching. And no one is at the helm.
>
> I am not a C coder, so really I'm not the best man for the job. But I
> am willing to coordinate and do everything I am able for the project.
> I really need a good, fast xml library for Ruby, and AFAIK bindings to
> libxml are the best choice. Unfortunately there are some serious
> issues with the current bindings --memory leaks and speed issues (I've
> seen libxml run slower than rexml, which really threw me for a loop).
> So I want to do something...
>
> So if you all agree, that a fork is a good idea, the question arises:
> would we be better off starting from scratch? (We can use some of the
> old code piecemeal as proves useful, of course.)  If so, who wants to
> take up that challenge?
>
> T.
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/libxml-devel/attachments/20070806/fe78df9f/attachment.html 

From transfire at gmail.com  Mon Aug  6 12:33:14 2007
From: transfire at gmail.com (TRANS)
Date: Mon, 6 Aug 2007 09:33:14 -0700
Subject: [libxml-devel] A New LibXml
In-Reply-To: <46B7479C.2010304@savagexi.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
Message-ID: <4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>

On 8/6/07, Charlie Savage <cfis at savagexi.com> wrote:
> > I think pat's right. rather than waste any more time, lets just fork
> > the dang thing.
>
> I agree its would be good to get going.  But do we really need to fork
> just to get checkin privileges?  Its just such an awful way of trying to
> reestablish checkin privileges.

Unfortunately, without Sean Chittenden there's no way to get admin
privileges, and no one can get hold of Sean (again). So we're stuck in
the water without a paddle. I don't know what else to do but fork.

Also, I should point out that this has happened twice before. In fact,
the last time I tried to move this project forward Sean appeared out
of NOWHERE to take admin control back. I wasn't happy about it. And
just as I suspected, here we are again w/o Sean. -- I don't want to
ever have to go through this again.

T.

From pat.eyler at gmail.com  Mon Aug  6 12:51:26 2007
From: pat.eyler at gmail.com (pat eyler)
Date: Mon, 6 Aug 2007 10:51:26 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
Message-ID: <6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>

On 8/6/07, TRANS <transfire at gmail.com> wrote:
> On 8/6/07, Charlie Savage <cfis at savagexi.com> wrote:
> > > I think pat's right. rather than waste any more time, lets just fork
> > > the dang thing.
> >
> > I agree its would be good to get going.  But do we really need to fork
> > just to get checkin privileges?  Its just such an awful way of trying to
> > reestablish checkin privileges.
>
> Unfortunately, without Sean Chittenden there's no way to get admin
> privileges, and no one can get hold of Sean (again). So we're stuck in
> the water without a paddle. I don't know what else to do but fork.
>
> Also, I should point out that this has happened twice before. In fact,
> the last time I tried to move this project forward Sean appeared out
> of NOWHERE to take admin control back. I wasn't happy about it. And
> just as I suspected, here we are again w/o Sean. -- I don't want to
> ever have to go through this again.

I think that plenty of goodwill has been shown.  It's time to start a
new project with a couple of admins (both of whom should have a
long history of hanging around).  Whether or not the new project
starts with libXML code is a decision for a different discussion (one
held among the admins and coders).

Trans, go ahead and start up a new project.  Let me know what
I can do to help.



>
> T.
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel
>


-- 
thanks,
-pate
-------------------------
   Duty makes us do things, Love make us do things well.
http://on-ruby.blogspot.com       http://on-erlang.blogspot.com
          http://on-soccer.blogspot.com

From transfire at gmail.com  Mon Aug  6 13:56:12 2007
From: transfire at gmail.com (TRANS)
Date: Mon, 6 Aug 2007 10:56:12 -0700
Subject: [libxml-devel] Fwd: RubyForge Project Approved
In-Reply-To: <20070806175122.3CCFF5240D2A@rubyforge.org>
References: <20070806175122.3CCFF5240D2A@rubyforge.org>
Message-ID: <4b6f054f0708061056r38458343y177ffd7428cd6c29@mail.gmail.com>

---------- Forwarded message ----------
From: noreply at rubyforge.org <noreply at rubyforge.org>
Date: Aug 6, 2007 10:51 AM
Subject: RubyForge Project Approved
To: transfire at gmail.com


Your project registration for RubyForge has been approved!

Project Full Name:  libxml2
Project Unix Name:  libxml2
Virtual host:       libxml2.rubyforge.org

If after an hour your CVS/Subversion account doesn't work, please
open a support ticket
(http://rubyforge.org/tracker/?atid=102&group_id=5&func=browse)
so that we can look at the problem.  Please note that all shell accounts
are restricted to CVS/Subversion/SCP/SFTP.

There's a bunch of good information on configuring mailing lists, twiddling
your project settings to save bandwidth, setting up commit emails for CVS and
Subversion, and much more in the project admin guide here:

http://rubyforge.org/docman/?group_id=5

Enjoy, and please tell others about RubyForge. Let us know
if there is anything we can do to help you.  Welcome aboard!

-- The RubyForge admin team (Tom Copeland, Rich Kilmer)


-- 
O  trans
^^ transfire at gmail.com

If there's one thing I learned from watching sitcoms it's this:
whenever someone abruptly says "don't be silly", be silly! It could
cost you your life.

From cfis at savagexi.com  Mon Aug  6 14:05:01 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Mon, 06 Aug 2007 12:05:01 -0600
Subject: [libxml-devel] Fwd: RubyForge Project Approved
In-Reply-To: <4b6f054f0708061056r38458343y177ffd7428cd6c29@mail.gmail.com>
References: <20070806175122.3CCFF5240D2A@rubyforge.org>
	<4b6f054f0708061056r38458343y177ffd7428cd6c29@mail.gmail.com>
Message-ID: <46B762CD.6050406@savagexi.com>

Ok, some questions:

1.  Are we going to change to a new mailing list then?  If so, maybe you 
can send an email to this list Trans?

2.  Trans, are you just going to check in the libxml existing code to 
libxml2?  That would be my preference.

3.  If you add me in as a devel, I'll fix the obvious windows issues and 
package up a windows gem.

4.  We need tests cases that show the memory failures...anyone have some?


Charlie

TRANS wrote:
> ---------- Forwarded message ----------
> From: noreply at rubyforge.org <noreply at rubyforge.org>
> Date: Aug 6, 2007 10:51 AM
> Subject: RubyForge Project Approved
> To: transfire at gmail.com
> 
> 
> Your project registration for RubyForge has been approved!
> 
> Project Full Name:  libxml2
> Project Unix Name:  libxml2
> Virtual host:       libxml2.rubyforge.org
> 
> If after an hour your CVS/Subversion account doesn't work, please
> open a support ticket
> (http://rubyforge.org/tracker/?atid=102&group_id=5&func=browse)
> so that we can look at the problem.  Please note that all shell accounts
> are restricted to CVS/Subversion/SCP/SFTP.
> 
> There's a bunch of good information on configuring mailing lists, twiddling
> your project settings to save bandwidth, setting up commit emails for CVS and
> Subversion, and much more in the project admin guide here:
> 
> http://rubyforge.org/docman/?group_id=5
> 
> Enjoy, and please tell others about RubyForge. Let us know
> if there is anything we can do to help you.  Welcome aboard!
> 
> -- The RubyForge admin team (Tom Copeland, Rich Kilmer)
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070806/5bce10bd/attachment.bin 

From pat.eyler at gmail.com  Mon Aug  6 14:09:29 2007
From: pat.eyler at gmail.com (pat eyler)
Date: Mon, 6 Aug 2007 12:09:29 -0600
Subject: [libxml-devel] Fwd: RubyForge Project Approved
In-Reply-To: <46B762CD.6050406@savagexi.com>
References: <20070806175122.3CCFF5240D2A@rubyforge.org>
	<4b6f054f0708061056r38458343y177ffd7428cd6c29@mail.gmail.com>
	<46B762CD.6050406@savagexi.com>
Message-ID: <6fd0654b0708061109r4fcd4a23x5606492a6cfdb915@mail.gmail.com>

On 8/6/07, Charlie Savage <cfis at savagexi.com> wrote:
> Ok, some questions:
>
> 1.  Are we going to change to a new mailing list then?  If so, maybe you
> can send an email to this list Trans?

If possible, it would be nice to continue to use this list as it will
help to maintain some continuity and maybe catch some of the
folks that miss the word of the fork/fresh start.   Alternatively,
this list should be monitored and people redirected to the new
list.


>
> Charlie
>

-- 
thanks,
-pate
-------------------------
   Duty makes us do things, Love make us do things well.
http://on-ruby.blogspot.com       http://on-erlang.blogspot.com
          http://on-soccer.blogspot.com

From transfire at gmail.com  Mon Aug  6 14:11:33 2007
From: transfire at gmail.com (TRANS)
Date: Mon, 6 Aug 2007 11:11:33 -0700
Subject: [libxml-devel] A New LibXml
In-Reply-To: <6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
Message-ID: <4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>

Okay, the libxml2 project has been created. (it happened so fast I
think Tom Copeland must have been sitting the "create libxml2 project
button" ;).

Couple of things to do first...

I assume it's best to create a new mailing list. I suppose I can send
out invitations to all the old list members (anyone see any reason not
to do that?)

I'll announce the new project on ruby-talk once we have the repo checked in.

Unless someone wants to put up a damn good argument as to why we
should start from scratch, then we'll go ahead move forward with with
the current code base.

The new repo is SVN, not CVS. I'd like to have a hands on libxml coder
do the initial check-in. Someone with a fair idea how to migrate from
CVS to SVN. Any takers? I figure we need to check in the current
stable release as a tag/ and Ross' development version as the current
trunk. Sound right?

Anything else I'm forgetting?

A big thanks to everyone for their input on this! I'm looking very
much forward to seeing this project blossom.

T.

From cfis at savagexi.com  Mon Aug  6 14:17:11 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Mon, 06 Aug 2007 12:17:11 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>	<46B6421D.4050006@savagexi.com>	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>	<46B6773B.4000704@savagexi.com>	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>	<46B7479C.2010304@savagexi.com>	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
	<4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
Message-ID: <46B765A7.309@savagexi.com>

> I assume it's best to create a new mailing list. I suppose I can send
> out invitations to all the old list members (anyone see any reason not
> to do that?)

Sounds good to me - I think a new mailing list is the way to go now that 
we've decided to fork.  Can you just move everyone to the new mailing 
list, instead of having to have people re-subscribe?

> The new repo is SVN, not CVS. I'd like to have a hands on libxml coder
> do the initial check-in. Someone with a fair idea how to migrate from
> CVS to SVN. Any takers? I figure we need to check in the current
> stable release as a tag/ and Ross' development version as the current
> trunk. Sound right?

SVN sounds good.  I guess the question is if anyone wants to migrate the 
old history from CVS to SVN.  If no, then a checkin would be simple.  If 
yes, then someone will have to dig into how that works (I don't know 
myself, but would hope its not too hard).

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070806/0869a6d7/attachment-0001.bin 

From transfire at gmail.com  Mon Aug  6 14:25:21 2007
From: transfire at gmail.com (TRANS)
Date: Mon, 6 Aug 2007 11:25:21 -0700
Subject: [libxml-devel] A New LibXml
In-Reply-To: <46B765A7.309@savagexi.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
	<4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
	<46B765A7.309@savagexi.com>
Message-ID: <4b6f054f0708061125y52f7101bic6f421af4c514721@mail.gmail.com>

On 8/6/07, Charlie Savage <cfis at savagexi.com> wrote:
> > I assume it's best to create a new mailing list. I suppose I can send
> > out invitations to all the old list members (anyone see any reason not
> > to do that?)
>
> Sounds good to me - I think a new mailing list is the way to go now that
> we've decided to fork.  Can you just move everyone to the new mailing
> list, instead of having to have people re-subscribe?
>
> > The new repo is SVN, not CVS. I'd like to have a hands on libxml coder
> > do the initial check-in. Someone with a fair idea how to migrate from
> > CVS to SVN. Any takers? I figure we need to check in the current
> > stable release as a tag/ and Ross' development version as the current
> > trunk. Sound right?
>
> SVN sounds good.  I guess the question is if anyone wants to migrate the
> old history from CVS to SVN.  If no, then a checkin would be simple.  If
> yes, then someone will have to dig into how that works (I don't know
> myself, but would hope its not too hard).

We can either start fresh, or use this tool

  http://cvs2svn.tigris.org/

T.

From danj at 3skel.com  Mon Aug  6 15:12:23 2007
From: danj at 3skel.com (Dan Janowski)
Date: Mon, 6 Aug 2007 15:12:23 -0400
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
Message-ID: <87F9D946-FF78-41D0-807E-67996B049E9F@3skel.com>

It is summer, is Sean just on vacation?

Dan

On Aug 5, 2007, at 09:08, TRANS wrote:

> Hi All,
>
> I'm considering forking the libxml project. Once again I can't get a
> hold of Sean Chittenden. So we can't get any new developers on the
> project with commit rights. No one, as far as I know, is actively
> patching. And no one is at the helm.
>
> I am not a C coder, so really I'm not the best man for the job. But I
> am willing to coordinate and do everything I am able for the project.
> I really need a good, fast xml library for Ruby, and AFAIK bindings to
> libxml are the best choice. Unfortunately there are some serious
> issues with the current bindings --memory leaks and speed issues (I've
> seen libxml run slower than rexml, which really threw me for a loop).
> So I want to do something...
>
> So if you all agree, that a fork is a good idea, the question arises:
> would we be better off starting from scratch? (We can use some of the
> old code piecemeal as proves useful, of course.)  If so, who wants to
> take up that challenge?
>
> T.
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel


From rosco at roscopeco.co.uk  Mon Aug  6 15:44:00 2007
From: rosco at roscopeco.co.uk (Ross Bamford)
Date: Mon, 06 Aug 2007 20:44:00 +0100
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<46B6421D.4050006@savagexi.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
	<4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
Message-ID: <op.twnk3mllkc9f7k@localhost.localdomain>

On Mon, 06 Aug 2007 19:11:33 +0100, TRANS <transfire at gmail.com> wrote:

>
> Unless someone wants to put up a damn good argument as to why we
> should start from scratch, then we'll go ahead move forward with with
> the current code base.
>

I'd say there's too much good code in there to just throw it away - a lot  
of the current code works well, and is tested to at least some extent.  
Plus, there seem to be a lot of patches floating around again.

In terms of moving forward, IMO it'd be worth making reference to  
ruby-xml-smart, which does some of what we do, without the bugs...  
http://raa.ruby-lang.org/project/ruby-xml-smart/

> The new repo is SVN, not CVS. I'd like to have a hands on libxml coder
> do the initial check-in. Someone with a fair idea how to migrate from
> CVS to SVN. Any takers? I figure we need to check in the current
> stable release as a tag/ and Ross' development version as the current
> trunk. Sound right?
>

cvs2svn would be great, but I think it's best to have filesystem access to  
the repository. I'm pretty sure it's running on rubyforge already though,  
since I've converted other projects from CVS without (IIRC) losing the  
history, but I think you'd have to ask Tom to run it cross-project...

-- 
Ross Bamford - rosco at roscopeco.co.uk

From transfire at gmail.com  Mon Aug  6 16:23:12 2007
From: transfire at gmail.com (TRANS)
Date: Mon, 6 Aug 2007 13:23:12 -0700
Subject: [libxml-devel] A New LibXml
In-Reply-To: <87F9D946-FF78-41D0-807E-67996B049E9F@3skel.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<87F9D946-FF78-41D0-807E-67996B049E9F@3skel.com>
Message-ID: <4b6f054f0708061323y3c774ba7q87f6237e8bd146fc@mail.gmail.com>

On 8/6/07, Dan Janowski <danj at 3skel.com> wrote:
> It is summer, is Sean just on vacation?

Well, he may be, but since I haven't been able to get a hold of him in
well over a year, I'd say it's a little more than that.

And when's the last time he posted to this list... Has he ever posted
to this list?

T.

From transfire at gmail.com  Mon Aug  6 16:29:21 2007
From: transfire at gmail.com (TRANS)
Date: Mon, 6 Aug 2007 13:29:21 -0700
Subject: [libxml-devel] A New LibXml
In-Reply-To: <op.twnk3mllkc9f7k@localhost.localdomain>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
	<4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
	<op.twnk3mllkc9f7k@localhost.localdomain>
Message-ID: <4b6f054f0708061329t30db00daw136d454fbfaf3218@mail.gmail.com>

On 8/6/07, Ross Bamford <rosco at roscopeco.co.uk> wrote:

> I'd say there's too much good code in there to just throw it away - a lot
> of the current code works well, and is tested to at least some extent.
> Plus, there seem to be a lot of patches floating around again.
>
> In terms of moving forward, IMO it'd be worth making reference to
> ruby-xml-smart, which does some of what we do, without the bugs...
> http://raa.ruby-lang.org/project/ruby-xml-smart/

Cool, looks like they made some progress since I last looked at it.
Yes, that's good to know about --its not a full binding, not intended
to be, but could still be a good resource.

> cvs2svn would be great, but I think it's best to have filesystem access to
> the repository. I'm pretty sure it's running on rubyforge already though,
> since I've converted other projects from CVS without (IIRC) losing the
> history, but I think you'd have to ask Tom to run it cross-project...

Oh, cool. I'll ask Tom if he can do that. If he can, we should be all
set. If not, well we can just start with a fresh repo. I don't think
it matters too too much.

T.

From danj at 3skel.com  Wed Aug  8 23:07:23 2007
From: danj at 3skel.com (Dan Janowski)
Date: Wed, 8 Aug 2007 23:07:23 -0400
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708061329t30db00daw136d454fbfaf3218@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<2fff50390708051557x5a6f8daei638b3a048215cacd@mail.gmail.com>
	<46B6773B.4000704@savagexi.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
	<4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
	<op.twnk3mllkc9f7k@localhost.localdomain>
	<4b6f054f0708061329t30db00daw136d454fbfaf3218@mail.gmail.com>
Message-ID: <D9EE9EBF-CEAC-4933-B2CA-48A39CE8DFE4@3skel.com>

If it is possible to have both CVS and SVN under the project, then  
one option is to get a full historical copy of the libxml CVS repo  
and put it into libxml2 and just leave it for reference. Then import  
the latest into SVN and just start from there. As long as the logs  
are accessible, it matters less if they are in the same repo.

I have made some contributions to libxml (am a C programmer) and have  
done this very fork thing before when I started bdb2 on rubyforge. Is  
there some way that the C programmers can coordinate so we can all  
understand the problems and we can unify a strategy for dealing with  
them?

Dan


On Aug 6, 2007, at 16:29, TRANS wrote:

> On 8/6/07, Ross Bamford <rosco at roscopeco.co.uk> wrote:
>
>> I'd say there's too much good code in there to just throw it away  
>> - a lot
>> of the current code works well, and is tested to at least some  
>> extent.
>> Plus, there seem to be a lot of patches floating around again.
>>
>> In terms of moving forward, IMO it'd be worth making reference to
>> ruby-xml-smart, which does some of what we do, without the bugs...
>> http://raa.ruby-lang.org/project/ruby-xml-smart/
>
> Cool, looks like they made some progress since I last looked at it.
> Yes, that's good to know about --its not a full binding, not intended
> to be, but could still be a good resource.
>
>> cvs2svn would be great, but I think it's best to have filesystem  
>> access to
>> the repository. I'm pretty sure it's running on rubyforge already  
>> though,
>> since I've converted other projects from CVS without (IIRC) losing  
>> the
>> history, but I think you'd have to ask Tom to run it cross-project...
>
> Oh, cool. I'll ask Tom if he can do that. If he can, we should be all
> set. If not, well we can just start with a fresh repo. I don't think
> it matters too too much.
>
> T.
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel


From transfire at gmail.com  Thu Aug  9 07:31:20 2007
From: transfire at gmail.com (TRANS)
Date: Thu, 9 Aug 2007 04:31:20 -0700
Subject: [libxml-devel] A New LibXml
In-Reply-To: <D9EE9EBF-CEAC-4933-B2CA-48A39CE8DFE4@3skel.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<6fd0654b0708060631n72af8dc1o2d8dbcef7196b1db@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
	<4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
	<op.twnk3mllkc9f7k@localhost.localdomain>
	<4b6f054f0708061329t30db00daw136d454fbfaf3218@mail.gmail.com>
	<D9EE9EBF-CEAC-4933-B2CA-48A39CE8DFE4@3skel.com>
Message-ID: <4b6f054f0708090431h4010b196u44b131549b8eb39a@mail.gmail.com>

On 8/8/07, Dan Janowski <danj at 3skel.com> wrote:
> If it is possible to have both CVS and SVN under the project, then
> one option is to get a full historical copy of the libxml CVS repo
> and put it into libxml2 and just leave it for reference. Then import
> the latest into SVN and just start from there. As long as the logs
> are accessible, it matters less if they are in the same repo.

Yes. I was thinking exactly that yesterday. But we don't need to have
a CVS copy at all IMO. We can just dump a ChangeLog and keep that for
reference. The CVS repo will still be available from the old libxml
site. So we can go ahead a start with a fresh copy.

Does any one want to do the import, or should I? I'm thinking of this layout:

  branches/
  tags/
    0.3.8.4/
  trunk/

Where trunk contains Ross' 0.4.0pre01 version. Does that work for everyone?

> I have made some contributions to libxml (am a C programmer) and have
> done this very fork thing before when I started bdb2 on rubyforge.
> Is there some way that the C programmers can coordinate so we can all
> understand the problems and we can unify a strategy for dealing with
> them?

There's the new mailing list (libxml2-discuss at rubyforge.org), but do
you mean IRC or a Wiki or?

T.

From transfire at gmail.com  Thu Aug  9 10:02:13 2007
From: transfire at gmail.com (TRANS)
Date: Thu, 9 Aug 2007 07:02:13 -0700
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708090431h4010b196u44b131549b8eb39a@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
	<4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
	<op.twnk3mllkc9f7k@localhost.localdomain>
	<4b6f054f0708061329t30db00daw136d454fbfaf3218@mail.gmail.com>
	<D9EE9EBF-CEAC-4933-B2CA-48A39CE8DFE4@3skel.com>
	<4b6f054f0708090431h4010b196u44b131549b8eb39a@mail.gmail.com>
Message-ID: <4b6f054f0708090702t6a06f4cah4b2f8a62329127e7@mail.gmail.com>

Tom Copeland just told me he can convert/transfer the current libxml
CVS repo to libxml2's SVN repo. Do we want to do that? Or would it be
better to start fresh?

T.

From rosco at roscopeco.co.uk  Thu Aug  9 10:17:32 2007
From: rosco at roscopeco.co.uk (Ross Bamford)
Date: Thu, 09 Aug 2007 15:17:32 +0100
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708090702t6a06f4cah4b2f8a62329127e7@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>
	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>
	<46B7479C.2010304@savagexi.com>
	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>
	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>
	<4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>
	<op.twnk3mllkc9f7k@localhost.localdomain>
	<4b6f054f0708061329t30db00daw136d454fbfaf3218@mail.gmail.com>
	<D9EE9EBF-CEAC-4933-B2CA-48A39CE8DFE4@3skel.com>
	<4b6f054f0708090431h4010b196u44b131549b8eb39a@mail.gmail.com>
	<4b6f054f0708090702t6a06f4cah4b2f8a62329127e7@mail.gmail.com>
Message-ID: <op.twspzifwkc9f7k@localhost.localdomain>

On Thu, 09 Aug 2007 15:02:13 +0100, TRANS <transfire at gmail.com> wrote:

> Tom Copeland just told me he can convert/transfer the current libxml
> CVS repo to libxml2's SVN repo. Do we want to do that? Or would it be
> better to start fresh?
>

IMO converting across would be preferable - the libxml CVS is set up  
similarly to what you described now (0.3.8 is a branch, trunk is 0.4. The  
DEV_0_4 branch is dead). That said, I guess it's not the end of the world  
starting from scratch, but it just seems 'wrong' to me :)

Btw, if you want to add me on as a developer I'll be happy to help out  
where I can.

Cheers,
Ross

-- 
Ross Bamford - rosco at roscopeco.co.uk

From cfis at savagexi.com  Thu Aug  9 12:07:51 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 09 Aug 2007 10:07:51 -0600
Subject: [libxml-devel] A New LibXml
In-Reply-To: <4b6f054f0708090702t6a06f4cah4b2f8a62329127e7@mail.gmail.com>
References: <4b6f054f0708050608j96bb687pfe55f590144f717b@mail.gmail.com>	<4b6f054f0708060647u351bf545m3eee3a954404f53e@mail.gmail.com>	<46B7479C.2010304@savagexi.com>	<4b6f054f0708060933p7b03dc0cn53651032cba0c4@mail.gmail.com>	<6fd0654b0708060951n615e4006ld55e8fdcb84abad0@mail.gmail.com>	<4b6f054f0708061111y54be7fa4jd42c00450d5665d4@mail.gmail.com>	<op.twnk3mllkc9f7k@localhost.localdomain>	<4b6f054f0708061329t30db00daw136d454fbfaf3218@mail.gmail.com>	<D9EE9EBF-CEAC-4933-B2CA-48A39CE8DFE4@3skel.com>	<4b6f054f0708090431h4010b196u44b131549b8eb39a@mail.gmail.com>
	<4b6f054f0708090702t6a06f4cah4b2f8a62329127e7@mail.gmail.com>
Message-ID: <46BB3BD7.9060300@savagexi.com>


TRANS wrote:
> Tom Copeland just told me he can convert/transfer the current libxml
> CVS repo to libxml2's SVN repo. Do we want to do that? Or would it be
> better to start fresh?

I'd say go ahead and do the convert/transfer - it certainly can't hurt. 
  And then rearrange the code to fit the proposal you mentioned earlier 
(if the conversion doesn't already do that).

   branches/
   tags/
     0.3.8.4/
   trunk/

Charlie

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070809/7e1039e4/attachment-0001.bin 

From transfire at gmail.com  Fri Aug 10 10:01:02 2007
From: transfire at gmail.com (TRANS)
Date: Fri, 10 Aug 2007 07:01:02 -0700
Subject: [libxml-devel] New List
Message-ID: <4b6f054f0708100701l1d3ee40ao6fb9b698227e5ecd@mail.gmail.com>

Hi-

I created the new mailing list and sent out invitations to everyone of
the old list. But just in case, here's a reminder of where you can
sign-up:

  http://rubyforge.org/mailman/listinfo/libxml2-discuss

We should start transitioning to the new list.... NOW! ;)

BTW, I used the suffix "discuss" rather than "devel" simply b/c I
figure this project is more fitted to a single list rather than a
split between developer and a user lists.

T.

From transfire at gmail.com  Thu Aug 16 08:50:49 2007
From: transfire at gmail.com (TRANS)
Date: Thu, 16 Aug 2007 05:50:49 -0700
Subject: [libxml-devel] Fwd: RubyForge Project Approved
In-Reply-To: <6fd0654b0708061109r4fcd4a23x5606492a6cfdb915@mail.gmail.com>
References: <20070806175122.3CCFF5240D2A@rubyforge.org>
	<4b6f054f0708061056r38458343y177ffd7428cd6c29@mail.gmail.com>
	<46B762CD.6050406@savagexi.com>
	<6fd0654b0708061109r4fcd4a23x5606492a6cfdb915@mail.gmail.com>
Message-ID: <4b6f054f0708160550o496f8778jda071669b195db2e@mail.gmail.com>

On 8/6/07, pat eyler <pat.eyler at gmail.com> wrote:
> On 8/6/07, Charlie Savage <cfis at savagexi.com> wrote:
> > Ok, some questions:
> >
> > 1.  Are we going to change to a new mailing list then?  If so, maybe you
> > can send an email to this list Trans?
>
> If possible, it would be nice to continue to use this list as it will
> help to maintain some continuity and maybe catch some of the
> folks that miss the word of the fork/fresh start.   Alternatively,
> this list should be monitored and people redirected to the new
> list.

Yes, redirect to the new list.

T.

From sean at ruby-lang.org  Thu Aug 16 11:42:17 2007
From: sean at ruby-lang.org (Sean Chittenden)
Date: Thu, 16 Aug 2007 08:42:17 -0700
Subject: [libxml-devel] Fwd: RubyForge Project Approved
In-Reply-To: <4b6f054f0708160550o496f8778jda071669b195db2e@mail.gmail.com>
References: <20070806175122.3CCFF5240D2A@rubyforge.org>
	<4b6f054f0708061056r38458343y177ffd7428cd6c29@mail.gmail.com>
	<46B762CD.6050406@savagexi.com>
	<6fd0654b0708061109r4fcd4a23x5606492a6cfdb915@mail.gmail.com>
	<4b6f054f0708160550o496f8778jda071669b195db2e@mail.gmail.com>
Message-ID: <959B871D-A82D-421C-9FDC-8D5772D83F1B@ruby-lang.org>

>>> Ok, some questions:
>>>
>>> 1.  Are we going to change to a new mailing list then?  If so,  
>>> maybe you
>>> can send an email to this list Trans?
>>
>> If possible, it would be nice to continue to use this list as it will
>> help to maintain some continuity and maybe catch some of the
>> folks that miss the word of the fork/fresh start.   Alternatively,
>> this list should be monitored and people redirected to the new
>> list.
>
> Yes, redirect to the new list.

What's the new list name?  -sc

From transfire at gmail.com  Fri Aug 17 13:33:53 2007
From: transfire at gmail.com (TRANS)
Date: Fri, 17 Aug 2007 10:33:53 -0700
Subject: [libxml-devel] libxml vs. libxml2
Message-ID: <4b6f054f0708171033m4bf5eb5dw6b711fb07eb2ae74@mail.gmail.com>

Crazy, crazy. Sean appeared out the blue aether yesterday, and gave me
admin rights to the original  libxml project. So I suppose the fork
isn't necessary after all. But I'll put it to the community just the
same to be sure. Should we stick to the old project or go forward with
the fork?

T.


P.S. If we do stick with the original, I will still have the
repository converted to SVN and do some clean up.

From cjbottaro at alumni.cs.utexas.edu  Fri Aug 17 14:19:55 2007
From: cjbottaro at alumni.cs.utexas.edu (Christopher J. Bottaro)
Date: Fri, 17 Aug 2007 13:19:55 -0500
Subject: [libxml-devel] [Libxml2] libxml vs. libxml2
In-Reply-To: <4b6f054f0708171033m4bf5eb5dw6b711fb07eb2ae74@mail.gmail.com>
References: <4b6f054f0708171033m4bf5eb5dw6b711fb07eb2ae74@mail.gmail.com>
Message-ID: <c19e6adb0708171119p19307783we5bbc9da8fe1e623@mail.gmail.com>

I don't care, either way... as long a some gem is made official and I
don't have to worry about it.  I dunno, maybe continue with the orig
project since already when you search for libxml on rubyforge, two
projects come up (libxml and libxml2) and I think that might confuse
people.

My company is considering moving our app to Rails and that decision
it's very dependent on the performance and reliability of this
project.  :)

Thanks for continuing on with this project.

-- Christopher

On 8/17/07, TRANS <transfire at gmail.com> wrote:
> Crazy, crazy. Sean appeared out the blue aether yesterday, and gave me
> admin rights to the original  libxml project. So I suppose the fork
> isn't necessary after all. But I'll put it to the community just the
> same to be sure. Should we stick to the old project or go forward with
> the fork?
>
> T.
>
>
> P.S. If we do stick with the original, I will still have the
> repository converted to SVN and do some clean up.
> _______________________________________________
> Libxml2-discuss mailing list
> Libxml2-discuss at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml2-discuss
>

From pat.eyler at gmail.com  Wed Aug 22 12:17:03 2007
From: pat.eyler at gmail.com (pat eyler)
Date: Wed, 22 Aug 2007 10:17:03 -0600
Subject: [libxml-devel] libxml vs. libxml2
In-Reply-To: <4b6f054f0708171033m4bf5eb5dw6b711fb07eb2ae74@mail.gmail.com>
References: <4b6f054f0708171033m4bf5eb5dw6b711fb07eb2ae74@mail.gmail.com>
Message-ID: <6fd0654b0708220917g6c4b7919t1ef3796736d993c9@mail.gmail.com>

For consistency, stick to the original.

On 8/17/07, TRANS <transfire at gmail.com> wrote:
> Crazy, crazy. Sean appeared out the blue aether yesterday, and gave me
> admin rights to the original  libxml project. So I suppose the fork
> isn't necessary after all. But I'll put it to the community just the
> same to be sure. Should we stick to the old project or go forward with
> the fork?
>
> T.
>
>
> P.S. If we do stick with the original, I will still have the
> repository converted to SVN and do some clean up.
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel
>


-- 
thanks,
-pate
-------------------------
   Duty makes us do things, Love make us do things well.
http://on-ruby.blogspot.com       http://on-erlang.blogspot.com
          http://on-soccer.blogspot.com

From cjbottaro at alumni.cs.utexas.edu  Wed Aug 22 14:59:28 2007
From: cjbottaro at alumni.cs.utexas.edu (Christopher J. Bottaro)
Date: Wed, 22 Aug 2007 13:59:28 -0500
Subject: [libxml-devel] cannot call find() on a "copied" node
Message-ID: <c19e6adb0708221159n247f5da7j50a8c51ea0286db4@mail.gmail.com>

copied_node = a_node.copy(true)
copied_node.find('.//*') # => TypeError: wrong argument type nil (expected Data)

What am I doing wrong?

Thank you.

From cjbottaro at alumni.cs.utexas.edu  Wed Aug 22 17:38:17 2007
From: cjbottaro at alumni.cs.utexas.edu (Christopher J. Bottaro)
Date: Wed, 22 Aug 2007 16:38:17 -0500
Subject: [libxml-devel] cannot call find() on a "copied" node
In-Reply-To: <c19e6adb0708221159n247f5da7j50a8c51ea0286db4@mail.gmail.com>
References: <c19e6adb0708221159n247f5da7j50a8c51ea0286db4@mail.gmail.com>
Message-ID: <c19e6adb0708221438h4822cac0td628fe9086a20944@mail.gmail.com>

On 8/22/07, Christopher J. Bottaro <cjbottaro at alumni.cs.utexas.edu> wrote:
> copied_node = a_node.copy(true)
> copied_node.find('.//*') # => TypeError: wrong argument type nil (expected Data)
>
> What am I doing wrong?
>
> Thank you.

Ok, it looks like the copied node is not associated with any
XML::Document.  How do make this association?

Thanks.

From marc at bloodnok.com  Wed Aug 22 12:46:19 2007
From: marc at bloodnok.com (Marc Munro)
Date: Wed, 22 Aug 2007 09:46:19 -0700
Subject: [libxml-devel] libxml crash
Message-ID: <1187801179.12172.13.camel@bloodnok.com>

Folks,
I have a reproducible crash using libxml.  I'd like some suggestions on
how I can track this down as it's become a show-stopper for my current
project.  This is from a libxml gem created from svn this morning.

I can write and debug C but I'd like some suggestions on what to look
for, and what to try.  I am inclined to modify all calls to free() to
zero or set the allocated memory chunk before freeing it in the hope
that this will make it obvious where things are breaking.  Having not
hacked ruby before I am not sure how easy this will be.  I'm also
wondering whether any of the standard C memory testing tools can be
easily incorporated into the ruby build.

I'd welcome any suggestions/random thoughts/shared experiences/anything
from anyone at this stage :-)

Here is the backtrace, in case it helps.

*** glibc detected *** ruby: double free or corruption (fasttop):
0x081f98d8 ***
======= Backtrace: =========
/lib/libc.so.6[0xb7d08824]
/lib/libc.so.6(cfree+0x90)[0xb7d0be90]
/usr/lib/libruby1.8.so.1.8(ruby_xfree+0x37)[0xb7e99d67]
/usr/lib/libxml2.so.2(xmlFreeNodeList+0x11d)[0xb7b0eadd]
/usr/lib/libxml2.so.2(xmlFreeProp+0x52)[0xb7b0ec12]
/usr/lib/libxml2.so.2(xmlFreePropList+0x1b)[0xb7b0ee8b]
/usr/lib/libxml2.so.2(xmlFreeNodeList+0xba)[0xb7b0ea7a]
/usr/lib/libxml2.so.2(xmlFreeNodeList+0x96)[0xb7b0ea56]
/usr/lib/libxml2.so.2(xmlFreeNode+0x7b)[0xb7b0ef1b]
/usr/lib/ruby/1.8/i486-linux/xml/libxml_so.so(ruby_xml_node_free
+0x5d)[0xb7c0abdd]
/usr/lib/libruby1.8.so.1.8(rb_gc_call_finalizer_at_exit
+0xa7)[0xb7e9a027]
/usr/lib/libruby1.8.so.1.8[0xb7e802a7]
/usr/lib/libruby1.8.so.1.8(ruby_cleanup+0x103)[0xb7e88293]
/usr/lib/libruby1.8.so.1.8(ruby_stop+0x1d)[0xb7e8839d]
/usr/lib/libruby1.8.so.1.8[0xb7e93181]
ruby[0x80486bd]
/lib/libc.so.6(__libc_start_main+0xe0)[0xb7cb5030]
ruby[0x80485e1]
======= Memory map: ========
08048000-08049000 r-xp 00000000 03:01 21071      /usr/bin/ruby1.8
08049000-0804a000 rw-p 00000000 03:01 21071      /usr/bin/ruby1.8
0804a000-088f6000 rw-p 0804a000 00:00 0          [heap]
b7800000-b7821000 rw-p b7800000 00:00 0 
b7821000-b7900000 ---p b7821000 00:00 0 
b79de000-b79e6000 rw-p b79de000 00:00 0 
b79e9000-b79f3000 r-xp 00000000 03:01 472100     /lib/libgcc_s.so.1
b79f3000-b79f4000 rw-p 00009000 03:01 472100     /lib/libgcc_s.so.1
b7a07000-b7a56000 r-xp 00000000 03:01
19373      /usr/lib/libgcrypt.so.11.2.3
b7a56000-b7a58000 rw-p 0004f000 03:01
19373      /usr/lib/libgcrypt.so.11.2.3
b7a58000-b7a68000 r-xp 00000000 03:01
17836      /usr/lib/libexslt.so.0.8.13
b7a68000-b7a69000 rw-p 0000f000 03:01
17836      /usr/lib/libexslt.so.0.8.13
b7a69000-b7a9c000 r-xp 00000000 03:01
21872      /usr/lib/libxslt.so.1.1.21
b7a9c000-b7a9d000 rw-p 00032000 03:01
21872      /usr/lib/libxslt.so.1.1.21
b7a9d000-b7ab1000 r-xp 00000000 03:01 233293     /lib/libnsl-2.6.so
b7ab1000-b7ab3000 rw-p 00013000 03:01 233293     /lib/libnsl-2.6.so
b7ab3000-b7ab5000 rw-p b7ab3000 00:00 0 
b7ab5000-b7ac9000 r-xp 00000000 03:01
18146      /usr/lib/libz.so.1.2.3.3
b7ac9000-b7aca000 rw-p 00013000 03:01
18146      /usr/lib/libz.so.1.2.3.3
b7aca000-b7be1000 r-xp 00000000 03:01
16657      /usr/lib/libxml2.so.2.6.29
b7be1000-b7be6000 rw-p 00116000 03:01
16657      /usr/lib/libxml2.so.2.6.29
b7be6000-b7be8000 rw-p b7be6000 00:00 0 
b7be8000-b7bfa000 rw-p b7be8000 00:00 0 
b7bfa000-b7c14000 r-xp 00000000 03:01
146364     /usr/lib/ruby/1.8/i486-linux/xml/libxml_so.so
b7c14000-b7c15000 rw-p 0001a000 03:01
146364     /usr/lib/ruby/1.8/i486-linux/xml/libxml_so.so
b7c15000-b7c9f000 rw-p b7c15000 00:00 0 
b7c9f000-b7de0000 r-xp 00000000 03:01 233283     /lib/libc-2.6.so
b7de0000-b7de1000 r--p 00141000 03:01 233283     /lib/libc-2.6.so
b7de1000-b7de3000 rw-p 00142000 03:01 233283     /lib/libc-2.6.so
b7de3000-b7de6000 rw-p b7de3000 00:00 0 
b7de6000-b7e0b000 r-xp 00000000 03:01 233287     /lib/libm-2.6.so
b7e0b000-b7e0d000 rw-p 00024000 03:01 233287     /lib/libm-2.6.so
b7e0d000-b7e12000 r-xp 00000000 03:01 233285     /lib/libcrypt-2.6.so
b7e12000-b7e14000 rw-p 00004000 03:01 233285     /lib/libcrypt-2.6.so
b7e14000-b7e3b000 rw-p b7e14000 00:00 0 
b7e3b000-b7e3d000 r-xp 00000000 03:01 233286     /lib/libdl-2.6.so
b7e3d000-b7e3f000 rw-p 00001000 03:01 233286     /lib/libdl-2.6.so
b7e3f000-b7e40000 rw-p b7e3f000 00:00 0 
b7e40000-b7e53000 r-xp 00000000 03:01 233301     /lib/libpthread-2.6.so
b7e53000-b7e55000 rw-p 00012000 03:01 233301     /lib/libpthread-2.6.so
b7e55000-b7e57000 rw-p b7e55000 00:00 0 
b7e57000-b7f14000 r-xp 00000000 03:01
29683      /usr/lib/libruby1.8.so.1.8.6
b7f14000-b7f17000 rw-p 000bc000 03:01
29683      /usr/lib/libruby1.8.so.1.8.6
b7f17000-b7f26000 rw-p b7f17000 00:00 0 
b7f28000-b7f2a000 rw-p b7f28000 00:00 0 
b7f2a000-b7f2d000 r-xp 00000000 03:01
18898      /usr/lib/libgpg-error.so.0.3.0
b7f2d000-b7f2e000 rw-p 00002000 03:01
18898      /usr/lib/libgpg-error.so.0.3.0
b7f2e000-b7f33000 r-xp 00000000 03:01
146769     /usr/lib/ruby/1.8/i486-linux/xml/libxslt.so
b7f33000-b7f34000 rw-p 00004000 03:01
146769     /usr/lib/ruby/1.8/i486-linux/xml/libxslt.so
b7f34000-b7f38000 r-xp 00000000 03:01
150566     /usr/lib/ruby/1.8/i486-linux/stringio.so
b7f38000-b7f39000 rw-p 00003000 03:01
150566     /usr/lib/ruby/1.8/i486-linux/stringio.so
b7f39000-b7f3b000 rw-p b7f39000 00:00 0 
b7f3b000-b7f57000 r-xp 00000000 03:01 991671     /lib/ld-2.6.so
b7f57000-b7f59000 rw-p 0001b000 03:01 991671     /lib/ld-2.6.so
bf9f5000-bfa57000 rw-p bf9f5000 00:00 0          [stack]
ffffe000-fffff000 ---p 00000000 00:00 0          [vdso]
Aborted


__
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070822/d95e664d/attachment.bin 

From transfire at gmail.com  Wed Aug 22 19:41:05 2007
From: transfire at gmail.com (TRANS)
Date: Wed, 22 Aug 2007 16:41:05 -0700
Subject: [libxml-devel] Developers
Message-ID: <4b6f054f0708221641y548b1dafi5d826322ea812d6@mail.gmail.com>

Hi--

These are the people I have as developers on the libxml project.

  http://rubyforge.org/project/admin/index.php?group_id=494

The libxml2 fork project has been removed, so I couldn't double check
the developer list. Let me know if II forgot to add you to the list.

I'll try to get the new SVN repo cleaned up tomorrow.

T.

From cfis at savagexi.com  Wed Aug 22 20:21:59 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Wed, 22 Aug 2007 18:21:59 -0600
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187801179.12172.13.camel@bloodnok.com>
References: <1187801179.12172.13.camel@bloodnok.com>
Message-ID: <46CCD327.1080805@savagexi.com>

Marc,

> I have a reproducible crash using libxml.  I'd like some suggestions on
> how I can track this down as it's become a show-stopper for my current
> project.  This is from a libxml gem created from svn this morning.

Can you provide the test data?  That would be hugely helpful.

Also what OS, platform, etc?

> I can write and debug C but I'd like some suggestions on what to look
> for, and what to try.  I am inclined to modify all calls to free() to
> zero or set the allocated memory chunk before freeing it in the hope
> that this will make it obvious where things are breaking.


> Having not
> hacked ruby before I am not sure how easy this will be.  I'm also
> wondering whether any of the standard C memory testing tools can be
> easily incorporated into the ruby build.

I've successfully used Valgrind.

> 
> I'd welcome any suggestions/random thoughts/shared experiences/anything
> from anyone at this stage :-)

Did you try these patches?

http://rubyforge.org/pipermail/libxml-devel/2007-June/000314.html

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070822/02f6027c/attachment-0001.bin 

From marc at bloodnok.com  Wed Aug 22 21:05:36 2007
From: marc at bloodnok.com (Marc Munro)
Date: Wed, 22 Aug 2007 18:05:36 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CCD327.1080805@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<46CCD327.1080805@savagexi.com>
Message-ID: <1187831136.12172.34.camel@bloodnok.com>

On Wed, 2007-22-08 at 18:21 -0600, Charlie Savage wrote:
> Marc,
> 
> > I have a reproducible crash using libxml...

> Can you provide the test data?  That would be hugely helpful.

I could but it's a bit of a monster, probably timing critical, and
likely to run differently on different platforms.  For the next day I'd
like to bash on on my own.  If I get nowhere, I'll package up a test
case and make it available.  I'll also try on another platform to see if
I can make it crash reliably.

> Also what OS, platform, etc?

Linux bloodnok 2.6.15.6 #4 SMP PREEMPT Sat Aug 5 20:40:37 PDT 2006 i686
GNU/Linux

> I've successfully used Valgrind.

That's what I've been trying to use.  I've also added some
instrumentation to the free calls in libxml which seem to be helping.

I have discovered that commenting out the call: xmlFreeNode(rxn->node);
in ruby_xml_node_free (in ruby_xml_node.c) removes my crash.
Consequently, I suspect the logic for determining whether to free the
node is faulty.  This is by no means definite as almost every minor
change I make seems to affect the timing and thereby the crash
behaviour, but it is a reasonable working hypothesis.  Right now I'm
suspicious of the reference counting in the _private field of the
xmlNode structures.

> Did you try these patches?

> http://rubyforge.org/pipermail/libxml-devel/2007-June/000314.html

No.  I'll give them a look.  Thanks.

__
Marc

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070822/654697bd/attachment.bin 

From marc at bloodnok.com  Wed Aug 22 21:08:39 2007
From: marc at bloodnok.com (Marc Munro)
Date: Wed, 22 Aug 2007 18:08:39 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CCD327.1080805@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<46CCD327.1080805@savagexi.com>
Message-ID: <1187831319.12172.38.camel@bloodnok.com>

On Wed, 2007-22-08 at 18:21 -0600, Charlie Savage wrote:
> 
> Did you try these patches?
> 
> http://rubyforge.org/pipermail/libxml-devel/2007-June/000314.html

Hmmm.  I can apply the leak2 patch but not the leak1 patch which seems
way more significant:

marc:[libxml]$ patch -p1 </home/marc/My\ Downloads/libxml-leak2.patch 
patching file ext/xml/ruby_xml_document.c
marc:[libxml]$ patch -p1 </home/marc/My\ Downloads/libxml-leak1.patch 
patching file ext/xml/libxml.c
patching file ext/xml/ruby_xml_attr.c
patching file ext/xml/ruby_xml_attr.h
patching file ext/xml/ruby_xml_document.c
patch: **** malformed patch at line 354: diff -urw
libxml-ruby-0.3.8.4/ext/xml/ruby_xml_node.c
libxml-ruby-fixed/ext/xml/ruby_xml_node.c

marc:[libxml]$ 

I have tried in both .../libxml/tags/REL_0_3_8_4/libxml and
in .../libxml/trunk/libxml but get the same error each time.  Any ideas?

__
Marc

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070822/558f1a25/attachment.bin 

From tom at compton.nu  Thu Aug 23 03:31:15 2007
From: tom at compton.nu (Tom Hughes)
Date: Thu, 23 Aug 2007 08:31:15 +0100
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187801179.12172.13.camel@bloodnok.com> (Marc Munro's message of
	"Wed\, 22 Aug 2007 09\:46\:19 -0700")
References: <1187801179.12172.13.camel@bloodnok.com>
Message-ID: <yekps1epv0c.fsf@dellow.uk.cyberscience.com>

In message <1187801179.12172.13.camel at bloodnok.com>
        Marc Munro <marc at bloodnok.com> wrote:

> I have a reproducible crash using libxml.  I'd like some suggestions on
> how I can track this down as it's become a show-stopper for my current
> project.  This is from a libxml gem created from svn this morning.

Your crash is a typical one - it looks like it is caused by ruby
still having a handle to a libxml object that has been attached to
another libxml object that has then been freed.

When ruby then gets around to freeing the first object it crashes
trying to free the (already freed) underlying libxml object.

My patches (to which I see you have already been pointed) are quite
likely to help with this. They're only a band aid really though - what
is really needed is a ground up review/redesign of the way the two
object models are tied together as the current scheme basically
doesn't work at all.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/

From bryant.doug at gmail.com  Thu Aug 23 07:33:45 2007
From: bryant.doug at gmail.com (Doug Bryant)
Date: Thu, 23 Aug 2007 07:33:45 -0400
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187801179.12172.13.camel@bloodnok.com>
References: <1187801179.12172.13.camel@bloodnok.com>
Message-ID: <177c686f0708230433y6624fdffm6b877ebd249839a3@mail.gmail.com>

This is most likely related to a copy node bug which has been fixed in a .4
release. Ross fixed per an issue I opened.   Although the .4 release is
labled as a pre-release, it is stable with the exception of the memory bug.

Upgrade and it will most likely fix the problem.

See issue #4635 (now closed)
http://rubyforge.org/tracker/index.php?func=detail&aid=4635&group_id=494&atid=1971

Doug

On 8/22/07, Marc Munro <marc at bloodnok.com> wrote:
>
> Folks,
> I have a reproducible crash using libxml.  I'd like some suggestions on
> how I can track this down as it's become a show-stopper for my current
> project.  This is from a libxml gem created from svn this morning.
>
> I can write and debug C but I'd like some suggestions on what to look
> for, and what to try.  I am inclined to modify all calls to free() to
> zero or set the allocated memory chunk before freeing it in the hope
> that this will make it obvious where things are breaking.  Having not
> hacked ruby before I am not sure how easy this will be.  I'm also
> wondering whether any of the standard C memory testing tools can be
> easily incorporated into the ruby build.
>
> I'd welcome any suggestions/random thoughts/shared experiences/anything
> from anyone at this stage :-)
>
> Here is the backtrace, in case it helps.
>
> *** glibc detected *** ruby: double free or corruption (fasttop):
> 0x081f98d8 ***
> ======= Backtrace: =========
> /lib/libc.so.6[0xb7d08824]
> /lib/libc.so.6(cfree+0x90)[0xb7d0be90]
> /usr/lib/libruby1.8.so.1.8(ruby_xfree+0x37)[0xb7e99d67]
> /usr/lib/libxml2.so.2(xmlFreeNodeList+0x11d)[0xb7b0eadd]
> /usr/lib/libxml2.so.2(xmlFreeProp+0x52)[0xb7b0ec12]
> /usr/lib/libxml2.so.2(xmlFreePropList+0x1b)[0xb7b0ee8b]
> /usr/lib/libxml2.so.2(xmlFreeNodeList+0xba)[0xb7b0ea7a]
> /usr/lib/libxml2.so.2(xmlFreeNodeList+0x96)[0xb7b0ea56]
> /usr/lib/libxml2.so.2(xmlFreeNode+0x7b)[0xb7b0ef1b]
> /usr/lib/ruby/1.8/i486-linux/xml/libxml_so.so(ruby_xml_node_free
> +0x5d)[0xb7c0abdd]
> /usr/lib/libruby1.8.so.1.8(rb_gc_call_finalizer_at_exit
> +0xa7)[0xb7e9a027]
> /usr/lib/libruby1.8.so.1.8[0xb7e802a7]
> /usr/lib/libruby1.8.so.1.8(ruby_cleanup+0x103)[0xb7e88293]
> /usr/lib/libruby1.8.so.1.8(ruby_stop+0x1d)[0xb7e8839d]
> /usr/lib/libruby1.8.so.1.8[0xb7e93181]
> ruby[0x80486bd]
> /lib/libc.so.6(__libc_start_main+0xe0)[0xb7cb5030]
> ruby[0x80485e1]
> ======= Memory map: ========
> 08048000-08049000 r-xp 00000000 03:01 21071      /usr/bin/ruby1.8
> 08049000-0804a000 rw-p 00000000 03:01 21071      /usr/bin/ruby1.8
> 0804a000-088f6000 rw-p 0804a000 00:00 0          [heap]
> b7800000-b7821000 rw-p b7800000 00:00 0
> b7821000-b7900000 ---p b7821000 00:00 0
> b79de000-b79e6000 rw-p b79de000 00:00 0
> b79e9000-b79f3000 r-xp 00000000 03:01 472100     /lib/libgcc_s.so.1
> b79f3000-b79f4000 rw-p 00009000 03:01 472100     /lib/libgcc_s.so.1
> b7a07000-b7a56000 r-xp 00000000 03:01
> 19373      /usr/lib/libgcrypt.so.11.2.3
> b7a56000-b7a58000 rw-p 0004f000 03:01
> 19373      /usr/lib/libgcrypt.so.11.2.3
> b7a58000-b7a68000 r-xp 00000000 03:01
> 17836      /usr/lib/libexslt.so.0.8.13
> b7a68000-b7a69000 rw-p 0000f000 03:01
> 17836      /usr/lib/libexslt.so.0.8.13
> b7a69000-b7a9c000 r-xp 00000000 03:01
> 21872      /usr/lib/libxslt.so.1.1.21
> b7a9c000-b7a9d000 rw-p 00032000 03:01
> 21872      /usr/lib/libxslt.so.1.1.21
> b7a9d000-b7ab1000 r-xp 00000000 03:01 233293     /lib/libnsl-2.6.so
> b7ab1000-b7ab3000 rw-p 00013000 03:01 233293     /lib/libnsl-2.6.so
> b7ab3000-b7ab5000 rw-p b7ab3000 00:00 0
> b7ab5000-b7ac9000 r-xp 00000000 03:01
> 18146      /usr/lib/libz.so.1.2.3.3
> b7ac9000-b7aca000 rw-p 00013000 03:01
> 18146      /usr/lib/libz.so.1.2.3.3
> b7aca000-b7be1000 r-xp 00000000 03:01
> 16657      /usr/lib/libxml2.so.2.6.29
> b7be1000-b7be6000 rw-p 00116000 03:01
> 16657      /usr/lib/libxml2.so.2.6.29
> b7be6000-b7be8000 rw-p b7be6000 00:00 0
> b7be8000-b7bfa000 rw-p b7be8000 00:00 0
> b7bfa000-b7c14000 r-xp 00000000 03:01
> 146364     /usr/lib/ruby/1.8/i486-linux/xml/libxml_so.so
> b7c14000-b7c15000 rw-p 0001a000 03:01
> 146364     /usr/lib/ruby/1.8/i486-linux/xml/libxml_so.so
> b7c15000-b7c9f000 rw-p b7c15000 00:00 0
> b7c9f000-b7de0000 r-xp 00000000 03:01 233283     /lib/libc-2.6.so
> b7de0000-b7de1000 r--p 00141000 03:01 233283     /lib/libc-2.6.so
> b7de1000-b7de3000 rw-p 00142000 03:01 233283     /lib/libc-2.6.so
> b7de3000-b7de6000 rw-p b7de3000 00:00 0
> b7de6000-b7e0b000 r-xp 00000000 03:01 233287     /lib/libm-2.6.so
> b7e0b000-b7e0d000 rw-p 00024000 03:01 233287     /lib/libm-2.6.so
> b7e0d000-b7e12000 r-xp 00000000 03:01 233285     /lib/libcrypt-2.6.so
> b7e12000-b7e14000 rw-p 00004000 03:01 233285     /lib/libcrypt-2.6.so
> b7e14000-b7e3b000 rw-p b7e14000 00:00 0
> b7e3b000-b7e3d000 r-xp 00000000 03:01 233286     /lib/libdl-2.6.so
> b7e3d000-b7e3f000 rw-p 00001000 03:01 233286     /lib/libdl-2.6.so
> b7e3f000-b7e40000 rw-p b7e3f000 00:00 0
> b7e40000-b7e53000 r-xp 00000000 03:01 233301     /lib/libpthread-2.6.so
> b7e53000-b7e55000 rw-p 00012000 03:01 233301     /lib/libpthread-2.6.so
> b7e55000-b7e57000 rw-p b7e55000 00:00 0
> b7e57000-b7f14000 r-xp 00000000 03:01
> 29683      /usr/lib/libruby1.8.so.1.8.6
> b7f14000-b7f17000 rw-p 000bc000 03:01
> 29683      /usr/lib/libruby1.8.so.1.8.6
> b7f17000-b7f26000 rw-p b7f17000 00:00 0
> b7f28000-b7f2a000 rw-p b7f28000 00:00 0
> b7f2a000-b7f2d000 r-xp 00000000 03:01
> 18898      /usr/lib/libgpg-error.so.0.3.0
> b7f2d000-b7f2e000 rw-p 00002000 03:01
> 18898      /usr/lib/libgpg-error.so.0.3.0
> b7f2e000-b7f33000 r-xp 00000000 03:01
> 146769     /usr/lib/ruby/1.8/i486-linux/xml/libxslt.so
> b7f33000-b7f34000 rw-p 00004000 03:01
> 146769     /usr/lib/ruby/1.8/i486-linux/xml/libxslt.so
> b7f34000-b7f38000 r-xp 00000000 03:01
> 150566     /usr/lib/ruby/1.8/i486-linux/stringio.so
> b7f38000-b7f39000 rw-p 00003000 03:01
> 150566     /usr/lib/ruby/1.8/i486-linux/stringio.so
> b7f39000-b7f3b000 rw-p b7f39000 00:00 0
> b7f3b000-b7f57000 r-xp 00000000 03:01 991671     /lib/ld-2.6.so
> b7f57000-b7f59000 rw-p 0001b000 03:01 991671     /lib/ld-2.6.so
> bf9f5000-bfa57000 rw-p bf9f5000 00:00 0          [stack]
> ffffe000-fffff000 ---p 00000000 00:00 0          [vdso]
> Aborted
>
>
> __
> Marc
>
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/b6f6ad55/attachment.html 

From marc at bloodnok.com  Thu Aug 23 12:40:15 2007
From: marc at bloodnok.com (Marc Munro)
Date: Thu, 23 Aug 2007 09:40:15 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <177c686f0708230433y6624fdffm6b877ebd249839a3@mail.gmail.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<177c686f0708230433y6624fdffm6b877ebd249839a3@mail.gmail.com>
Message-ID: <1187887215.4827.5.camel@bloodnok.com>

On Thu, 2007-23-08 at 07:33 -0400, Doug Bryant wrote:
> This is most likely related to a copy node bug which has been fixed in
> a .4 release. Ross fixed per an issue I opened.   Although the .4
> release is labled as a pre-release, it is stable with the exception of
> the memory bug. 
> 
> Upgrade and it will most likely fix the problem.
> 
> See issue #4635 (now closed)
> http://rubyforge.org/tracker/index.php?func=detail&aid=4635&group_id=494&atid=1971
> 

Nope, that doesn't fix it.  I have now produced a nice small test case,
which is documented in this bug:
http://rubyforge.org/tracker/index.php?func=detail&aid=13310&group_id=494&atid=1971

Since I now understand the bug, I can probably work round it.  I can see
that fixing this might be kinda tricky.  I think that prior to freeing
an xmlNode, libxml is going to have to walk the tree and remove any
nodes that are referenced by ruby objects.

__
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/64fe4764/attachment.bin 

From cfis at savagexi.com  Thu Aug 23 12:45:19 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 23 Aug 2007 10:45:19 -0600
Subject: [libxml-devel] libxml crash
In-Reply-To: <yekps1epv0c.fsf@dellow.uk.cyberscience.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
Message-ID: <46CDB99F.9050608@savagexi.com>

Tom,

> Your crash is a typical one - it looks like it is caused by ruby
> still having a handle to a libxml object that has been attached to
> another libxml object that has then been freed.

Would the solution be a simple as having ruby nodes *never* freeing 
libxml objects except the toplevel document (since it sounds like libxml 
cleans up after itself).

> My patches (to which I see you have already been pointed) are quite
> likely to help with this. They're only a band aid really though - what
> is really needed is a ground up review/redesign of the way the two
> object models are tied together as the current scheme basically
> doesn't work at all.

It seems like a fundamental decision has to be made - who is responsible 
for freeing allocated nodes.  Ruby?  Libxml?  Having both do it, which 
sounds like the current situation, is obviously totally non-workable.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/4f40673a/attachment-0001.bin 

From marc at bloodnok.com  Thu Aug 23 12:53:15 2007
From: marc at bloodnok.com (Marc Munro)
Date: Thu, 23 Aug 2007 09:53:15 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CDB99F.9050608@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<46CDB99F.9050608@savagexi.com>
Message-ID: <1187887995.4827.11.camel@bloodnok.com>

On Thu, 2007-23-08 at 10:45 -0600, Charlie Savage wrote:

> Would the solution be a simple as having ruby nodes *never* freeing 
> libxml objects except the toplevel document (since it sounds like libxml 
> cleans up after itself).

Not all nodes wind up in documents, so unless you're prepared for
possibly large memory leaks that is not going to work.

> It seems like a fundamental decision has to be made - who is responsible 
> for freeing allocated nodes.  Ruby?  Libxml?  Having both do it, which 
> sounds like the current situation, is obviously totally non-workable.

The problem is that some nodes are allocated by libxml by parsing
documents, processing xsl, etc, and some directly by ruby.  I think the
hybrid approach is necessary in order to get the speed benefits of using
libxml, and I think a hybrid approach to freeing is therefore necessary.
It does seem to demand a bit more smarts though, from the memory freeing
mechanisms.

__
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/99a716b8/attachment.bin 

From cfis at savagexi.com  Thu Aug 23 13:13:55 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 23 Aug 2007 11:13:55 -0600
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187887995.4827.11.camel@bloodnok.com>
References: <1187801179.12172.13.camel@bloodnok.com>	
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>	
	<46CDB99F.9050608@savagexi.com>
	<1187887995.4827.11.camel@bloodnok.com>
Message-ID: <46CDC053.7000604@savagexi.com>

> The problem is that some nodes are allocated by libxml by parsing
> documents, processing xsl, etc, and some directly by ruby.  I think the
> hybrid approach is necessary in order to get the speed benefits of using
> libxml, and I think a hybrid approach to freeing is therefore necessary.
> It does seem to demand a bit more smarts though, from the memory freeing
> mechanisms.

Ah - interesting.  A hybrid approach is extremely difficult because it 
means you have to keep track of the ruby to libxml object mapping.  For 
example, say you have some ruby code that create a libxml node.  That 
ruby object would then implement a free method that frees the libxml 
node.  But let's say you end up with a 2nd ruby object pointing to the 
same libxml object (which is really easy to do) - now you have a problem.

For SWIG I implemented support for mapping between C pointers and Ruby 
objects by using a Hash Table (ruby's built in one, st_table).   That 
allows you to enforce a one-to-one Ruby to C object mapping.  Do we want 
to do something like that here?  The couple issues with it are:

* You have to hook it into every single binding method (easy in SWIG 
since its all auto-generated)

* Ruby object ids get reused, so you have to clean out old objects from 
the hash table as soon as they are freed.

If you have such a mapping, then a hybrid approach becomes doable as 
long as you know absolutely for sure when a Ruby object owns a libxml 
object and when it doesn't.  When it does, you assign a free method. 
When it doesn't, you don't.

If we go down this approach, it might be worth using SWIG.  My only 
problem with SWIG is that the SWIG mapping language is very complex, and 
almost no one is going to understand it.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/fca5b2dd/attachment.bin 

From marc at bloodnok.com  Thu Aug 23 14:07:47 2007
From: marc at bloodnok.com (Marc Munro)
Date: Thu, 23 Aug 2007 11:07:47 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CDC053.7000604@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<46CDB99F.9050608@savagexi.com> <1187887995.4827.11.camel@bloodnok.com>
	<46CDC053.7000604@savagexi.com>
Message-ID: <1187892467.4827.28.camel@bloodnok.com>

On Thu, 2007-23-08 at 11:13 -0600, Charlie Savage wrote:
> Ah - interesting.  A hybrid approach is extremely difficult because it 
> means you have to keep track of the ruby to libxml object mapping.  For 
> example, say you have some ruby code that create a libxml node.  That 
> ruby object would then implement a free method that frees the libxml 
> node.  But let's say you end up with a 2nd ruby object pointing to the 
> same libxml object (which is really easy to do) - now you have a problem.

The current implementation uses a reference counting scheme within the
xmlNode elements returned from libxml.  It seems to me that it *should*
be possible for this to work.

The problems that I have discovered stem from libxml freeing entire
document trees while there are still ruby objects referencing the nodes.
When those ruby objects are subsequently garbage-collected, the xmlNodes
in those objects have already been freed and sadness ensues.  But I
think the current mechanism is so close to working that it does not
warrant a complete rewrite.  

As far as I can see, all trees should simply be walked prior to calling
XMLFree to check for nodes referenced from ruby.  Any such nodes should
be removed from the tree, as they will be freed later when their
referencing objects are garbage-collected.  This probably also applies
to attributes but that hasn't bitten me yet.

> For SWIG I implemented support for mapping between C pointers and Ruby 
> objects by using a Hash Table (ruby's built in one, st_table).   That 
> allows you to enforce a one-to-one Ruby to C object mapping.  Do we want 
> to do something like that here?  The couple issues with it are:
> 
> * You have to hook it into every single binding method (easy in SWIG 
> since its all auto-generated)
> 
> * Ruby object ids get reused, so you have to clean out old objects from 
> the hash table as soon as they are freed.
> 
> If you have such a mapping, then a hybrid approach becomes doable as 
> long as you know absolutely for sure when a Ruby object owns a libxml 
> object and when it doesn't.  When it does, you assign a free method. 
> When it doesn't, you don't.
> 
> If we go down this approach, it might be worth using SWIG.  My only 
> problem with SWIG is that the SWIG mapping language is very complex, and 
> almost no one is going to understand it.

If it weren't so close to working already I'd probably be keen to see
such a proven approach used, even if it is tricky.  As it is, I'd prefer
to see reasonably simple bug-fixes applied.  Not my choice though - I'm
not one of the project developers so I'll butt out now :-)

> Charlie

__
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/056ef192/attachment.bin 

From cfis at savagexi.com  Thu Aug 23 14:14:38 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 23 Aug 2007 12:14:38 -0600
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187892467.4827.28.camel@bloodnok.com>
References: <1187801179.12172.13.camel@bloodnok.com>	
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>	
	<46CDB99F.9050608@savagexi.com>
	<1187887995.4827.11.camel@bloodnok.com>	
	<46CDC053.7000604@savagexi.com>
	<1187892467.4827.28.camel@bloodnok.com>
Message-ID: <46CDCE8E.7080805@savagexi.com>

> The problems that I have discovered stem from libxml freeing entire
> document trees while there are still ruby objects referencing the nodes.
> When those ruby objects are subsequently garbage-collected, the xmlNodes
> in those objects have already been freed and sadness ensues.  But I
> think the current mechanism is so close to working that it does not
> warrant a complete rewrite.

Why would a Ruby object "own" a xmlnode in a document tree?  Can that be 
avoided?

If not, can you install a callback into libxml to be alerted when a node 
is freed?  If so, then you could decrement the reference count.  But 
you'd still have to keep some long-term memory around so that any 
existing Ruby objects pointing to the node could check the reference 
count and see if it is zero. How do you know when to free that memory? 
I see no way, unless you go to the Ruby object <-> libxml object mapping 
like I described in the last email.

> As far as I can see, all trees should simply be walked prior to calling
> XMLFree to check for nodes referenced from ruby. 

Yes, that is another approach.  It would still require mapping ruby 
objects to libxml objects.  Unless you want to use object space and 
iterate over Ruby objects that way.

It would be easier to avoid the whole issue, and not have the Ruby 
objects own the libxml objects at all.  You would have an issue that a 
programmer would have to make sure not to use a Ruby object pointing to 
an invalid libxml object though.


> If it weren't so close to working already I'd probably be keen to see
> such a proven approach used, even if it is tricky.  As it is, I'd prefer
> to see reasonably simple bug-fixes applied.  Not my choice though - I'm
> not one of the project developers so I'll butt out now :-)

Your insights are much appreciated, please keep contributing your 
thoughts :)

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/e3d596cb/attachment.bin 

From cfis at savagexi.com  Thu Aug 23 14:32:44 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 23 Aug 2007 12:32:44 -0600
Subject: [libxml-devel] Python and the LibXml bindings
Message-ID: <46CDD2CC.7070608@savagexi.com>

Seems the python community has some of the same issues.

http://web.archive.org/web/20041010012053/www.dwerg.net/2004/articles/libxml
http://web.archive.org/web/20041010012802/www.dwerg.net/2004/articles/morelibxml
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/fc6a5c69/attachment-0001.bin 

From transfire at gmail.com  Thu Aug 23 14:40:11 2007
From: transfire at gmail.com (TRANS)
Date: Thu, 23 Aug 2007 11:40:11 -0700
Subject: [libxml-devel] Repo layout
Message-ID: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>

Dear Devs,

Need  to make a decision about the Repo layout. The two cases are:

  branches/
  tags/
  trunk/
    libxml/
    libxslt/

or

  libxml/
    branches/
    tags/
    trunk/
  libxslt/
    branches/
    tags/
    trunk/

If the first case, the versions of libxml and libxslt would always be
in sync (eg. libxml-0.4 and libxslt-0.4), and there would always be
both libxml and libxslt in every tag and branch. This is convenient in
that we'd always have the right version of libxml to go with libxslt,
but it means toting around libxslt when it may not be needed.

In the second case the version numbers can diverge. So it allows a
little more independence between the two libs. The downside is that
one has to keep track of which version of libxml to use for a
particular version of libxslt. Not a big deal, really, but there it is
nonetheless. Also, something to consider, for this last option, it
would be possible to move the libxslt lib over to it's own Rubyforge
project (which already exists actually). Maybe that's a better idea?

What do you all think?

T.

From marc at bloodnok.com  Thu Aug 23 14:44:36 2007
From: marc at bloodnok.com (Marc Munro)
Date: Thu, 23 Aug 2007 11:44:36 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CDCE8E.7080805@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<46CDB99F.9050608@savagexi.com> <1187887995.4827.11.camel@bloodnok.com>
	<46CDC053.7000604@savagexi.com> <1187892467.4827.28.camel@bloodnok.com>
	<46CDCE8E.7080805@savagexi.com>
Message-ID: <1187894676.4827.47.camel@bloodnok.com>

Dang!  And I really intended to butt out of this discussion :-)

On Thu, 2007-23-08 at 12:14 -0600, Charlie Savage wrote:
> > The problems that I have discovered stem from libxml freeing entire
> > document trees while there are still ruby objects referencing the nodes.
> > When those ruby objects are subsequently garbage-collected, the xmlNodes
> > in those objects have already been freed and sadness ensues.  But I
> > think the current mechanism is so close to working that it does not
> > warrant a complete rewrite.
> 
> Why would a Ruby object "own" a xmlnode in a document tree?  Can that be 
> avoided?

I don't see how it can be avoided.  My app creates xml documents by
processing other documents.  Each node in the created document is added
by first creating it as a ruby object, then adding attributes, etc, and
finally adding it to the document.

> If not, can you install a callback into libxml to be alerted when a node 
> is freed?  If so, then you could decrement the reference count.  But 
> you'd still have to keep some long-term memory around so that any 
> existing Ruby objects pointing to the node could check the reference 
> count and see if it is zero. How do you know when to free that memory? 
> I see no way, unless you go to the Ruby object <-> libxml object mapping 
> like I described in the last email.

I don't know if libxml2 provides such callbacks, but I don't see how
this would work anyway.  The xmlNode has no idea what ruby objects are
referencing it.  Instead of trying to ensure that we free the ruby
object when the xmlNode is freed, we have to ensure that such nodes
referenced by ruby objects are not freed at all by xmlFree.

> > As far as I can see, all trees should simply be walked prior to calling
> > XMLFree to check for nodes referenced from ruby. 
> 
> Yes, that is another approach.  It would still require mapping ruby 
> objects to libxml objects.  Unless you want to use object space and 
> iterate over Ruby objects that way.

I don't think it requires any more mapping than is in place now, with
each ruby object knowing the xmlNode it represents.  Beyond that the
reference counting should be enough.

Right now, each xmlNode knows how many ruby objects it is referenced
from (assuming the reference counting works).  When a ruby object is
freed, it decrements the reference count and if it is the last object
referencing the xmlNode, and that xmlNode is not referenced by parent
nodes, then the xmlNode is freed using xmlFree.  The problem is that
xmlFree will free child nodes without checking whether they are
referenced from ruby.  That's why I think we need to walk the tree and
remove such nodes prior to letting xmlFree do its stuff.

> It would be easier to avoid the whole issue, and not have the Ruby 
> objects own the libxml objects at all.  You would have an issue that a 
> programmer would have to make sure not to use a Ruby object pointing to 
> an invalid libxml object though.

I don't see how this would be possible.  My target documents are built
one node at a time from ruby objects.  I think ruby needs to own those
objects.

> > If it weren't so close to working already I'd probably be keen to see
> > such a proven approach used, even if it is tricky.  As it is, I'd prefer
> > to see reasonably simple bug-fixes applied.  Not my choice though - I'm
> > not one of the project developers so I'll butt out now :-)
> 
> Your insights are much appreciated, please keep contributing your 
> thoughts :)

Thanks.

__
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/7a9c99a5/attachment.bin 

From cfis at savagexi.com  Thu Aug 23 14:49:05 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 23 Aug 2007 12:49:05 -0600
Subject: [libxml-devel] Repo layout
In-Reply-To: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>
References: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>
Message-ID: <46CDD6A1.8060701@savagexi.com>

> In the second case the version numbers can diverge. So it allows a
> little more independence between the two libs. The downside is that
> one has to keep track of which version of libxml to use for a
> particular version of libxslt. Not a big deal, really, but there it is
> nonetheless. Also, something to consider, for this last option, it
> would be possible to move the libxslt lib over to it's own Rubyforge
> project (which already exists actually). Maybe that's a better idea?

I'd vote a separate project.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/e41c396f/attachment.bin 

From tom at compton.nu  Thu Aug 23 14:29:37 2007
From: tom at compton.nu (Tom Hughes)
Date: Thu, 23 Aug 2007 19:29:37 +0100
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CDC053.7000604@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<46CDB99F.9050608@savagexi.com> 
	<1187887995.4827.11.camel@bloodnok.com> <46CDC053.7000604@savagexi.com>
Message-ID: <e378ff164f.tom@loxley.compton.nu>

In message <46CDC053.7000604 at savagexi.com>
          Charlie Savage <cfis at savagexi.com> wrote:

> > The problem is that some nodes are allocated by libxml by parsing
> > documents, processing xsl, etc, and some directly by ruby.  I think the
> > hybrid approach is necessary in order to get the speed benefits of using
> > libxml, and I think a hybrid approach to freeing is therefore necessary.
> > It does seem to demand a bit more smarts though, from the memory freeing
> > mechanisms.
> 
> Ah - interesting.  A hybrid approach is extremely difficult because it
> means you have to keep track of the ruby to libxml object mapping.  For
> example, say you have some ruby code that create a libxml node.  That
> ruby object would then implement a free method that frees the libxml
> node.  But let's say you end up with a 2nd ruby object pointing to the
> same libxml object (which is really easy to do) - now you have a problem.

Keeping track of the mapping is exactly what I wound up doing in
my patch - just for attributes and elements I think although the
problem probably exists with other objects as well and could do
with extending.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/

From tom at compton.nu  Thu Aug 23 14:27:09 2007
From: tom at compton.nu (Tom Hughes)
Date: Thu, 23 Aug 2007 19:27:09 +0100
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187892467.4827.28.camel@bloodnok.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<46CDB99F.9050608@savagexi.com> 
	<1187887995.4827.11.camel@bloodnok.com> <46CDC053.7000604@savagexi.com>
	<1187892467.4827.28.camel@bloodnok.com>
Message-ID: <113fff164f.tom@loxley.compton.nu>

In message <1187892467.4827.28.camel at bloodnok.com>
          Marc Munro <marc at bloodnok.com> wrote:

> On Thu, 2007-23-08 at 11:13 -0600, Charlie Savage wrote:
> > Ah - interesting.  A hybrid approach is extremely difficult because it
> > means you have to keep track of the ruby to libxml object mapping.  For
> > example, say you have some ruby code that create a libxml node.  That
> > ruby object would then implement a free method that frees the libxml
> > node.  But let's say you end up with a 2nd ruby object pointing to the
> > same libxml object (which is really easy to do) - now you have a problem.
> 
> The current implementation uses a reference counting scheme within the
> xmlNode elements returned from libxml.  It seems to me that it *should*
> be possible for this to work.

Reference counting can't work, because the references held by
other libxml objects are not counted, and libxml does not respect
them so will free things that still have a non-zero reference
count.

As I recall my patch works round this by setting a callback in
libxml so that it finds out when a libxml object is being
destroyed and can duplicate it if it is still being referenced
by ruby objects.

> The problems that I have discovered stem from libxml freeing entire
> document trees while there are still ruby objects referencing the nodes.
> When those ruby objects are subsequently garbage-collected, the xmlNodes
> in those objects have already been freed and sadness ensues.  But I
> think the current mechanism is so close to working that it does not
> warrant a complete rewrite.

That is precisely the problem.

> As far as I can see, all trees should simply be walked prior to calling
> XMLFree to check for nodes referenced from ruby.  Any such nodes should
> be removed from the tree, as they will be freed later when their
> referencing objects are garbage-collected.  This probably also applies
> to attributes but that hasn't bitten me yet.

I was trying to avoid having to do anything that expensive I think
although I certainly considered it.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/

From tom at compton.nu  Thu Aug 23 14:28:39 2007
From: tom at compton.nu (Tom Hughes)
Date: Thu, 23 Aug 2007 19:28:39 +0100
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CDCE8E.7080805@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<46CDB99F.9050608@savagexi.com> 
	<1187887995.4827.11.camel@bloodnok.com> <46CDC053.7000604@savagexi.com>
	<1187892467.4827.28.camel@bloodnok.com>
	<46CDCE8E.7080805@savagexi.com>
Message-ID: <fd61ff164f.tom@loxley.compton.nu>

In message <46CDCE8E.7080805 at savagexi.com>
          Charlie Savage <cfis at savagexi.com> wrote:

> > The problems that I have discovered stem from libxml freeing entire
> > document trees while there are still ruby objects referencing the nodes.
> > When those ruby objects are subsequently garbage-collected, the xmlNodes
> > in those objects have already been freed and sadness ensues.  But I
> > think the current mechanism is so close to working that it does not
> > warrant a complete rewrite.
> 
> Why would a Ruby object "own" a xmlnode in a document tree?  Can that be
> avoided?

Simple - you create an element object in ruby, then add that element
as the child of another element. The ruby element object still exists
and still points at the same element though (and can be used to make
changes to that element in the tree).

> If not, can you install a callback into libxml to be alerted when a node
> is freed?  If so, then you could decrement the reference count.  But
> you'd still have to keep some long-term memory around so that any
> existing Ruby objects pointing to the node could check the reference
> count and see if it is zero. How do you know when to free that memory?
> I see no way, unless you go to the Ruby object <-> libxml object mapping
> like I described in the last email.

You can install callbacks, yes - that is what my patch does.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/

From cfis at savagexi.com  Thu Aug 23 14:58:38 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 23 Aug 2007 12:58:38 -0600
Subject: [libxml-devel] libxml crash
In-Reply-To: <fd61ff164f.tom@loxley.compton.nu>
References: <1187801179.12172.13.camel@bloodnok.com>	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>	<46CDB99F.9050608@savagexi.com>
	<1187887995.4827.11.camel@bloodnok.com>
	<46CDC053.7000604@savagexi.com>	<1187892467.4827.28.camel@bloodnok.com>	<46CDCE8E.7080805@savagexi.com>
	<fd61ff164f.tom@loxley.compton.nu>
Message-ID: <46CDD8DE.9070304@savagexi.com>

> Simple - you create an element object in ruby, then add that element
> as the child of another element. The ruby element object still exists
> and still points at the same element though (and can be used to make
> changes to that element in the tree).

Is this avoidable by unsetting the Ruby object's free method when it is 
added to a node (and I suppose adding it back in if you remove it from 
the tree)?  Then you'd only have the Ruby object pointing to the top 
level document having a free method (to free the libxml document).

To be really safe, you'd still want to go find all outstanding Ruby 
objects pointing to the freed libxml nodes and mark them as "invalid." 
Thus when a user tries to access one, they'd get an exception saying 
"the underlying libxml node has been freed" instead of segmentation 
fault.  For what its worth, that's how I made SWIG work....

> You can install callbacks, yes - that is what my patch does.

Ok, thanks. Sounds like your patch is the place to start.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/1e6a4359/attachment-0001.bin 

From cfis at savagexi.com  Thu Aug 23 15:04:41 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 23 Aug 2007 13:04:41 -0600
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187894676.4827.47.camel@bloodnok.com>
References: <1187801179.12172.13.camel@bloodnok.com>	
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>	
	<46CDB99F.9050608@savagexi.com>
	<1187887995.4827.11.camel@bloodnok.com>	
	<46CDC053.7000604@savagexi.com>
	<1187892467.4827.28.camel@bloodnok.com>	
	<46CDCE8E.7080805@savagexi.com>
	<1187894676.4827.47.camel@bloodnok.com>
Message-ID: <46CDDA49.8050105@savagexi.com>

> I don't know if libxml2 provides such callbacks, but I don't see how
> this would work anyway.  The xmlNode has no idea what ruby objects are
> referencing it. 

Right - you have to add in mapping between Ruby objects and libxml 
objects (its a hash table keyed on the C pointer having pointers to the 
Ruby objects as values).  Its easy to do, just tedious.

  Instead of trying to ensure that we free the ruby
> object when the xmlNode is freed, we have to ensure that such nodes
> referenced by ruby objects are not freed at all by xmlFree.

Boy - that's tough in a tree.  What if you have a reference to a leaf 
node, but no longer to the root.  How are you supposed to free the whole 
tree when you are done with the leaf?


> I don't think it requires any more mapping than is in place now, with
> each ruby object knowing the xmlNode it represents.  Beyond that the
> reference counting should be enough.

Not if libxml frees the memory from under you.  So your point of view is 
that Ruby entirely controls the memory allocation/deallocation.  But see 
my point above for why that could be tough.

> Right now, each xmlNode knows how many ruby objects it is referenced
> from (assuming the reference counting works).  When a ruby object is
> freed, it decrements the reference count and if it is the last object
> referencing the xmlNode, and that xmlNode is not referenced by parent
> nodes, then the xmlNode is freed using xmlFree.  The problem is that
> xmlFree will free child nodes without checking whether they are
> referenced from ruby.  That's why I think we need to walk the tree and
> remove such nodes prior to letting xmlFree do its stuff.

Walk what tree?  The Ruby tree (thereby creating a bunch of new ruby 
objects and unneeded references)?  The libxml c tree?  If so, you are 
back to having to know which Ruby objects point to which libxml object.

> I don't see how this would be possible.  My target documents are built
> one node at a time from ruby objects.  I think ruby needs to own those
> objects.

When you add the Ruby objects to a tree, the Ruby objects can give up 
ownership (that's simple, set the free method to nil).

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/e4aa229a/attachment.bin 

From marc at bloodnok.com  Thu Aug 23 15:37:08 2007
From: marc at bloodnok.com (Marc Munro)
Date: Thu, 23 Aug 2007 12:37:08 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CDDA49.8050105@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<46CDB99F.9050608@savagexi.com> <1187887995.4827.11.camel@bloodnok.com>
	<46CDC053.7000604@savagexi.com> <1187892467.4827.28.camel@bloodnok.com>
	<46CDCE8E.7080805@savagexi.com> <1187894676.4827.47.camel@bloodnok.com>
	<46CDDA49.8050105@savagexi.com>
Message-ID: <1187897828.4827.79.camel@bloodnok.com>

On Thu, 2007-23-08 at 13:04 -0600, Charlie Savage wrote:
> > 
>   Instead of trying to ensure that we free the ruby
> > object when the xmlNode is freed, we have to ensure that such nodes
> > referenced by ruby objects are not freed at all by xmlFree.
> 
> Boy - that's tough in a tree.  What if you have a reference to a leaf 
> node, but no longer to the root.  How are you supposed to free the whole 
> tree when you are done with the leaf?

I'm not sure why this seems tough.  If we have a reference to a leaf
when we want to garbage collect the rest of the tree, I am suggesting
removing that leaf from the tree prior to freeing it.

> > I don't think it requires any more mapping than is in place now, with
> > each ruby object knowing the xmlNode it represents.  Beyond that the
> > reference counting should be enough.
> 
> Not if libxml frees the memory from under you.  So your point of view is 
> that Ruby entirely controls the memory allocation/deallocation.  But see 
> my point above for why that could be tough.

Ruby controls which libxml trees are freed, but has no say about how.
If I understand things correctly, calling xmlFree on a node, frees that
node and all of its descendants, so removing any referenced node from
the tree before freeing it should be enough.

> > Right now, each xmlNode knows how many ruby objects it is referenced
> > from (assuming the reference counting works).  When a ruby object is
> > freed, it decrements the reference count and if it is the last object
> > referencing the xmlNode, and that xmlNode is not referenced by parent
> > nodes, then the xmlNode is freed using xmlFree.  The problem is that
> > xmlFree will free child nodes without checking whether they are
> > referenced from ruby.  That's why I think we need to walk the tree and
> > remove such nodes prior to letting xmlFree do its stuff.
> 
> Walk what tree?  The Ruby tree (thereby creating a bunch of new ruby 
> objects and unneeded references)?  The libxml c tree?  If so, you are 
> back to having to know which Ruby objects point to which libxml object.

I meant walk the libxml c tree.  I don't think we need to care *which*
ruby objects reference an xmlNode, just whether *any* ruby object does.
If any ruby object references a node, that node must not be be freed by
xmlFree.  This, of course, assumes that the ruby libxml reference
counting approach works which may not be true.

> > I don't see how this would be possible.  My target documents are built
> > one node at a time from ruby objects.  I think ruby needs to own those
> > objects.
> 
> When you add the Ruby objects to a tree, the Ruby objects can give up 
> ownership (that's simple, set the free method to nil).

That would be quite a change to the API.  As it is I can do either of
these things and both seem reasonable:

  doc = XML::Document.new 
  root = XML::node.new('node')
  doc.root = root
  root['wibble'] = 'wibble'

or

  doc = XML::Document.new
  root = XML::node.new('node')
  root['wibble'] = 'wibble'
  doc.root = root

I suspect my app does both, so I wouldn't like to see that change.

__
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/9f1509fd/attachment.bin 

From cfis at savagexi.com  Thu Aug 23 15:48:27 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 23 Aug 2007 13:48:27 -0600
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187897828.4827.79.camel@bloodnok.com>
References: <1187801179.12172.13.camel@bloodnok.com>	
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>	
	<46CDB99F.9050608@savagexi.com>
	<1187887995.4827.11.camel@bloodnok.com>	
	<46CDC053.7000604@savagexi.com>
	<1187892467.4827.28.camel@bloodnok.com>	
	<46CDCE8E.7080805@savagexi.com>
	<1187894676.4827.47.camel@bloodnok.com>	
	<46CDDA49.8050105@savagexi.com>
	<1187897828.4827.79.camel@bloodnok.com>
Message-ID: <46CDE48B.6000506@savagexi.com>

> I'm not sure why this seems tough.  If we have a reference to a leaf
> when we want to garbage collect the rest of the tree, I am suggesting
> removing that leaf from the tree prior to freeing it.

You don't view the tree as atomic?  Seems to me it is.

>> When you add the Ruby objects to a tree, the Ruby objects can give up 
>> ownership (that's simple, set the free method to nil).
> 
> That would be quite a change to the API.  As it is I can do either of
> these things and both seem reasonable:
> 
>   doc = XML::Document.new 
>   root = XML::node.new('node')
>   doc.root = root
>   root['wibble'] = 'wibble'
> 
> or
> 
>   doc = XML::Document.new
>   root = XML::node.new('node')
>   root['wibble'] = 'wibble'
>   doc.root = root
> 
> I suspect my app does both, so I wouldn't like to see that change.

I don't think your Ruby code would change at all.  Under the covers, the 
bindings would see that you are adding root to to doc, and so the Ruby 
object pointing to root would give up ownership.

You'd then setup a mark function for each node.  The mark function would 
alert the Ruby garbage collector that the doc Ruby object is still in 
use (because you have Ruby objects pointing to child nodes).

When eventually all Ruby objects pointing to nodes are no longer in use, 
Ruby's garbage collector will collect the doc object which in turn will 
free the libxml tree.

Seems nice and simple to me, avoids all reference counting, and 
leverages Ruby's mark-and-sweep garbage collector.

So am I missing something?

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/ad270725/attachment.bin 

From marc at bloodnok.com  Thu Aug 23 16:12:21 2007
From: marc at bloodnok.com (Marc Munro)
Date: Thu, 23 Aug 2007 13:12:21 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CDE48B.6000506@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<46CDB99F.9050608@savagexi.com> <1187887995.4827.11.camel@bloodnok.com>
	<46CDC053.7000604@savagexi.com> <1187892467.4827.28.camel@bloodnok.com>
	<46CDCE8E.7080805@savagexi.com> <1187894676.4827.47.camel@bloodnok.com>
	<46CDDA49.8050105@savagexi.com> <1187897828.4827.79.camel@bloodnok.com>
	<46CDE48B.6000506@savagexi.com>
Message-ID: <1187899941.4827.100.camel@bloodnok.com>

On Thu, 2007-23-08 at 13:48 -0600, Charlie Savage wrote:

> I don't think your Ruby code would change at all.  Under the covers, the 
> bindings would see that you are adding root to to doc, and so the Ruby 
> object pointing to root would give up ownership.
> 
> You'd then setup a mark function for each node.  The mark function would 
> alert the Ruby garbage collector that the doc Ruby object is still in 
> use (because you have Ruby objects pointing to child nodes).
> 
> When eventually all Ruby objects pointing to nodes are no longer in use, 
> Ruby's garbage collector will collect the doc object which in turn will 
> free the libxml tree.
> 
> Seems nice and simple to me, avoids all reference counting, and 
> leverages Ruby's mark-and-sweep garbage collector.
> 
> So am I missing something?

No, I think you're right.  The more I think about this, the more I
realise that the reference counting here is a can of worms.  It could
probably be made to work but I keep seeing more and more corner cases.

So, when can you have it ready ;-)
__
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/b9991f34/attachment.bin 

From cfis at savagexi.com  Thu Aug 23 16:20:07 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Thu, 23 Aug 2007 14:20:07 -0600
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187899941.4827.100.camel@bloodnok.com>
References: <1187801179.12172.13.camel@bloodnok.com>	
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>	
	<46CDB99F.9050608@savagexi.com>
	<1187887995.4827.11.camel@bloodnok.com>	
	<46CDC053.7000604@savagexi.com>
	<1187892467.4827.28.camel@bloodnok.com>	
	<46CDCE8E.7080805@savagexi.com>
	<1187894676.4827.47.camel@bloodnok.com>	
	<46CDDA49.8050105@savagexi.com>
	<1187897828.4827.79.camel@bloodnok.com>	
	<46CDE48B.6000506@savagexi.com>
	<1187899941.4827.100.camel@bloodnok.com>
Message-ID: <46CDEBF7.9050305@savagexi.com>

> So, when can you have it ready ;-)

Hah.  Let's what other people think - I could be way off base since I 
haven't looked in-depth at the bindings code.  Thoughts everyone?

If this idea has merit, then the trickiest bit would be making sure 
there is a one-to-one Ruby document object to libxml document object 
mapping.  Otherwise the libxml document object would be freed multiple 
times.  Or maybe you could stick to reference counting just the document 
object.

My cursory glance at the bindings also shows a lot of alloced and freed 
structures - I'd have to look to see how those fit in.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070823/250131e8/attachment-0001.bin 

From sean at chittenden.org  Thu Aug 23 14:25:55 2007
From: sean at chittenden.org (Sean Chittenden)
Date: Thu, 23 Aug 2007 11:25:55 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <yekps1epv0c.fsf@dellow.uk.cyberscience.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
Message-ID: <357A4F57-C426-49DF-B517-FD8509716E29@chittenden.org>

> My patches (to which I see you have already been pointed) are quite
> likely to help with this. They're only a band aid really though - what
> is really needed is a ground up review/redesign of the way the two
> object models are tied together as the current scheme basically
> doesn't work at all.

Having spent a great deal of time with this, I'm now of the following  
opinion of this library: when initially writing this module, I  
exposed too much of libxml to Ruby and the memory models and object/ 
tree dependencies and interactions between memory and objects is...  
problematic and should be reviewed.  $0.02.  -sc

From sean at chittenden.org  Thu Aug 23 14:28:32 2007
From: sean at chittenden.org (Sean Chittenden)
Date: Thu, 23 Aug 2007 11:28:32 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <46CDC053.7000604@savagexi.com>
References: <1187801179.12172.13.camel@bloodnok.com>	
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>	
	<46CDB99F.9050608@savagexi.com>
	<1187887995.4827.11.camel@bloodnok.com>
	<46CDC053.7000604@savagexi.com>
Message-ID: <E597678A-0100-4615-ABD8-0AE1CDAE79B5@chittenden.org>

> For SWIG I implemented support for mapping between C pointers and  
> Ruby objects by using a Hash Table (ruby's built in one,  
> st_table).   That allows you to enforce a one-to-one Ruby to C  
> object mapping.  Do we want to do something like that here?

It doesn't exist now and I'm inclined to say that adding it would be  
a bad idea.  Sadly, I think the library needs to be more aggressive  
about dup'ing data.

-sc

PS  This is one of the primary reasons I use D as a programming  
language these days instead of Ruby.  <:~)

From tom at compton.nu  Thu Aug 23 17:26:19 2007
From: tom at compton.nu (Tom Hughes)
Date: Thu, 23 Aug 2007 22:26:19 +0100
Subject: [libxml-devel] libxml crash
In-Reply-To: <1187894676.4827.47.camel@bloodnok.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<46CDB99F.9050608@savagexi.com> 
	<1187887995.4827.11.camel@bloodnok.com> <46CDC053.7000604@savagexi.com>
	<1187892467.4827.28.camel@bloodnok.com>
	<46CDCE8E.7080805@savagexi.com> 
	<1187894676.4827.47.camel@bloodnok.com>
Message-ID: <31a60f174f.tom@loxley.compton.nu>

In message <1187894676.4827.47.camel at bloodnok.com>
          Marc Munro <marc at bloodnok.com> wrote:

> On Thu, 2007-23-08 at 12:14 -0600, Charlie Savage wrote:
> 
> > If not, can you install a callback into libxml to be alerted when a node
> > is freed?  If so, then you could decrement the reference count.  But
> > you'd still have to keep some long-term memory around so that any
> > existing Ruby objects pointing to the node could check the reference
> > count and see if it is zero. How do you know when to free that memory?
> > I see no way, unless you go to the Ruby object <-> libxml object mapping
> > like I described in the last email.
> 
> I don't know if libxml2 provides such callbacks, but I don't see how
> this would work anyway.  The xmlNode has no idea what ruby objects are
> referencing it.  Instead of trying to ensure that we free the ruby
> object when the xmlNode is freed, we have to ensure that such nodes
> referenced by ruby objects are not freed at all by xmlFree.

The way I did it is to have the ruby object point at a structure
belonging to the wrapper which in turn points to the real libxml
object. That intermediate structure also has a reference count
which records how many ruby objects (but not libxml objects) point
at it.

In addition the _private member of the libxml object is set to
point back at the intermediate structure so that when libxml calls
the callback you can find that additional information and see if
any ruby objects still reference it.

If a ruby object does still reference it then the libxml object
is cloned (as we can't stop libxml deleting the one it has called
us back about) and the intermediate structure is changed to point
at that clone instead of the original copy that is about the be
deleted.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/

From danj at 3skel.com  Thu Aug 23 19:27:52 2007
From: danj at 3skel.com (Dan Janowski)
Date: Thu, 23 Aug 2007 19:27:52 -0400
Subject: [libxml-devel] Python and the LibXml bindings
In-Reply-To: <46CDD2CC.7070608@savagexi.com>
References: <46CDD2CC.7070608@savagexi.com>
Message-ID: <B5BF50A7-E2EA-4AD2-B2E4-0F4CBC577370@3skel.com>

This is good foreign intelligence. In your searching, did you find  
anything to indicate that the py interface had been changed to cope  
with these issues from 2004?

Dan

On Aug 23, 2007, at 14:32, Charlie Savage wrote:

> Seems the python community has some of the same issues.
>
> http://web.archive.org/web/20041010012053/www.dwerg.net/2004/ 
> articles/libxml
> http://web.archive.org/web/20041010012802/www.dwerg.net/2004/ 
> articles/morelibxml
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel


From danj at 3skel.com  Fri Aug 24 11:47:58 2007
From: danj at 3skel.com (Dan Janowski)
Date: Fri, 24 Aug 2007 11:47:58 -0400
Subject: [libxml-devel] libxml crash
In-Reply-To: <357A4F57-C426-49DF-B517-FD8509716E29@chittenden.org>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<357A4F57-C426-49DF-B517-FD8509716E29@chittenden.org>
Message-ID: <0D0D57E8-20C0-4CD4-889F-7886704BE7F0@3skel.com>

My analysis leads me to a similar conclusion. Reference counting is  
out and there is no need for it. This is my hypothesis:

If things are being freed unintentionally, then there are not enough  
ruby to ruby cross-references to maintain integrity during mark and  
sweep. A single ruby object holding on to an xmlNode should inhibit  
collection of an entire tree. Individual ruby xml nodes can be  
reaped, but there should be no xmlFree done by anything but the  
document level object. As for an xmlNode that is not associated with  
a document, it must call xmlFree (the xmlNode already indicates if it  
is associated with a document) if it has no parent. Which effectively  
translates into the same document semantics.

One operational change should be once an xmlNode is associated with a  
document it cannot be associated with another one (at least using the  
implicit copy mechanism provided in ruby_xml_node_child_set). The  
ruby xmlNode becomes changed and reflects its association with the  
document. However, using xmlCopyNode, it is possible to get a copy of  
the node without an association to the document and also libxml2  
handles propagation of namespaces. This new copy preserves the  
document based free condition above, and also makes it easy to put it  
into another document. Not sure if this should be automatic (thinking  
not) or an overload of .dup.

This should establish a bright line between libxml2 and ruby for  
managing memory.

This is from only a few hours looking at the docs and source code for  
libxml2, the python interface and the ruby interface.  If these seem  
to ignore some known pitfall, please let me know. If not, this should  
fix this scope of the memory problem.

I intend to do this within a week. Post your warnings or wish me luck.

Dan


On Aug 23, 2007, at 14:25, Sean Chittenden wrote:

> Having spent a great deal of time with this, I'm now of the following
> opinion of this library: when initially writing this module, I
> exposed too much of libxml to Ruby and the memory models and object/
> tree dependencies and interactions between memory and objects is...
> problematic and should be reviewed.  $0.02.  -sc

From transfire at gmail.com  Fri Aug 24 18:39:36 2007
From: transfire at gmail.com (TRANS)
Date: Fri, 24 Aug 2007 15:39:36 -0700
Subject: [libxml-devel] Repo layout
In-Reply-To: <46CDD6A1.8060701@savagexi.com>
References: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>
	<46CDD6A1.8060701@savagexi.com>
Message-ID: <4b6f054f0708241539s59770d36yb07a085ae25c9de@mail.gmail.com>

On 8/23/07, Charlie Savage <cfis at savagexi.com> wrote:
> > In the second case the version numbers can diverge. So it allows a
> > little more independence between the two libs. The downside is that
> > one has to keep track of which version of libxml to use for a
> > particular version of libxslt. Not a big deal, really, but there it is
> > nonetheless. Also, something to consider, for this last option, it
> > would be possible to move the libxslt lib over to it's own Rubyforge
> > project (which already exists actually). Maybe that's a better idea?
>
> I'd vote a separate project.

Sean told me he was indifferent, but half-suggested that we might
merge the two libs completely. I'm not qualified enough to make that
call. But Charlie's makes at least one vote for the separate
project/repo. So unless any of you have anything else to add, I'm
leaning toward that approach. Let me know as soon as you can. I'd like
to get this taken care of over the weekend.

Thanks,
T.

From danj at 3skel.com  Fri Aug 24 21:29:03 2007
From: danj at 3skel.com (Dan Janowski)
Date: Fri, 24 Aug 2007 21:29:03 -0400
Subject: [libxml-devel] Repo layout
In-Reply-To: <4b6f054f0708241539s59770d36yb07a085ae25c9de@mail.gmail.com>
References: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>
	<46CDD6A1.8060701@savagexi.com>
	<4b6f054f0708241539s59770d36yb07a085ae25c9de@mail.gmail.com>
Message-ID: <28211B0A-67EE-4517-981E-5F7E7D068D71@3skel.com>

If they are to be separate units, I vote for separate modules. A  
common virtual release tag can be applied to both to deal with the  
synchronization issue.

Dan

On Aug 24, 2007, at 18:39, TRANS wrote:

> On 8/23/07, Charlie Savage <cfis at savagexi.com> wrote:
>>> In the second case the version numbers can diverge. So it allows a
>>> little more independence between the two libs. The downside is that
>>> one has to keep track of which version of libxml to use for a
>>> particular version of libxslt. Not a big deal, really, but there  
>>> it is
>>> nonetheless. Also, something to consider, for this last option, it
>>> would be possible to move the libxslt lib over to it's own Rubyforge
>>> project (which already exists actually). Maybe that's a better idea?
>>
>> I'd vote a separate project.
>
> Sean told me he was indifferent, but half-suggested that we might
> merge the two libs completely. I'm not qualified enough to make that
> call. But Charlie's makes at least one vote for the separate
> project/repo. So unless any of you have anything else to add, I'm
> leaning toward that approach. Let me know as soon as you can. I'd like
> to get this taken care of over the weekend.
>
> Thanks,
> T.


From rosco at roscopeco.co.uk  Sat Aug 25 04:57:40 2007
From: rosco at roscopeco.co.uk (Ross Bamford)
Date: Sat, 25 Aug 2007 09:57:40 +0100
Subject: [libxml-devel] libxml crash
In-Reply-To: <0D0D57E8-20C0-4CD4-889F-7886704BE7F0@3skel.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<357A4F57-C426-49DF-B517-FD8509716E29@chittenden.org>
	<0D0D57E8-20C0-4CD4-889F-7886704BE7F0@3skel.com>
Message-ID: <op.txlxuecmkc9f7k@localhost.localdomain>

On Fri, 24 Aug 2007 16:47:58 +0100, Dan Janowski <danj at 3skel.com> wrote:

> If things are being freed unintentionally, then there are not enough
> ruby to ruby cross-references to maintain integrity during mark and
> sweep. A single ruby object holding on to an xmlNode should inhibit
> collection of an entire tree. Individual ruby xml nodes can be
> reaped, but there should be no xmlFree done by anything but the
> document level object. As for an xmlNode that is not associated with
> a document, it must call xmlFree (the xmlNode already indicates if it
> is associated with a document) if it has no parent. Which effectively
> translates into the same document semantics.

Sounds good to me. I take it you'll also mark the ruby-side document  
object from the node's mark function?

>
> One operational change should be once an xmlNode is associated with a
> document it cannot be associated with another one (at least using the
> implicit copy mechanism provided in ruby_xml_node_child_set). The
> ruby xmlNode becomes changed and reflects its association with the
> document. However, using xmlCopyNode, it is possible to get a copy of
> the node without an association to the document and also libxml2
> handles propagation of namespaces. This new copy preserves the
> document based free condition above, and also makes it easy to put it
> into another document. Not sure if this should be automatic (thinking
> not) or an overload of .dup.
>

I agree with Sean that we're going to have to be more willing to duplicate  
libxml objects within the binding. The whole reference counting thing was  
aimed really at avoiding copying, with the rationale that the binding  
should be as fast and low-footprint as possible. However, with the  
problems it's caused I'm now of the opinion that we should remove  
reference counting, make copies everywhere it's even remotely necessary,  
and then when the damn thing works look again at optimizing out some of  
that copying.

> This should establish a bright line between libxml2 and ruby for
> managing memory.
>
> This is from only a few hours looking at the docs and source code for
> libxml2, the python interface and the ruby interface.  If these seem
> to ignore some known pitfall, please let me know. If not, this should
> fix this scope of the memory problem.
>
> I intend to do this within a week. Post your warnings or wish me luck.
>

Good luck!


> Dan
>
>
> On Aug 23, 2007, at 14:25, Sean Chittenden wrote:
>
>> Having spent a great deal of time with this, I'm now of the following
>> opinion of this library: when initially writing this module, I
>> exposed too much of libxml to Ruby and the memory models and object/
>> tree dependencies and interactions between memory and objects is...
>> problematic and should be reviewed.  $0.02.  -sc
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel
>

-- 
Ross Bamford - rosco at roscopeco.co.uk

From rosco at roscopeco.co.uk  Sat Aug 25 05:07:18 2007
From: rosco at roscopeco.co.uk (Ross Bamford)
Date: Sat, 25 Aug 2007 10:07:18 +0100
Subject: [libxml-devel] Repo layout
In-Reply-To: <4b6f054f0708241539s59770d36yb07a085ae25c9de@mail.gmail.com>
References: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>
	<46CDD6A1.8060701@savagexi.com>
	<4b6f054f0708241539s59770d36yb07a085ae25c9de@mail.gmail.com>
Message-ID: <op.txlyag06kc9f7k@localhost.localdomain>

On Fri, 24 Aug 2007 23:39:36 +0100, TRANS <transfire at gmail.com> wrote:

> On 8/23/07, Charlie Savage <cfis at savagexi.com> wrote:
>> > In the second case the version numbers can diverge. So it allows a
>> > little more independence between the two libs. The downside is that
>> > one has to keep track of which version of libxml to use for a
>> > particular version of libxslt. Not a big deal, really, but there it is
>> > nonetheless. Also, something to consider, for this last option, it
>> > would be possible to move the libxslt lib over to it's own Rubyforge
>> > project (which already exists actually). Maybe that's a better idea?
>>
>> I'd vote a separate project.
>
> Sean told me he was indifferent, but half-suggested that we might
> merge the two libs completely. I'm not qualified enough to make that
> call. But Charlie's makes at least one vote for the separate
> project/repo. So unless any of you have anything else to add, I'm
> leaning toward that approach. Let me know as soon as you can. I'd like
> to get this taken care of over the weekend.

The only problem I can see with a separate project is that we're then  
introducing C dependencies between two totally separate ruby binding  
projects - this seems to me to be bad, but for no reason that I can put my  
finger on right now. Completely merging the two is interesting, and really  
only depends on how common it is to have libxml2 installed, but not  
libxslt.

Although even if is quite common, extconf could check for libxslt,  
defining a HAVE_LIBXSLT or whatever. We can then selectively ignore the  
xslt stuff when compiling the c (making libxslt an optional dependency,  
obviously).

-- 
Ross Bamford - rosco at roscopeco.co.uk

From cfis at savagexi.com  Sat Aug 25 05:15:51 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Sat, 25 Aug 2007 03:15:51 -0600
Subject: [libxml-devel] libxml crash
In-Reply-To: <0D0D57E8-20C0-4CD4-889F-7886704BE7F0@3skel.com>
References: <1187801179.12172.13.camel@bloodnok.com>	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>	<357A4F57-C426-49DF-B517-FD8509716E29@chittenden.org>
	<0D0D57E8-20C0-4CD4-889F-7886704BE7F0@3skel.com>
Message-ID: <46CFF347.2010103@savagexi.com>

 > A single ruby object holding on to an xmlNode should inhibit
> collection of an entire tree. Individual ruby xml nodes can be  
> reaped, but there should be no xmlFree done by anything but the  
> document level object. As for an xmlNode that is not associated with  
> a document, it must call xmlFree (the xmlNode already indicates if it  
> is associated with a document) if it has no parent. Which effectively  
> translates into the same document semantics.

Agreed.

I theorized this can be done by making a Ruby object give up ownership 
of its node when that node is added to a document (by setting the Ruby 
object's free method to null).  Then during the mark phase the Ruby 
object should note it is dependent on the top level Ruby document object by:

ruby node -> xmlnode -> xmldocument -> ruby document.

This means caching a back reference to the ruby document from the xml 
document in its private field.

If that all works, its should neatly solve the memory issues.

> One operational change should be once an xmlNode is associated with a  
> document it cannot be associated with another one (at least using the  
> implicit copy mechanism provided in ruby_xml_node_child_set). 

Do you mean a node can't be moved from one document to another (I can't 
see why not) or do you mean a node can't be in 2 documents at once 
(which makes perfect sense).

> This should establish a bright line between libxml2 and ruby for  
> managing memory.

Which is that ruby document objects hold the keys to the kingdom - once 
they go out of scope they free their associated xmldocuments which in 
turn free all its child nodes.

> I intend to do this within a week. Post your warnings or wish me luck.

Good luck.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070825/eb26ac40/attachment.bin 

From transfire at gmail.com  Sat Aug 25 08:01:38 2007
From: transfire at gmail.com (TRANS)
Date: Sat, 25 Aug 2007 05:01:38 -0700
Subject: [libxml-devel] Repo layout
In-Reply-To: <op.txlyag06kc9f7k@localhost.localdomain>
References: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>
	<46CDD6A1.8060701@savagexi.com>
	<4b6f054f0708241539s59770d36yb07a085ae25c9de@mail.gmail.com>
	<op.txlyag06kc9f7k@localhost.localdomain>
Message-ID: <4b6f054f0708250501s489f4695jc4f8f4ce8360a5fc@mail.gmail.com>

On 8/25/07, Ross Bamford <rosco at roscopeco.co.uk> wrote:
> On Fri, 24 Aug 2007 23:39:36 +0100, TRANS <transfire at gmail.com> wrote:
>
> > On 8/23/07, Charlie Savage <cfis at savagexi.com> wrote:
> >> > In the second case the version numbers can diverge. So it allows a
> >> > little more independence between the two libs. The downside is that
> >> > one has to keep track of which version of libxml to use for a
> >> > particular version of libxslt. Not a big deal, really, but there it is
> >> > nonetheless. Also, something to consider, for this last option, it
> >> > would be possible to move the libxslt lib over to it's own Rubyforge
> >> > project (which already exists actually). Maybe that's a better idea?
> >>
> >> I'd vote a separate project.
> >
> > Sean told me he was indifferent, but half-suggested that we might
> > merge the two libs completely. I'm not qualified enough to make that
> > call. But Charlie's makes at least one vote for the separate
> > project/repo. So unless any of you have anything else to add, I'm
> > leaning toward that approach. Let me know as soon as you can. I'd like
> > to get this taken care of over the weekend.
>
> The only problem I can see with a separate project is that we're then
> introducing C dependencies between two totally separate ruby binding
> projects - this seems to me to be bad, but for no reason that I can put my
> finger on right now.

I'm not sure it's any different than the current arrangement. We still
have two packages. So for the end-user there is a dependency already.
Or am I missing your point?

> Completely merging the two is interesting, and really
> only depends on how common it is to have libxml2 installed, but not
> libxslt.
>
> Although even if is quite common, extconf could check for libxslt,
> defining a HAVE_LIBXSLT or whatever. We can then selectively ignore the
> xslt stuff when compiling the c (making libxslt an optional dependency,
> obviously).

Considering libxslt is a whole 6 files of code and maybe 10 files for
tests, it certainly wouldn't get in the way, so to speak.

Anyone see a reason we ought not to merge them?

T.

From sean at chittenden.org  Sat Aug 25 10:59:48 2007
From: sean at chittenden.org (Sean Chittenden)
Date: Sat, 25 Aug 2007 07:59:48 -0700
Subject: [libxml-devel] libxml crash
In-Reply-To: <0D0D57E8-20C0-4CD4-889F-7886704BE7F0@3skel.com>
References: <1187801179.12172.13.camel@bloodnok.com>
	<yekps1epv0c.fsf@dellow.uk.cyberscience.com>
	<357A4F57-C426-49DF-B517-FD8509716E29@chittenden.org>
	<0D0D57E8-20C0-4CD4-889F-7886704BE7F0@3skel.com>
Message-ID: <1B0AFAF9-D4DF-4B59-A74F-0A59FAA20EA8@chittenden.org>

> If things are being freed unintentionally, then there are not enough
> ruby to ruby cross-references to maintain integrity during mark and
> sweep.

I actually think this could be done, but an external hash table of  
ptr's of both ruby and libxml obj's must be setup/maintained and it's  
problematic and I'd strongly avoiding doing so at this time.  At one  
point in time I started down this path and I caved under the  
complexity of managing references for two different dependency models.

> However, using xmlCopyNode, it is possible to get a copy of
> the node without an association to the document and also libxml2
> handles propagation of namespaces.

Exactly.  I went to great lengths initially to avoid this and am now  
of the opinion that copy'ing can't be practically avoided.  Setting a  
flag stating that a node is copied may not be a bad idea.

0) Create a series of hash tables:
   a) rubyCopiedNodes[ruby_node_obj's] = xmlNodeObjs
   b) rubyCopiedDocs[ruby_node_obj's] = xmlDocObjs
   c) libxmlCopiedNodes[xmlNodeObj] = ruby_node_obj;
   d) libxmlCopiedDocs[xmlDocObj] = ruby_doc_obj;
1) Make a copy of a node/doc
2) Make an entry in the above hash table for reference back to the  
original libxml obj
3) Make an entry in the above hash table for reference back to the  
original ruby obj
4) All node lookups that have the copy flag set resolve dependencies  
through the above hash table if the above returns something not  
null.  If null, returns null.

On deallocation of the object, remove the hash table entry.  Doing  
this only on copy is pretty easy to maintain and would give us a huge  
swatch of the originally intended/desired functionality.

> I intend to do this within a week. Post your warnings or wish me luck.

Warnings/suggestions posted above, but your plan is as good of one as  
exists - good luck!  -sc

--
Sean Chittenden
sean at chittenden.org



From transfire at gmail.com  Mon Aug 27 14:31:24 2007
From: transfire at gmail.com (TRANS)
Date: Mon, 27 Aug 2007 11:31:24 -0700
Subject: [libxml-devel] Repo layout
In-Reply-To: <4b6f054f0708250501s489f4695jc4f8f4ce8360a5fc@mail.gmail.com>
References: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>
	<46CDD6A1.8060701@savagexi.com>
	<4b6f054f0708241539s59770d36yb07a085ae25c9de@mail.gmail.com>
	<op.txlyag06kc9f7k@localhost.localdomain>
	<4b6f054f0708250501s489f4695jc4f8f4ce8360a5fc@mail.gmail.com>
Message-ID: <4b6f054f0708271131k3a1935abl2576e8f9f9628258@mail.gmail.com>

On 8/25/07, TRANS <transfire at gmail.com> wrote:

> Anyone see a reason we ought not to merge them?

Gave this a little more thought. I think it should be a separate
project after all, b/c it occurs to that the libxslt bindings also
require that the LibXSLT library be installed. But, of course, we
don't need that to use LibXML. I fear we might run into some issues
with distribution where the install/test process moans about not
finding libxslt libs, even though the end-user only cares about
libxml.

Does that make sense?

T.

From keith at oreilly.com  Mon Aug 27 15:20:52 2007
From: keith at oreilly.com (Keith Fahlgren)
Date: Mon, 27 Aug 2007 15:20:52 -0400
Subject: [libxml-devel] Repo layout
In-Reply-To: <op.txlyag06kc9f7k@localhost.localdomain>
References: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>	<46CDD6A1.8060701@savagexi.com>	<4b6f054f0708241539s59770d36yb07a085ae25c9de@mail.gmail.com>
	<op.txlyag06kc9f7k@localhost.localdomain>
Message-ID: <46D32414.5060105@oreilly.com>

On 8/25/07 5:07 AM, Ross Bamford wrote:
> Completely merging the two is interesting, and really  
> only depends on how common it is to have libxml2 installed, but not  
> libxslt.

While I certainly haven't run into _that_ many boxen, I've never seen one that
hand libxml and did _not_ have libxslt as well.



Keith

From cfis at savagexi.com  Mon Aug 27 16:36:14 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Mon, 27 Aug 2007 14:36:14 -0600
Subject: [libxml-devel] Repo layout
In-Reply-To: <46D32414.5060105@oreilly.com>
References: <4b6f054f0708231140q2477adabh82a4fb06b1c375f2@mail.gmail.com>	<46CDD6A1.8060701@savagexi.com>	<4b6f054f0708241539s59770d36yb07a085ae25c9de@mail.gmail.com>	<op.txlyag06kc9f7k@localhost.localdomain>
	<46D32414.5060105@oreilly.com>
Message-ID: <46D335BE.6020504@savagexi.com>

> On 8/25/07 5:07 AM, Ross Bamford wrote:
>> Completely merging the two is interesting, and really  
>> only depends on how common it is to have libxml2 installed, but not  
>> libxslt.

Windows....

I can build libxml and the ruby bindings, but totally failed last time I 
tried libxslt and the ruby bindings.

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070827/478ce982/attachment.bin 

From danj at 3skel.com  Tue Aug 28 20:36:00 2007
From: danj at 3skel.com (Dan Janowski)
Date: Tue, 28 Aug 2007 20:36:00 -0400
Subject: [libxml-devel] Memory changes, tests needed
Message-ID: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>

Hi all,

I am going to go out on a limb and say, tentatively, that my new  
approach to memory management is working. It is so far only applied  
to the ruby side of tree.c functions (ruby_xml_node...). My simple  
example of creating the same document 1M times, loosing references to  
the priors and watching the mark and free run, is at least no worse.

Since I have not propagated the changes to other parts, the test  
suite is broken. I suspect that they did not expose the memory  
problems anyway.

What I need now is a few concise examples that have blown up  
previously. I am particularly looking for ones that just use the node  
operations.

Looking forward to your dastardly tests.

Dan


From cfis at savagexi.com  Tue Aug 28 22:03:12 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Tue, 28 Aug 2007 20:03:12 -0600
Subject: [libxml-devel] Memory changes, tests needed
In-Reply-To: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>
References: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>
Message-ID: <46D4D3E0.8020201@savagexi.com>

Sounds like great work Dan.  Since this is so tricky, can you be sure to 
write up what you've done?  Maybe add it in as a Readme file in the 
source tree, or perhaps directly as comments in the code?

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070828/07f3edf5/attachment.bin 

From marc at bloodnok.com  Tue Aug 28 22:13:49 2007
From: marc at bloodnok.com (Marc Munro)
Date: Tue, 28 Aug 2007 19:13:49 -0700
Subject: [libxml-devel] Memory changes, tests needed
In-Reply-To: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>
References: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>
Message-ID: <1188353629.7451.19.camel@bloodnok.com>

Dan,
Here is something, not strictly memory related, that doesn't work right
now, that probably should:

class XML::Node
  attr_reader :extra
  attr_writer :extra
end

parent = XML::Node.new('parent')
child = XML::Node.new('child')
parent.extra = 'wibble'

parent.child = child

puts parent.extra         # There it is
puts child.parent.extra	  # Huh, where'd it go?

I discovered this a couple of days ago.  My app extends XML::Node quite
significantly so this is of some concern to me.

Also, if you'd like to send me your latest code, I'd be pleased to run
it aginst my app as a sanity check.  I'm afraid that I'm not prepared to
put my app into the wild yet as the quality of the code is still too
embarassing.

__
Marc


On Tue, 2007-28-08 at 20:36 -0400, Dan Janowski wrote:
> Hi all,
> 
> I am going to go out on a limb and say, tentatively, that my new  
> approach to memory management is working. It is so far only applied  
> to the ruby side of tree.c functions (ruby_xml_node...). My simple  
> example of creating the same document 1M times, loosing references to  
> the priors and watching the mark and free run, is at least no worse.
> 
> Since I have not propagated the changes to other parts, the test  
> suite is broken. I suspect that they did not expose the memory  
> problems anyway.
> 
> What I need now is a few concise examples that have blown up  
> previously. I am particularly looking for ones that just use the node  
> operations.
> 
> Looking forward to your dastardly tests.
> 
> Dan
> 
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070828/53f464d7/attachment.bin 

From tom at compton.nu  Wed Aug 29 04:01:04 2007
From: tom at compton.nu (Tom Hughes)
Date: Wed, 29 Aug 2007 09:01:04 +0100
Subject: [libxml-devel] Memory changes, tests needed
In-Reply-To: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com> (Dan Janowski's
	message of "Tue\, 28 Aug 2007 20\:36\:00 -0400")
References: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>
Message-ID: <yekk5rekbwf.fsf@dellow.uk.cyberscience.com>

An embedded and charset-unspecified text was scrubbed...
Name: xmltest.rb
Url: http://rubyforge.org/pipermail/libxml-devel/attachments/20070829/43a61157/attachment.pl 

From danj at 3skel.com  Wed Aug 29 13:26:22 2007
From: danj at 3skel.com (Dan Janowski)
Date: Wed, 29 Aug 2007 13:26:22 -0400
Subject: [libxml-devel] Memory changes, tests needed
In-Reply-To: <yekk5rekbwf.fsf@dellow.uk.cyberscience.com>
References: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>
	<yekk5rekbwf.fsf@dellow.uk.cyberscience.com>
Message-ID: <E0C7D535-B9B4-4D8D-891C-BD7F1F4CFF6F@3skel.com>

Tom,

This is now memory stable. The script completes using only 15MB.

On Aug 29, 2007, at 04:01, Tom Hughes wrote:

> In message <7A5C5895-2A58-47AD-B548-A55561B42194 at 3skel.com>
>         Dan Janowski <danj at 3skel.com> wrote:
>
>> What I need now is a few concise examples that have blown up
>> previously. I am particularly looking for ones that just use the node
>> operations.
>
> Attached is roughly the test case I was using when I was working on
> this before - it broadly mirrors one things that the OpenStreetMap
> site does when answering requests.
>
> Basically it creates 100 GPX files, each with 10000 points in - in
> real life each one would be streamed back to the client but in this
> test we just go on to the next one.
>
> With libxml-ruby 0.3.8.4 this grows rapidly to a resident size of
> about 650Mb to 750Mb (total virtual size is about 50Mb more than
> that) and with my patches it is stable at a resident size of
> about 35Mb (about 95Mb total).
>
> Tom
>
> -- 
> Tom Hughes (tom at compton.nu)
> http://www.compton.nu/
>
>
> require 'rubygems'
> require 'xml/libxml'
>
> 100.times do
>   doc = XML::Document.new
>   doc.encoding = 'UTF-8'
>
>   root = XML::Node.new 'gpx'
>   root['version'] = '1.0'
>   root['creator'] = 'OpenStreetMap.org'
>   root['xmlns'] = "http://www.topografix.com/GPX/1/0/"
>
>   doc.root = root
>
>   track = XML::Node.new 'trk'
>   doc.root << track
>
>   trkseg = XML::Node.new 'trkseg'
>   track << trkseg
>
>   1.upto(10000) do |n|
>     trkpt = XML::Node.new 'trkpt'
>     trkpt['lat'] = n.to_s
>     trkpt['lon'] = n.to_s
>     trkseg << trkpt
>   end
> end


From danj at 3skel.com  Wed Aug 29 14:21:59 2007
From: danj at 3skel.com (Dan Janowski)
Date: Wed, 29 Aug 2007 14:21:59 -0400
Subject: [libxml-devel] MEM2 for viewing
Message-ID: <085D56C7-A7E5-4E32-B237-917FD0FED209@3skel.com>

Hi all,

I have just committed the current state of my memory overhaul to a  
branch MEM2. It is not complete, but it includes rewritten node,  
document and attr. I also removed attribute. It does pass the tests  
run during rake, so it may be good to try out generally, but I make  
no guarantees. Memory problems will still be present in modules I  
have not modified, but the creation rate in those is much lower, so  
the risk is far less.

I encourage you to check it out and see. Here are some steps:

1. svn co http://libxml.rubyforge.org/svn/branches/MEM2
2. cd MEM2
3. rake

(run the node exercises)

4. cd ext
5. ruby ../rwtest/gpx.rb
6. ruby ../rwtest/tsr.rb

I do not have ruby libxml installed in my ruby either as a gem or  
not, so you may want to disable gem (if you use the env var). gpx  
runs consuming no more than ~15M and tsr ~8M on my system.

Since it passes the build tests, you may be able to try it with your  
apps, in a development sandbox.

Please post feedback, if you test, what you test, how it went, etc.  
so I know if it is getting some eval.

Dan

From tom at compton.nu  Wed Aug 29 14:13:44 2007
From: tom at compton.nu (Tom Hughes)
Date: Wed, 29 Aug 2007 19:13:44 +0100
Subject: [libxml-devel] Memory changes, tests needed
In-Reply-To: <E0C7D535-B9B4-4D8D-891C-BD7F1F4CFF6F@3skel.com>
References: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>
	<yekk5rekbwf.fsf@dellow.uk.cyberscience.com> 
	<E0C7D535-B9B4-4D8D-891C-BD7F1F4CFF6F@3skel.com>
Message-ID: <6408151a4f.tom@loxley.compton.nu>

In message <E0C7D535-B9B4-4D8D-891C-BD7F1F4CFF6F at 3skel.com>
          Dan Janowski <danj at 3skel.com> wrote:

> This is now memory stable. The script completes using only 15MB.

Excellent.

Incidentally, now people are actively working on the wrapper again
you might want to look at my other patch which is a simple one that
fixes a leak when converting a DOM tree to text.

The patch for that is the one at:

  http://trac.openstreetmap.org/attachment/ticket/482/libxml-leak2.patch

That leak has also been raised on the bug tracker:

  http://rubyforge.org/tracker/index.php?func=detail&aid=7945&group_id=494&atid=1971

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/

From marc at bloodnok.com  Wed Aug 29 14:30:49 2007
From: marc at bloodnok.com (Marc Munro)
Date: Wed, 29 Aug 2007 11:30:49 -0700
Subject: [libxml-devel] MEM2 for viewing
In-Reply-To: <085D56C7-A7E5-4E32-B237-917FD0FED209@3skel.com>
References: <085D56C7-A7E5-4E32-B237-917FD0FED209@3skel.com>
Message-ID: <1188412249.16177.5.camel@bloodnok.com>

Dan,
Just tried to build, per your instructions.  For me it fails some of the
self tests:
-----------------------------------------------
. . .
Loaded suite /usr/lib/ruby/1.8/rake/rake_test_loader
Started
...........................F.F..............................................
EXPECTING: TWO ERRORS:
Entity: line 1: parser error : Opening and ending tag mismatch: foo line
1 and foz
<foo><bar/></foz>
                 ^
Entity: line 1: parser error : Opening and ending tag mismatch: foo line
1 and foz
<foo><bar/></foz>
                 ^
...........................
Finished in 0.191201 seconds.

  1) Failure:
test_xml_node_eql?(TC_XML_Node4)
[/home/marc/libxml/MEM2/tests/tc_xml_node4.rb:25]:
<-606539058> expected to be != to
<-606539058>.

  2) Failure:
test_xml_node_hash(TC_XML_Node4)
[/home/marc/libxml/MEM2/tests/tc_xml_node4.rb:36]:
<-606626528> expected to be != to
<-606626528>.

103 tests, 705 assertions, 2 failures, 0 errors
rake aborted!
Command failed with status (1): [/usr/bin/ruby1.8 -Ilib
"/usr/lib/ruby/1.8/...]

(See full trace by running task with --trace)
-----------------------------------------------

Let me know if you need more info/diagnostics, etc.


__
Marc


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070829/c2399759/attachment.bin 

From cfis at savagexi.com  Wed Aug 29 14:31:12 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Wed, 29 Aug 2007 12:31:12 -0600
Subject: [libxml-devel] MEM2 for viewing
In-Reply-To: <085D56C7-A7E5-4E32-B237-917FD0FED209@3skel.com>
References: <085D56C7-A7E5-4E32-B237-917FD0FED209@3skel.com>
Message-ID: <46D5BB70.2050902@savagexi.com>

Dan,

If you have a second, could you fix the free/xfee issue (or I can do it 
once you are done, I just don't want to interfere with what you are 
doing).

The problem is that various structures in libxml are allocated with the 
macro ALLOC, which means the ruby executable is creating the memory. 
But the structures are then deallocated using free, which means the dll 
is trying to delete them.  On windows this doesn't work since each dll 
likely has its own stack (depending on the runtime library it is using).

The solution is simple - use xfree (which is what you are supposed to 
use anyway for memory created via ALLOCA) and not free.  There are about 
10 to 15 instances of this in the current code base.

Thanks,

Charlie


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070829/17ff2bb5/attachment-0001.bin 

From transfire at gmail.com  Wed Aug 29 14:33:01 2007
From: transfire at gmail.com (TRANS)
Date: Wed, 29 Aug 2007 11:33:01 -0700
Subject: [libxml-devel] Finishing Repo Reorganization
Message-ID: <4b6f054f0708291133s36bfcd10ra97fa5f5461e74a2@mail.gmail.com>

Hi--

Haven't heard from Ross on the matter, I guess he's busy. So I'm just
going to go ahead and move the libxslt binding code over to the libxsl
project (actually an up-to-date copy is already there). If we later
confirm that it is a good idea to merge the two libs we can do so in
the future. In the meantime we'll just play it safe.

With that out of the way, I can finish reorganizing the libxml repo.
Could those of you currently working on the lib let me know when you
come to "break" point? This change will be a significant as I will be
moving all the the files in trunk/libxml/ to trunk/. Please get
anything you have going committed and let me know. Or, let me know if
this will be a problem at the moment and I can hold off.

Thanks,
T.

From danj at 3skel.com  Wed Aug 29 15:02:57 2007
From: danj at 3skel.com (Dan Janowski)
Date: Wed, 29 Aug 2007 15:02:57 -0400
Subject: [libxml-devel] MEM2 for viewing
In-Reply-To: <46D5BB70.2050902@savagexi.com>
References: <085D56C7-A7E5-4E32-B237-917FD0FED209@3skel.com>
	<46D5BB70.2050902@savagexi.com>
Message-ID: <B35C6A76-E2B5-42E5-8526-07B35C535A70@3skel.com>

Charlie,

If they are the ones I am thinking of, they are going away entirely  
(no longer needed). If they were in ruby_xml_{node,document,attr}.c,  
there are gone now. They will be in modules I have not touched yet,  
but they will go too.

There is no code carried forward for ruby_xml_*_new or  
ruby_xml_*_free, the memory management is completely different.  
Everything is GC friendly and one xmlNode will only have one ruby  
VALUE object associated with it at a time.

Dan

On Aug 29, 2007, at 14:31, Charlie Savage wrote:

> Dan,
>
> If you have a second, could you fix the free/xfee issue (or I can  
> do it once you are done, I just don't want to interfere with what  
> you are doing).
>
> The problem is that various structures in libxml are allocated with  
> the macro ALLOC, which means the ruby executable is creating the  
> memory. But the structures are then deallocated using free, which  
> means the dll is trying to delete them.  On windows this doesn't  
> work since each dll likely has its own stack (depending on the  
> runtime library it is using).
>
> The solution is simple - use xfree (which is what you are supposed  
> to use anyway for memory created via ALLOCA) and not free.  There  
> are about 10 to 15 instances of this in the current code base.
>
> Thanks,
>
> Charlie
>
>
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel


From cfis at savagexi.com  Wed Aug 29 15:08:56 2007
From: cfis at savagexi.com (Charlie Savage)
Date: Wed, 29 Aug 2007 13:08:56 -0600
Subject: [libxml-devel] MEM2 for viewing
In-Reply-To: <B35C6A76-E2B5-42E5-8526-07B35C535A70@3skel.com>
References: <085D56C7-A7E5-4E32-B237-917FD0FED209@3skel.com>	<46D5BB70.2050902@savagexi.com>
	<B35C6A76-E2B5-42E5-8526-07B35C535A70@3skel.com>
Message-ID: <46D5C448.90208@savagexi.com>

> If they are the ones I am thinking of, they are going away entirely  
> (no longer needed). If they were in ruby_xml_{node,document,attr}.c,  
> there are gone now. They will be in modules I have not touched yet,  
> but they will go too.

Excellent - that's even better then.

> There is no code carried forward for ruby_xml_*_new or  
> ruby_xml_*_free, the memory management is completely different.  
> Everything is GC friendly and one xmlNode will only have one ruby  
> VALUE object associated with it at a time.

Ok, sounds good - thanks for your hard work.

Thanks,

Charlie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070829/1fbb80b7/attachment.bin 

From rosco at roscopeco.co.uk  Thu Aug 30 04:53:44 2007
From: rosco at roscopeco.co.uk (Ross Bamford)
Date: Thu, 30 Aug 2007 09:53:44 +0100
Subject: [libxml-devel] Memory changes, tests needed
In-Reply-To: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>
References: <7A5C5895-2A58-47AD-B548-A55561B42194@3skel.com>
Message-ID: <op.txu6zurwkc9f7k@localhost.localdomain>

On Wed, 29 Aug 2007 01:36:00 +0100, Dan Janowski <danj at 3skel.com> wrote:

> Hi all,
>
> I am going to go out on a limb and say, tentatively, that my new
> approach to memory management is working. It is so far only applied
> to the ruby side of tree.c functions (ruby_xml_node...). My simple
> example of creating the same document 1M times, loosing references to
> the priors and watching the mark and free run, is at least no worse.
>

Sounds cool!

> Since I have not propagated the changes to other parts, the test
> suite is broken. I suspect that they did not expose the memory
> problems anyway.
>
> What I need now is a few concise examples that have blown up
> previously. I am particularly looking for ones that just use the node
> operations.
>

You're right that the unit tests don't really hit the memory bugs, but  
there's also a few additional tests in there that specifically test some  
of the difficult areas (node copying, etc). They're especially useful with  
valgrind.

Cheers,
-- 
Ross Bamford - rosco at roscopeco.co.uk

From danj at 3skel.com  Thu Aug 30 09:41:16 2007
From: danj at 3skel.com (Dan Janowski)
Date: Thu, 30 Aug 2007 09:41:16 -0400
Subject: [libxml-devel] MEM2 development release (0.5.0)
Message-ID: <F4071274-B40C-48A0-A89F-C817B0CED6A6@3skel.com>

libxml at rubyforge (http://rubyforge.org/projects/libxml/) now has a  
packaged development release from the MEM2 branch (New Memory Model)  
http://rubyforge.org/frs/shownotes.php?release_id=14118

The downloads are highlighted here:

http://rubyforge.org/frs/?group_id=494&release_id=14118

Please be sure to read the brief release notes about running the  
tests in 'rwtest' without having to install this release.

If anyone has amd64/freeBSD the rwtest/doc_file.rb seems to be  
specific to that platform. If anyone can test it, please report back.

The next release should increase the coverage of MEM2 as well as  
integrating the tests in rwtest into the rake file.

This release of MEM2 resolves a wide variety of bugs.

Dan


From transfire at gmail.com  Thu Aug 30 11:32:09 2007
From: transfire at gmail.com (TRANS)
Date: Thu, 30 Aug 2007 08:32:09 -0700
Subject: [libxml-devel] MEM2 development release (0.5.0)
In-Reply-To: <F4071274-B40C-48A0-A89F-C817B0CED6A6@3skel.com>
References: <F4071274-B40C-48A0-A89F-C817B0CED6A6@3skel.com>
Message-ID: <4b6f054f0708300832g5e8d9742h7ed3c4d50c37687c@mail.gmail.com>

On 8/30/07, Dan Janowski <danj at 3skel.com> wrote:
> libxml at rubyforge (http://rubyforge.org/projects/libxml/) now has a
> packaged development release from the MEM2 branch (New Memory Model)
> http://rubyforge.org/frs/shownotes.php?release_id=14118
>
> The downloads are highlighted here:
>
> http://rubyforge.org/frs/?group_id=494&release_id=14118
>
> Please be sure to read the brief release notes about running the
> tests in 'rwtest' without having to install this release.
>
> If anyone has amd64/freeBSD the rwtest/doc_file.rb seems to be
> specific to that platform. If anyone can test it, please report back.
>
> The next release should increase the coverage of MEM2 as well as
> integrating the tests in rwtest into the rake file.
>
> This release of MEM2 resolves a wide variety of bugs.

W00t!

Nice work, Dan.

T.

From marc at bloodnok.com  Thu Aug 30 19:33:54 2007
From: marc at bloodnok.com (Marc Munro)
Date: Thu, 30 Aug 2007 16:33:54 -0700
Subject: [libxml-devel] wierd lock-up
Message-ID: <1188516834.16688.28.camel@bloodnok.com>

I have been getting some odd lock-ups from the old version of libxml
(Debian testing, claiming to be version 0.3.8-1).  The odd thing is that
the app stops in an uninterruptible state yet consumes no resources.

From the behaviour I would guess it is stuck on a semaphore or some such
thing but I can't imagine why that would be.  Running under gdb prevents
the problem from manifesting.

I assume this is just another incarnation of the various memory issues
but I thought I'd

a) report it;
b) see if anyone had an suggestions.

__
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070830/aacc14f7/attachment.bin 

From marc at bloodnok.com  Thu Aug 30 18:28:20 2007
From: marc at bloodnok.com (Marc Munro)
Date: Thu, 30 Aug 2007 15:28:20 -0700
Subject: [libxml-devel] MEM2 development release (0.5.0)
In-Reply-To: <F4071274-B40C-48A0-A89F-C817B0CED6A6@3skel.com>
References: <F4071274-B40C-48A0-A89F-C817B0CED6A6@3skel.com>
Message-ID: <1188512900.16688.9.camel@bloodnok.com>

Dan,
I just tried the new version and it breaks my app in every test I
have :-(

This is kinda typical:

marc:[skituls]$ make utgen
========== Difftree tests ====================
tests/generate.rb | tools/utest_status
Loaded suite tests/generate
Started
./classes/xmlfile.rb:288: [BUG] Segmentation fault
ruby 1.8.6 (2007-06-07) [i486-linux]


Running simple tests, including those reported previously as bugs, in
irb shows no problems so I don't think this is a build problem on my
machine.

FYI: 
marc:[libxml-ruby-0.5.0]$ uname -a
Linux bloodnok 2.6.15.6 #4 SMP PREEMPT Sat Aug 5 20:40:37 PDT 2006 i686 GNU/Linux

Suggestions?

__
Marc



On Thu, 2007-30-08 at 09:41 -0400, Dan Janowski wrote:
> libxml at rubyforge (http://rubyforge.org/projects/libxml/) now has a  
> packaged development release from the MEM2 branch (New Memory Model)  
> http://rubyforge.org/frs/shownotes.php?release_id=14118
> 
> The downloads are highlighted here:
> 
> http://rubyforge.org/frs/?group_id=494&release_id=14118
> 
> Please be sure to read the brief release notes about running the  
> tests in 'rwtest' without having to install this release.
> 
> If anyone has amd64/freeBSD the rwtest/doc_file.rb seems to be  
> specific to that platform. If anyone can test it, please report back.
> 
> The next release should increase the coverage of MEM2 as well as  
> integrating the tests in rwtest into the rake file.
> 
> This release of MEM2 resolves a wide variety of bugs.
> 
> Dan
> 
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20070830/ce5a8fc9/attachment.bin 

From rosco at roscopeco.co.uk  Fri Aug 31 08:54:40 2007
From: rosco at roscopeco.co.uk (Ross Bamford)
Date: Fri, 31 Aug 2007 13:54:40 +0100
Subject: [libxml-devel] MEM2 development release (0.5.0)
In-Reply-To: <F4071274-B40C-48A0-A89F-C817B0CED6A6@3skel.com>
References: <F4071274-B40C-48A0-A89F-C817B0CED6A6@3skel.com>
Message-ID: <op.txxcte2kkc9f7k@localhost.localdomain>

On Thu, 30 Aug 2007 14:41:16 +0100, Dan Janowski <danj at 3skel.com> wrote:

> libxml at rubyforge (http://rubyforge.org/projects/libxml/) now has a
> packaged development release from the MEM2 branch (New Memory Model)
> http://rubyforge.org/frs/shownotes.php?release_id=14118
>

Very cool, Dan! :)

On my box though, it seems to break the previous bug tests - both  
tests/copy_bug.rb and tests/copy_bug2.rb bomb out with a doublefree error.  
FYI this is compiled from the gem, with GCC 4.1.1 (Red Hat 4.1.1-1). I  
included a trace below. Also, running in valgrind shows rather a lot of  
memory errors (but no leaks, so looking good there!)




# ruby bug2.rb
[... 1..90 elided ... ]
91
92
*** glibc detected *** ruby: double free or corruption (fasttop):  
0x09ab4a38 ***
======= Backtrace: =========
/lib/libc.so.6[0x2bdf18]
/lib/libc.so.6(__libc_free+0x78)[0x2c13ef]
/usr/lib/libxml2.so.2(xmlFreeNodeList+0x16e)[0x7c0aeae]
/usr/lib/libxml2.so.2(xmlFreeNodeList+0x96)[0x7c0add6]
/usr/lib/libxml2.so.2(xmlFreeDoc+0xcb)[0x7c0abfb]
./../ext/xml/libxml_so.so(ruby_xml_document_free+0x2e)[0x9562fe]
ruby[0x8070286]
ruby(rb_newobj+0x46)[0x8070396]
ruby(rb_data_object_alloc+0x10)[0x80703b0]
./../ext/xml/libxml_so.so(ruby_xml_node2_wrap+0x67)[0x9537b7]
./../ext/xml/libxml_so.so(ruby_xml_node_copy+0x6f)[0x95382f]
ruby[0x805dd8d]
ruby[0x805e981]
ruby[0x805c404]
ruby[0x805c396]
ruby[0x8059b73]
ruby(rb_yield+0x21)[0x805a3f1]
ruby[0x8080b91]
ruby[0x805dd8d]
ruby[0x805e981]
ruby[0x805c404]
ruby[0x805d69b]
ruby[0x805e57f]
ruby[0x805e981]
ruby[0x805c50b]
ruby[0x805adaf]
ruby[0x805e57f]
ruby[0x805e981]
ruby[0x805c50b]
ruby[0x8059b73]
ruby(rb_yield+0x21)[0x805a3f1]
ruby[0x8080b91]
ruby[0x805dd8d]
ruby[0x805e981]
ruby[0x805c404]
ruby[0x805d69b]
ruby[0x806940d]
ruby(ruby_exec+0x16)[0x8069446]
ruby(ruby_run+0x20)[0x8069470]
ruby[0x805278f]
/lib/libc.so.6(__libc_start_main+0xdc)[0x26f724]
ruby[0x80526c1]
======= Memory map: ========
0023c000-0023d000 r-xp 0023c000 00:00 0          [vdso]
0023d000-00256000 r-xp 00000000 fd:01 997991     /lib/ld-2.4.so
00256000-00257000 r-xp 00018000 fd:01 997991     /lib/ld-2.4.so
00257000-00258000 rwxp 00019000 fd:01 997991     /lib/ld-2.4.so
0025a000-00387000 r-xp 00000000 fd:01 997993     /lib/libc-2.4.so
00387000-00389000 r-xp 0012d000 fd:01 997993     /lib/libc-2.4.so
00389000-0038a000 rwxp 0012f000 fd:01 997993     /lib/libc-2.4.so
0038a000-0038d000 rwxp 0038a000 00:00 0
0038f000-003b2000 r-xp 00000000 fd:01 998148     /lib/libm-2.4.so
003b2000-003b3000 r-xp 00022000 fd:01 998148     /lib/libm-2.4.so
003b3000-003b4000 rwxp 00023000 fd:01 998148     /lib/libm-2.4.so
003b6000-003b8000 r-xp 00000000 fd:01 998149     /lib/libdl-2.4.so
003b8000-003b9000 r-xp 00001000 fd:01 998149     /lib/libdl-2.4.so
003b9000-003ba000 rwxp 00002000 fd:01 998149     /lib/libdl-2.4.so
003bc000-003ce000 r-xp 00000000 fd:01 104393     /usr/lib/libz.so.1.2.3
003ce000-003cf000 rwxp 00011000 fd:01 104393     /usr/lib/libz.so.1.2.3
00651000-0065c000 r-xp 00000000 fd:01 998151      
/lib/libgcc_s-4.1.1-20060525.so.1
0065c000-0065d000 rwxp 0000a000 fd:01 998151      
/lib/libgcc_s-4.1.1-20060525.so.1
00943000-0095f000 r-xp 00000000 fd:00 6427242     
/home/usr/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.5.0/ext/xml/libxml_so.so
0095f000-00960000 rwxp 0001c000 fd:00 6427242     
/home/usr/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.5.0/ext/xml/libxml_so.so
00d27000-00d39000 r-xp 00000000 fd:01 998161     /lib/libnsl-2.4.so
00d39000-00d3a000 r-xp 00011000 fd:01 998161     /lib/libnsl-2.4.so
00d3a000-00d3b000 rwxp 00012000 fd:01 998161     /lib/libnsl-2.4.so
00d3b000-00d3d000 rwxp 00d3b000 00:00 0
07bcb000-07cef000 r-xp 00000000 fd:01 104471     /usr/lib/libxml2.so.2.6.23
07cef000-07cf7000 rwxp 00124000 fd:01 104471     /usr/lib/libxml2.so.2.6.23
07cf7000-07cf8000 rwxp 07cf7000 00:00 0
07e2d000-07e32000 r-xp 00000000 fd:01 998156     /lib/libcrypt-2.4.so
07e32000-07e33000 r-xp 00004000 fd:01 998156     /lib/libcrypt-2.4.so
07e33000-07e34000 rwxp 00005000 fd:01 998156     /lib/libcrypt-2.4.so
07e34000-07e5b000 rwxp 07e34000 00:00 0
08048000-080ef000 r-xp 00000000 fd:00 5030041    /home/usr/local/bin/ruby
080ef000-080f1000 rw-p 000a6000 fd:00 5030041    /home/usr/local/bin/ruby
080f1000-08100000 rw-p 080f1000 00:00 0
09972000-09add000 rw-p 09972000 00:00 0          [heap]
b7e00000-b7e21000 rw-p b7e00000 00:00 0
b7e21000-b7f00000 ---p b7e21000 00:00 0
b7f3c000-b7f6f000 rw-p b7f3c000 00:00 0
b7f8d000-b7f8f000 rw-p b7f8d000 00:00 0
bf7f5000-bf80b000 rw-p bf7f5000 00:00 0          [stack]
Aborted

-- 
Ross Bamford - rosco at roscopeco.co.uk

From danj at 3skel.com  Fri Aug 31 11:52:31 2007
From: danj at 3skel.com (Dan Janowski)
Date: Fri, 31 Aug 2007 11:52:31 -0400
Subject: [libxml-devel] MEM2-0.5.0.1 development release (svn 137)
Message-ID: <CD2CE2D0-C997-406C-89CA-AF03D5A7E7AE@3skel.com>

libxml at rubyforge (http://rubyforge.org/projects/libxml/)

A new packaged development release from the MEM2 branch (New Memory  
Model) is available:

http://rubyforge.org/frs/?group_id=494&release_id=14142

Release notes at:

http://rubyforge.org/frs/shownotes.php?release_id=14142

Dan


From danj at 3skel.com  Fri Aug 31 11:53:52 2007
From: danj at 3skel.com (Dan Janowski)
Date: Fri, 31 Aug 2007 11:53:52 -0400
Subject: [libxml-devel] MEM2 development release (0.5.0)
In-Reply-To: <op.txxcte2kkc9f7k@localhost.localdomain>
References: <F4071274-B40C-48A0-A89F-C817B0CED6A6@3skel.com>
	<op.txxcte2kkc9f7k@localhost.localdomain>
Message-ID: <64EA1073-9337-4311-AB05-780CFD36B58C@3skel.com>

Ross,

Try this again with the patch release just announced.

Dan

On Aug 31, 2007, at 08:54, Ross Bamford wrote:

> On Thu, 30 Aug 2007 14:41:16 +0100, Dan Janowski <danj at 3skel.com>  
> wrote:
>
>> libxml at rubyforge (http://rubyforge.org/projects/libxml/) now has a
>> packaged development release from the MEM2 branch (New Memory Model)
>> http://rubyforge.org/frs/shownotes.php?release_id=14118
>>
>
> Very cool, Dan! :)
>
> On my box though, it seems to break the previous bug tests - both
> tests/copy_bug.rb and tests/copy_bug2.rb bomb out with a doublefree  
> error.
> FYI this is compiled from the gem, with GCC 4.1.1 (Red Hat 4.1.1-1). I
> included a trace below. Also, running in valgrind shows rather a  
> lot of
> memory errors (but no leaks, so looking good there!)
>
>
>
>
> # ruby bug2.rb
> [... 1..90 elided ... ]
> 91
> 92
> *** glibc detected *** ruby: double free or corruption (fasttop):
> 0x09ab4a38 ***
> ======= Backtrace: =========
> /lib/libc.so.6[0x2bdf18]
> /lib/libc.so.6(__libc_free+0x78)[0x2c13ef]
> /usr/lib/libxml2.so.2(xmlFreeNodeList+0x16e)[0x7c0aeae]
> /usr/lib/libxml2.so.2(xmlFreeNodeList+0x96)[0x7c0add6]
> /usr/lib/libxml2.so.2(xmlFreeDoc+0xcb)[0x7c0abfb]
> ./../ext/xml/libxml_so.so(ruby_xml_document_free+0x2e)[0x9562fe]
> ruby[0x8070286]
> ruby(rb_newobj+0x46)[0x8070396]
> ruby(rb_data_object_alloc+0x10)[0x80703b0]
> ./../ext/xml/libxml_so.so(ruby_xml_node2_wrap+0x67)[0x9537b7]
> ./../ext/xml/libxml_so.so(ruby_xml_node_copy+0x6f)[0x95382f]
> ruby[0x805dd8d]
> ruby[0x805e981]
> ruby[0x805c404]
> ruby[0x805c396]
> ruby[0x8059b73]
> ruby(rb_yield+0x21)[0x805a3f1]
> ruby[0x8080b91]
> ruby[0x805dd8d]
> ruby[0x805e981]
> ruby[0x805c404]
> ruby[0x805d69b]
> ruby[0x805e57f]
> ruby[0x805e981]
> ruby[0x805c50b]
> ruby[0x805adaf]
> ruby[0x805e57f]
> ruby[0x805e981]
> ruby[0x805c50b]
> ruby[0x8059b73]
> ruby(rb_yield+0x21)[0x805a3f1]
> ruby[0x8080b91]
> ruby[0x805dd8d]
> ruby[0x805e981]
> ruby[0x805c404]
> ruby[0x805d69b]
> ruby[0x806940d]
> ruby(ruby_exec+0x16)[0x8069446]
> ruby(ruby_run+0x20)[0x8069470]
> ruby[0x805278f]
> /lib/libc.so.6(__libc_start_main+0xdc)[0x26f724]
> ruby[0x80526c1]
> ======= Memory map: ========
> 0023c000-0023d000 r-xp 0023c000 00:00 0          [vdso]
> 0023d000-00256000 r-xp 00000000 fd:01 997991     /lib/ld-2.4.so
> 00256000-00257000 r-xp 00018000 fd:01 997991     /lib/ld-2.4.so
> 00257000-00258000 rwxp 00019000 fd:01 997991     /lib/ld-2.4.so
> 0025a000-00387000 r-xp 00000000 fd:01 997993     /lib/libc-2.4.so
> 00387000-00389000 r-xp 0012d000 fd:01 997993     /lib/libc-2.4.so
> 00389000-0038a000 rwxp 0012f000 fd:01 997993     /lib/libc-2.4.so
> 0038a000-0038d000 rwxp 0038a000 00:00 0
> 0038f000-003b2000 r-xp 00000000 fd:01 998148     /lib/libm-2.4.so
> 003b2000-003b3000 r-xp 00022000 fd:01 998148     /lib/libm-2.4.so
> 003b3000-003b4000 rwxp 00023000 fd:01 998148     /lib/libm-2.4.so
> 003b6000-003b8000 r-xp 00000000 fd:01 998149     /lib/libdl-2.4.so
> 003b8000-003b9000 r-xp 00001000 fd:01 998149     /lib/libdl-2.4.so
> 003b9000-003ba000 rwxp 00002000 fd:01 998149     /lib/libdl-2.4.so
> 003bc000-003ce000 r-xp 00000000 fd:01 104393     /usr/lib/libz.so. 
> 1.2.3
> 003ce000-003cf000 rwxp 00011000 fd:01 104393     /usr/lib/libz.so. 
> 1.2.3
> 00651000-0065c000 r-xp 00000000 fd:01 998151
> /lib/libgcc_s-4.1.1-20060525.so.1
> 0065c000-0065d000 rwxp 0000a000 fd:01 998151
> /lib/libgcc_s-4.1.1-20060525.so.1
> 00943000-0095f000 r-xp 00000000 fd:00 6427242
> /home/usr/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.5.0/ext/xml/ 
> libxml_so.so
> 0095f000-00960000 rwxp 0001c000 fd:00 6427242
> /home/usr/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.5.0/ext/xml/ 
> libxml_so.so
> 00d27000-00d39000 r-xp 00000000 fd:01 998161     /lib/libnsl-2.4.so
> 00d39000-00d3a000 r-xp 00011000 fd:01 998161     /lib/libnsl-2.4.so
> 00d3a000-00d3b000 rwxp 00012000 fd:01 998161     /lib/libnsl-2.4.so
> 00d3b000-00d3d000 rwxp 00d3b000 00:00 0
> 07bcb000-07cef000 r-xp 00000000 fd:01 104471     /usr/lib/ 
> libxml2.so.2.6.23
> 07cef000-07cf7000 rwxp 00124000 fd:01 104471     /usr/lib/ 
> libxml2.so.2.6.23
> 07cf7000-07cf8000 rwxp 07cf7000 00:00 0
> 07e2d000-07e32000 r-xp 00000000 fd:01 998156     /lib/libcrypt-2.4.so
> 07e32000-07e33000 r-xp 00004000 fd:01 998156     /lib/libcrypt-2.4.so
> 07e33000-07e34000 rwxp 00005000 fd:01 998156     /lib/libcrypt-2.4.so
> 07e34000-07e5b000 rwxp 07e34000 00:00 0
> 08048000-080ef000 r-xp 00000000 fd:00 5030041    /home/usr/local/ 
> bin/ruby
> 080ef000-080f1000 rw-p 000a6000 fd:00 5030041    /home/usr/local/ 
> bin/ruby
> 080f1000-08100000 rw-p 080f1000 00:00 0
> 09972000-09add000 rw-p 09972000 00:00 0          [heap]
> b7e00000-b7e21000 rw-p b7e00000 00:00 0
> b7e21000-b7f00000 ---p b7e21000 00:00 0
> b7f3c000-b7f6f000 rw-p b7f3c000 00:00 0
> b7f8d000-b7f8f000 rw-p b7f8d000 00:00 0
> bf7f5000-bf80b000 rw-p bf7f5000 00:00 0          [stack]
> Aborted
>
> -- 
> Ross Bamford - rosco at roscopeco.co.uk
> _______________________________________________
> libxml-devel mailing list
> libxml-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/libxml-devel