From stephen.bannasch at deanbrook.org  Thu Jan 13 15:37:50 2011
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Thu, 13 Jan 2011 15:37:50 -0500
Subject: spurious <li> generated when list followed by 3
Message-ID: <p06240810c9550efc734d@[192.168.1.122]>

RedCloth master generates a spurios <li> element when a numbered or undumbered list is followed by three or more newlines.

RedCloth.new("* one\n* two\n* three \n\n\n").to_html
#=> "<ul>\n\t<li>one</li>\n\t<li>two</li>\n\t<li>three</li>\n</ul>\n<li>"
# note extra trailing <li> outside <ul>

A colleague generated a pull request with a new rspec test that exposes this bug:

https://github.com/jgarber/redcloth/pull/1

I checked and the bug is present on master.

I just took a look at the ragel code but it's been quite a while since I've used ragel and it's not quite as easy to parse as 
Ruby ;-)

Seems like li_open in lib/redcloth/formatters/html.rb is being called incorrectly when the thrid newline is parsed.

Here are the locations in the code where li_open is referenced:

$ ack --all-types li_open

   ext/redcloth_scan/redcloth.h
   202:    ASET("type", "li_open");

   ext/redcloth_scan/RedclothScanService.java
   95:     ASET("type", "li_open");

   lib/redcloth/formatters/html.rb
   50:  def li_open(opts)

   lib/redcloth/formatters/latex.rb
   103:  def li_open(opts)

   ragel/redcloth_scan.java.rl
   94:     ASET("type", "li_open");

   ragel/redcloth_scan.rb.rl
   309:      ASET("type", "li_open")



From stephen.bannasch at deanbrook.org  Thu Jan 13 17:24:39 2011
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Thu, 13 Jan 2011 17:24:39 -0500
Subject: spurious <li> generated when list followed by 3
In-Reply-To: <p06240810c9550efc734d@[192.168.1.122]>
References: <p06240810c9550efc734d@[192.168.1.122]>
Message-ID: <p06240811c955289c74e9@[192.168.1.122]>

>RedCloth master generates a spurios <li> element when a numbered or undumbered list is followed by three or more newlines.
>
>RedCloth.new("* one\n* two\n* three \n\n\n").to_html
>#=> "<ul>\n\t<li>one</li>\n\t<li>two</li>\n\t<li>three</li>\n</ul>\n<li>"
># note extra trailing <li> outside <ul>
>
>A colleague generated a pull request with a new rspec test that exposes this bug:
>
>https://github.com/jgarber/redcloth/pull/1
>
>I checked and the bug is present on master.
>
>I just took a look at the ragel code but it's been quite a while since I've used ragel and it's not quite as easy to parse as Ruby ;-)
>
>Seems like li_open in lib/redcloth/formatters/html.rb is being called incorrectly when the thrid newline is parsed.
>
>Here are the locations in the code where li_open is referenced:
>
>$ ack --all-types li_open
>
>  ext/redcloth_scan/redcloth.h
>  202:    ASET("type", "li_open");
>
>  ext/redcloth_scan/RedclothScanService.java
>  95:     ASET("type", "li_open");
>
>  lib/redcloth/formatters/html.rb
>  50:  def li_open(opts)
>
>  lib/redcloth/formatters/latex.rb
>  103:  def li_open(opts)
>
>  ragel/redcloth_scan.java.rl
>  94:     ASET("type", "li_open");
>
>  ragel/redcloth_scan.rb.rl
>  309:      ASET("type", "li_open")
>

Here's the new spec test:

  it "should not add spurious li tags to the end of markup" do
    input         = "* one\n* two\n* three \n\n"
    failing_input = "* one\n* two\n* three \n\n\n"
    RedCloth.new(input).to_html.should_not match /<li>$/
    RedCloth.new(failing_input).to_html.should_not match /<li>$/
  end

When I run the spec test in the debugger:

  $ rspec -d spec/parser_spec.rb

RedCloth::Formatters::HTML.li_open(opts) is called once for each of the three list items in the first test:

    RedCloth.new(input).to_html.should_not match /<li>$/

and then called again once for each list item in the second test:

    RedCloth.new(failing_input).to_html.should_not match /<li>$/

and then it is called a seventh time with this argument: {:type=>"li_open", :nest=>0, :text=>""}


   50    def li_open(opts)
   51      debugger
=> 52      "#{"\t" * opts[:nest]}<li#{pba(opts)}>#{opts[:text]}"
   53    end

values of argument opts:

  1: {:type=>"li_open", :nest=>1, :text=>"one"}
  2: {:type=>"li_open", :nest=>1, :text=>"two"}
  3: {:type=>"li_open", :nest=>1, :text=>"three"}
  4: {:type=>"li_open", :nest=>1, :text=>"one"}
  5: {:type=>"li_open", :nest=>1, :text=>"two"}
  6: {:type=>"li_open", :nest=>1, :text=>"three"}
  7: {:type=>"li_open", :nest=>0, :text=>""}

The nesting level is 0 and the text is an empty string ...

Is there a way to tell more about what the ragel machine has been doing and why li_open was called this last time?

From stephen.bannasch at deanbrook.org  Fri Jan 14 02:21:29 2011
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Fri, 14 Jan 2011 02:21:29 -0500
Subject: spurious <li> generated when list followed by 3
In-Reply-To: <p06240811c955289c74e9@[192.168.1.122]>
References: <p06240810c9550efc734d@[192.168.1.122]>
	<p06240811c955289c74e9@[192.168.1.122]>
Message-ID: <p06240812c955a8ea4734@[192.168.1.122]>

found the bug in a lighthouse ticket from last February:

http://jgarber.lighthouseapp.com/projects/13054/tickets/183-three-newlines-after-a-list-at-end-of-file-make-dangling-li

From jg at jasongarber.com  Fri Jan 14 08:07:49 2011
From: jg at jasongarber.com (Jason Garber)
Date: Fri, 14 Jan 2011 07:07:49 -0600
Subject: spurious <li> generated when list followed by 3
In-Reply-To: <p06240812c955a8ea4734@192.168.1.122>
References: <p06240810c9550efc734d@192.168.1.122>
	<p06240811c955289c74e9@192.168.1.122>
	<p06240812c955a8ea4734@192.168.1.122>
Message-ID: <AANLkTikJk2p6HjZJSJN3A0ZyEv4FXNWAc3PyxVsK7Bpw@mail.gmail.com>

Yep, it's an ugly one. I've tried to figure it out last year, but have never
been able to. Ragel is just too much of a black box?at least to me it is.
I've printed state machine diagrams that take up the whole floor trying to
figure out this stuff. It's impossible!

I don't think crazy edge cases like this one will get resolved until I
switch RedCloth to a different architecture that has a little more
transparency.

Jason

On Fri, Jan 14, 2011 at 1:21 AM, Stephen Bannasch <
stephen.bannasch at deanbrook.org> wrote:

> found the bug in a lighthouse ticket from last February:
>
>
> http://jgarber.lighthouseapp.com/projects/13054/tickets/183-three-newlines-after-a-list-at-end-of-file-make-dangling-li
>
> _______________________________________________
> Redcloth-upwards mailing list
> Redcloth-upwards at rubyforge.org
> http://rubyforge.org/mailman/listinfo/redcloth-upwards
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/redcloth-upwards/attachments/20110114/bf6a25f8/attachment.html>

From stephen.bannasch at deanbrook.org  Fri Jan 14 12:54:36 2011
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Fri, 14 Jan 2011 12:54:36 -0500
Subject: spurious <li> generated when list followed by 3
In-Reply-To: <AANLkTikJk2p6HjZJSJN3A0ZyEv4FXNWAc3PyxVsK7Bpw@mail.gmail.com>
References: <p06240810c9550efc734d@192.168.1.122>	<p06240811c955289c74e9@192.168.1.122>	<p06240812c955a8ea4734@192.168.1.122>
	<AANLkTikJk2p6HjZJSJN3A0ZyEv4FXNWAc3PyxVsK7Bpw@mail.gmail.com>
Message-ID: <p06240826c9563e41d6b6@[192.168.1.122]>

>Yep, it's an ugly one. I've tried to figure it out last year, but have never been able to. Ragel isjust too much of a black box-at least to me it is. I've printed state machine diagrams that take up the whole floor trying to figure out this stuff. It's impossible!
>
>I don't think crazy edge cases like this one will get resolved until I switch RedCloth to a different architecture that has a little more transparency.

Fixed in new pull request: https://github.com/jgarber/redcloth/pull/2

Ticket updated also.

From biz at sunnyhome.org  Tue Jan 18 20:22:57 2011
From: biz at sunnyhome.org (Douglas Pearson)
Date: Tue, 18 Jan 2011 17:22:57 -0800
Subject: Installing on CentOS
Message-ID: <AANLkTinHJ+8VRkVqWuxXPNBtbti+5_5HgY2kbOHCywWV@mail.gmail.com>

Hi,

I'm trying to install RedCloth on CentOS.  It came with ruby 1.8.5
pre-installed so I installed rubygems-1.2.0 since that seemed compatible
(later versions require ruby 1.8.6).

All looks good but when I try:

[root at dev4 rubygems-1.2.0]# gem install RedCloth
ERROR:  could not find gem RedCloth locally or in a repository

Can anyone tell me what I'm missing?  Do I need to configure some repository
information?

Thanks,

Doug

P.S. Total Ruby/RubyGems novice - so I could be doing all manner of dumb
stuff...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/redcloth-upwards/attachments/20110118/ec67017a/attachment.html>

From jg at jasongarber.com  Sun Jan 23 16:03:32 2011
From: jg at jasongarber.com (Jason Garber)
Date: Sun, 23 Jan 2011 15:03:32 -0600
Subject: Installing on CentOS
In-Reply-To: <AANLkTinHJ+8VRkVqWuxXPNBtbti+5_5HgY2kbOHCywWV@mail.gmail.com>
References: <AANLkTinHJ+8VRkVqWuxXPNBtbti+5_5HgY2kbOHCywWV@mail.gmail.com>
Message-ID: <AANLkTikEoJ2K0d0LWXyRKjxAfMST_LDWdpQGf5wwc1Q6@mail.gmail.com>

Doug,
Sorry, I was away on vacation. I'm not sure why it's not working. I
certainly have never tested it with Ruby or RubyGems that old. Even if you
do get it to find it, it's going to want to install a bunch of dependencies
that aren't necessary because older RubyGems didn't have the concept of
development dependencies.

If it were me, I'd use RVM to install a recent version of Ruby and RubyGems.
Otherwise, I imagine you'll have problems with libraries other than RedCloth
as well and it's probably not worth figuring out. Good luck!

Jason

On Tue, Jan 18, 2011 at 7:22 PM, Douglas Pearson <biz at sunnyhome.org> wrote:

> Hi,
>
> I'm trying to install RedCloth on CentOS.  It came with ruby 1.8.5
> pre-installed so I installed rubygems-1.2.0 since that seemed compatible
> (later versions require ruby 1.8.6).
>
> All looks good but when I try:
>
> [root at dev4 rubygems-1.2.0]# gem install RedCloth
> ERROR:  could not find gem RedCloth locally or in a repository
>
> Can anyone tell me what I'm missing?  Do I need to configure some
> repository information?
>
> Thanks,
>
> Doug
>
> P.S. Total Ruby/RubyGems novice - so I could be doing all manner of dumb
> stuff...
>
>
> _______________________________________________
> Redcloth-upwards mailing list
> Redcloth-upwards at rubyforge.org
> http://rubyforge.org/mailman/listinfo/redcloth-upwards
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/redcloth-upwards/attachments/20110123/c8a8944e/attachment.html>

From biz at sunnyhome.org  Sun Jan 23 18:47:16 2011
From: biz at sunnyhome.org (Douglas Pearson)
Date: Sun, 23 Jan 2011 15:47:16 -0800
Subject: Installing on CentOS
In-Reply-To: <AANLkTikEoJ2K0d0LWXyRKjxAfMST_LDWdpQGf5wwc1Q6@mail.gmail.com>
References: <AANLkTinHJ+8VRkVqWuxXPNBtbti+5_5HgY2kbOHCywWV@mail.gmail.com>
	<AANLkTikEoJ2K0d0LWXyRKjxAfMST_LDWdpQGf5wwc1Q6@mail.gmail.com>
Message-ID: <AANLkTi=hTCEywYpT7PV4vuLoBzox60u9DeB+E4KK3aco@mail.gmail.com>

Hey Jason,

Thanks a lot for following up.  I actually ended up discovering BitNami (
http://bitnami.org) and installing a completely new Ruby stack to an EC2
instance and abandoning the whole CentOS physical server I was working
with.  I've not tried RedCloth again yet but I suspect it'll go much easier
this time since I'm now running with current code.

One lesson I've learned is that Ruby is very version sensitive - so best not
to swim upstream against that.

Thanks for the help,

Doug

On Sun, Jan 23, 2011 at 1:03 PM, Jason Garber <jg at jasongarber.com> wrote:

> Doug,
> Sorry, I was away on vacation. I'm not sure why it's not working. I
> certainly have never tested it with Ruby or RubyGems that old. Even if you
> do get it to find it, it's going to want to install a bunch of dependencies
> that aren't necessary because older RubyGems didn't have the concept of
> development dependencies.
>
> If it were me, I'd use RVM to install a recent version of Ruby and
> RubyGems. Otherwise, I imagine you'll have problems with libraries other
> than RedCloth as well and it's probably not worth figuring out. Good luck!
>
> Jason
>
> On Tue, Jan 18, 2011 at 7:22 PM, Douglas Pearson <biz at sunnyhome.org>wrote:
>
>> Hi,
>>
>> I'm trying to install RedCloth on CentOS.  It came with ruby 1.8.5
>> pre-installed so I installed rubygems-1.2.0 since that seemed compatible
>> (later versions require ruby 1.8.6).
>>
>> All looks good but when I try:
>>
>> [root at dev4 rubygems-1.2.0]# gem install RedCloth
>> ERROR:  could not find gem RedCloth locally or in a repository
>>
>> Can anyone tell me what I'm missing?  Do I need to configure some
>> repository information?
>>
>> Thanks,
>>
>> Doug
>>
>> P.S. Total Ruby/RubyGems novice - so I could be doing all manner of dumb
>> stuff...
>>
>>
>> _______________________________________________
>> Redcloth-upwards mailing list
>> Redcloth-upwards at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/redcloth-upwards
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/redcloth-upwards/attachments/20110123/4d40b24e/attachment-0001.html>

From jg at jasongarber.com  Sun Jan 23 19:41:23 2011
From: jg at jasongarber.com (Jason Garber)
Date: Sun, 23 Jan 2011 18:41:23 -0600
Subject: spurious <li> generated when list followed by 3
In-Reply-To: <p06240826c9563e41d6b6@192.168.1.122>
References: <p06240810c9550efc734d@192.168.1.122>
	<p06240811c955289c74e9@192.168.1.122>
	<p06240812c955a8ea4734@192.168.1.122>
	<AANLkTikJk2p6HjZJSJN3A0ZyEv4FXNWAc3PyxVsK7Bpw@mail.gmail.com>
	<p06240826c9563e41d6b6@192.168.1.122>
Message-ID: <AANLkTincW+hTEx7YzKyLNVXpDziAGeBEP=Z76knMFizq@mail.gmail.com>

Thanks so much for the patch! Applied.

On Fri, Jan 14, 2011 at 11:54 AM, Stephen Bannasch <
stephen.bannasch at deanbrook.org> wrote:

> >Yep, it's an ugly one. I've tried to figure it out last year, but have
> never been able to. Ragel isjust too much of a black box-at least to me it
> is. I've printed state machine diagrams that take up the whole floor trying
> to figure out this stuff. It's impossible!
> >
> >I don't think crazy edge cases like this one will get resolved until I
> switch RedCloth to a different architecture that has a little more
> transparency.
>
> Fixed in new pull request: https://github.com/jgarber/redcloth/pull/2
>
> Ticket updated also.
> _______________________________________________
> Redcloth-upwards mailing list
> Redcloth-upwards at rubyforge.org
> http://rubyforge.org/mailman/listinfo/redcloth-upwards
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/redcloth-upwards/attachments/20110123/b762f514/attachment.html>

From stephen.bannasch at deanbrook.org  Sun Jan 23 23:00:02 2011
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Sun, 23 Jan 2011 23:00:02 -0500
Subject: spurious <li> generated when list followed by 3
In-Reply-To: <AANLkTincW+hTEx7YzKyLNVXpDziAGeBEP=Z76knMFizq@mail.gmail.com>
References: <p06240810c9550efc734d@192.168.1.122>	<p06240811c955289c74e9@192.168.1.122>	<p06240812c955a8ea4734@192.168.1.122>
	<AANLkTikJk2p6HjZJSJN3A0ZyEv4FXNWAc3PyxVsK7Bpw@mail.gmail.com>	<p06240826c9563e41d6b6@192.168.1.122>
	<AANLkTincW+hTEx7YzKyLNVXpDziAGeBEP=Z76knMFizq@mail.gmail.com>
Message-ID: <p0624088ec962a7d9cf04@[192.168.1.122]>

At 6:41 PM -0600 1/23/11, Jason Garber wrote:
>Thanks so much for the patch! Applied.

Your welcome!

I was surprised that I was able to find the place to fix the bug as quickly as I did -- I didn't want to spend much time and wasn't sure ragel would come back to me fast enough.

I was very glad there are a bunch of tests -- because I sure couldn't easily tell that my change wouldn't break something else important.

What's your plan for releasing 4.2.4?

In terms of changes in operation since 4.2.3 it looks like just:

  Fix double-push attempt.
  https://github.com/jgarber/redcloth/commit/25360cdc922bea6f5f956ec3ddfbfc6c1ec54018

and

  fix dangling <li>
  https://github.com/jgarber/redcloth/commit/ade0bfd1a44dedc2498112a069d0c77505ceb5e4

Some other changes to the build process.

From jg at jasongarber.com  Mon Jan 24 08:01:11 2011
From: jg at jasongarber.com (Jason Garber)
Date: Mon, 24 Jan 2011 07:01:11 -0600
Subject: spurious <li> generated when list followed by 3
In-Reply-To: <p0624088ec962a7d9cf04@192.168.1.122>
References: <p06240810c9550efc734d@192.168.1.122>
	<p06240811c955289c74e9@192.168.1.122>
	<p06240812c955a8ea4734@192.168.1.122>
	<AANLkTikJk2p6HjZJSJN3A0ZyEv4FXNWAc3PyxVsK7Bpw@mail.gmail.com>
	<p06240826c9563e41d6b6@192.168.1.122>
	<AANLkTincW+hTEx7YzKyLNVXpDziAGeBEP=Z76knMFizq@mail.gmail.com>
	<p0624088ec962a7d9cf04@192.168.1.122>
Message-ID: <AANLkTi=WExfB_XrFNcL5jWDHR=zo3BEfsLPxZ2S=2X+y@mail.gmail.com>

I was waiting to release until multi-byte characters in Java were
fixed, but seems like that's not likely to happen anytime soon. I'll
go ahead and release. Thanks for the nudge.

Jason

On Sun, Jan 23, 2011 at 10:00 PM, Stephen Bannasch
<stephen.bannasch at deanbrook.org> wrote:
> At 6:41 PM -0600 1/23/11, Jason Garber wrote:
>>Thanks so much for the patch! Applied.
>
> Your welcome!
>
> I was surprised that I was able to find the place to fix the bug as quickly as I did -- I didn't want to spend much time and wasn't sure ragel would come back to me fast enough.
>
> I was very glad there are a bunch of tests -- because I sure couldn't easily tell that my change wouldn't break something else important.
>
> What's your plan for releasing 4.2.4?
>
> In terms of changes in operation since 4.2.3 it looks like just:
>
> ?Fix double-push attempt.
> ?https://github.com/jgarber/redcloth/commit/25360cdc922bea6f5f956ec3ddfbfc6c1ec54018
>
> and
>
> ?fix dangling <li>
> ?https://github.com/jgarber/redcloth/commit/ade0bfd1a44dedc2498112a069d0c77505ceb5e4
>
> Some other changes to the build process.
> _______________________________________________
> Redcloth-upwards mailing list
> Redcloth-upwards at rubyforge.org
> http://rubyforge.org/mailman/listinfo/redcloth-upwards
>

From 4ux6as402 at sneakemail.com  Thu Jan 27 07:59:25 2011
From: 4ux6as402 at sneakemail.com (=?ISO-8859-1?Q?=22Peter_Valdemar_M=F8rch_=28Lists=29=22?=)
Date: Thu, 27 Jan 2011 13:59:25 +0100
Subject: Installing on CentOS
In-Reply-To: <AANLkTi=hTCEywYpT7PV4vuLoBzox60u9DeB+E4KK3aco@mail.gmail.com>
References: <AANLkTinHJ+8VRkVqWuxXPNBtbti+5_5HgY2kbOHCywWV@mail.gmail.com>
	<AANLkTikEoJ2K0d0LWXyRKjxAfMST_LDWdpQGf5wwc1Q6@mail.gmail.com>
	<AANLkTi=hTCEywYpT7PV4vuLoBzox60u9DeB+E4KK3aco@mail.gmail.com>
Message-ID: <4D416C2D.5010603@sneakemail.com>

On 2011-01-24 00:47, Douglas Pearson biz-at-sunnyhome.org |Lists/Send to 
lists| wrote:
> One lesson I've learned is that Ruby is very version sensitive - so best
> not to swim upstream against that.

I've also had to learn this over and over with different projects. It 
sucks and is the main reason I'm hesitant to use Ruby.

Peter