From jg at jasongarber.com Thu Aug 23 14:37:04 2007 From: jg at jasongarber.com (Jason Garber) Date: Thu, 23 Aug 2007 14:37:04 -0400 Subject: Eliminating bad tests from SuperRedCloth Message-ID: Hi, all. I haven't felt at liberty to go axing test cases, but some of the 33 that are failing now are ones I don't think we want to pass? cruft coming from Markdown or made up to fill a gap some years ago. I'd like to float the proposition that we make Super (and thus RedCloth 4) behave as identically as possible to Textile2 as working on http://textile.thresholdstate.com/ (the textpattern site seems to be running an older version of this). There are some new things Textile2 does that SRC does not and I'd like to impliment those and also strip out the cruft I mentioned. This may break backwards compatibility for some people's content (I'm thinking perhaps the poignant test cases), but I think that's okay with a major release (3 -> 4) such as this would be. You can't ever move ahead if you don't own up to mistakes and leave them behind. What do you all think? Can I start cleaning up test cases to be how they ought to be? I'll do my research carefully and be testing it with real content (in a university CMS) over the next few months. I want the end result to be as stable, predictable, and intuitive as anyone. Thanks! Jason Garber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/redcloth-upwards/attachments/20070823/974f1689/attachment.html From stephen.bannasch at deanbrook.org Thu Aug 23 15:48:40 2007 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Thu, 23 Aug 2007 15:48:40 -0400 Subject: Eliminating bad tests from SuperRedCloth In-Reply-To: References: Message-ID: >Hi, all. I haven't felt at liberty to go axing test cases, but some >of the 33 that are failing now are ones I don't think we want to >pass-cruft coming from Markdown or made up to fill a gap some years >ago. > >I'd like to float the proposition that we make Super (and thus >RedCloth 4) behave as identically as possible to Textile2 as working >on >http://textile.thresholdstate.com/ >(the textpattern site seems to be running an older version of this). >There are some new things Textile2 does that SRC does not and I'd >like to impliment those and also strip out the cruft I mentioned. >This may break backwards compatibility for some people's content >(I'm thinking perhaps the poignant test cases), Can you categorize the way the sets of functionality relate: I'm most interested in the sets in this order: * functionality in existing SRC that is incompatible with T2. By this I mean that just because this functionality exists SRC could not in principle pass a complete suite of T2 tests. * functionality in T2 that is NOT in existing SRC * functionality in existing SRC that not in T2. Is there a document that clearly specifies the language rules/spec of T2? In general I think it's great that you want to clean up the test cases. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/redcloth-upwards/attachments/20070823/ccaa81fc/attachment.html From jg at jasongarber.com Thu Aug 23 16:37:16 2007 From: jg at jasongarber.com (Jason Garber) Date: Thu, 23 Aug 2007 16:37:16 -0400 Subject: Eliminating bad tests from SuperRedCloth In-Reply-To: References: Message-ID: Unfortunately, I can't come up with a list of the sets you request, Stephen. Off-hand, I know definition lists are done differently in RedCloth (not implemented in SRC) than in T2; backticks to denote code were in RC (don't know about SRC). What I'd like to do is hook up another test that runs all the cases (yml files) through T2 (php) and compares the output. I don't necessarily want that to be a permanent part of SRC's test suite, but it would be interesting to see exactly how they handle things differently. I think I've encountered a few cases where T2 has undesirable output but SRC does it right. Obviously we'd want to preserve SRC's way. Language ref? The Movable Type Textile2 manual is the best I've found. http://www.bradchoate.com/mt/docs/mtmanual_textile2.html Certainly wish Textile2 (php) had some test cases that I could just port over to Ruby! On Aug 23, 2007, at 3:48 PM, Stephen Bannasch wrote: >> Hi, all. I haven't felt at liberty to go axing test cases, but >> some of the 33 that are failing now are ones I don't think we want >> to pass-cruft coming from Markdown or made up to fill a gap some >> years ago. >> >> I'd like to float the proposition that we make Super (and thus >> RedCloth 4) behave as identically as possible to Textile2 as >> working on http://textile.thresholdstate.com/ (the textpattern >> site seems to be running an older version of this). There are some >> new things Textile2 does that SRC does not and I'd like to >> impliment those and also strip out the cruft I mentioned. This may >> break backwards compatibility for some people's content (I'm >> thinking perhaps the poignant test cases), > > Can you categorize the way the sets of functionality relate: > > I'm most interested in the sets in this order: > > * functionality in existing SRC that is incompatible with T2. By > this I mean that just because this functionality exists SRC could > not in principle pass a complete suite of T2 tests. > * functionality in T2 that is NOT in existing SRC > * functionality in existing SRC that not in T2. > > Is there a document that clearly specifies the language rules/spec > of T2? > > In general I think it's great that you want to clean up the test > cases. > _______________________________________________ > Redcloth-upwards mailing list > Redcloth-upwards at rubyforge.org > http://rubyforge.org/mailman/listinfo/redcloth-upwards -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/redcloth-upwards/attachments/20070823/5db4daa7/attachment-0001.html From stephen.bannasch at deanbrook.org Thu Aug 23 16:49:08 2007 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Thu, 23 Aug 2007 16:49:08 -0400 Subject: Eliminating bad tests from SuperRedCloth In-Reply-To: References: Message-ID: At 4:37 PM -0400 8/23/07, Jason Garber wrote: >Unfortunately, I can't come up with a list of the sets you request, Stephen. Off-hand, I know definition lists are done differently in RedCloth (not implemented in SRC) than in T2; backticks to denote code were in RC (don't know about SRC). You just did come up with a list -- it's partial and informal -- but that is a great place to start! >What I'd like to do is hook up another test that runs all the cases (yml files) through T2 (php) and compares the output. I don't necessarily want that to be a permanent part of SRC's test suite, but it would be interesting to see exactly how they handle things differently. I think I've encountered a few cases where T2 has undesirable output but SRC does it right. Obviously we'd want to preserve SRC's way. I'd be interested in your tests and the results. Please post them to the list. From jg at jasongarber.com Fri Aug 24 11:21:17 2007 From: jg at jasongarber.com (Jason Garber) Date: Fri, 24 Aug 2007 11:21:17 -0400 Subject: Eliminating bad tests from SuperRedCloth In-Reply-To: References: Message-ID: Okay, last night I hooked up the tests to Textile-2.0.0. Lots of failures, but it's hard to talk about them in a meaningful way on this list (there's a 40KB limit, so I can't even attach the test failures). Rather than contriving examples and lots of copying and pasting into emails, I think I'd rather make changes to the tests and then everyone can look at the changeset on Trac or look through the YAML files if they want and make sure it's agreeable to all. I will be quick to roll back some or all of my changes if there's opposition, especially from _why. I'm not trying to do any code fisting here. :-) On Aug 23, 2007, at 4:49 PM, Stephen Bannasch wrote: > At 4:37 PM -0400 8/23/07, Jason Garber wrote: >> Unfortunately, I can't come up with a list of the sets you >> request, Stephen. Off-hand, I know definition lists are done >> differently in RedCloth (not implemented in SRC) than in T2; >> backticks to denote code were in RC (don't know about SRC). > > You just did come up with a list -- it's partial and informal -- > but that is a great place to start! > >> What I'd like to do is hook up another test that runs all the >> cases (yml files) through T2 (php) and compares the output. I >> don't necessarily want that to be a permanent part of SRC's test >> suite, but it would be interesting to see exactly how they handle >> things differently. I think I've encountered a few cases where T2 >> has undesirable output but SRC does it right. Obviously we'd want >> to preserve SRC's way. > > I'd be interested in your tests and the results. Please post them > to the list. > _______________________________________________ > Redcloth-upwards mailing list > Redcloth-upwards at rubyforge.org > http://rubyforge.org/mailman/listinfo/redcloth-upwards From jg at jasongarber.com Fri Aug 24 12:49:24 2007 From: jg at jasongarber.com (Jason Garber) Date: Fri, 24 Aug 2007 12:49:24 -0400 Subject: Test cleanup: code.yml In-Reply-To: References: Message-ID: <95CB3A75-D4E4-4926-8AA1-42DAD694366F@jasongarber.com> I'm going to be working one YAML file at a time and posting my comments to the list. No substitute for reading the diffs and yml files yourself, but it helps to have a summary. I want to be very open about the changes. Changes: * Escaping needs to be different within code tags (or @ or bc.) * Removed triple backticks as a code signature * Switched tests that used 
HTML.  I don't  
see how it's useful, but Textile2 does it.  SRC doesn't yet.
* Added a test to ensure hard breaks aren't inserted between HTML  
elements (like LIs or 
\n)

Remaining differences from Textile2:
* Textile2 doesn't seem to like multiple ==ignores== on one line
* Textile2 uses numerical entities.  I prefer SRC's already-working  
named ones.
* Textile2 doesn't observe hard breaks inside HTML elements (see:  
"and it will not put a 
 between them")
* If something is wrapped in  tags, Textile2 still puts it  
in a paragraph

You can see how actual SRC output measures up to expected output with
   rake test:parser:code
and see how the expected output measures up to Textile 2.0.0's output  
with
   rake test:textile2:code

My AIM name is JasonGarberEMU if you want to chat.

From jg at jasongarber.com  Fri Aug 24 16:41:18 2007
From: jg at jasongarber.com (Jason Garber)
Date: Fri, 24 Aug 2007 16:41:18 -0400
Subject: Test cleanup: definitions.yml
In-Reply-To: 
References: 
	
	
	
Message-ID: <382EAE0D-9F2C-46F1-8779-921416F6B114@jasongarber.com>

Changes: none

Differences:
* Textile-2.0.0 (php) doesn't support definition lists.  MT-Textile  
(perl) does support a dl. term:definition format.  RedCloth already  
has its own definition list format.  Neither are implemented in SRC,  
but one ought to be.  The question for you all is: which one?

From jg at jasongarber.com  Fri Aug 24 16:49:24 2007
From: jg at jasongarber.com (Jason Garber)
Date: Fri, 24 Aug 2007 16:49:24 -0400
Subject: Test cleanup: images.yml
In-Reply-To: 
References: 
	
	
	
Message-ID: 

SRC's image handling works really well.  All tests pass for the  
parser.  Not many do against Textile2 because it includes an empty  
title attribute and it doesn't handle image URLs near as well as SRC.

From jg at jasongarber.com  Fri Aug 24 17:39:24 2007
From: jg at jasongarber.com (Jason Garber)
Date: Fri, 24 Aug 2007 17:39:24 -0400
Subject: Test cleanup: instiki.yml
In-Reply-To: 
References: 
	
	
	
Message-ID: <4521F2EE-0577-447A-AA2F-4C9975EEA134@jasongarber.com>

Changes:
* Fixed missing hard breaks in expected output

Questions:
* A single dash - surrounded by spaces - comes out as &8211; but eats  
the spaces on either side.  I know _why talked about this with em  
dashes and I'm so glad he had it eat spaces surrounding the em dash.   
Should en dashes also eat the surrounding spaces?
* Take a look at how divs are opened (and closed?) in test_instiki_7  
& 8.  Textile2 nor MT-Textile support this and the way of closing  
seems unintuitive.  I'd like to remove these tests and let div(foo).  
work just like p(foo).  Anything more complex and you can just use  
HTML.  Anyone disagree?

From stephen.bannasch at deanbrook.org  Fri Aug 24 17:45:53 2007
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Fri, 24 Aug 2007 17:45:53 -0400
Subject: Test cleanup: code.yml
In-Reply-To: <95CB3A75-D4E4-4926-8AA1-42DAD694366F@jasongarber.com>
References: 
	
	
	
	<95CB3A75-D4E4-4926-8AA1-42DAD694366F@jasongarber.com>
Message-ID: 

Hi Jason,

Here's a comment:
>* Textile2 uses numerical entities.  I prefer SRC's already-working 
>named ones.

Just which entities are you referring to?

I've got a patch which fixes some problems I've been having and they both relate to entities in different situations.

When an ampersand is used in a url and SRC translates it to the entity & this causes problems later for me with some html rendering engines.  The following patch instead replaces the ampersand with the entity &.

The second problem is that the I am also sending the xhtml output through some xml processing and enitites like the right arrow cause a problem unless I also declare the entity at the top of he xml document (which contains SRC output along with lost of other stuff). Instead I use the Unicode entity → for the arrow.

Neither of these changes appear to cause any problems:

Index: lib/superredcloth.rb
===================================================================
--- lib/superredcloth.rb        (revision 180)
+++ lib/superredcloth.rb        (working copy)
@@ -82,7 +82,7 @@
   def image opts
     p_opts = {:float => opts.delete(:align)} if opts[:align]
     opts[:alt] = opts[:title]
-    img = "\"#{opts[:alt]}\"" 
+    img = "\"#{opts[:alt]}\"" 
     img = "#{img}" if opts[:href]
     img = "#{img}
" if p_opts
     img
@@ -115,7 +115,7 @@
     "–"
   end
   def arrow opts
-    "→"
+    '→'
   end
   def dim opts
     "#{opts[:x]}×#{opts[:y]}"
@@ -133,6 +133,6 @@
     "&#{opts[:text]};"
   end
   def urlesc txt
-    txt.gsub(/&/, '&')
+    txt.gsub(/&/, '&')
   end
 end

From why at hobix.com  Sat Aug 25 23:47:52 2007
From: why at hobix.com (why the lucky stiff)
Date: Sat, 25 Aug 2007 22:47:52 -0500
Subject: Eliminating bad tests from SuperRedCloth
In-Reply-To: 
References: 
	
	
	
	
Message-ID: <20070826034752.GW1705@beekeeper.hobix.com>

On Fri, Aug 24, 2007 at 11:21:17AM -0400, Jason Garber wrote:
> I will be quick to roll back some or all of my changes if there's  
> opposition, especially from _why.  I'm not trying to do any code  
> fisting here. :-)

Toosk toosk.  On the contrary.  You're doing great and I'm verrry
mentally occupied with Shoes and Hackety Hack.  Just go with it, I'm
really totally happy about it.

_why

From jg at jasongarber.com  Mon Aug 27 11:55:13 2007
From: jg at jasongarber.com (Jason Garber)
Date: Mon, 27 Aug 2007 11:55:13 -0400
Subject: Test cleanup: code.yml
In-Reply-To: 
References: 
	
	
	
	<95CB3A75-D4E4-4926-8AA1-42DAD694366F@jasongarber.com>
	
Message-ID: 

Stephen, I was referring to the translation of "Barnes & Noble" into  
"Barnes & Noble" in regular text.  Textile2 does it as "Barnes  
& Noble" so I prefer the SuperRedCloth way of converting certain  
entities into the named entities instead of numerical ones (see  
superredcloth_inline.rl about line 209).  It still doesn't convert  
all entities though, so if I write a pound sign (?), it is not  
converted to £ or £.  Should it be?

I applied your patch.  Thanks!

On Aug 24, 2007, at 5:45 PM, Stephen Bannasch wrote:

> Hi Jason,
>
> Here's a comment:
>> * Textile2 uses numerical entities.  I prefer SRC's already-working
>> named ones.
>
> Just which entities are you referring to?
>
> I've got a patch which fixes some problems I've been having and  
> they both relate to entities in different situations.
>
> When an ampersand is used in a url and SRC translates it to the  
> entity & this causes problems later for me with some html  
> rendering engines.  The following patch instead replaces the  
> ampersand with the entity &.
>
> The second problem is that the I am also sending the xhtml output  
> through some xml processing and enitites like the right arrow cause  
> a problem unless I also declare the entity at the top of he xml  
> document (which contains SRC output along with lost of other  
> stuff). Instead I use the Unicode entity → for the arrow.
>
> Neither of these changes appear to cause any problems:
>
> Index: lib/superredcloth.rb
> ===================================================================
> --- lib/superredcloth.rb        (revision 180)
> +++ lib/superredcloth.rb        (working copy)
> @@ -82,7 +82,7 @@
>    def image opts
>      p_opts = {:float => opts.delete(:align)} if opts[:align]
>      opts[:alt] = opts[:title]
> -    img = "\"#{opts [:alt]}\" />"
> +    img = "\"# {opts[:alt]}\" />"
>      img = "#{img}" if opts 
> [:href]
>      img = "#{img}
" if p_opts
>      img
> @@ -115,7 +115,7 @@
>      "–"
>    end
>    def arrow opts
> -    "→"
> +    '→'
>    end
>    def dim opts
>      "#{opts[:x]}×#{opts[:y]}"
> @@ -133,6 +133,6 @@
>      "&#{opts[:text]};"
>    end
>    def urlesc txt
> -    txt.gsub(/&/, '&')
> +    txt.gsub(/&/, '&')
>    end
>  end
> _______________________________________________
> Redcloth-upwards mailing list
> Redcloth-upwards at rubyforge.org
> http://rubyforge.org/mailman/listinfo/redcloth-upwards


From jg at jasongarber.com  Mon Aug 27 12:19:52 2007
From: jg at jasongarber.com (Jason Garber)
Date: Mon, 27 Aug 2007 12:19:52 -0400
Subject: Test cleanup: links.yml and lists.yml
In-Reply-To: 
References: 
	
	
	
Message-ID: <6D088631-723A-4209-AEA7-9CC23FB3B87C@jasongarber.com>

links.yml: nothing to change!

lists.yml:
Test 3 ("Funky...") doesn't make a whole lot of sense to me.  I get  
confused with all the nesting and type switching.  Textile2 doesn't  
have a clue either.  Someone else want to take a look at it and make  
a recommendation?

Starting/continuation of numbering is the biggest thing broken.   
test_lists_10 and 11 are failing because they try to continue  
numbering within a list (not using the start attribute but handling  
it with paragraph tags within the LI).  I'm removing those parts  
because I don't think it's something Super should do.  Take a look at  
the diff and let me know if you disagree.


From jg at jasongarber.com  Mon Aug 27 14:12:03 2007
From: jg at jasongarber.com (Jason Garber)
Date: Mon, 27 Aug 2007 14:12:03 -0400
Subject: Test cleanup: poignant.yml & important alignment modifier question
In-Reply-To: 
References: 
	
	
	
Message-ID: <5A4C0E76-E03A-489E-8BF7-6E81123B3014@jasongarber.com>

Other than being crazy long, this test is okay.  It raises a question  
I've had for awhile, though: what should alignment modifiers do?

!>/image.gif! right-aligned image


 right-aligned image  (Current SRC expected output)

 right- 
aligned image
  (Current SRC actual output)

 right-aligned image
   
(What Textile2 (Textism, WordPress, and MovableType-non-CSS do)

 right-aligned image
   
(What I would prefer)

The align attribute is deprecated, but forcing a right float gives me  
the willies. What if you want to have some left margin on right- 
floating images and right margin on left-floating ones?  You can't  
really detect or override the float in your CSS.

Is anyone opposed to changing the alignment modifiers to have effect  
on class and leaving it to the designer to write styles for those  
classes?  Is there a better solution?

From jg at jasongarber.com  Mon Aug 27 14:53:17 2007
From: jg at jasongarber.com (Jason Garber)
Date: Mon, 27 Aug 2007 14:53:17 -0400
Subject: Test cleanup: table.yml
In-Reply-To: 
References: 
	
	
	
Message-ID: <2189F2D0-FB76-48A3-96E2-058AE502CBFA@jasongarber.com>

I cleaned up a bunch of duplicate rows in tables that were  
unnecessary for testing purposes.

There's one test that uses a period between vertical pipes to  
represent an empty table cell.  The expected output shows |.| is  
rendered as   but actual output leaves the dot.   
Textile2 leaves the dot too.  If you remove the period from the input  
and just have two vertical pipes ||, Textile2 ignores the cell (and  
leaves the row one short) and SRC puts the pipe with the next cell,  
so that won't work.

The empty cell "feature" (|.|) is not documented anywhere and is  
unique to RedCloth. And it's not implemented in SRC.  Can it be dropped?

From why at hobix.com  Mon Aug 27 16:02:19 2007
From: why at hobix.com (why the lucky stiff)
Date: Mon, 27 Aug 2007 15:02:19 -0500
Subject: Test cleanup: poignant.yml & important alignment modifier question
In-Reply-To: <5A4C0E76-E03A-489E-8BF7-6E81123B3014@jasongarber.com>
References: 
	
	
	
	<5A4C0E76-E03A-489E-8BF7-6E81123B3014@jasongarber.com>
Message-ID: <20070827200219.GA1705@beekeeper.hobix.com>

On Mon, Aug 27, 2007 at 02:12:03PM -0400, Jason Garber wrote:
> Is anyone opposed to changing the alignment modifiers to have effect  
> on class and leaving it to the designer to write styles for those  
> classes?  Is there a better solution?

What if we leave the default behavior to float?  And then we allow
an override to hook up to a CSS class, if they like.  I the idea
of using classes much better, but I'm worried about people not
seeing instant results when they haven't got the right CSS, you
know?

_why

From jg at jasongarber.com  Mon Aug 27 17:25:43 2007
From: jg at jasongarber.com (Jason Garber)
Date: Mon, 27 Aug 2007 17:25:43 -0400
Subject: Test cleanup: textism.yml & removing sanitize, filter,
	and hard_breaks tests
In-Reply-To: 
References: 
	
	
	
Message-ID: <2E154DBB-BA9E-4FB8-9921-18D0BEF123A0@jasongarber.com>

h2. textism.yml
Broke the huge sample test from http://textism.com/tools/textile/? 
sample=2 into manageable chunks.
Re-enabled the commented-out test of providing a named URL anywhere  
in the document and linking to it.
Added footnote classes that are present in Textile2 but not in SRC.

h2. Removing sanitize, filter, and hard_breaks tests
I removed sanitize and filter tests since these are not options  
(yet?) in SuperRedCloth.  Someone can reinstate them when they  
implement these features.  I integrated the hard_breaks tests into  
textism.yml and removed it as a separate set of tests.

This is the end of my cleaning up tests.  Now to fix bugs!  IM me if  
you're going to be fixing some too.  I'd love help!

AIM: JasonGarberEMU

And please give me your input on all the questions I asked in  
previous emails!

From jg at jasongarber.com  Mon Aug 27 17:26:31 2007
From: jg at jasongarber.com (Jason Garber)
Date: Mon, 27 Aug 2007 17:26:31 -0400
Subject: Test cleanup: poignant.yml & important alignment modifier question
In-Reply-To: <20070827200219.GA1705@beekeeper.hobix.com>
References: 
	
	
	
	<5A4C0E76-E03A-489E-8BF7-6E81123B3014@jasongarber.com>
	<20070827200219.GA1705@beekeeper.hobix.com>
Message-ID: <45C6852F-5AFA-4A50-8E42-A308436D49CE@jasongarber.com>

Yes, that crossed my mind.  Excellent suggestion!

On Aug 27, 2007, at 4:02 PM, why the lucky stiff wrote:

> On Mon, Aug 27, 2007 at 02:12:03PM -0400, Jason Garber wrote:
>> Is anyone opposed to changing the alignment modifiers to have effect
>> on class and leaving it to the designer to write styles for those
>> classes?  Is there a better solution?
>
> What if we leave the default behavior to float?  And then we allow
> an override to hook up to a CSS class, if they like.  I the idea
> of using classes much better, but I'm worried about people not
> seeing instant results when they haven't got the right CSS, you
> know?
>
> _why
> _______________________________________________
> Redcloth-upwards mailing list
> Redcloth-upwards at rubyforge.org
> http://rubyforge.org/mailman/listinfo/redcloth-upwards


From why at hobix.com  Mon Aug 27 21:31:39 2007
From: why at hobix.com (why the lucky stiff)
Date: Mon, 27 Aug 2007 20:31:39 -0500
Subject: Test cleanup: table.yml
In-Reply-To: <2189F2D0-FB76-48A3-96E2-058AE502CBFA@jasongarber.com>
References: 
	
	
	
	<2189F2D0-FB76-48A3-96E2-058AE502CBFA@jasongarber.com>
Message-ID: <20070828013139.GB1705@beekeeper.hobix.com>

On Mon, Aug 27, 2007 at 02:53:17PM -0400, Jason Garber wrote:
> The empty cell "feature" (|.|) is not documented anywhere and is  
> unique to RedCloth. And it's not implemented in SRC.  Can it be dropped?

I may have taken this from PyTextile, from which I pulled some of
these tests.  It's so obscure and minor, I really don't mind if it
goes it away.

_why

From stephen.bannasch at deanbrook.org  Mon Aug 27 21:57:23 2007
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Mon, 27 Aug 2007 21:57:23 -0400
Subject: empty cell, was: Test cleanup: table.yml
In-Reply-To: <20070828013139.GB1705@beekeeper.hobix.com>
References: 
	
	
	
	<2189F2D0-FB76-48A3-96E2-058AE502CBFA@jasongarber.com>
	<20070828013139.GB1705@beekeeper.hobix.com>
Message-ID: 

At 8:31 PM -0500 8/27/07, why the lucky stiff wrote:
>On Mon, Aug 27, 2007 at 02:53:17PM -0400, Jason Garber wrote:
> > The empty cell "feature" (|.|) is not documented anywhere and is 
>> unique to RedCloth. And it's not implemented in SRC.  Can it be dropped?
>
>I may have taken this from PyTextile, from which I pulled some of
>these tests.  It's so obscure and minor, I really don't mind if it
>goes it away.

I can't think of a reason to need an empty cell:

  '|.|' 

that can't be filled by a cell with one space:

  '| |' =>  


From stephen.bannasch at deanbrook.org  Tue Aug 28 00:25:24 2007
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Tue, 28 Aug 2007 00:25:24 -0400
Subject: entities, was: Test cleanup: code.yml
In-Reply-To: 
References: 
	
	
	
	<95CB3A75-D4E4-4926-8AA1-42DAD694366F@jasongarber.com>
	
	
Message-ID: 

At 11:55 AM -0400 8/27/07, Jason Garber wrote:
>Stephen, I was referring to the translation of "Barnes & Noble" into 
>"Barnes & Noble" in regular text.  Textile2 does it as "Barnes 
>& Noble" so I prefer the SuperRedCloth way of converting certain 
>entities into the named entities instead of numerical ones (see
>superredcloth_inline.rl about line 209).

I agree, however I'm wrestling with whether SRC should only use named entities for the first four of the five standard XML character entity references

http://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references#Character_entities_in_XML

  &  & (ampersand, U+0026)
  <   < (left angle bracket, less-than sign, U+003C)
  >   > (right angle bracket, greater-than sign, U+003E)
  " " (quotation mark, U+0022)
  ' ' (apostrophe, U+0027)

These are the only named entities that xml documents can contain unless the character entities are declared. The apostrophe entity isn't actually a standard html entity so only the first four should be used.

If the output of SRC is going to be processed by any other general XML processor any other special character references should be converted to numerical Unicode character references.

We are using an XML processing path that chokes on entity references like →. However we are specifically including XHTML content in our XML documents and right now we are not including the XHTML DTD character entity declarations -- and now I'm thinking we should include these declarations.

See: http://www.w3.org/TR/xhtml1/dtds.html#h-A2

Which references these three DTD entity files:

  http://www.w3.org/TR/xhtml1/DTD/xhtml-lat1.ent
  http://www.w3.org/TR/xhtml1/DTD/xhtml-special.ent
  http://www.w3.org/TR/xhtml1/DTD/xhtml-symbol.ent

You applied my patch which removes → (which fixes my current problem).  The only other character entity in SRC is in test/table.yml in the output around line 180. The nbsp is the output for '|.|'.  However I don't even see any code for producing the nbsp enity anymore so you could probably replace the test with '| |' and look for  .

>It still doesn't convert
>all entities though, so if I write a pound sign (?), it is not 
>converted to £ or £.  Should it be?

I'm not sure what is best. When should a 'special' character like ? be converted to an entity?

It's not part of the document structure like '<' so we don't need to convert it for that reason.

I don't think it's valid in a url so it shouldn't appear there at all.

In a document:

If you convert it to a character entity it will be human parsable but it will cause a generic XML processor to choke.

If you convert it to a numerical character referenceI think it will work everywhere but it won't be easily parsable.

If you leave it un-translated AND transport protocols don't mess with it AND it's displayed properly to users then it's probably best not to translate it at all.

From timcharper at gmail.com  Wed Aug 29 19:20:34 2007
From: timcharper at gmail.com (Tim Harper)
Date: Wed, 29 Aug 2007 17:20:34 -0600
Subject: Patch - bug fix for RedCloth when using !image! and :filter_html in
	combination
Message-ID: <001101c7ea93$34ceab00$9e6c0100$@com>

RedCloth.new("!image!", [:filter_html]).to_html will cause the following
error:

 

"error occurred while evaluating nil.gsub"

 

This patch will fix it.  Can you apply this please to next release of
RedCloth?

 

Thanks!  

 

Tim

 

--- redcloth_old.rb     2007-08-29 17:02:51.537099600 -0600

+++ redcloth.rb 2007-08-29 17:04:34.666099600 -0600

@@ -1115,7 +1115,7 @@

                         if raw[3] =~ /#{prop}\s*=\s*#{q}([^#{q2}]+)#{q}/i

                             attrv = $1

                             next if prop == 'src' and attrv =~
%r{^(?!http)\w+:}

-                            pcs << "#{prop}=\"#{$1.gsub('"', '\\"')}\""

+                            pcs << "#{prop}=\"#{attrv.gsub('"', '\\"')}\""

                             break

                         end

                     end

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/redcloth-upwards/attachments/20070829/a8690a2e/attachment-0001.html