From elk9r9q02 at sneakemail.com  Thu Sep  1 13:05:29 2005
From: elk9r9q02 at sneakemail.com (Venkat)
Date: Thu Sep  1 12:58:43 2005
Subject: [Instiki-devel] Instiki-AR critial bug
Message-ID: <16419-26443@sneakemail.com>

Dear all:

Summary: In some cases, the page would be rendered with all links
showing some internal meaningless string.

Details:

Behavior in v 0.10.1

Each chunk (content part) was given a mask as the following:

    def id
      @id ||= "#{@content.page_id}n#{@content.chunk_id}"
    end

Where the page_id and chunk_id were defined as positive integers
starting from 0. The pattern used for the above is as following:

    def Abstract::mask_re(chunk_types)
      tmp = chunk_types.map{|klass| klass.mask_string}.join("|")
      Regexp.new("chunk([0-9a-f]+n\\d+)(#{tmp})chunk")
    end

Behavior in Instiki-AR:

Chunk mask is 

    # should contain only [a-z0-9]
    def mask
      @mask ||= "chunk#{self.object_id}#{self.class.mask_string}chunk"
    end

We should note here that the object_id in Ruby can be a positive or
NEGATIVE INTEGER.

Pattern is

    def Abstract::mask_re(chunk_types)
      chunk_classes = chunk_types.map{|klass| klass.mask_string}.join("|")
      /chunk(\d+)(#{chunk_classes})chunk/
    end

The above pattern won't match negative integers and hence the
rendering will be left with masks.

Fix:

The short term fix for this is to change the above pattern to look for
negative numbers also

      #Before \d+ added -*
      /chunk(-*\d+)(#{chunk_classes})chunk/   

PS: 1. It took sometime for me to figure this out because, in Win XP,
things were working as object_ids for some reason were always a
postive number. In my shared host Linux machine, the number was always
negative.
2. For some reason, from here in Bangalore-India, the instiki.org
server seems to be down most of the times. It would help if Alex can
investigate this. I don't have this problem with other sites like ROR
, etc. I have a very good broadband connection here.

Hope this helps in fixing this problem.

Cheers,
Venkat.

From alex at verk.info  Sun Sep 11 00:37:35 2005
From: alex at verk.info (Alexey Verkhovsky)
Date: Sun Sep 11 00:30:49 2005
Subject: [Instiki-devel] Caching strategy - just caching pre-rendered pages
	not enough
Message-ID: <4323B48F.9090201@verk.info>

I finally got to the point of trying to run Instiki AR branch on a 
reasonably large amount of data.

As I always knew, merely caching prerendered pages is not sufficient. 
Basically, rendering is required to establish which page references 
which, as well as categories.
Therefore, every attempt to display All Pages, or Recently Revised takes 
several minutes. Too much, even if we cache it all.

What I am going to do then is to create tables page_categories and 
references:

create table page_categories (
  number id,
  number page_id,
  varchar category
)

create table references (
  number id,
  number page_id,
  varchar referenced_page_name, -- it has to be by name because of 
"wanted page" links
  char reference_type
)

where reference type can be L (linked page), W (wanted page) or I 
(included page)

PageRenderer instance will be able to return lists of Reference and 
Category objects after rendering a revision. Page#revise will clear old 
entries (if any) and insert new ones. It will also update 'Wanted' 
references to 'Linked' ones for a particular page.

Thoughts?

Alex

From alex at verk.info  Sun Sep 11 13:09:06 2005
From: alex at verk.info (Alexey Verkhovsky)
Date: Sun Sep 11 13:02:24 2005
Subject: [Instiki-devel] Caching strategy - just caching pre-rendered
	pages	not enough
In-Reply-To: <4323B48F.9090201@verk.info>
References: <4323B48F.9090201@verk.info>
Message-ID: <432464B2.307@verk.info>

Alexey Verkhovsky wrote:

> What I am going to do then is to create tables page_categories and 
> references:

Done. Avoided creation of a separate page_categories table by making it 
a wiki_references row with link_type = 'C'.
Performance of All Pages and Revised Pages is now reasonable (since 
there is no more rendering involved), at least for a 70-pages wiki that 
I am testing with.

Rendering of includes is broken.

The next step regarding performance is to properly implement page 
caching (and sweeping) for everything, especially show and RSS feeds.

Alex

From alex at verk.info  Sun Sep 11 14:04:15 2005
From: alex at verk.info (Alexey Verkhovsky)
Date: Sun Sep 11 13:57:32 2005
Subject: [Instiki-devel] Caching strategy - just caching pre-rendered
	pages	not enough
In-Reply-To: <432464B2.307@verk.info>
References: <4323B48F.9090201@verk.info> <432464B2.307@verk.info>
Message-ID: <4324719F.9080707@verk.info>

Alexey Verkhovsky wrote:

> The next step regarding performance is to properly implement page 
> caching (and sweeping) for everything, especially show and RSS feeds.

Done! Gosh, this was embarrasingly easy :)

Alex

From elk9r9q02 at sneakemail.com  Tue Sep 13 08:42:13 2005
From: elk9r9q02 at sneakemail.com (Venkat)
Date: Tue Sep 13 08:35:05 2005
Subject: [Instiki-devel] Disappointed
In-Reply-To: <16419-26443@sneakemail.com>
References: <16419-26443@sneakemail.com>
Message-ID: <14589-98402@sneakemail.com>

I am a little disappointed with the lack of feedback I am getting from
this list.

In responding to a call to try out the new AR version, here I went
tried it out, troubleshoot a serious bug and even post a fix for it
and none of the developers respond with some cursory message.

Perhaps they don't welcome my contribution.

On 9/1/05, Venkat elk9r9q02-at-sneakemail.com |instiki-dev|
<...> wrote:
> Dear all:
> 
> Summary: In some cases, the page would be rendered with all links
> showing some internal meaningless string.
> 
> Details:
> 
> Behavior in v 0.10.1
> 
> Each chunk (content part) was given a mask as the following:
> 
>     def id
>       @id ||= "#{@content.page_id}n#{@content.chunk_id}"
>     end
> 
> Where the page_id and chunk_id were defined as positive integers
> starting from 0. The pattern used for the above is as following:
> 
>     def Abstract::mask_re(chunk_types)
>       tmp = chunk_types.map{|klass| klass.mask_string}.join("|")
>       Regexp.new("chunk([0-9a-f]+n\\d+)(#{tmp})chunk")
>     end
> 
> Behavior in Instiki-AR:
> 
> Chunk mask is
> 
>     # should contain only [a-z0-9]
>     def mask
>       @mask ||= "chunk#{self.object_id}#{self.class.mask_string}chunk"
>     end
> 
> We should note here that the object_id in Ruby can be a positive or
> NEGATIVE INTEGER.
> 
> Pattern is
> 
>     def Abstract::mask_re(chunk_types)
>       chunk_classes = chunk_types.map{|klass| klass.mask_string}.join("|")
>       /chunk(\d+)(#{chunk_classes})chunk/
>     end
> 
> The above pattern won't match negative integers and hence the
> rendering will be left with masks.
> 
> Fix:
> 
> The short term fix for this is to change the above pattern to look for
> negative numbers also
> 
>       #Before \d+ added -*
>       /chunk(-*\d+)(#{chunk_classes})chunk/
> 
> PS: 1. It took sometime for me to figure this out because, in Win XP,
> things were working as object_ids for some reason were always a
> postive number. In my shared host Linux machine, the number was always
> negative.
> 2. For some reason, from here in Bangalore-India, the instiki.org
> server seems to be down most of the times. It would help if Alex can
> investigate this. I don't have this problem with other sites like ROR
> , etc. I have a very good broadband connection here.
> 
> Hope this helps in fixing this problem.
> 
> Cheers,
> Venkat.
> 
> _______________________________________________
> Instiki-devel mailing list
> Instiki-devel@rubyforge.org
> http://rubyforge.org/mailman/listinfo/instiki-devel
>

From alex at verk.info  Tue Sep 13 22:06:47 2005
From: alex at verk.info (Alexey Verkhovsky)
Date: Tue Sep 13 21:59:52 2005
Subject: [Instiki-devel] Disappointed
In-Reply-To: <14589-98402@sneakemail.com>
References: <16419-26443@sneakemail.com> <14589-98402@sneakemail.com>
Message-ID: <432785B7.3020001@verk.info>

Venkat wrote:

>I am a little disappointed with the lack of feedback I am getting from
>this list.
>
>In responding to a call to try out the new AR version, here I went
>tried it out, troubleshoot a serious bug and even post a fix for it
>and none of the developers respond with some cursory message.
>
>Perhaps they don't welcome my contribution.
>  
>
Pouring ashes on my head, in shame. I just missed your email. Sorry.

Your contribution is certainly very much appreciated, as it saves us 
from a big embarrassment. Thanks a lot for it.

I've modified the mask from (-*\d+) to (-?\d+), assuming that there is 
either no dash or a single dash.
Just in case, can you please check out revision 389 or later and confirm 
that it works on youor PC now?

Best regards,
Alex

From alex at verk.info  Tue Sep 13 22:37:50 2005
From: alex at verk.info (Alexey Verkhovsky)
Date: Tue Sep 13 22:30:56 2005
Subject: [Instiki-devel] SQL-based Instiki - where are we
Message-ID: <43278CFE.7080903@verk.info>

Hi all,

In recent days several people asked me how is the work on instiki-ar 
branch getting on. It is getting on fairly well.
By now, we have a version that seems quite fast enough for a small wiki 
(say, upto 100-200 pages). We also have a conversion script that can 
port the contents of all revisions from Madeleine storage into the wiki.
Issues that need to be addressed before we can release it are:

1. At least one know critical problem with the [[!include]] tag try to 
include a page with any WikiLink on it, and see how it doesn't work. 
Once this is fixed (and some more testing is performed), instiki-ar 
should become a usable software. We are probably just a few days away 
from this point now!
2. Storing cached fragments in database. Shouldn't be too difficult, we 
just didn't quite get to it yet.
3. Some other optimizations (such as caching web and web.pages for the 
duration of request, doing something about Web#has_page?(name) method, 
which gets called very often and makes a roundtrip to the database on 
every call).
4. A sizable effort is still required on prepackaging Instiki with 
SQLite for all supported platforms (to observe the No Step Three dogma).
5. Deployment on Lighttpd and Apache needs to be tested and sorted out.
6. More testing, including scalability for large wikis (e.g., 
http://rubyonrails.com)

If you want SQL-based Instiki to happen sooner than later, please look 
into one of the issues listed above. A prompt fix for the Include bug 
would be particularly apreciated (I wont have time to work on it untill 
weekend myself).

To run the conversion script:

1. Put the particulars of your databases into config/database.yml
2. Execute script/create_db to create the database objects (tables and 
indices)
3. Execute script/import_storage (do "ruby script/import_storage -h" for 
te list of options)
4. Logon to the database and run the SQL script created by import_storage

Remember that "instiki -e development" does not cache anything, and 
reloads a lot of source files before each request - as opposed to 
"instiki -e production" which caches a lot of things, including almost 
all rendered pages.

Best regards,
Alex