From dbalmain.ml at gmail.com Sat Apr 1 00:15:40 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Sat, 1 Apr 2006 14:15:40 +0900 Subject: [Ferret-talk] undefined method `<=>' for :id:Symbol In-Reply-To: <9c5f400c2ea80223977a3b0b50299359@ruby-forum.com> References: <9c5f400c2ea80223977a3b0b50299359@ruby-forum.com> Message-ID: This bug has been fixed in the subversion repository. I'll release 0.9.1 soon. On 4/1/06, etienne.durand at woa.hu wrote: > Upgrading to 0.9.0, I have the following error. Anybody? > > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/term.rb:35:in > `<=>': undefined method `<=>' for :id:Symbol > (NoMethodError) > from > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/term_infos_io.rb:263:in > `get_index_offset' > from > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/term_infos_io.rb:162:in > `get_term_info' > from > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_reader.rb:176:in > `doc_freq' > from > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/search/index_searcher.rb:47:in > `doc_freq' > from > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/search/term_query.rb:13:in > `initialize' > from > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/search/term_query.rb:99:in > `create_weight' > from > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/search/query.rb:51:in > `weight' > from > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/search/index_searcher.rb:151:in > `search_each' > ... 23 levels... > from > c:/ruby/lib/ruby/gems/1.8/gems/activerecord-1.14.0/lib/active_record/base.rb:1850:in > `each_with_index' > from C:/tmp/agora/trunk/db/content/import.rb:27 > from C:/tmp/agora/trunk/db/content/import.rb:20 > from C:/tmp/agora/trunk/db/content/import.rb:13 > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From etienne.durand at woa.hu Sat Apr 1 00:52:14 2006 From: etienne.durand at woa.hu (etienne.durand at woa.hu) Date: Sat, 1 Apr 2006 07:52:14 +0200 Subject: [Ferret-talk] undefined method `<=>' for :id:Symbol In-Reply-To: References: <9c5f400c2ea80223977a3b0b50299359@ruby-forum.com> Message-ID: Thanks David, tests now pass with dev version. When do you plan to release 0.9.1? -- Posted via http://www.ruby-forum.com/. From dbalmain.ml at gmail.com Sat Apr 1 02:01:03 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Sat, 1 Apr 2006 16:01:03 +0900 Subject: [Ferret-talk] undefined method `<=>' for :id:Symbol In-Reply-To: References: <9c5f400c2ea80223977a3b0b50299359@ruby-forum.com> Message-ID: On 4/1/06, etienne.durand at woa.hu wrote: > Thanks David, tests now pass with dev version. > When do you plan to release 0.9.1? Monday. It will include UTF-8 support and the standard analyzer should work on any language that uses spaces as word delimeters. It will also include the snowball stemmer. http://snowball.tartarus.org/ It'll still be an alpha release though. > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From no137a at gmail.com Sun Apr 2 01:53:30 2006 From: no137a at gmail.com (Mr) Date: Sat, 01 Apr 2006 23:53:30 -0700 Subject: [Ferret-talk] EdgeRails: "undefined method `weight' for # References: <562a35c10603290621y53749d54jbde89d7a1e72005d@mail.gmail.com> Message-ID: I posted a resolution to the "undefined method `weight'" error message on the rails website: http://wiki.rubyonrails.com/rails/pages/HowToIntegrateFerretWithRails MikeR Alain Ravet wrote: > I started from scratch on EdgeRails, and it worked. > It's strange though, that changing the Rails version has an impact on > Ferret (that doesn't require Rails). > > Alain > From atomgiant at gmail.com Mon Apr 3 08:24:14 2006 From: atomgiant at gmail.com (Tom Davies) Date: Mon, 3 Apr 2006 08:24:14 -0400 Subject: [Ferret-talk] Installing Ferret locally on TextDrive Message-ID: I would like to give the 0.9.0 version of Ferret a try on my application hosted on TextDrive. I am currently running on the 0.3.2 version there. Does anyone have any tips on installing it locally there? I know just enough about Ruby gems to get by... but I am thinking it could be as easy as passing a -i flag to specify the install location for ferret. Then, the only thing I am not sure about is how to modify the ruby require library search path. Any tips are appreciated. Thanks, -- Tom http://blog.atomgiant.com http://gifthat.com From kraemer at webit.de Mon Apr 3 09:33:52 2006 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 3 Apr 2006 15:33:52 +0200 Subject: [Ferret-talk] Installing Ferret locally on TextDrive In-Reply-To: References: Message-ID: <20060403133352.GA15917@cordoba.webit.de> Hi Tom, I don't know anything about Textdrive, but maybe this helps. There is a 'freeze_other_gems' rake task at http://nubyonrails.com/articles/2005/12/22/freeze-other-gems-to-rails-lib-directory , that basically takes the gems you specify in line 4 of the file from your local gem repository (on your development machine) and installs them into lib/ of your Rails project. You can also copy the relevant files into RAILS_ROOT/lib manually. I have done this with Ferret 0.3.2, works like a charm and looks like this: RAILS_ROOT/ lib/ ferret_ext.so ferret.rb ferret/ analysis/ analysis.rb document/ document.rb ... hth, Jens On Mon, Apr 03, 2006 at 08:24:14AM -0400, Tom Davies wrote: > I would like to give the 0.9.0 version of Ferret a try on my > application hosted on TextDrive. I am currently running on the 0.3.2 > version there. > > Does anyone have any tips on installing it locally there? I know just > enough about Ruby gems to get by... but I am thinking it could be as > easy as passing a -i flag to specify the install location for ferret. > Then, the only thing I am not sure about is how to modify the ruby > require library search path. > > Any tips are appreciated. > > Thanks, > -- Tom > > http://blog.atomgiant.com > http://gifthat.com > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From nathaniel at talbott.ws Mon Apr 3 09:39:01 2006 From: nathaniel at talbott.ws (Nathaniel Talbott) Date: Mon, 3 Apr 2006 15:39:01 +0200 Subject: [Ferret-talk] Stop words in queries Message-ID: I've run in to an issue that I'm not sure how to address. Basically, I'm building queries with occur_default Search::BooleanClause::Occur::MUST, and using the StandardAnalyzer which does stop filtering. The stop filtering is working beautifully on the indexing side. The problem is that when the query parser parses through a query with a stop word in it, say "the oregon trail", it builds a query that looks something like this: MUST title: MUST title: oregon MUST title: trail Which unfortunately fails when searching for the previously indexed "The Oregon Trail" because it doesn't have a blank title term in it. Is there a good way to deal with this issue besides filtering stop words before handing the query string off to the parser? Thanks! Nathaniel P.S. I'm using the pure Ruby part of Ferret 0.9.0 on Ruby 1.8.4. -- Posted via http://www.ruby-forum.com/. From atomgiant at gmail.com Mon Apr 3 11:05:18 2006 From: atomgiant at gmail.com (Tom Davies) Date: Mon, 3 Apr 2006 11:05:18 -0400 Subject: [Ferret-talk] Installing Ferret locally on TextDrive In-Reply-To: <20060403133352.GA15917@cordoba.webit.de> References: <20060403133352.GA15917@cordoba.webit.de> Message-ID: Hi Jens, That looks promising and I will dig in to it when I have a chance over the next couple of days. However, looking through the comments people have had problems running it on Windows. Also, I am not sure if I can freeze a ferret gem that will work on both Windows and Linux. Do you think this would be possible? If not, i am leaning towards just tweaking my TextDrive setup to look for gems in an additional location that is outside of my deploy path... so I don't have to reinstall ferret every time I upload a new version of my application. Tom On 4/3/06, Jens Kraemer wrote: > Hi Tom, > > I don't know anything about Textdrive, but maybe this helps. > > There is a 'freeze_other_gems' rake task at > http://nubyonrails.com/articles/2005/12/22/freeze-other-gems-to-rails-lib-directory > , that basically takes the gems you specify in line 4 of the file from > your local gem repository (on your development machine) and installs > them into lib/ of your Rails project. > > You can also copy the relevant files into RAILS_ROOT/lib manually. > I have done this with Ferret 0.3.2, works like a charm and looks like > this: > > RAILS_ROOT/ > lib/ > ferret_ext.so > ferret.rb > ferret/ > analysis/ > analysis.rb > document/ > document.rb > ... > > > hth, > Jens > > On Mon, Apr 03, 2006 at 08:24:14AM -0400, Tom Davies wrote: > > I would like to give the 0.9.0 version of Ferret a try on my > > application hosted on TextDrive. I am currently running on the 0.3.2 > > version there. > > > > Does anyone have any tips on installing it locally there? I know just > > enough about Ruby gems to get by... but I am thinking it could be as > > easy as passing a -i flag to specify the install location for ferret. > > Then, the only thing I am not sure about is how to modify the ruby > > require library search path. > > > > Any tips are appreciated. > > > > Thanks, > > -- Tom > > > > http://blog.atomgiant.com > > http://gifthat.com > > > > _______________________________________________ > > Ferret-talk mailing list > > Ferret-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/ferret-talk > > -- > webit! Gesellschaft f?r neue Medien mbH www.webit.de > Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de > Schnorrstra?e 76 Tel +49 351 46766 0 > D-01069 Dresden Fax +49 351 46766 66 > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Tom Davies http://blog.atomgiant.com http://gifthat.com From kraemer at webit.de Mon Apr 3 12:49:58 2006 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 3 Apr 2006 18:49:58 +0200 Subject: [Ferret-talk] Installing Ferret locally on TextDrive In-Reply-To: References: <20060403133352.GA15917@cordoba.webit.de> Message-ID: <20060403164958.GB15917@cordoba.webit.de> On Mon, Apr 03, 2006 at 11:05:18AM -0400, Tom Davies wrote: > Hi Jens, > > That looks promising and I will dig in to it when I have a chance over > the next couple of days. However, looking through the comments people > have had problems running it on Windows. Also, I am not sure if I can > freeze a ferret gem that will work on both Windows and Linux. Do you > think this would be possible? no, I think this won't be possible because of the compiled parts. The Ruby-only version of ferret should work, though. Easy solution: switch to Linux for development ;-) > If not, i am leaning towards just tweaking my TextDrive setup to look > for gems in an additional location that is outside of my deploy > path... so I don't have to reinstall ferret every time I upload a new > version of my application. don't know how to configure rubygems for that, Imho a rubygems installation only looks for gems in the location specified at installation time. I once had to re-install all gems after installing rubygems below /usr/local, because it only looked for installed gems there. I had another version of rubygems in /usr, with plenty of gems installed, but the new rubygems did not find them. What about installing ferret into a lib/ directory inside your home directory, and symlinking the contents of this directory into your RAILS_ROOT/lib/ on deploy ? Should be easy to automate with capistrano, formerly known as switchtower. Adding that lib dir to some Ruby library search path environment variable could work, too. Installing your very own rubygems into your home dir would be another option (see http://docs.rubygems.org/read/chapter/3 , section 3.2). That would then look for gems in the location you specified as GEM_HOME, but nowhere else, so you'd have to install all gems that you might need into this private gems installation. Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From atomgiant at gmail.com Tue Apr 4 12:27:26 2006 From: atomgiant at gmail.com (Tom Davies) Date: Tue, 4 Apr 2006 12:27:26 -0400 Subject: [Ferret-talk] Installing Ferret locally on TextDrive In-Reply-To: <20060403164958.GB15917@cordoba.webit.de> References: <20060403133352.GA15917@cordoba.webit.de> <20060403164958.GB15917@cordoba.webit.de> Message-ID: Thanks Jens. That is a solid idea about the symlink. It also has the added benefit of being easy to rollback if Ferret 0.9 is not stable enough. It may take me a few days to get around to testing this out, but I will post the details when I am finished in case others would like to give it a try. Tom On 4/3/06, Jens Kraemer wrote: > On Mon, Apr 03, 2006 at 11:05:18AM -0400, Tom Davies wrote: > > Hi Jens, > > > > That looks promising and I will dig in to it when I have a chance over > > the next couple of days. However, looking through the comments people > > have had problems running it on Windows. Also, I am not sure if I can > > freeze a ferret gem that will work on both Windows and Linux. Do you > > think this would be possible? > > no, I think this won't be possible because of the compiled parts. The > Ruby-only version of ferret should work, though. > > Easy solution: switch to Linux for development ;-) > > > If not, i am leaning towards just tweaking my TextDrive setup to look > > for gems in an additional location that is outside of my deploy > > path... so I don't have to reinstall ferret every time I upload a new > > version of my application. > > don't know how to configure rubygems for that, Imho a rubygems > installation only looks for gems in the location specified at > installation time. > > I once had to re-install all gems after installing rubygems below > /usr/local, because it only looked for installed gems there. > I had another version of rubygems in /usr, with plenty of gems > installed, but the new rubygems did not find them. > > What about installing ferret into a lib/ directory inside your home > directory, and symlinking the contents of this directory into your > RAILS_ROOT/lib/ on deploy ? Should be easy to automate with capistrano, > formerly known as switchtower. > > Adding that lib dir to some Ruby library search path environment > variable could work, too. > > Installing your very own rubygems into your home dir would be another > option (see http://docs.rubygems.org/read/chapter/3 , section 3.2). > That would then look for gems in the location you specified as GEM_HOME, > but nowhere else, so you'd have to install all gems that you might need > into this private gems installation. > > Jens > > -- > webit! Gesellschaft f?r neue Medien mbH www.webit.de > Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de > Schnorrstra?e 76 Tel +49 351 46766 0 > D-01069 Dresden Fax +49 351 46766 66 > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Tom Davies http://blog.atomgiant.com http://gifthat.com From srackham at methods.co.nz Tue Apr 4 21:19:33 2006 From: srackham at methods.co.nz (Stuart Rackham) Date: Wed, 5 Apr 2006 03:19:33 +0200 Subject: [Ferret-talk] Missing Ferret 0.9.0 Field methods Message-ID: <71c7b9d889ca69bb4955d51d5b03e941@ruby-forum.com> The following instance methods seem to be missing from the Ferret::Document::Field class in Ferret 0.9.0 using compiled C extensions: #string_value, #reader_value and #binary_value. They are in the pure ruby implementation). I got round it by mixing in hacked versions of the pure Ruby methods (@data replaced by self.data). Many thanks for Ferret it's shaping up to being a killer app for Ruby. -- Posted via http://www.ruby-forum.com/. From jmcgrath at fryolator.com Wed Apr 5 00:09:37 2006 From: jmcgrath at fryolator.com (John McGrath) Date: Wed, 5 Apr 2006 00:09:37 -0400 Subject: [Ferret-talk] duplicate search results Message-ID: <1144210177.443343019c359@webmail.whoi.edu> i'm using Ferret 0.9.0 with acts_as_ferret (the one from svn.jkraemer.net), and i'm getting duplicate results, as described in this thread: http://rubyforge.org/pipermail/ferret-talk/2005-December/000048.html is there a way to configure the indexes created by acts_as_ferret to use :key => :id, as described in that thread? i've poked around in the code, and had trouble even finding where the index was being created. or could something else be causing this? every time i re-save (or update) a model, a new document is being created in the index. ferret is rad, many thanks for all the work on it. best, john _____________________ John McGrath http://fryolator.com ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From kraemer at webit.de Wed Apr 5 05:10:28 2006 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 5 Apr 2006 11:10:28 +0200 Subject: [Ferret-talk] duplicate search results In-Reply-To: <1144210177.443343019c359@webmail.whoi.edu> References: <1144210177.443343019c359@webmail.whoi.edu> Message-ID: <20060405091028.GH15917@cordoba.webit.de> Hi John, On Wed, Apr 05, 2006 at 12:09:37AM -0400, John McGrath wrote: > i'm using Ferret 0.9.0 with acts_as_ferret (the one from svn.jkraemer.net), and > i'm getting duplicate results, as described in this thread: > http://rubyforge.org/pipermail/ferret-talk/2005-December/000048.html > > is there a way to configure the indexes created by acts_as_ferret to use :key => > :id, as described in that thread? i've poked around in the code, and had trouble > even finding where the index was being created. acts_as_ferret uses the id as a key by default. Could you please check if this problem still occurs with the current trunk (located at svn://projects.jkraemer.net/acts_as_ferret/trunk/plugin/acts_as_ferret ) Cheers, Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From jmcgrath at fryolator.com Wed Apr 5 05:35:44 2006 From: jmcgrath at fryolator.com (John McGrath) Date: Wed, 5 Apr 2006 05:35:44 -0400 Subject: [Ferret-talk] duplicate search results In-Reply-To: <20060405091028.GH15917@cordoba.webit.de> References: <1144210177.443343019c359@webmail.whoi.edu> <20060405091028.GH15917@cordoba.webit.de> Message-ID: <1144229744.44338f701f1d9@webmail.whoi.edu> hi jens, updated to trunk, but still having the same issue in my Book.rb model i have the acts_as_ferret line with no attributes: acts_as_ferret. to keep things simple it's the only model using acts_as_ferret right now. my controller code is based on the example in the comments above find_id_by_contents: @res << Book.find_id_by_contents(params[:search][:search_term]) if i zero both the index and the db, then add a single book, one (and only one) book appears in the 'books' table, but the line above from the controller returns 3 results from the newly-created index at index/development/Book. it's coming from somewhere in the plugin, i think, because when i add this line to find_id_by_contents, right after the call to ferret_index.search(q, option): logger.debug("hits is: #{hits.size}") this shows up in the log: hits is: 3 best, john Quoting Jens Kraemer : > Hi John, > > On Wed, Apr 05, 2006 at 12:09:37AM -0400, John McGrath wrote: > > i'm using Ferret 0.9.0 with acts_as_ferret (the one from svn.jkraemer.net), > and > > i'm getting duplicate results, as described in this thread: > > http://rubyforge.org/pipermail/ferret-talk/2005-December/000048.html > > > > is there a way to configure the indexes created by acts_as_ferret to use > :key => > > :id, as described in that thread? i've poked around in the code, and had > trouble > > even finding where the index was being created. > > acts_as_ferret uses the id as a key by default. Could you please check > if this problem still occurs with the current trunk (located at > svn://projects.jkraemer.net/acts_as_ferret/trunk/plugin/acts_as_ferret ) > > > Cheers, > Jens > > > -- > webit! Gesellschaft f?r neue Medien mbH www.webit.de > Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de > Schnorrstra?e 76 Tel +49 351 46766 0 > D-01069 Dresden Fax +49 351 46766 66 > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From jmcgrath at fryolator.com Wed Apr 5 23:18:07 2006 From: jmcgrath at fryolator.com (John McGrath) Date: Wed, 5 Apr 2006 23:18:07 -0400 Subject: [Ferret-talk] duplicate search results In-Reply-To: <1144229744.44338f701f1d9@webmail.whoi.edu> References: <1144210177.443343019c359@webmail.whoi.edu> <20060405091028.GH15917@cordoba.webit.de> <1144229744.44338f701f1d9@webmail.whoi.edu> Message-ID: <1144293487.4434886fe74ee@webmail.whoi.edu> still banging my head against this. at a suggestion from jens i checked the index size with irb, and it is indeed larger than it should be -- extra entries are being put in. i also got the demo from svn and ran the unit tests, some of which failed. the failing tests look like they're related to multiple id issues, too -- i'll paste them in below. i'm slowly working my way through the code, but ferret is new to me (ruby too, really), so it might take a while. if anyone has any pointers in the meanwhile, i'd really appreciate it. since my problems have been so consistent, and since i get similar issues with the demo, i'm guessing it might have something to do with changes to ActiveRecord in Rails 1.1. fwiw, my env details again: mac os 10.4.6, mysql 5.0.18, ruby 1.8.4, rails 1.1, ferret 0.9, acts_as_ferret from trunk. i removed all other plugins from my app in case they were conflicting, and still had the same problems. here's the rake output from the unit tests: dewey:~/user/squirl/aaf_demo/aaf_demo john$ rake (in /Users/john/user/squirl/aaf_demo/aaf_demo) /opt/local/bin/ruby -Ilib:test "/opt/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader.rb" "test/unit/comment_test.rb" "test/unit/content_test.rb" Loaded suite /opt/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader Started ........FFF.F. Finished in 22.529063 seconds. 1) Failure: test_id_multi_search(ContentTest) [./test/unit/content_test.rb:96]: <5> expected but was <6>. 2) Failure: test_indexed_method(ContentTest) [./test/unit/content_test.rb:43]: <1> expected but was <0>. 3) Failure: test_multi_index(ContentTest) [./test/unit/content_test.rb:67]: <5> expected but was <6>. 4) Failure: test_multi_search(ContentTest) [./test/unit/content_test.rb:85]: <5> expected but was <6>. 14 tests, 70 assertions, 4 failures, 0 errors /opt/local/bin/ruby -Ilib:test "/opt/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader.rb" "test/functional/content_controller_test.rb" Loaded suite /opt/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader Started ......... Finished in 3.15368 seconds. 9 tests, 36 assertions, 0 failures, 0 errors rake aborted! Test failures Quoting John McGrath : > hi jens, updated to trunk, but still having the same issue > > in my Book.rb model i have the acts_as_ferret line with no attributes: > acts_as_ferret. to keep things simple it's the only model using > acts_as_ferret > right now. > > my controller code is based on the example in the comments above > find_id_by_contents: > > @res << Book.find_id_by_contents(params[:search][:search_term]) > > if i zero both the index and the db, then add a single book, one (and only > one) > book appears in the 'books' table, but the line above from the controller > returns 3 results from the newly-created index at index/development/Book. > > it's coming from somewhere in the plugin, i think, because when i add this > line > to find_id_by_contents, right after the call to ferret_index.search(q, > option): > logger.debug("hits is: #{hits.size}") > > this shows up in the log: > hits is: 3 > > best, > john > > > Quoting Jens Kraemer : > > > Hi John, > > > > On Wed, Apr 05, 2006 at 12:09:37AM -0400, John McGrath wrote: > > > i'm using Ferret 0.9.0 with acts_as_ferret (the one from > svn.jkraemer.net), > > and > > > i'm getting duplicate results, as described in this thread: > > > http://rubyforge.org/pipermail/ferret-talk/2005-December/000048.html > > > > > > is there a way to configure the indexes created by acts_as_ferret to use > > :key => > > > :id, as described in that thread? i've poked around in the code, and had > > trouble > > > even finding where the index was being created. > > > > acts_as_ferret uses the id as a key by default. Could you please check > > if this problem still occurs with the current trunk (located at > > svn://projects.jkraemer.net/acts_as_ferret/trunk/plugin/acts_as_ferret ) > > > > > > Cheers, > > Jens > > > > > > -- > > webit! Gesellschaft f?r neue Medien mbH www.webit.de > > Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de > > Schnorrstra?e 76 Tel +49 351 46766 0 > > D-01069 Dresden Fax +49 351 46766 66 > > _______________________________________________ > > Ferret-talk mailing list > > Ferret-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/ferret-talk ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From weibel at gmail.com Thu Apr 6 03:56:52 2006 From: weibel at gmail.com (Kasper Weibel) Date: Thu, 6 Apr 2006 09:56:52 +0200 Subject: [Ferret-talk] duplicate search results In-Reply-To: <1144293487.4434886fe74ee@webmail.whoi.edu> References: <1144210177.443343019c359@webmail.whoi.edu> <20060405091028.GH15917@cordoba.webit.de> <1144229744.44338f701f1d9@webmail.whoi.edu> <1144293487.4434886fe74ee@webmail.whoi.edu> Message-ID: Hmmm, strange happenings are afoot ... Tell me, do you get the same results with ferret 0.3.2? Kasper John McGrath wrote: > get similar issues with the demo, i'm guessing it might have something > to do > with changes to ActiveRecord in Rails 1.1. > > fwiw, my env details again: mac os 10.4.6, mysql 5.0.18, ruby 1.8.4, > rails 1.1, > ferret 0.9, acts_as_ferret from trunk. i removed all other plugins from > my app > in case they were conflicting, and still had the same problems. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Apr 6 04:35:27 2006 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 6 Apr 2006 10:35:27 +0200 Subject: [Ferret-talk] duplicate search results In-Reply-To: References: <1144210177.443343019c359@webmail.whoi.edu> <20060405091028.GH15917@cordoba.webit.de> <1144229744.44338f701f1d9@webmail.whoi.edu> <1144293487.4434886fe74ee@webmail.whoi.edu> Message-ID: <20060406083527.GJ15917@cordoba.webit.de> Hi again, I could reproduce these Results with Ferret 0.9 (Ruby-Version). The C extension bails out with errors instead failures on the same tests, because the MultiReader class, which is used to search across indexes is not accessible from outside of ferret. Using Ferret 0.3.2, all tests except the test_indexed_method (which is expected to fail for now and has nothing to do with the problem here) pass both with Rails 1.0 and Rails 1.1. All the failures happen on an OR query accross multiple Indexes that returns more results than expected, simple one-term queries run fine. A test I added locally, that updates a single Object, saves it again and checks if it's still only one in the index runs fine in all scenarios (Ferret 0.3.2, 0.9-ruby, 0.9-C, Rails 1.1, Linux). To sum up: - it doesn't seem to be a Rails issue. - Ferret 0.9/Ruby behaves somewhat different when using OR queries across multiple Indexes, but that should not be related to your problem. - tests creating and searching on a single index seem to pass in your environment (e.g. the content_controller_test) very strange... here's a test method I just added to content_test.rb (can't commit it right now) that passes with all variations of Ferret, too. def test_update contents_from_ferret = Content.find_by_contents('useless') assert_equal 1, contents_from_ferret.size assert_equal @content.id, contents_from_ferret.first.id @content.description = 'Updated description, still useless' @content.save contents_from_ferret = Content.find_by_contents('useless') assert_equal 1, contents_from_ferret.size assert_equal @content.id, contents_from_ferret.first.id contents_from_ferret = Content.find_by_contents('updated AND description') assert_equal 1, contents_from_ferret.size assert_equal @content.id, contents_from_ferret.first.id contents_from_ferret = Content.find_by_contents('updated OR description') assert_equal 1, contents_from_ferret.size assert_equal @content.id, contents_from_ferret.first.id end hope we can solve this somehow... Jens On Thu, Apr 06, 2006 at 09:56:52AM +0200, Kasper Weibel wrote: > Hmmm, strange happenings are afoot ... > > Tell me, do you get the same results with ferret 0.3.2? > > Kasper > > John McGrath wrote: > > > get similar issues with the demo, i'm guessing it might have something > > to do > > with changes to ActiveRecord in Rails 1.1. > > > > fwiw, my env details again: mac os 10.4.6, mysql 5.0.18, ruby 1.8.4, > > rails 1.1, > > ferret 0.9, acts_as_ferret from trunk. i removed all other plugins from > > my app > > in case they were conflicting, and still had the same problems. > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From tim at karmacrash.com Thu Apr 6 16:19:46 2006 From: tim at karmacrash.com (Tim Case) Date: Thu, 6 Apr 2006 22:19:46 +0200 Subject: [Ferret-talk] Installing Ferret locally on TextDrive In-Reply-To: References: <20060403133352.GA15917@cordoba.webit.de> <20060403164958.GB15917@cordoba.webit.de> Message-ID: <8a519dd2f12591c72fd93fe6901dad2d@ruby-forum.com> Hi, I'm having this problem myself, I notice that my Txd deployed Rails application is not working, like my locally installed Rails application. I froze the gem and it's in my vendor/lib and still doesn't work. After reading this email: http://forum.textdrive.com/viewtopic.php?id=7880 I'm coming to the conclusion that you can't freeze ferret like other gems because of the C extensions. Installing the gem also builds the C extensions? So it's a no go for a gem freeze for Ferret? Am I wrong on this one? Thanks, Tim Case -- Posted via http://www.ruby-forum.com/. From progressions at gmail.com Thu Apr 6 22:07:25 2006 From: progressions at gmail.com (Jeff Coleman) Date: Fri, 7 Apr 2006 04:07:25 +0200 Subject: [Ferret-talk] How do you create the first index? Message-ID: <4518af42026d8e6c5ec7d3894a54f87b@ruby-forum.com> I'm attempting to implement acts_as_ferret, but I haven't been able to find out how to create the first index. I see the "rebuild_index.rb" file, but it tells me "Uninitialized constant: Content" if I try to run it by itself, and I'm not sure if I can invoke it some other way. Any clues? Thanks, Jeff Coleman -- Posted via http://www.ruby-forum.com/. From jmcgrath at fryolator.com Fri Apr 7 03:30:53 2006 From: jmcgrath at fryolator.com (John McGrath) Date: Fri, 7 Apr 2006 03:30:53 -0400 Subject: [Ferret-talk] How do you create the first index? In-Reply-To: <4518af42026d8e6c5ec7d3894a54f87b@ruby-forum.com> References: <4518af42026d8e6c5ec7d3894a54f87b@ruby-forum.com> Message-ID: <1144395053.4436152db0b17@webmail.whoi.edu> i'm new to this too, but i think if just put acts_as_ferret at the top of your model, when you save or update the model the index is created automatically in #{RAILS_ROOT}/index. john Quoting Jeff Coleman : > > I'm attempting to implement acts_as_ferret, but I haven't been able to > find out how to create the first index. I see the "rebuild_index.rb" > file, but it tells me "Uninitialized constant: Content" if I try to run > it by itself, and I'm not sure if I can invoke it some other way. > > Any clues? > > Thanks, > Jeff Coleman ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From progressions at gmail.com Fri Apr 7 03:35:00 2006 From: progressions at gmail.com (Jeff Coleman) Date: Fri, 7 Apr 2006 09:35:00 +0200 Subject: [Ferret-talk] How do you create the first index? In-Reply-To: <1144395053.4436152db0b17@webmail.whoi.edu> References: <4518af42026d8e6c5ec7d3894a54f87b@ruby-forum.com> <1144395053.4436152db0b17@webmail.whoi.edu> Message-ID: <14bc97bf7da180dac2e2e8ceeecbcbea@ruby-forum.com> John McGrath wrote: > i'm new to this too, but i think if just put acts_as_ferret at the top > of your > model, when you save or update the model the index is created > automatically in > #{RAILS_ROOT}/index. > > john Thanks, John. I was transferring from another search plugin that required some more elaborate setup to get going, so I didn't realize it would be that simple! Jeff -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Apr 7 04:33:08 2006 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 7 Apr 2006 10:33:08 +0200 Subject: [Ferret-talk] Installing Ferret locally on TextDrive In-Reply-To: <8a519dd2f12591c72fd93fe6901dad2d@ruby-forum.com> References: <20060403133352.GA15917@cordoba.webit.de> <20060403164958.GB15917@cordoba.webit.de> <8a519dd2f12591c72fd93fe6901dad2d@ruby-forum.com> Message-ID: <20060407083308.GN15917@cordoba.webit.de> On Thu, Apr 06, 2006 at 10:19:46PM +0200, Tim Case wrote: > Hi, > > I'm having this problem myself, I notice that my Txd deployed Rails > application is not working, like my locally installed Rails application. > I froze the gem and it's in my vendor/lib and still doesn't work. After > reading this email: > > http://forum.textdrive.com/viewtopic.php?id=7880 > > I'm coming to the conclusion that you can't freeze ferret like other > gems because of the C extensions. Installing the gem also builds the C > extensions? So it's a no go for a gem freeze for Ferret? just copy the compiled C extension (ferret_ext.so) to your lib dir, too. works fine for me. Regards, Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From dbalmain.ml at gmail.com Mon Apr 10 19:53:30 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 11 Apr 2006 08:53:30 +0900 Subject: [Ferret-talk] Stop words in queries In-Reply-To: References: Message-ID: Hi Nathaniel, This is a bug. I might get around to fixing it but I can't promise anything. I'm focusing entirely on the C extension version of Ferret (which doesn't have this bug). Cheers, Dave PS: Sorry for the slow reply. It's been a tough few weeks here. On 4/3/06, Nathaniel Talbott wrote: > I've run in to an issue that I'm not sure how to address. Basically, I'm > building queries with occur_default Search::BooleanClause::Occur::MUST, > and using the StandardAnalyzer which does stop filtering. The stop > filtering is working beautifully on the indexing side. The problem is > that when the query parser parses through a query with a stop word in > it, say "the oregon trail", it builds a query that looks something like > this: > > MUST title: > MUST title: oregon > MUST title: trail > Which unfortunately fails when searching for the previously indexed "The > Oregon Trail" because it doesn't have a blank title term in it. > > Is there a good way to deal with this issue besides filtering stop words > before handing the query string off to the parser? > > Thanks! > > > Nathaniel > > P.S. I'm using the pure Ruby part of Ferret 0.9.0 on Ruby 1.8.4. > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Mon Apr 10 20:01:27 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 11 Apr 2006 09:01:27 +0900 Subject: [Ferret-talk] Missing Ferret 0.9.0 Field methods In-Reply-To: <71c7b9d889ca69bb4955d51d5b03e941@ruby-forum.com> References: <71c7b9d889ca69bb4955d51d5b03e941@ruby-forum.com> Message-ID: Hi Stuart, Just curious. What do you use those methods for? I didn't bother including them because I didn't think anyone would need them. It seemed simpler just to have the one method that returned the data as a string. I hope I can help. Cheers, Dave On 4/5/06, Stuart Rackham wrote: > The following instance methods seem to be missing from the > Ferret::Document::Field class in Ferret 0.9.0 using compiled C > extensions: #string_value, #reader_value and #binary_value. They are in > the pure ruby implementation). > > I got round it by mixing in hacked versions of the pure Ruby methods > (@data replaced by self.data). > > Many thanks for Ferret it's shaping up to being a killer app for Ruby. > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From roop at pookmail.com Tue Apr 11 16:42:35 2006 From: roop at pookmail.com (roop) Date: Tue, 11 Apr 2006 22:42:35 +0200 Subject: [Ferret-talk] setting the default operator on a QueryParser Message-ID: <4e1a1830d68a481c1b5be51ce904bf59@ruby-forum.com> Hi all, Thanks to the authors for the amazing work on Ferret! I have a question about setting up the standard operators on a QueryParser. I would like my parser to default to AND queries rather than the default OR. In Java I would simply do: QueryParser parser = new QueryParser("contents", new StandardAnalyzer()); parser.setDefaultOperator(QueryParser.Operator.AND); But I can't find the corresponding method in the Ferret API. Have I missed it or is there a workaround? Thanks, Roop -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Apr 12 03:13:34 2006 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 12 Apr 2006 09:13:34 +0200 Subject: [Ferret-talk] setting the default operator on a QueryParser In-Reply-To: <4e1a1830d68a481c1b5be51ce904bf59@ruby-forum.com> References: <4e1a1830d68a481c1b5be51ce904bf59@ruby-forum.com> Message-ID: <20060412071334.GB1985@cordoba.webit.de> Hi! On Tue, Apr 11, 2006 at 10:42:35PM +0200, roop wrote: > Hi all, > > Thanks to the authors for the amazing work on Ferret! I have a question > about setting up the standard operators on a QueryParser. I would like > my parser to default to AND queries rather than the default OR. > > In Java I would simply do: > > QueryParser parser = new QueryParser("contents", new > StandardAnalyzer()); > parser.setDefaultOperator(QueryParser.Operator.AND); Afair the option is named :occur_default and takes one of Ferret::Search::BooleanClause::Occur::MUST or Ferret::Search::BooleanClause::Occur::SHOULD. You can use this option in the Ferret::Index::Index constructor (which will internally create a query parser) or directly when constructing the query parser yourself. Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From onurturgay at labristeknoloji.com Wed Apr 12 08:17:33 2006 From: onurturgay at labristeknoloji.com (Onur Turgay) Date: Wed, 12 Apr 2006 15:17:33 +0300 Subject: [Ferret-talk] testing tokenizers Message-ID: Hi, is there a way to test tokenizers? I mean, I want to give input stream and see the output tokens. AND is there a way to see an indexed document's index tokens? Which words in the document are used to index this document? Thanks in advance Onur From roop at pookmail.com Wed Apr 12 14:55:40 2006 From: roop at pookmail.com (roop) Date: Wed, 12 Apr 2006 20:55:40 +0200 Subject: [Ferret-talk] setting the default operator on a QueryParser In-Reply-To: <20060412071334.GB1985@cordoba.webit.de> References: <4e1a1830d68a481c1b5be51ce904bf59@ruby-forum.com> <20060412071334.GB1985@cordoba.webit.de> Message-ID: <6a00efdb4b2cb69bfec5c7906a034353@ruby-forum.com> Thanks. The difference between setting the AND/OR default operator versus setting the MUST/SHOULD default occur appears to be redundant then? In any case, thanks for the tip. -- Posted via http://www.ruby-forum.com/. From john at likealightbulb.com Wed Apr 12 15:37:11 2006 From: john at likealightbulb.com (John Butler) Date: Wed, 12 Apr 2006 12:37:11 -0700 Subject: [Ferret-talk] Location of match? Message-ID: <8E50D95E-0C46-4901-B461-63A03AE6FE8C@likealightbulb.com> Is it possible with Ferret to find the location of the matches in a document? For example imagine I have 100 documents and I search with the phrase "bob~0.5" and that returns 3 matching documents. How can I then find all locations in a specific document where it matched "bob~0.5". What I need is something like an array that contains the start index and length for each match within a given document. Does this exact? Should I break up my matching document into subdocuments then search on that? Also for my application I will be searching for fairly large pieces of text ( many sentences long ) and doing fuzzy matching. I suppose what I am doing is very similar to trying to find matching phrases within an essay to catch people plagiarizing (that's not what I'm doing at all, but it's close enough in terms of methods). Are both of these possible with Ferret? Is there another technology I should look at for doing this? I will have a relatively small index size ( somewhere between 100 and 500 ) and so I'm not really concerned with speed issues. Thanks so much for any help! -John From srackham at methods.co.nz Wed Apr 12 23:46:07 2006 From: srackham at methods.co.nz (Stuart Rackham) Date: Thu, 13 Apr 2006 05:46:07 +0200 Subject: [Ferret-talk] [ANN] FerretHelper module and Ferret Finder utility Message-ID: <3268119addd011aa6dc70da1a23fc1ff@ruby-forum.com> Hi I've written ferret_helper.rb (FerretHelper module) containing wrapper methods for converting PDF, HTML, Open Document and Microsoft Word files to plain text for the Ferret index analysers. To help test it I wrote a simple ff.rb (Ferret Finder) command-line utility to index and search document files. You can download the code at http://www.methods.co.nz/ff/ Cheers, Stuart -- Stuart Rackham -- Posted via http://www.ruby-forum.com/. From roop at pookmail.com Thu Apr 13 05:34:39 2006 From: roop at pookmail.com (roop) Date: Thu, 13 Apr 2006 11:34:39 +0200 Subject: [Ferret-talk] QueryParser doesn't use StandardAnalyzer correctly? Message-ID: <78f54b406bfb221ff44a7164b90b468e@ruby-forum.com> I am having a bit of a problem with my search queries being parsed correctly it seems, and I wonder if anyone else has experienced this. I have written an index using StandardAnalyzer for analysis. I want to search that index by passing my user query through a QueryParser instance which is also using a StandardAnalyzer. However the resultant query does not seem to be a valid term query and therefore the search produces no hits. Specifically I have a bunch of docs with the phrase "museum of art" in the source text. A query 'museum art' gets parsed into '+contents:museum +contents:art' which works just fine and produces hits. A query of 'museum of art' gets parsed into '+contents:museum +contents: +contents:art' which produces no hits. The resulting term query itself seems to be malformed, containing an extraneous term for a stop word which was (correctly) filtered out. Using the Luke gui tool for Lucene, I have verified that passing my query through StandardAnalyzer should indeed work, as it produces the expected term query and the expected hits in that environment. But as for the same query in Ferret, I'm at a loss. This should be easily reproducible with the following code fragment: require 'ferret' parser = Ferret::QueryParser.new('contents', :analyzer => Ferret::Analysis::StandardAnalyzer.new, :occur_default => Ferret::Search::BooleanClause::Occur::MUST) q1 = parser.parse('museum art') q2 = parser.parse('museum of art') puts q1, q2 Thanks for any insight. -Roop -- Posted via http://www.ruby-forum.com/. From JanPrill at blauton.de Thu Apr 13 05:42:22 2006 From: JanPrill at blauton.de (Jan Prill) Date: Thu, 13 Apr 2006 11:42:22 +0200 Subject: [Ferret-talk] [ANN] FerretHelper module and Ferret Finder utility In-Reply-To: <3268119addd011aa6dc70da1a23fc1ff@ruby-forum.com> References: <3268119addd011aa6dc70da1a23fc1ff@ruby-forum.com> Message-ID: <562a35c10604130242h5753fa8bse2f89c9eb21e247f@mail.gmail.com> Wow, great step for making something like desktop and email search available with ruby and ferret. I'll definitly try this out. Thank you so much! Just as a pointer for something I've used for email searching before I've converted to gmail: http://zoe.nu/ Regards Jan Prill On 4/13/06, Stuart Rackham wrote: > > Hi > > I've written ferret_helper.rb (FerretHelper module) containing wrapper > methods > for converting PDF, HTML, Open Document and Microsoft Word files to > plain text > for the Ferret index analysers. > > To help test it I wrote a simple ff.rb (Ferret Finder) command-line > utility to > index and search document files. > > You can download the code at http://www.methods.co.nz/ff/ > > > Cheers, Stuart > -- > Stuart Rackham > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20060413/ebc44f2e/attachment.htm From mailinglist at moribund.ath.cx Thu Apr 13 09:52:42 2006 From: mailinglist at moribund.ath.cx (Dave Slusher) Date: Thu, 13 Apr 2006 15:52:42 +0200 Subject: [Ferret-talk] Problems with Unicode and acts_as_ferret Message-ID: Thanks to Dave and everyone who has contributed to ferret and acts_as_ferret! Man, can I not wait until I get this up and working on my project. Just the fuzzy search alone will be worth it. I'm having problems with rebuilding the index. My database includes Unicode entries, and I have configured the rest of Rails to correctly use it. On the initial index creation, I get this exception: : Error occured at :678 Error: exception 2 not handled: Error decoding input string. Check that you have the locale set correctly I tried the methods that Albert Delamednolls wrote about on his blog for Unicode and acts_as_rails but that failed for me. It seems like that might have been based on older versions of acts_as_rails but I just couldn't make that work. I've got two levels of solution that I'm looking for. The best would be to figure out the errors and be able to index even when the content contains Unicode characters. As a workaround, if I could just trap the exception during indexing so that it would skip the problematic entries but index everything else, that would get me going. I tried to catch the exception in this line from rebuild_index: self.find(:all, :order=>"id" ).each { |content| index << content.to_doc } But even though I tried to rescue there, the exception still is thrown. Does anyone have any insight on this? -- Posted via http://www.ruby-forum.com/. From joergd at pobox.com Thu Apr 13 11:57:06 2006 From: joergd at pobox.com (Joerg Diekmann) Date: Thu, 13 Apr 2006 17:57:06 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror Message-ID: Hi, I'm running the edge Typo and the latest acts_as_ferret plugin and ferret gems (as of 11 April). In my Content model I put acts_as_ferret :fields => [:title, :body] In the console, I call Content.rebuild_index to index all of my existing content. All the directories get created etc, but I get the following error back: Loading development environment. >> Content.rebuild_index NameError: undefined method `title' for class `Comment' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:104:in `method' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:104:in `title_to_ferret' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:103:in `title_to_ferret' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:439:in `send' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:439:in `to_doc' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:438:in `each' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:438:in `to_doc' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:217:in `rebuild_index' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:217:in `each' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a cts_as_ferret.rb:217:in `rebuild_index' from (irb):1 >> Has anybody got this to work before? Thanks Joerg -- Posted via http://www.ruby-forum.com/. From weibel at gmail.com Thu Apr 13 12:31:31 2006 From: weibel at gmail.com (Kasper Weibel) Date: Thu, 13 Apr 2006 18:31:31 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: References: Message-ID: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> I'm not exactly sure why the error is about an undefined method for the class Comment, when you run rebuild_index on Content. 1) Does Comment have the fields 'title' and 'body' defined somewhere? 2) Didn't you mean Comment.rebuild_index? Joerg Diekmann wrote: > In my Content model I put acts_as_ferret :fields => [:title, :body] > Loading development environment. >>> Content.rebuild_index > NameError: undefined method `title' for class `Comment' > from > ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a > cts_as_ferret.rb:104:in `method' -- Posted via http://www.ruby-forum.com/. From joergd at pobox.com Thu Apr 13 14:13:03 2006 From: joergd at pobox.com (Joerg Diekmann) Date: Thu, 13 Apr 2006 20:13:03 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> References: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> Message-ID: Sorry - I wasn't clear enough. Comment is defined as: class Comment < Content and class Content < ActiveRecord::Base The contents table has fields like title, body, extended etc ... so the Comment model will have them too then. The error isn't particular to the :title field either ... if I were to have :extended as part of the fields list, it'll through the same error but wrt extended. Kasper Weibel wrote: > I'm not exactly sure why the error is about an undefined method for the > class Comment, when you run rebuild_index on Content. > > 1) Does Comment have the fields 'title' and 'body' defined somewhere? > 2) Didn't you mean Comment.rebuild_index? > > Joerg Diekmann wrote: >> In my Content model I put acts_as_ferret :fields => [:title, :body] >> Loading development environment. >>>> Content.rebuild_index >> NameError: undefined method `title' for class `Comment' >> from >> ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a >> cts_as_ferret.rb:104:in `method' -- Posted via http://www.ruby-forum.com/. From srackham at methods.co.nz Fri Apr 14 00:36:32 2006 From: srackham at methods.co.nz (Stuart Rackham) Date: Fri, 14 Apr 2006 06:36:32 +0200 Subject: [Ferret-talk] Missing Ferret 0.9.0 Field methods In-Reply-To: References: <71c7b9d889ca69bb4955d51d5b03e941@ruby-forum.com> Message-ID: <38305be134bfe89f81e8fe4c89f70096@ruby-forum.com> David Balmain wrote: > Hi Stuart, > > Just curious. What do you use those methods for? I didn't bother > including them because I didn't think anyone would need them. It > seemed simpler just to have the one method that returned the data as a > string. I hope I can help. The #string_value is used in the Rails HowToIntegrateFerretWithRails demo application on the Rails Wiki http://wiki.rubyonrails.com/rails/pages/HowToIntegrateFerretWithRails It is used in the ./app/views/search/get_results.rhtml template for example: I don't know Lucene well enough to know wether there's an alternative way of doing the the same thing. I've not come across any usage of the other two methods. Apologies for the time taken to reply, I've only just spotted your reply. Cheers, Stuart > > Cheers, > Dave -- Posted via http://www.ruby-forum.com/. From nathaniel at talbott.ws Fri Apr 14 09:31:34 2006 From: nathaniel at talbott.ws (Nathaniel Talbott) Date: Fri, 14 Apr 2006 15:31:34 +0200 Subject: [Ferret-talk] Stop words in queries In-Reply-To: References: Message-ID: <58b153ea16c61670e7742648dce8ed5d@ruby-forum.com> David Balmain wrote: > This is a bug. I might get around to fixing it but I can't promise > anything. I'm focusing entirely on the C extension version of Ferret > (which doesn't have this bug). Bummer... I've got a hack to avoid the problem for the time being, but it's _really_ ugly :-/ This brings up another issue, though, that I'll go ahead and broach... I'm kind of sad that you've made the jump to C so soon. Ferret is brimming with potential, but it still feels a lot like, well, Java. The API is still pretty heavy, and when I dig in to the underlying code it feels over-designed. I'm guessing a lot of that is due to the straight translation from Java, which while it's a good first step, it's also not surprising that it would initially result in a library that feels pretty alien. While I understand the performance reasons for using C, doing so also makes it much harder to refactor and refine the API, and my feeling is that for most problems, the pure-Ruby performance isn't a show-stopper. Putting everything in C also makes it harder for folks such as myself, who don't do much C, to hack on the internals ourselves and push patches back up to you. I hope this comes off the right way - it's open-source, and you're of course free to take the project where you will. I'm also extremely grateful for the project - it's helping me out a lot. I just have doubts about the long-term viability of Ferret within the Ruby community when an API (and underlying code) that I find I spend a lot of time fighting is getting set in stone so early. I'd hate to see you spend a lot of time on it to only have it be a prototype for a more Ruby-ish library that comes along later. I want Ferret to be the standard by which other indexing tools are measured, in Ruby and elsewhere, and I don't think that raw benchmarks are going to drive that. > PS: Sorry for the slow reply. It's been a tough few weeks here. No problem! So what exactly do you do? Are you a student? Freelancer? Employee? Astronaut? Thanks a ton for the great library, Nathaniel -- Posted via http://www.ruby-forum.com/. From nathaniel at talbott.ws Fri Apr 14 09:33:23 2006 From: nathaniel at talbott.ws (Nathaniel Talbott) Date: Fri, 14 Apr 2006 15:33:23 +0200 Subject: [Ferret-talk] QueryParser doesn't use StandardAnalyzer correctly? In-Reply-To: <78f54b406bfb221ff44a7164b90b468e@ruby-forum.com> References: <78f54b406bfb221ff44a7164b90b468e@ruby-forum.com> Message-ID: <38c357f91d4791916a3359ee915cc02c@ruby-forum.com> roop wrote: > I am having a bit of a problem with my search queries being parsed > correctly it seems, and I wonder if anyone else has experienced this. See this recent thread: "Stop words in queries" (http://www.ruby-forum.com/topic/60599). HTH, Nathaniel -- Posted via http://www.ruby-forum.com/. From roop at pookmail.com Fri Apr 14 16:29:07 2006 From: roop at pookmail.com (roop) Date: Fri, 14 Apr 2006 22:29:07 +0200 Subject: [Ferret-talk] QueryParser doesn't use StandardAnalyzer correctly? In-Reply-To: <38c357f91d4791916a3359ee915cc02c@ruby-forum.com> References: <78f54b406bfb221ff44a7164b90b468e@ruby-forum.com> <38c357f91d4791916a3359ee915cc02c@ruby-forum.com> Message-ID: Nathaniel, thanks for the info. I will await the bug fix. In the meantime my own workaround looks like this. In my QueryParser subclass, I override parse() so that it filters out stopwords first: class SafeQueryParser < Ferret::QueryParser def initialize(default_field, options) my_options = { :analyzer => Ferret::Analysis::StandardAnalyzer.new }.update(options) super(default_field, my_options) # breaking encapsulation here, but whaddya gonna do... @stop_words = my_options[:analyzer].instance_variable_get(:@stop_words) end def parse(query) @stop_words.each do |word| query.gsub!(/\b#{word}\b\s*/, '') end super(query) end end -- Posted via http://www.ruby-forum.com/. From mailinglist at moribund.ath.cx Sat Apr 15 14:07:53 2006 From: mailinglist at moribund.ath.cx (Dave Slusher) Date: Sat, 15 Apr 2006 20:07:53 +0200 Subject: [Ferret-talk] Segmentation Faults with Fuzzy Queries Message-ID: There are certain queries that I can execute that will cause a completely repeatable segmentation fault at the point where the search or search_each leaves my control and goes into the ferret libraries. I tried deleting my indexes and recreating them, and even so the exact same queries seg fault repeatedly after. Usually a one character change will make them succeed. Changing away from a Fuzzy Query will also make it succeed. What would be a good way to report this back? What do I need to do to gather information of use to help with this problem? -- Posted via http://www.ruby-forum.com/. From carl at youngbloods.org Sun Apr 16 01:35:29 2006 From: carl at youngbloods.org (Carl Youngblood) Date: Sat, 15 Apr 2006 23:35:29 -0600 Subject: [Ferret-talk] Error upgrading to 0.9.1 on freebsd Message-ID: I got some errors when running gem update to update my version of ferret on freebsd: Attempting remote upgrade of ferret Attempting remote installation of 'ferret' Building native extensions. This could take a while... In file included from /usr/include/sys/time.h:40, from /usr/local/lib/ruby/1.8/i386-freebsd4.10/missing.h:16, from /usr/local/lib/ruby/1.8/i386-freebsd4.10/ruby.h:670, from lang.h:4, from global.h:19, from ferret.h:4, from ferret.c:1: /usr/include/sys/types.h:57: redefinition of `uint' global.h:15: `uint' previously declared here In file included from /usr/include/sys/time.h:40, from /usr/local/lib/ruby/1.8/i386-freebsd4.10/missing.h:16, from /usr/local/lib/ruby/1.8/i386-freebsd4.10/ruby.h:670, from lang.h:4, from global.h:19, from ferret.h:4, from ferret.c:1: /usr/include/sys/types.h:57: redefinition of `uint' global.h:15: `uint' previously declared here ruby extconf.rb update creating Makefile make gcc -fPIC -g -O2 -I. -I/usr/local/lib/ruby/1.8/i386-freebsd4.10 -I/usr/local/lib/ruby/1.8/i386-freebsd4.10 -I. -c ferret.c *** Error code 1 Stop in /usr/local/lib/ruby/gems/1.8/gems/ferret-0.9.1/ext. make install gcc -fPIC -g -O2 -I. -I/usr/local/lib/ruby/1.8/i386-freebsd4.10 -I/usr/local/lib/ruby/1.8/i386-freebsd4.10 -I. -c ferret.c *** Error code 1 Stop in /usr/local/lib/ruby/gems/1.8/gems/ferret-0.9.1/ext. Successfully installed ferret-0.9.1 Did the upgrade really work? It seems doubtful. Any suggestions on how I can fix this? Thanks, Carl From carl at youngbloods.org Sun Apr 16 01:38:02 2006 From: carl at youngbloods.org (Carl Youngblood) Date: Sat, 15 Apr 2006 23:38:02 -0600 Subject: [Ferret-talk] Error upgrading to 0.9.1 on freebsd In-Reply-To: References: Message-ID: Update: I got the same errors installing from scratch, FWIW. On 4/15/06, Carl Youngblood wrote: > I got some errors when running gem update to update my version of > ferret on freebsd: > > Attempting remote upgrade of ferret > Attempting remote installation of 'ferret' > Building native extensions. This could take a while... > In file included from /usr/include/sys/time.h:40, > from /usr/local/lib/ruby/1.8/i386-freebsd4.10/missing.h:16, > from /usr/local/lib/ruby/1.8/i386-freebsd4.10/ruby.h:670, > from lang.h:4, > from global.h:19, > from ferret.h:4, > from ferret.c:1: > /usr/include/sys/types.h:57: redefinition of `uint' > global.h:15: `uint' previously declared here > In file included from /usr/include/sys/time.h:40, > from /usr/local/lib/ruby/1.8/i386-freebsd4.10/missing.h:16, > from /usr/local/lib/ruby/1.8/i386-freebsd4.10/ruby.h:670, > from lang.h:4, > from global.h:19, > from ferret.h:4, > from ferret.c:1: > /usr/include/sys/types.h:57: redefinition of `uint' > global.h:15: `uint' previously declared here > ruby extconf.rb update > creating Makefile > > make > gcc -fPIC -g -O2 -I. -I/usr/local/lib/ruby/1.8/i386-freebsd4.10 > -I/usr/local/lib/ruby/1.8/i386-freebsd4.10 -I. -c ferret.c > *** Error code 1 > > Stop in /usr/local/lib/ruby/gems/1.8/gems/ferret-0.9.1/ext. > > make install > gcc -fPIC -g -O2 -I. -I/usr/local/lib/ruby/1.8/i386-freebsd4.10 > -I/usr/local/lib/ruby/1.8/i386-freebsd4.10 -I. -c ferret.c > *** Error code 1 > > Stop in /usr/local/lib/ruby/gems/1.8/gems/ferret-0.9.1/ext. > Successfully installed ferret-0.9.1 > > Did the upgrade really work? It seems doubtful. Any suggestions on > how I can fix this? > > Thanks, > Carl > From kraemer at webit.de Mon Apr 17 11:52:13 2006 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 17 Apr 2006 17:52:13 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: References: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> Message-ID: <20060417155213.GB7420@cordoba.webit.de> Hi, I could reproduce the error, but I can't explain why it happens ;-) I just committed a fix that rescues from the failure and indexes an empty string for the field in question. It seems the errors are raised when aaf tries to index a field that isn't used by the particular model class, i.e. :title in a Comment object. Could you please try if the change fixes this for you, too ? Greetings, Jens On Thu, Apr 13, 2006 at 08:13:03PM +0200, Joerg Diekmann wrote: > Sorry - I wasn't clear enough. Comment is defined as: > > class Comment < Content > > and > > class Content < ActiveRecord::Base > > The contents table has fields like title, body, extended etc ... so the > Comment model will have them too then. The error isn't particular to the > :title field either ... if I were to have :extended as part of the > fields list, it'll through the same error but wrt extended. > > > > Kasper Weibel wrote: > > I'm not exactly sure why the error is about an undefined method for the > > class Comment, when you run rebuild_index on Content. > > > > 1) Does Comment have the fields 'title' and 'body' defined somewhere? > > 2) Didn't you mean Comment.rebuild_index? > > > > Joerg Diekmann wrote: > >> In my Content model I put acts_as_ferret :fields => [:title, :body] > >> Loading development environment. > >>>> Content.rebuild_index > >> NameError: undefined method `title' for class `Comment' > >> from > >> ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/a > >> cts_as_ferret.rb:104:in `method' > > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From joergd at pobox.com Mon Apr 17 12:18:39 2006 From: joergd at pobox.com (Joerg Diekmann) Date: Mon, 17 Apr 2006 18:18:39 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: <20060417155213.GB7420@cordoba.webit.de> References: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> <20060417155213.GB7420@cordoba.webit.de> Message-ID: I can test it ... do I just get the latest acts_as_ferret.rb file? http://www.jkraemer.net/files/acts_as_ferret.rb Jens Kraemer wrote: > Hi, > > I could reproduce the error, but I can't explain why it happens ;-) > I just committed a fix that rescues from the failure and indexes an > empty string for the field in question. -- Posted via http://www.ruby-forum.com/. From weibel at gmail.com Mon Apr 17 12:25:48 2006 From: weibel at gmail.com (Kasper Weibel) Date: Mon, 17 Apr 2006 18:25:48 +0200 Subject: [Ferret-talk] Problems with Unicode and acts_as_ferret In-Reply-To: References: Message-ID: <6aa3833b8f6f3420ce00874a6abe40e7@ruby-forum.com> Have you looked at changeset 302 for Ferret http://ferret.davebalmain.com/trac/changeset/302 It seems to adress your problem. Dave Slusher wrote: > I'm having problems with rebuilding the index. My database includes > Unicode entries, and I have configured the rest of Rails to correctly > use it. On the initial index creation, I get this exception: -- Posted via http://www.ruby-forum.com/. From weibel at gmail.com Mon Apr 17 12:39:00 2006 From: weibel at gmail.com (Kasper Weibel) Date: Mon, 17 Apr 2006 18:39:00 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: References: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> <20060417155213.GB7420@cordoba.webit.de> Message-ID: That's an old location. Try here instead (or use SVN) http://projects.jkraemer.net/acts_as_ferret/browser/trunk/plugin/acts_as_ferret/lib/acts_as_ferret.rb?format=raw Joerg Diekmann wrote: > I can test it ... do I just get the latest acts_as_ferret.rb file? > http://www.jkraemer.net/files/acts_as_ferret.rb > > > Jens Kraemer wrote: >> Hi, >> >> I could reproduce the error, but I can't explain why it happens ;-) >> I just committed a fix that rescues from the failure and indexes an >> empty string for the field in question. -- Posted via http://www.ruby-forum.com/. From joergd at pobox.com Mon Apr 17 12:57:36 2006 From: joergd at pobox.com (Joerg Diekmann) Date: Mon, 17 Apr 2006 18:57:36 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: References: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> <20060417155213.GB7420@cordoba.webit.de> Message-ID: Ok got it thanks. Tried it out. Error is gone. Return value from rebuild_index is false. Is that right? Kasper Weibel wrote: > That's an old location. Try here instead (or use SVN) > http://projects.jkraemer.net/acts_as_ferret/browser/trunk/plugin/acts_as_ferret/lib/acts_as_ferret.rb?format=raw > > -- Posted via http://www.ruby-forum.com/. From weibel at gmail.com Mon Apr 17 13:08:21 2006 From: weibel at gmail.com (Kasper Weibel) Date: Mon, 17 Apr 2006 19:08:21 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: References: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> <20060417155213.GB7420@cordoba.webit.de> Message-ID: <17a40d34189b5e8d5c1708b7ac143b39@ruby-forum.com> According to the API false is the correct value http://ferret.davebalmain.com/api/classes/Ferret/Index/Index.html#M000041 Joerg Diekmann wrote: > Ok got it thanks. Tried it out. Error is gone. Return value from > rebuild_index is false. Is that right? > -- Posted via http://www.ruby-forum.com/. From joergd at pobox.com Mon Apr 17 13:51:07 2006 From: joergd at pobox.com (Joerg Diekmann) Date: Mon, 17 Apr 2006 19:51:07 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: <17a40d34189b5e8d5c1708b7ac143b39@ruby-forum.com> References: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> <20060417155213.GB7420@cordoba.webit.de> <17a40d34189b5e8d5c1708b7ac143b39@ruby-forum.com> Message-ID: <1e0dac750945f59a93af698f683b8000@ruby-forum.com> K great. Well - I tested it by doing searches, and my content was correctly indexed. Thanks a lot for your help! Joerg Kasper Weibel wrote: > According to the API false is the correct value > http://ferret.davebalmain.com/api/classes/Ferret/Index/Index.html#M000041 > > > Joerg Diekmann wrote: >> Ok got it thanks. Tried it out. Error is gone. Return value from >> rebuild_index is false. Is that right? >> -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Mon Apr 17 16:59:44 2006 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 17 Apr 2006 22:59:44 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: <1e0dac750945f59a93af698f683b8000@ruby-forum.com> References: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> <20060417155213.GB7420@cordoba.webit.de> <17a40d34189b5e8d5c1708b7ac143b39@ruby-forum.com> <1e0dac750945f59a93af698f683b8000@ruby-forum.com> Message-ID: <20060417205944.GA3079@cordoba.webit.de> On Mon, Apr 17, 2006 at 07:51:07PM +0200, Joerg Diekmann wrote: > K great. Well - I tested it by doing searches, and my content was > correctly indexed. > > Thanks a lot for your help! glad to help :-) Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From dbalmain.ml at gmail.com Mon Apr 17 21:56:03 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 18 Apr 2006 10:56:03 +0900 Subject: [Ferret-talk] testing tokenizers In-Reply-To: References: Message-ID: Hey Onur, just got back from a trip around Japan. You've probably already worked out the answer to this question but here is how I test tokenizers; require 'ferret' $stdin.each do |line| stk = Ferret::Analysis::StandardTokenizer.new(line) while tk = stk.next() puts " <#{tk.text}> from #{tk.start_offset} to #{tk.end_offset}" end end And I run it like this; ruby -r rubygems tz_tester.rb < file_to_tokenize.txt You can just change the tokenizer to whaterver tokenizer you want to test. Hope that helps, Dave On 4/12/06, Onur Turgay wrote: > Hi, > is there a way to test tokenizers? I mean, I want to give input stream > and see the output tokens. > > AND is there a way to see an indexed document's index tokens? Which > words in the document are used to index this document? > > Thanks in advance > Onur > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Mon Apr 17 22:53:51 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 18 Apr 2006 11:53:51 +0900 Subject: [Ferret-talk] Location of match? In-Reply-To: <8E50D95E-0C46-4901-B461-63A03AE6FE8C@likealightbulb.com> References: <8E50D95E-0C46-4901-B461-63A03AE6FE8C@likealightbulb.com> Message-ID: Hi John, On 4/13/06, John Butler wrote: > Is it possible with Ferret to find the location of the matches in a > document? For example imagine I have 100 documents and I search with > the phrase "bob~0.5" and that returns 3 matching documents. How can I > then find all locations in a specific document where it matched > "bob~0.5". What I need is something like an array that contains the > start index and length for each match within a given document. Does > this exact? Should I break up my matching document into subdocuments > then search on that? A search result highlighter is coming in a future version of Ferret. This will enable you to find the position of the match in a document. I can't say when. > Also for my application I will be searching for fairly large pieces > of text ( many sentences long ) and doing fuzzy matching. I suppose > what I am doing is very similar to trying to find matching phrases > within an essay to catch people plagiarizing (that's not what I'm > doing at all, but it's close enough in terms of methods). > > Are both of these possible with Ferret? Is there another technology I > should look at for doing this? I will have a relatively small index > size ( somewhere between 100 and 500 ) and so I'm not really > concerned with speed issues. > At University we had an assignment to write a program that would find similar documents with the purpose of catching people plagiarizing. I used a running hash, something similar to this; require 'ferret' NUM_WORDS = 5 def hash_doc(filename) stk = Ferret::Analysis::StandardTokenizer.new("") words = [] hashes = [] File.open(filename) do |f| f.each do |line| stk.text = line while tk = stk.next() words << tk.text if words.size == NUM_WORDS hashes << words.hash words.shift end end end end return hashes.sort! end def hash_cmp(hash1, hash2) same = 0 size_avg = (hash1.size + hash2.size)/2 h1 = hash1.pop h2 = hash2.pop while (not hash1.empty? and not hash2.empty?) if (h2 == h1) same += 1 h1 = hash1.pop h2 = hash2.pop else if (h1 > h2) h1 = hash1.pop else h2 = hash2.pop end end end return same.to_f/size_avg end puts hash_cmp(hash_doc(ARGV[0]), hash_doc(ARGV[1])) I'm not sure if this would work better for you than using a really long phrase query. Cheers, Dave > Thanks so much for any help! > > -John > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Mon Apr 17 22:57:36 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 18 Apr 2006 11:57:36 +0900 Subject: [Ferret-talk] Missing Ferret 0.9.0 Field methods In-Reply-To: <38305be134bfe89f81e8fe4c89f70096@ruby-forum.com> References: <71c7b9d889ca69bb4955d51d5b03e941@ruby-forum.com> <38305be134bfe89f81e8fe4c89f70096@ruby-forum.com> Message-ID: On 4/14/06, Stuart Rackham wrote: > David Balmain wrote: > > Hi Stuart, > > > > Just curious. What do you use those methods for? I didn't bother > > including them because I didn't think anyone would need them. It > > seemed simpler just to have the one method that returned the data as a > > string. I hope I can help. > > The #string_value is used in the Rails HowToIntegrateFerretWithRails > demo application on the Rails Wiki > http://wiki.rubyonrails.com/rails/pages/HowToIntegrateFerretWithRails > > It is used in the ./app/views/search/get_results.rhtml template for > example: > > > > I don't know Lucene well enough to know wether there's an alternative > way of doing the the same thing. All add the string_value method for backwards compatibility but you can just use the data method instead. > I've not come across any usage of the other two methods. > > Apologies for the time taken to reply, I've only just spotted your > reply. No worries, I took even longer to reply. > Cheers, Stuart > > > > > Cheers, > > Dave > > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Mon Apr 17 23:14:47 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 18 Apr 2006 12:14:47 +0900 Subject: [Ferret-talk] Stop words in queries In-Reply-To: <58b153ea16c61670e7742648dce8ed5d@ruby-forum.com> References: <58b153ea16c61670e7742648dce8ed5d@ruby-forum.com> Message-ID: On 4/14/06, Nathaniel Talbott wrote: > David Balmain wrote: > > > This is a bug. I might get around to fixing it but I can't promise > > anything. I'm focusing entirely on the C extension version of Ferret > > (which doesn't have this bug). > > Bummer... I've got a hack to avoid the problem for the time being, but > it's _really_ ugly :-/ > > This brings up another issue, though, that I'll go ahead and broach... > > I'm kind of sad that you've made the jump to C so soon. Ferret is > brimming with potential, but it still feels a lot like, well, Java. The > API is still pretty heavy, and when I dig in to the underlying code it > feels over-designed. I'm guessing a lot of that is due to the straight > translation from Java, which while it's a good first step, it's also not > surprising that it would initially result in a library that feels pretty > alien. > > While I understand the performance reasons for using C, doing so also > makes it much harder to refactor and refine the API, and my feeling is > that for most problems, the pure-Ruby performance isn't a show-stopper. > Putting everything in C also makes it harder for folks such as myself, > who don't do much C, to hack on the internals ourselves and push patches > back up to you. The pure ruby version is still there and I'd love for someone to take over from me. I completely agree with you on the advantages of having a pure ruby version. I personally want the performance which is why I have taken the C route. And there is a huge difference. Somewhere around 100 times. There are people out there who were still using Java Lucene for indexing because of performance issues so I wasn't the only one concerned about the performance. As for refactoring the API, I understand it is very difficult for some Ruby programmers to get around the C code but you don't need to send me a patch. Just let me know what you think needs to be changed. > > I hope this comes off the right way - it's open-source, and you're of > course free to take the project where you will. I'm also extremely > grateful for the project - it's helping me out a lot. I just have doubts > about the long-term viability of Ferret within the Ruby community when > an API (and underlying code) that I find I spend a lot of time fighting > is getting set in stone so early. I'd hate to see you spend a lot of > time on it to only have it be a prototype for a more Ruby-ish library > that comes along later. I want Ferret to be the standard by which other > indexing tools are measured, in Ruby and elsewhere, and I don't think > that raw benchmarks are going to drive that. I want the same thing too. The other advantage to having the C version is that it won't be too much work to Ferret in Perl, Python, Tcl etc. > > > PS: Sorry for the slow reply. It's been a tough few weeks here. > > No problem! So what exactly do you do? Are you a student? Freelancer? > Employee? Astronaut? I'm currently an athlete. I'm practicing Judo in Japan and working on Ferret whenever I have time. > Thanks a ton for the great library, > > > Nathaniel > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Tue Apr 18 00:46:25 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 18 Apr 2006 13:46:25 +0900 Subject: [Ferret-talk] QueryParser doesn't use StandardAnalyzer correctly? In-Reply-To: References: <78f54b406bfb221ff44a7164b90b468e@ruby-forum.com> <38c357f91d4791916a3359ee915cc02c@ruby-forum.com> Message-ID: Hey guys, Since this was a pretty easy fix, I've fixed it in the pure ruby version. You'll have to get it out of the subversion repo if you want it. Cheers, Dave On 4/15/06, roop wrote: > Nathaniel, thanks for the info. I will await the bug fix. In the > meantime my own workaround looks like this. In my QueryParser subclass, > I override parse() so that it filters out stopwords first: > > class SafeQueryParser < Ferret::QueryParser > > def initialize(default_field, options) > my_options = { :analyzer => Ferret::Analysis::StandardAnalyzer.new > }.update(options) > super(default_field, my_options) > # breaking encapsulation here, but whaddya gonna do... > @stop_words = > my_options[:analyzer].instance_variable_get(:@stop_words) > end > > def parse(query) > @stop_words.each do |word| > query.gsub!(/\b#{word}\b\s*/, '') > end > super(query) > end > > end > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Tue Apr 18 00:49:56 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 18 Apr 2006 13:49:56 +0900 Subject: [Ferret-talk] Segmentation Faults with Fuzzy Queries In-Reply-To: References: Message-ID: Hi Dave, Using this list is fine or you could add a ticket here; http://ferret.davebalmain.com/trac/newticket Please submit a failing test if possible. Patches are even better but not required. Cheers, Dave On 4/16/06, Dave Slusher wrote: > There are certain queries that I can execute that will cause a > completely repeatable segmentation fault at the point where the search > or search_each leaves my control and goes into the ferret libraries. I > tried deleting my indexes and recreating them, and even so the exact > same queries seg fault repeatedly after. Usually a one character change > will make them succeed. Changing away from a Fuzzy Query will also make > it succeed. > > What would be a good way to report this back? What do I need to do to > gather information of use to help with this problem? > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From carl at youngbloods.org Wed Apr 19 01:32:36 2006 From: carl at youngbloods.org (Carl Youngblood) Date: Tue, 18 Apr 2006 23:32:36 -0600 Subject: [Ferret-talk] How to do case-sensitive searches Message-ID: Forgive me if this topic has already been discussed on the list. I googled but couldn't find much. I'd like to search through text for US state abbreviations that are written in capitals. What is the best way to do this? I read somewhere that tokenized fields are stored in the index in lowercase, so I am concerned that I will lose precision. What is the best way to store a field so that normal searches are case-insensitive but case-sensitive searches can still be made? Thanks, Carl From derek.mailer at iomart.com Wed Apr 19 10:25:54 2006 From: derek.mailer at iomart.com (Derek Mailer) Date: Wed, 19 Apr 2006 16:25:54 +0200 Subject: [Ferret-talk] Ferret EOFError creating index Message-ID: <482216f2ce11094dbce36e364dcce2a5@ruby-forum.com> I've been messing around with Ferret (no punn intended). After spending some time testing it out (indexing to file), I decided to index about 10% of the data I want to eventually index. It took several hours to complete the index on my local machine, but it was created without any problems and after optimising it the searches returned results at the sort of speed I was expecting. I then continuned to add a further 10% of the overall data to the index and it all ran through fine for a few hours until it bombed out with the error below. I've been running this on my local machine and the first run was left overnight, whereas the second run was going on while I was using my machine for other things. Which leads me to suspect that it's a memory-related issue that made it bail out. so I have 2 questions: 1) does anyone know what caused the error or have any suggestions to prevent it in future? 2) I can't search on the index without getting errors so is the index recoverable or will I need to recreate the index again? Thanks in advance, Derek EOFError c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:178:in `refill' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:94:in `read_byte' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:109:in `read_bytes' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:108:in `upto' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:108:in `read_bytes' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_reader.rb:286:in `get_norms_into' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_reader.rb:271:in `synchronize' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_reader.rb:271:in `get_norms_into' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:341:in `merge_norms' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:338:in `each' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:338:in `merge_norms' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:334:in `each_with_index' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/field_infos.rb:130:in `each_with_index' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/field_infos.rb:130:in `each' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/field_infos.rb:130:in `each_with_index' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/field_infos.rb:130:in `each_with_index' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:334:in `merge_norms' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:49:in `merge' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:403:in `merge_segments' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:351:in `flush_ram_segments' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:127:in `close' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:126:in `synchronize' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:126:in `close' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index.rb:173:in `close' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index.rb:168:in `synchronize' c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index.rb:168:in `close' -- Posted via http://www.ruby-forum.com/. From simon at tungstenconsulting.com Wed Apr 19 15:01:02 2006 From: simon at tungstenconsulting.com (Simon Wex) Date: Wed, 19 Apr 2006 21:01:02 +0200 Subject: [Ferret-talk] Shared host (Dreamhost) Message-ID: I am unable to install gems on my dreamhost server since it's just shared hosting. I tried unpacking the ferret gem into vendor/ferret, but to no avail. Also how would I recompile the C extensions for the new platform? Any ideas would be greatly appreciated. Cheers, Simon -- Posted via http://www.ruby-forum.com/. From carl at youngbloods.org Wed Apr 19 15:48:48 2006 From: carl at youngbloods.org (Carl Youngblood) Date: Wed, 19 Apr 2006 13:48:48 -0600 Subject: [Ferret-talk] Shared host (Dreamhost) In-Reply-To: References: Message-ID: Not true. I use dreamhost and I'm able to completely compile my own setup. Here's how I do it. The following bash script creates your own custom ruby environment, in which you can install whatever addons and gems you want. Then just make sure you have export PATH="$HOME/extra/bin:$PATH" in your .bash_profile script and also make sure your ruby scripts point to $HOME/extra/bin/ruby. --------------------------- #!/bin/sh # remove old stuff rm -rf $HOME/extra/bin rm -rf $HOME/extra/include rm -rf $HOME/extra/info rm -rf $HOME/extra/lib rm -rf $HOME/extra/man rm -rf $HOME/extra/src rm -rf $HOME/extra/ssl mkdir -p $HOME/extra/bin cp $HOME/extra/binbak/* $HOME/extra/bin # create directory for compiling source code mkdir -p $HOME/extra/src cd $HOME/extra/src # download and compile source tarballs wget http://www.openssl.org/source/openssl-0.9.8a.tar.gz tar zxvf openssl-0.9.8a.tar.gz rm -f openssl-0.9.8a.tar.gz cd openssl-0.9.8a ./config --prefix=$HOME/extra make make test make install cd .. wget http://www.fastcgi.com/dist/fcgi-2.4.0.tar.gz tar zxvf fcgi-2.4.0.tar.gz rm -f fcgi-2.4.0.tar.gz cd fcgi-2.4.0 ./configure --prefix=$HOME/extra make make install cd .. wget ftp://ftp.cwru.edu/pub/bash/readline-5.1.tar.gz tar zxvf readline-5.1.tar.gz rm -f readline-5.1.tar.gz cd readline-5.1 ./configure --prefix=$HOME/extra make make install cd .. wget ftp://ftp.ruby-lang.org/pub/ruby/ruby-1.8.4.tar.gz tar zxvf ruby-1.8.4.tar.gz rm -f ruby-1.8.4.tar.gz cd ruby-1.8.4 ./configure --prefix=$HOME/extra --with-openssl=$HOME/extra --with-readline-dir=$HOME/extra make make install cd .. wget http://tmtm.org/downloads/mysql/ruby/mysql-ruby-2.7.tar.gz tar zxvf mysql-ruby-2.7.tar.gz rm -f mysql-ruby-2.7.tar.gz cd mysql-ruby-2.7 ruby extconf.rb --with-mysql-config=/usr/bin/mysql_config make # PLACE CUSTOMIZED DATABASE PARAMETERS HERE: ruby ./test.rb DBHOST DBUSER DBPASS DBNAME DBPORT make install cd .. wget http://sugi.nemui.org/pub/ruby/fcgi/ruby-fcgi-0.8.6.tar.gz tar zxvf ruby-fcgi-0.8.6.tar.gz rm -f ruby-fcgi-0.8.6.tar.gz cd ruby-fcgi-0.8.6 ruby install.rb config ruby install.rb setup ruby install.rb install cd .. wget http://rubyforge.org/frs/download.php/5207/rubygems-0.8.11.tgz tar zxvf rubygems-0.8.11.tgz rm -f rubygems-0.8.11.tgz cd rubygems-0.8.11 ruby setup.rb cd .. gem install rails --include-dependencies On 4/19/06, Simon Wex wrote: > I am unable to install gems on my dreamhost server since it's just > shared hosting. I tried unpacking the ferret gem into vendor/ferret, but > to no avail. > > Also how would I recompile the C extensions for the new platform? > > Any ideas would be greatly appreciated. > > Cheers, Simon > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From simon at tungstenconsulting.com Wed Apr 19 20:45:42 2006 From: simon at tungstenconsulting.com (Simon Wex) Date: Thu, 20 Apr 2006 02:45:42 +0200 Subject: [Ferret-talk] Shared host (Dreamhost) In-Reply-To: References: Message-ID: <74a8055112f55ab234e3a0fc2567c676@ruby-forum.com> Carl Youngblood wrote: > Lots of good stuff. Nicely done. I'll try that out when I get home. Thanks bud. -- Posted via http://www.ruby-forum.com/. From Pedro.CorteReal at iantt.pt Thu Apr 20 11:25:05 2006 From: Pedro.CorteReal at iantt.pt (Pedro =?ISO-8859-1?Q?C=F4rte-Real?=) Date: Thu, 20 Apr 2006 16:25:05 +0100 Subject: [Ferret-talk] Creating my own analyzer Message-ID: <1145546705.9213.7.camel@localhost.localdomain> I created this analyzer: class DescriptionAnalyzer < Ferret::Analysis::Analyzer def token_stream(field, string) if field == "code" return CodeTokenStream.new(string) else return Ferret::Analysis::Analyzer.new.token_stream(field,string) end end end and created an IndexWriter with it: Ferret::Index::IndexWriter.new(get_index_path, :create => true, :analyzer => DescriptionAnalyzer.new) get_index_path is a method to calculate the path to use. The problem is that my token_stream method is never called. I'm using ferret 0.9.1 from rubygems, ruby is: ruby 1.8.4 (2005-12-24) [i486-linux] I'm running this in a rails app, but the script that creates the index only uses ActiveRecord to iterate all the records and index them. Any ideas? Greetings, Pedro. From joergd at pobox.com Thu Apr 20 11:39:11 2006 From: joergd at pobox.com (Joerg Diekmann) Date: Thu, 20 Apr 2006 17:39:11 +0200 Subject: [Ferret-talk] index_dir path Message-ID: <758d49a38138273a3dfc8e5df2c7ae47@ruby-forum.com> Hi, I am running several applications, and I need them to share their indexed data. So instead of putting my index in the default location of RAILS_ROOT/index/RAILS_ENV I want it to appear in RAILS_ROOT/../index/RAILS_ENV (ie in the parent directory of each of my root directories ... I have three apps working together) In the acts_as_ferret file, I have changed line 73 to: index_base = "#{RAILS_ROOT}/../index" In my console, I can see that the new path is reflected properly. >> MyModel.class_index_dir => "./script/../config/../config/../../index/development/MyModel" However, the index is STILL placed in the default location of RAILS_ROOT/index/RAILS_ENV Any ideas .... ? Thanks Joerg -- Posted via http://www.ruby-forum.com/. From Pedro.CorteReal at iantt.pt Fri Apr 21 10:50:38 2006 From: Pedro.CorteReal at iantt.pt (Pedro =?ISO-8859-1?Q?C=F4rte-Real?=) Date: Fri, 21 Apr 2006 15:50:38 +0100 Subject: [Ferret-talk] Sorting Search results Message-ID: <1145631039.9222.4.camel@localhost.localdomain> I tried sorting the search results in Ferret::Index::Index#search and what I found was that the sort is applied not to the whole search but to the returned results. Suppose I have these results: Text - Num foo - 2 bar - 3 far - 4 boo - 1 If I limit the search to the first two and sort by num I get: foo - 2 bar - 3 while I think the natural behaviour would be: boo - 1 foo - 2 Is this intended or a bug? Greetings, Pedro. From nathaniel at talbott.ws Fri Apr 21 11:34:29 2006 From: nathaniel at talbott.ws (Nathaniel Talbott) Date: Fri, 21 Apr 2006 17:34:29 +0200 Subject: [Ferret-talk] Stop words in queries In-Reply-To: References: <58b153ea16c61670e7742648dce8ed5d@ruby-forum.com> Message-ID: <2469371c104e99d933e9880ffb39d97c@ruby-forum.com> David Balmain wrote: > The pure ruby version is still there and I'd love for someone to take > over from me. I completely agree with you on the advantages of having > a pure ruby version. I personally want the performance which is why I > have taken the C route. And there is a huge difference. Somewhere > around 100 times. There are people out there who were still using Java > Lucene for indexing because of performance issues so I wasn't the only > one concerned about the performance. Understood, and I do look forward to improved performance. > As for refactoring the API, I > understand it is very difficult for some Ruby programmers to get > around the C code but you don't need to send me a patch. Just let me > know what you think needs to be changed. My big suggestion would be to cut down on the surface area of the API - it's almost overly flexible, and feels over-designed (probably due to the port from Java). Fewer (documented) classes, simplified options, etc. Basically, it's a bit overwhelming to someone coming at it for the first time, and I don't think that's strictly (or even mostly) a documentation issue. As I use it more I'll try to come up with specific examples. My small suggestion would be to use symbols (and booleans) for configuration instead of the constants currently being used. For instance: Ferret::Document::Field::Store::YES -> true Ferret::Document::Field::Store::NO -> false Ferret::Document::Field::Store::COMPRESS -> :compress and Ferret::Document::Field::Index::NO -> false Ferret::Document::Field::Index::TOKENIZED -> :tokenized Ferret::Document::Field::Index::UNTOKENIZED -> :untokenized I think this would help Ferret configuration feel much more Rubyish. >> that raw benchmarks are going to drive that. > I want the same thing too. The other advantage to having the C version > is that it won't be too much work to Ferret in Perl, Python, Tcl etc. But why share? (just kidding ;-) > I'm currently an athlete. I'm practicing Judo in Japan and working on > Ferret whenever I have time. Fascinating (and very cool). Best of luck with it! Thanks again for Ferret, Nathaniel Talbott -- Posted via http://www.ruby-forum.com/. From david.harrison at stress-free.co.nz Fri Apr 21 21:12:31 2006 From: david.harrison at stress-free.co.nz (Error with Ferret C Indexer) Date: Sat, 22 Apr 2006 03:12:31 +0200 Subject: [Ferret-talk] Ferret C Indexer Error: Fields not stored in index? Message-ID: Hello, I am trying to get Ferret's C indexer to work on OpenSUSE 10 and fastcgi. Indexing documents appears to work correctly but when I try to display the results I recieve the following error: ActionView::TemplateError (undefined method `string_value' for stored/uncompressed,indexed,tokenized,:Ferret::Document::Field) on line #17 of app/views/search/results.rhtml: 14: <%= render :partial => 'basic' %> 15:
    16: <% for result in @results 17: text = "" + result.field('title').string_value + "
    " 18: text += "Posted by " + result.field('created_by').string_value + " at " + result.field('created_on').string_value + " " 19: if !result.field('body').nil? 20: description = strip_tags(result.field('body').string_value) #{RAILS_ROOT}/app/views/search/results.rhtml:17:in `_run_rhtml_search_results' #{RAILS_ROOT}/app/views/search/results.rhtml:16:in `each' #{RAILS_ROOT}/app/views/search/results.rhtml:16:in `_run_rhtml_search_results' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_view/base.rb:314:in `send' It appears as if the indexer is not storing the actual field strings. Data for the index is being pulled from a MySQL database and then put into Ferret. If I delete the ferret_ext.so library file and rebuild the index using the pure Ruby indexer everything works perfectly so I am certain this is an issue with the C indexer. I have tried the gem file and compiling from source from svn (rev 325) and both produce the same error. Does anyone have any ideas? Regards, David -- Posted via http://www.ruby-forum.com/. From david.harrison at stress-free.co.nz Fri Apr 21 22:13:39 2006 From: david.harrison at stress-free.co.nz (David Harrison) Date: Sat, 22 Apr 2006 04:13:39 +0200 Subject: [Ferret-talk] Ferret C Indexer Error: Fields not stored in index? In-Reply-To: References: Message-ID: After a bit of experimenting and reflection I have found that when using the C indexer the function Field.data returns the value of the indexed field. Unfortunately using the pure Ruby indexer Field.data is not a function (its equivalent is Field.string_value). This is not too much of an issue though as I can do some presentation level reflection to call the correct function. There's probably a better workaround than this but at least things are now working nice and quickly with the C indexer and fastcgi. Regards, David -- Posted via http://www.ruby-forum.com/. From simon at tungstenconsulting.com Sat Apr 22 18:40:34 2006 From: simon at tungstenconsulting.com (Simon Wex) Date: Sun, 23 Apr 2006 00:40:34 +0200 Subject: [Ferret-talk] Shared host (Dreamhost) In-Reply-To: <74a8055112f55ab234e3a0fc2567c676@ruby-forum.com> References: <74a8055112f55ab234e3a0fc2567c676@ruby-forum.com> Message-ID: Carl, When I run make for openssl I get the following errors, any ideas? Cheers, Simon. make -f ../Makefile.shared -e \ APPNAME=openssl OBJECTS="openssl.o verify.o asn1pars.o req.o dgst.o dh.o dhparam.o enc.o passwd.o gendh.o errstr.o ca.o pkcs7.o crl2p7.o crl.o rsa.o rsautl.o dsa.o dsapa\ LIBDEPS=" $LIBRARIES -ldl" \ link_app.${shlib_target} make[2]: Entering directory `/home/.divot/mobiforge/extra/src/openssl-0.9.8a/apps' ( :; LIBDEPS="${LIBDEPS:--L.. -lssl -L.. -lcrypto -ldl}"; LDCMD="${LDCMD:-gcc}"; LDFLAGS="${LDFLAGS:--DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERM) speed.o(.text+0x764): In function `speed_main': : undefined reference to `BF_set_key' speed.o(.text+0x1420): In function `speed_main': : undefined reference to `BF_cbc_encrypt' speed.o(.text+0x1fde): In function `speed_main': : undefined reference to `BF_options' version.o(.text+0x173): In function `version_main': : undefined reference to `BF_options' ../libcrypto.a(e_bf.o)(.text+0x5d): In function `bf_init_key': : undefined reference to `BF_set_key' ../libcrypto.a(e_bf.o)(.text+0xa4): In function `bf_cbc_cipher': : undefined reference to `BF_cbc_encrypt' ../libcrypto.a(e_bf.o)(.text+0xfb): In function `bf_cfb64_cipher': : undefined reference to `BF_cfb64_encrypt' ../libcrypto.a(e_bf.o)(.text+0x154): In function `bf_ecb_cipher': : undefined reference to `BF_ecb_encrypt' ../libcrypto.a(e_bf.o)(.text+0x1a4): In function `bf_ofb_cipher': : undefined reference to `BF_ofb64_encrypt' collect2: ld returned 1 exit status make[2]: *** [link_app.] Error 1 make[2]: Leaving directory `/home/.divot/mobiforge/extra/src/openssl-0.9.8a/apps' make[1]: *** [openssl] Error 2 make[1]: Leaving directory `/home/.divot/mobiforge/extra/src/openssl-0.9.8a/apps' make: *** [build_apps] Error 1 -- Posted via http://www.ruby-forum.com/. From newsray at warmroom.com Sat Apr 22 20:59:59 2006 From: newsray at warmroom.com (Ray Baxter) Date: Sun, 23 Apr 2006 02:59:59 +0200 Subject: [Ferret-talk] Typo and acts_as_ferret rebuild_index errror In-Reply-To: <20060417155213.GB7420@cordoba.webit.de> References: <736d9aeb8cab047f75667a516f9de662@ruby-forum.com> <20060417155213.GB7420@cordoba.webit.de> Message-ID: <2393ff79623cbed4a7e9f914f80bbc7f@ruby-forum.com> Jens Kraemer wrote: > I could reproduce the error, but I can't explain why it happens ;-) > I just committed a fix that rescues from the failure and indexes an > empty string for the field in question. > > It seems the errors are raised when aaf tries to index a > field that isn't used by the particular model class, i.e. :title in a > Comment object. I was seeing a similar error on cases where there would be a single row that had NULL for the value of an indexed field. Rows before that row would index and then the exception would be encountered and indexing would stop. This fix also works for me. Thanks, Ray -- Posted via http://www.ruby-forum.com/. From simon at tungstenconsulting.com Sat Apr 22 22:48:41 2006 From: simon at tungstenconsulting.com (Simon Wex) Date: Sun, 23 Apr 2006 04:48:41 +0200 Subject: [Ferret-talk] Shared host (Dreamhost) In-Reply-To: References: <74a8055112f55ab234e3a0fc2567c676@ruby-forum.com> Message-ID: <4f0a75516f0c2492c778a7cbeec1ca57@ruby-forum.com> I think I figured out the issue. My script was being killed for some reason when I ran it interactively and the ./config didn't finish correctly. I pipped the output to a file and ran the script in the background with & I'm not sure why this fixed it, but it seems to have so far. -- Posted via http://www.ruby-forum.com/. From carl at youngbloods.org Sun Apr 23 01:45:50 2006 From: carl at youngbloods.org (Carl Youngblood) Date: Sat, 22 Apr 2006 23:45:50 -0600 Subject: [Ferret-talk] Shared host (Dreamhost) In-Reply-To: <4f0a75516f0c2492c778a7cbeec1ca57@ruby-forum.com> References: <74a8055112f55ab234e3a0fc2567c676@ruby-forum.com> <4f0a75516f0c2492c778a7cbeec1ca57@ruby-forum.com> Message-ID: I noticed that Dreamhost actually shut down my script occasionally while I was running it, probably because it ties up the CPU for extensive periods of time. I had to keep trying until it completed successfully. I'm sure this annoys them somewhat, but I don't have to set up my environment from scratch very often. Carl On 4/22/06, Simon Wex wrote: > I think I figured out the issue. My script was being killed for some > reason when I ran it interactively and the ./config didn't finish > correctly. I pipped the output to a file and ran the script in the > background with & > > I'm not sure why this fixed it, but it seems to have so far. > > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From kraemer at webit.de Sun Apr 23 06:22:30 2006 From: kraemer at webit.de (Jens Kraemer) Date: Sun, 23 Apr 2006 12:22:30 +0200 Subject: [Ferret-talk] index_dir path In-Reply-To: <758d49a38138273a3dfc8e5df2c7ae47@ruby-forum.com> References: <758d49a38138273a3dfc8e5df2c7ae47@ruby-forum.com> Message-ID: <20060423102230.GA3146@cordoba.webit.de> Hi Joerg, On Thu, Apr 20, 2006 at 05:39:11PM +0200, Joerg Diekmann wrote: [..] > In my console, I can see that the new path is reflected properly. > > >> MyModel.class_index_dir > => "./script/../config/../config/../../index/development/MyModel" > > > However, the index is STILL placed in the default location of > RAILS_ROOT/index/RAILS_ENV > > Any ideas .... ? just guessing, maybe your running app doesn't use your patched acts_as_ferret.rb ? I just tried to reproduce the problem, but everything worked fine here. regards, Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From kraemer at webit.de Sun Apr 23 06:41:53 2006 From: kraemer at webit.de (Jens Kraemer) Date: Sun, 23 Apr 2006 12:41:53 +0200 Subject: [Ferret-talk] How to do case-sensitive searches In-Reply-To: References: Message-ID: <20060423104153.GB3146@cordoba.webit.de> Hi Carl, On Tue, Apr 18, 2006 at 11:32:36PM -0600, Carl Youngblood wrote: > Forgive me if this topic has already been discussed on the list. I > googled but couldn't find much. I'd like to search through text for > US state abbreviations that are written in capitals. What is the best > way to do this? I read somewhere that tokenized fields are stored in > the index in lowercase, so I am concerned that I will lose precision. > What is the best way to store a field so that normal searches are > case-insensitive but case-sensitive searches can still be made? Are you sure this is a problem, i.e. do you get wrong hits because the lowercase variant of an abbreviation is used in another context ? I don't know what those abbrevs look like... To run case-sensitive and case-insensitive searches you'd need two fields, a tokenized one for normal case-insensitive searches, and an untokenized one for looking up the abbreviations. To reduce overhead in the index, you could filter the text for the known set of abbreviations at indexing time and only store those values in the untokenized field. Possibly this could be done in a custom analyzer. regards, Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From joergd at pobox.com Mon Apr 24 06:45:25 2006 From: joergd at pobox.com (Joerg Diekmann) Date: Mon, 24 Apr 2006 12:45:25 +0200 Subject: [Ferret-talk] index_dir path In-Reply-To: <20060423102230.GA3146@cordoba.webit.de> References: <758d49a38138273a3dfc8e5df2c7ae47@ruby-forum.com> <20060423102230.GA3146@cordoba.webit.de> Message-ID: <0e8ef253fbfd4cc6252b1c2edf1ae79c@ruby-forum.com> Just did an SVN update now, and I am still having the same problem. Maybe I was a bit vague ... Basically, I want this structure: index production content development content myapp_1 app components config db doc lib ... myapp_2 app components config db doc lib ... As opposed to having the index folder inside each application. For this I changed line 73 (inacts_as_ferret) to index_base = "#{RAILS_ROOT}/../index" This should work ... but it still creates the index folder inside the application folder. Using the console, however, reveals that it has picked up this index_base change - but seems to be ignoring it. I'm running XP. Let me know if I can give you any more information. Thanks Joerg > On Thu, Apr 20, 2006 at 05:39:11PM +0200, Joerg Diekmann wrote: > [..] >> In my console, I can see that the new path is reflected properly. >> >> >> MyModel.class_index_dir >> => "./script/../config/../config/../../index/development/MyModel" >> -- Posted via http://www.ruby-forum.com/. From johnpetterson at gmail.com Tue Apr 25 01:37:20 2006 From: johnpetterson at gmail.com (NN) Date: Tue, 25 Apr 2006 07:37:20 +0200 Subject: [Ferret-talk] Indexing Message-ID: Hi guys, I am trying to Index the doc/pdf files. If anybody could help me with the same. Give me some good links, if possible. Thanking you in advance. -- Posted via http://www.ruby-forum.com/. From etienne.durand at woa.hu Tue Apr 25 02:09:37 2006 From: etienne.durand at woa.hu (Jean-Etienne) Date: Tue, 25 Apr 2006 08:09:37 +0200 Subject: [Ferret-talk] Migrating to 0.9.1 Message-ID: <0af228cb1ab449e88a50228d909183eb@ruby-forum.com> After migrating to 0.9.1, I got: usr/local/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:123:in `const_missing': uninitialized constant TokenFilter (NameError) Here is a snapshot of my code: ... require 'ferret' class MyFilter < Analysis::TokenFilter ... I works fine on my dev machine, but not a production server (shared host). Any idea? -- Posted via http://www.ruby-forum.com/. From dbalmain.ml at gmail.com Tue Apr 25 03:20:12 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 25 Apr 2006 16:20:12 +0900 Subject: [Ferret-talk] Ferret EOFError creating index In-Reply-To: <482216f2ce11094dbce36e364dcce2a5@ruby-forum.com> References: <482216f2ce11094dbce36e364dcce2a5@ruby-forum.com> Message-ID: Hi Derek, I'm not sure why this happens. But it seems to only happen with the pure Ruby version so I'd recommend waiting for the Windows port of the C version. You'll also be done in minutes rather than hours with about 100 times speed up of the indexing. I will get around to trying to fix this bug in the pure Ruby version eventually but don't hold your breath. Patches welcome! Cheers, Dave On 4/19/06, Derek Mailer wrote: > I've been messing around with Ferret (no punn intended). After spending > some time testing it out (indexing to file), I decided to index about > 10% of the data I want to eventually index. It took several hours to > complete the index on my local machine, but it was created without any > problems and after optimising it the searches returned results at the > sort of speed I was expecting. I then continuned to add a further 10% of > the overall data to the index and it all ran through fine for a few > hours until it bombed out with the error below. > > I've been running this on my local machine and the first run was left > overnight, whereas the second run was going on while I was using my > machine for other things. Which leads me to suspect that it's a > memory-related issue that made it bail out. > > so I have 2 questions: > 1) does anyone know what caused the error or have any suggestions to > prevent it in future? > 2) I can't search on the index without getting errors so is the index > recoverable or will I need to recreate the index again? > > Thanks in advance, > Derek > > EOFError > > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:178:in > `refill' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:94:in > `read_byte' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:109:in > `read_bytes' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:108:in > `upto' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/store/buffered_index_io.rb:108:in > `read_bytes' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_reader.rb:286:in > `get_norms_into' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_reader.rb:271:in > `synchronize' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_reader.rb:271:in > `get_norms_into' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:341:in > `merge_norms' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:338:in > `each' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:338:in > `merge_norms' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:334:in > `each_with_index' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/field_infos.rb:130:in > `each_with_index' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/field_infos.rb:130:in > `each' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/field_infos.rb:130:in > `each_with_index' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/field_infos.rb:130:in > `each_with_index' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:334:in > `merge_norms' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_merger.rb:49:in > `merge' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:403:in > `merge_segments' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:351:in > `flush_ram_segments' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:127:in > `close' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:126:in > `synchronize' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index_writer.rb:126:in > `close' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index.rb:173:in > `close' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index.rb:168:in > `synchronize' > c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/index.rb:168:in > `close' > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Tue Apr 25 03:21:33 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 25 Apr 2006 16:21:33 +0900 Subject: [Ferret-talk] Creating my own analyzer In-Reply-To: <1145546705.9213.7.camel@localhost.localdomain> References: <1145546705.9213.7.camel@localhost.localdomain> Message-ID: Sorry Pedro, you can't create your own analyzer in 0.9.1. If you want to check out the latest version from subversion, this should work. Cheers, Dave On 4/21/06, Pedro C?rte-Real wrote: > I created this analyzer: > > class DescriptionAnalyzer < Ferret::Analysis::Analyzer > def token_stream(field, string) > if field == "code" > return CodeTokenStream.new(string) > else > return Ferret::Analysis::Analyzer.new.token_stream(field,string) > end > end > end > > and created an IndexWriter with it: > > Ferret::Index::IndexWriter.new(get_index_path, > :create => true, > :analyzer => DescriptionAnalyzer.new) > > get_index_path is a method to calculate the path to use. > > The problem is that my token_stream method is never called. > > I'm using ferret 0.9.1 from rubygems, ruby is: > > ruby 1.8.4 (2005-12-24) [i486-linux] > > I'm running this in a rails app, but the script that creates the index > only uses ActiveRecord to iterate all the records and index them. > > Any ideas? > > Greetings, > > Pedro. > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Tue Apr 25 09:20:17 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 25 Apr 2006 22:20:17 +0900 Subject: [Ferret-talk] Sorting Search results In-Reply-To: <1145631039.9222.4.camel@localhost.localdomain> References: <1145631039.9222.4.camel@localhost.localdomain> Message-ID: Thanks Pedro, This was a bug. It is now fixed. Regards, Dave On 4/21/06, Pedro C?rte-Real wrote: > I tried sorting the search results in Ferret::Index::Index#search and > what I found was that the sort is applied not to the whole search but to > the returned results. > > Suppose I have these results: > > Text - Num > > foo - 2 > bar - 3 > far - 4 > boo - 1 > > If I limit the search to the first two and sort by num I get: > > foo - 2 > bar - 3 > > while I think the natural behaviour would be: > > boo - 1 > foo - 2 > > Is this intended or a bug? > > Greetings, > > Pedro. > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Tue Apr 25 09:32:53 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 25 Apr 2006 22:32:53 +0900 Subject: [Ferret-talk] Ferret C Indexer Error: Fields not stored in index? In-Reply-To: References: Message-ID: Hi David, Please use Field#data from now on. data is actually an attribute of Field in the pure Ruby version so I'm not sure why it wasn't working for you. ie, you shouldn't need to do any reflection to access Field#data. Eventually I'll probably remove the string_value and binary_value methods but they'll stay for the moment. Sorry for the confusion. Cheers, Dave On 4/22/06, David Harrison wrote: > After a bit of experimenting and reflection I have found that when using > the C indexer the function Field.data returns the value of the indexed > field. > Unfortunately using the pure Ruby indexer Field.data is not a function > (its equivalent is Field.string_value). > > This is not too much of an issue though as I can do some presentation > level reflection to call the correct function. > > There's probably a better workaround than this but at least things are > now working nice and quickly with the C indexer and fastcgi. > > Regards, > > > David > > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Tue Apr 25 10:06:22 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 25 Apr 2006 23:06:22 +0900 Subject: [Ferret-talk] How to do case-sensitive searches In-Reply-To: <20060423104153.GB3146@cordoba.webit.de> References: <20060423104153.GB3146@cordoba.webit.de> Message-ID: Hey guys, This might help. The following code takes th input; "I used to live in NSW but now I live in the A.C.T." and produces; token["used":2:6:2] token["live":10:14:2] token["nsw":18:21:2] token["NSW":18:21:0] token["now":26:29:2] token["live":32:36:2] token["act":44:49:3] token["ACT":44:49:0] As you can see, the Australian states have been entered twice, once in upper case. The first two numbers are the start and end offsets, ie how many bytes from the start. The third digit is the position increment. So "live" occurs two words after "used" but "nsw" is in the same position as "NSW". Now you just have to make sure your query parser uses the correct Analyzer. Hope this helps. Dave require 'ferret' module Ferret::Analysis class TokenFilter < TokenStream protected # Construct a token stream filtering the given input. def initialize(input) @input = input end end class StateFilter < TokenFilter STATES = { "nsw" => "new south wales", "vic" => "victoria", "qld" => "queensland", "tas" => "tasmania", "sa" => "south australia", "wa" => "western australia", "nt" => "northern territory", "act" => "australian capital territory" } def next() if @state t = @state @state = nil return t end t = @input.next return nil if t.nil? if STATES[t.text] @state = Token.new(t.text.upcase, t.start_offset, t.end_offset, 0) end return t end end class StateAnalyzer < StandardAnalyzer def token_stream(field, text) StateFilter.new(super) end end analyzer = StateAnalyzer.new ts = analyzer.token_stream(nil, "I used to live in NSW but now I live in the A.C.T.") while t = ts.next puts t end end On 4/23/06, Jens Kraemer wrote: > Hi Carl, > > On Tue, Apr 18, 2006 at 11:32:36PM -0600, Carl Youngblood wrote: > > Forgive me if this topic has already been discussed on the list. I > > googled but couldn't find much. I'd like to search through text for > > US state abbreviations that are written in capitals. What is the best > > way to do this? I read somewhere that tokenized fields are stored in > > the index in lowercase, so I am concerned that I will lose precision. > > What is the best way to store a field so that normal searches are > > case-insensitive but case-sensitive searches can still be made? > > Are you sure this is a problem, i.e. do you get wrong hits because > the lowercase variant of an abbreviation is used in another context ? > I don't know what those abbrevs look like... > > To run case-sensitive and case-insensitive searches you'd need two > fields, a tokenized one for normal case-insensitive searches, and an > untokenized one for looking up the abbreviations. > > To reduce overhead in the index, you could filter the text for the > known set of abbreviations at indexing time and only store those > values in the untokenized field. Possibly this could be done in a > custom analyzer. > > regards, > Jens > > > -- > webit! Gesellschaft f?r neue Medien mbH www.webit.de > Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de > Schnorrstra?e 76 Tel +49 351 46766 0 > D-01069 Dresden Fax +49 351 46766 66 > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Tue Apr 25 10:08:36 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Tue, 25 Apr 2006 23:08:36 +0900 Subject: [Ferret-talk] Migrating to 0.9.1 In-Reply-To: <0af228cb1ab449e88a50228d909183eb@ruby-forum.com> References: <0af228cb1ab449e88a50228d909183eb@ruby-forum.com> Message-ID: You'll have to wait for 0.9.2. Sorry for the trouble. There are still a few things missing from the C version of Ferret. Regards, Dave On 4/25/06, Jean-Etienne wrote: > After migrating to 0.9.1, I got: > > usr/local/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:123:in > `const_missing': uninitialized constant TokenFilter (NameError) > > Here is a snapshot of my code: > ... > require 'ferret' > > class MyFilter < Analysis::TokenFilter > ... > > I works fine on my dev machine, but not a production server (shared > host). > Any idea? > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From frank at nullsense.net Wed Apr 26 06:41:10 2006 From: frank at nullsense.net (Frank Rosquin) Date: Wed, 26 Apr 2006 12:41:10 +0200 Subject: [Ferret-talk] Search multiple models Message-ID: Hello, Lets say you have a few models like Post, Article, Wiki, Comment, And you want to use ferret to search all of them at once. How would I set up the latest acts_as_ferret to accomplish this? And what would be fastest for searches? 1 index for all models, or have an index per model? Thank you -- Posted via http://www.ruby-forum.com/. From lmarlow at yahoo.com Wed Apr 26 17:52:16 2006 From: lmarlow at yahoo.com (Lee Marlow) Date: Wed, 26 Apr 2006 15:52:16 -0600 Subject: [Ferret-talk] Building native extensions fail on Macbook Pro Message-ID: <7968d7490604261452t6e31e6f4u5736ff3cdff09012@mail.gmail.com> I noticed that indexing seemed a little slow, so I tried reinstalling the 0.9.1 gem and noticed some errors while it was building the native extensions. Ferret still works with the pure ruby versions, but I'd love to take advantage of the c speed. I'm not real familiar with C, so I was wondering if anyone know what the errors are and if they can be fixed. Thanks -Lee ****** gem install output ******* $ sudo gem install ferret Attempting local installation of 'ferret' Local gem file not found: ferret*.gem Attempting remote installation of 'ferret' Building native extensions. This could take a while... /usr/bin/ld: multiple definitions of symbol _ranalyzer_key r_index_io.o definition of _ranalyzer_key in section (__DATA,__common) r_qparser.o definition of _ranalyzer_key in section (__DATA,__common) collect2: ld returned 1 exit status make: *** [ferret_ext.bundle] Error 1 /usr/bin/ld: multiple definitions of symbol _ranalyzer_key r_index_io.o definition of _ranalyzer_key in section (__DATA,__common) r_qparser.o definition of _ranalyzer_key in section (__DATA,__common) collect2: ld returned 1 exit status make: *** [ferret_ext.bundle] Error 1 ruby extconf.rb install ferret creating Makefile make gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c analysis.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c api.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c array.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c bitvector.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c compound_io.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c document.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c except.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c ferret.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c field.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c filter.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c fs_store.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c global.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c hash.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c hashset.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c helper.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c ind.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c index_io.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c index_rw.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c lang.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c libstemmer.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c priorityqueue.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_boolean.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_const_score.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_filtered_query.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_fuzzy.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_match_all.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_multi_phrase.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_parser.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_phrase.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_prefix.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_range.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_span.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_term.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c q_wildcard.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c r_analysis.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c r_doc.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c r_index_io.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c r_qparser.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c r_search.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c r_store.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c r_term.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c ram_store.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c search.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c similarity.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c sort.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_danish.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_dutch.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_english.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_finnish.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_french.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_german.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_italian.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_norwegian.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_porter.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_portuguese.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_spanish.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_ISO_8859_1_swedish.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_KOI8_R_russian.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_danish.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_dutch.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_english.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_finnish.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_french.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_german.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_italian.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_norwegian.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_porter.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_portuguese.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_russian.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_spanish.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stem_UTF_8_swedish.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c stopwords.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c store.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c term.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c termdocs.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c utilities.c gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe -I/opt/local/include -c vector.c cc -dynamic -bundle -undefined suppress -flat_namespace -L/opt/local/lib -L"/opt/local/lib" -o ferret_ext.bundle analysis.o api.o array.o bitvector.o compound_io.o document.o except.o ferret.o field.o filter.o fs_store.o global.o hash.o hashset.o helper.o ind.o index_io.o index_rw.o lang.o libstemmer.o priorityqueue.o q_boolean.o q_const_score.o q_filtered_query.o q_fuzzy.o q_match_all.o q_multi_phrase.o q_parser.o q_phrase.o q_prefix.o q_range.o q_span.o q_term.o q_wildcard.o r_analysis.o r_doc.o r_index_io.o r_qparser.o r_search.o r_store.o r_term.o ram_store.o search.o similarity.o sort.o stem_ISO_8859_1_danish.o stem_ISO_8859_1_dutch.o stem_ISO_8859_1_english.o stem_ISO_8859_1_finnish.o stem_ISO_8859_1_french.o stem_ISO_8859_1_german.o stem_ISO_8859_1_italian.o stem_ISO_8859_1_norwegian.o stem_ISO_8859_1_porter.o stem_ISO_8859_1_portuguese.o stem_ISO_8859_1_spanish.o stem_ISO_8859_1_swedish.o stem_KOI8_R_russian.o stem_UTF_8_danish.o stem_UTF_8_dutch.o stem_UTF_8_english.o stem_UTF_8_finnish.o stem_UTF_8_french.o stem_UTF_8_german.o stem_UTF_8_italian.o stem_UTF_8_norwegian.o stem_UTF_8_porter.o stem_UTF_8_portuguese.o stem_UTF_8_russian.o stem_UTF_8_spanish.o stem_UTF_8_swedish.o stopwords.o store.o term.o termdocs.o utilities.o vector.o -lruby -lpthread -ldl -lobjc make install cc -dynamic -bundle -undefined suppress -flat_namespace -L/opt/local/lib -L"/opt/local/lib" -o ferret_ext.bundle analysis.o api.o array.o bitvector.o compound_io.o document.o except.o ferret.o field.o filter.o fs_store.o global.o hash.o hashset.o helper.o ind.o index_io.o index_rw.o lang.o libstemmer.o priorityqueue.o q_boolean.o q_const_score.o q_filtered_query.o q_fuzzy.o q_match_all.o q_multi_phrase.o q_parser.o q_phrase.o q_prefix.o q_range.o q_span.o q_term.o q_wildcard.o r_analysis.o r_doc.o r_index_io.o r_qparser.o r_search.o r_store.o r_term.o ram_store.o search.o similarity.o sort.o stem_ISO_8859_1_danish.o stem_ISO_8859_1_dutch.o stem_ISO_8859_1_english.o stem_ISO_8859_1_finnish.o stem_ISO_8859_1_french.o stem_ISO_8859_1_german.o stem_ISO_8859_1_italian.o stem_ISO_8859_1_norwegian.o stem_ISO_8859_1_porter.o stem_ISO_8859_1_portuguese.o stem_ISO_8859_1_spanish.o stem_ISO_8859_1_swedish.o stem_KOI8_R_russian.o stem_UTF_8_danish.o stem_UTF_8_dutch.o stem_UTF_8_english.o stem_UTF_8_finnish.o stem_UTF_8_french.o stem_UTF_8_german.o stem_UTF_8_italian.o stem_UTF_8_norwegian.o stem_UTF_8_porter.o stem_UTF_8_portuguese.o stem_UTF_8_russian.o stem_UTF_8_spanish.o stem_UTF_8_swedish.o stopwords.o store.o term.o termdocs.o utilities.o vector.o -lruby -lpthread -ldl -lobjc Successfully installed ferret-0.9.1 From Pedro.CorteReal at iantt.pt Thu Apr 27 07:28:48 2006 From: Pedro.CorteReal at iantt.pt (Pedro =?ISO-8859-1?Q?C=F4rte-Real?=) Date: Thu, 27 Apr 2006 12:28:48 +0100 Subject: [Ferret-talk] Ferret crashing Ruby Message-ID: <1146137328.9225.13.camel@localhost.localdomain> I've gotten several problems with C ferret crashing ruby. Here's the stack trace of my latest case: #0 0x00000050 in ?? () #1 0xb74028ab in iw_close (iw=0x8685a40) at index_rw.c:947 #2 0xb7414359 in index_destroy (self=0x829ebc0) at ind.c:89 #3 0xb73f00bc in frt_ind_free (p=0x829ebc0) at r_search.c:1564 #4 0xb7f04dca in rb_gc_call_finalizer_at_exit () from /usr/lib/libruby1.8.so.1.8 #5 0xb7ee627d in is_ruby_native_thread () from /usr/lib/libruby1.8.so.1.8 #6 0xb7efe2b5 in ruby_cleanup () from /usr/lib/libruby1.8.so.1.8 #7 0xb7efe3cc in ruby_stop () from /usr/lib/libruby1.8.so.1.8 #8 0xb7efeabf in ruby_run () from /usr/lib/libruby1.8.so.1.8 #9 0x080486b1 in main () This was while doing several identical searches ordered by a field. The same code without the extensions throws this exception: RuntimeError: no terms in field start_date to sort by /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_cache.rb:189:in `get_auto_index' /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_sorted_hit_queue.rb:172:in `comparator_auto' /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_sorted_hit_queue.rb:123:in `get_cached_comparator' /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_sorted_hit_queue.rb:26:in `initialize' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/whiny_nil.rb:35:in `each_with_index' /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_sorted_hit_queue.rb:25:in `initialize' /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/index_searcher.rb:115:in `search' /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/index/index.rb:660:in `do_search' /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/index/index.rb:316:in `search' /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize' /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/index/index.rb:315:in `search' /home/pedrocr/eadexer/config/../app/models/description.rb:233:in `find_by_text' /home/pedrocr/eadexer/config/../app/controllers/descr_controller.rb:67:in `find' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/base.rb:910:in `perform_action_without_filters' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/filters.rb:368:in `perform_action_without_benchmark' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue' /usr/lib/ruby/1.8/benchmark.rb:293:in `measure' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/rescue.rb:82:in `perform_action' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/base.rb:381:in `process_without_filters' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/filters.rb:377:in `process_without_session_management_support' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/session_management.rb:117:in `process_without_test' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/test_process.rb:16:in `process' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/test_process.rb:363:in `process' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/test_process.rb:336:in `get' test/functional/descr_controller_test.rb:88:in `test_find_order_by' test/functional/descr_controller_test.rb:87:in `test_find_order_by' With the C extension this search actually works fine. But if I surround it with a 2.times do; ..search.. end; block it bombs. I'm using ferret 0.9.1. Should I be using SVN? Any ideas? Greetings, Pedro C?rte-Real. From dbalmain.ml at gmail.com Thu Apr 27 20:53:07 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Fri, 28 Apr 2006 09:53:07 +0900 Subject: [Ferret-talk] Building native extensions fail on Macbook Pro In-Reply-To: <7968d7490604261452t6e31e6f4u5736ff3cdff09012@mail.gmail.com> References: <7968d7490604261452t6e31e6f4u5736ff3cdff09012@mail.gmail.com> Message-ID: Thanks Lee, I'm aware of this and it will be fixed in the next version. Regards, Dave On 4/27/06, Lee Marlow wrote: > I noticed that indexing seemed a little slow, so I tried reinstalling > the 0.9.1 gem and noticed some errors while it was building the native > extensions. Ferret still works with the pure ruby versions, but I'd > love to take advantage of the c speed. > > I'm not real familiar with C, so I was wondering if anyone know what > the errors are and if they can be fixed. > > Thanks > > -Lee > > ****** gem install output ******* > > $ sudo gem install ferret > Attempting local installation of 'ferret' > Local gem file not found: ferret*.gem > Attempting remote installation of 'ferret' > Building native extensions. This could take a while... > /usr/bin/ld: multiple definitions of symbol _ranalyzer_key > r_index_io.o definition of _ranalyzer_key in section (__DATA,__common) > r_qparser.o definition of _ranalyzer_key in section (__DATA,__common) > collect2: ld returned 1 exit status > make: *** [ferret_ext.bundle] Error 1 > /usr/bin/ld: multiple definitions of symbol _ranalyzer_key > r_index_io.o definition of _ranalyzer_key in section (__DATA,__common) > r_qparser.o definition of _ranalyzer_key in section (__DATA,__common) > collect2: ld returned 1 exit status > make: *** [ferret_ext.bundle] Error 1 > ruby extconf.rb install ferret > creating Makefile > > make > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c analysis.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c api.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c array.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c bitvector.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c compound_io.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c document.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c except.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c ferret.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c field.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c filter.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c fs_store.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c global.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c hash.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c hashset.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c helper.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c ind.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c index_io.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c index_rw.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c lang.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c libstemmer.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c priorityqueue.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_boolean.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_const_score.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_filtered_query.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_fuzzy.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_match_all.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_multi_phrase.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_parser.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_phrase.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_prefix.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_range.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_span.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_term.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c q_wildcard.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c r_analysis.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c r_doc.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c r_index_io.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c r_qparser.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c r_search.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c r_store.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c r_term.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c ram_store.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c search.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c similarity.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c sort.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_danish.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_dutch.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_english.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_finnish.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_french.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_german.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_italian.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_norwegian.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_porter.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_portuguese.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_spanish.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_ISO_8859_1_swedish.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_KOI8_R_russian.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_danish.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_dutch.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_english.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_finnish.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_french.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_german.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_italian.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_norwegian.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_porter.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_portuguese.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_russian.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_spanish.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stem_UTF_8_swedish.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c stopwords.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c store.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c term.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c termdocs.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c utilities.c > gcc -fno-common -O -pipe -I/opt/local/include -fno-common -pipe > -fno-common -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 > -I/opt/local/lib/ruby/1.8/i686-darwin8.5.2 -I. -O -pipe > -I/opt/local/include -c vector.c > cc -dynamic -bundle -undefined suppress -flat_namespace > -L/opt/local/lib -L"/opt/local/lib" -o ferret_ext.bundle analysis.o > api.o array.o bitvector.o compound_io.o document.o except.o ferret.o > field.o filter.o fs_store.o global.o hash.o hashset.o helper.o ind.o > index_io.o index_rw.o lang.o libstemmer.o priorityqueue.o q_boolean.o > q_const_score.o q_filtered_query.o q_fuzzy.o q_match_all.o > q_multi_phrase.o q_parser.o q_phrase.o q_prefix.o q_range.o q_span.o > q_term.o q_wildcard.o r_analysis.o r_doc.o r_index_io.o r_qparser.o > r_search.o r_store.o r_term.o ram_store.o search.o similarity.o sort.o > stem_ISO_8859_1_danish.o stem_ISO_8859_1_dutch.o > stem_ISO_8859_1_english.o stem_ISO_8859_1_finnish.o > stem_ISO_8859_1_french.o stem_ISO_8859_1_german.o > stem_ISO_8859_1_italian.o stem_ISO_8859_1_norwegian.o > stem_ISO_8859_1_porter.o stem_ISO_8859_1_portuguese.o > stem_ISO_8859_1_spanish.o stem_ISO_8859_1_swedish.o > stem_KOI8_R_russian.o stem_UTF_8_danish.o stem_UTF_8_dutch.o > stem_UTF_8_english.o stem_UTF_8_finnish.o stem_UTF_8_french.o > stem_UTF_8_german.o stem_UTF_8_italian.o stem_UTF_8_norwegian.o > stem_UTF_8_porter.o stem_UTF_8_portuguese.o stem_UTF_8_russian.o > stem_UTF_8_spanish.o stem_UTF_8_swedish.o stopwords.o store.o term.o > termdocs.o utilities.o vector.o -lruby -lpthread -ldl -lobjc > > make install > cc -dynamic -bundle -undefined suppress -flat_namespace > -L/opt/local/lib -L"/opt/local/lib" -o ferret_ext.bundle analysis.o > api.o array.o bitvector.o compound_io.o document.o except.o ferret.o > field.o filter.o fs_store.o global.o hash.o hashset.o helper.o ind.o > index_io.o index_rw.o lang.o libstemmer.o priorityqueue.o q_boolean.o > q_const_score.o q_filtered_query.o q_fuzzy.o q_match_all.o > q_multi_phrase.o q_parser.o q_phrase.o q_prefix.o q_range.o q_span.o > q_term.o q_wildcard.o r_analysis.o r_doc.o r_index_io.o r_qparser.o > r_search.o r_store.o r_term.o ram_store.o search.o similarity.o sort.o > stem_ISO_8859_1_danish.o stem_ISO_8859_1_dutch.o > stem_ISO_8859_1_english.o stem_ISO_8859_1_finnish.o > stem_ISO_8859_1_french.o stem_ISO_8859_1_german.o > stem_ISO_8859_1_italian.o stem_ISO_8859_1_norwegian.o > stem_ISO_8859_1_porter.o stem_ISO_8859_1_portuguese.o > stem_ISO_8859_1_spanish.o stem_ISO_8859_1_swedish.o > stem_KOI8_R_russian.o stem_UTF_8_danish.o stem_UTF_8_dutch.o > stem_UTF_8_english.o stem_UTF_8_finnish.o stem_UTF_8_french.o > stem_UTF_8_german.o stem_UTF_8_italian.o stem_UTF_8_norwegian.o > stem_UTF_8_porter.o stem_UTF_8_portuguese.o stem_UTF_8_russian.o > stem_UTF_8_spanish.o stem_UTF_8_swedish.o stopwords.o store.o term.o > termdocs.o utilities.o vector.o -lruby -lpthread -ldl -lobjc > Successfully installed ferret-0.9.1 > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From dbalmain.ml at gmail.com Thu Apr 27 20:55:21 2006 From: dbalmain.ml at gmail.com (David Balmain) Date: Fri, 28 Apr 2006 09:55:21 +0900 Subject: [Ferret-talk] Ferret crashing Ruby In-Reply-To: <1146137328.9225.13.camel@localhost.localdomain> References: <1146137328.9225.13.camel@localhost.localdomain> Message-ID: Hi Pedro, I believe this is fixed in my working copy. I'm going to get a release out as soon as possible. I'm currently trying to build on Windows. When that's done I'll put 0.9.2 out. Cheers, Dave On 4/27/06, Pedro C?rte-Real wrote: > I've gotten several problems with C ferret crashing ruby. Here's the > stack trace of my latest case: > > #0 0x00000050 in ?? () > #1 0xb74028ab in iw_close (iw=0x8685a40) at index_rw.c:947 > #2 0xb7414359 in index_destroy (self=0x829ebc0) at ind.c:89 > #3 0xb73f00bc in frt_ind_free (p=0x829ebc0) at r_search.c:1564 > #4 0xb7f04dca in rb_gc_call_finalizer_at_exit () > from /usr/lib/libruby1.8.so.1.8 > #5 0xb7ee627d in is_ruby_native_thread () > from /usr/lib/libruby1.8.so.1.8 > #6 0xb7efe2b5 in ruby_cleanup () from /usr/lib/libruby1.8.so.1.8 > #7 0xb7efe3cc in ruby_stop () from /usr/lib/libruby1.8.so.1.8 > #8 0xb7efeabf in ruby_run () from /usr/lib/libruby1.8.so.1.8 > #9 0x080486b1 in main () > > This was while doing several identical searches ordered by a field. > > The same code without the extensions throws this exception: > > RuntimeError: no terms in field start_date to sort by > /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_cache.rb:189:in `get_auto_index' > /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_sorted_hit_queue.rb:172:in `comparator_auto' > /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_sorted_hit_queue.rb:123:in `get_cached_comparator' > /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_sorted_hit_queue.rb:26:in `initialize' > /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/whiny_nil.rb:35:in `each_with_index' > /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/field_sorted_hit_queue.rb:25:in `initialize' > /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/search/index_searcher.rb:115:in `search' > /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/index/index.rb:660:in `do_search' > /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/index/index.rb:316:in `search' > /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize' > /usr/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/index/index.rb:315:in `search' > /home/pedrocr/eadexer/config/../app/models/description.rb:233:in > `find_by_text' > /home/pedrocr/eadexer/config/../app/controllers/descr_controller.rb:67:in `find' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/base.rb:910:in `perform_action_without_filters' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/filters.rb:368:in `perform_action_without_benchmark' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue' > /usr/lib/ruby/1.8/benchmark.rb:293:in `measure' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/rescue.rb:82:in `perform_action' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/base.rb:381:in `process_without_filters' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/filters.rb:377:in `process_without_session_management_support' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/session_management.rb:117:in `process_without_test' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/test_process.rb:16:in `process' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/test_process.rb:363:in `process' > /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/test_process.rb:336:in `get' > test/functional/descr_controller_test.rb:88:in `test_find_order_by' > test/functional/descr_controller_test.rb:87:in `test_find_order_by' > > With the C extension this search actually works fine. But if I surround > it with a 2.times do; ..search.. end; block it bombs. > > I'm using ferret 0.9.1. Should I be using SVN? Any ideas? > > Greetings, > > Pedro C?rte-Real. > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From shubhra.maini at gmail.com Fri Apr 28 03:20:24 2006 From: shubhra.maini at gmail.com (SM) Date: Fri, 28 Apr 2006 09:20:24 +0200 Subject: [Ferret-talk] *Bug* Message-ID: <4afcb86109bcf45a9a687f6592c037f6@ruby-forum.com> My application uses Ferret to search through the database records on the system. The problem is that there is some bug in my application which I have not been able to fix. What it actually does is that it returns the records on its own sweet will. I mean, it gives me results for some records and for some it doesn't. I have done things like dropped the database and created it again, creating index all over again and everything I can. But I don't know why isn't it working. If anybody could understand what I am tryin to convey and help. Thankyou in advance. -- Posted via http://www.ruby-forum.com/. From jan.prill at gmail.com Fri Apr 28 03:56:18 2006 From: jan.prill at gmail.com (Jan Prill) Date: Fri, 28 Apr 2006 09:56:18 +0200 Subject: [Ferret-talk] *Bug* In-Reply-To: <4afcb86109bcf45a9a687f6592c037f6@ruby-forum.com> References: <4afcb86109bcf45a9a687f6592c037f6@ruby-forum.com> Message-ID: <562a35c10604280056l7e4ad47q3ae28af8a8e9d8e7@mail.gmail.com> Hi, Shubra, I'm quite sure the information you gave is not sufficient to give qualified answers. It lacks information such as: Which version of ferret are you using? Are you using rails or any other webapp framework and more important an object relational mapper to access your database, and if so which ones and which versions? Which database are you using? If you are using rails, are you using acts_as_ferret or any other technique to integrate ferret? Could you provide code snippets on the code that actually isn't working? Regards Jan Prill On 4/28/06, SM wrote: > > > My application uses Ferret to search through the database records on the > system. > The problem is that there is some bug in my application which I have not > been able to fix. What it actually does is that it returns the records > on its own sweet will. > I mean, it gives me results for some records and for some it doesn't. > I have done things like dropped the database and created it again, > creating index all over again and everything I can. > But I don't know why isn't it working. > If anybody could understand what I am tryin to convey and help. > Thankyou in advance. > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20060428/31a0097e/attachment.htm From onurturgay at labristeknoloji.com Fri Apr 28 04:22:45 2006 From: onurturgay at labristeknoloji.com (Onur Turgay) Date: Fri, 28 Apr 2006 11:22:45 +0300 Subject: [Ferret-talk] Migrating to 0.9.1 In-Reply-To: References: <0af228cb1ab449e88a50228d909183eb@ruby-forum.com> Message-ID: hi david, will 0.9.2 contain defining our own analyzers, filters and tokenizers? David Balmain wrote: > You'll have to wait for 0.9.2. Sorry for the trouble. There are still > a few things missing from the C version of Ferret. > > Regards, > Dave > > On 4/25/06, Jean-Etienne wrote: >> After migrating to 0.9.1, I got: >> >> usr/local/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:123:in >> `const_missing': uninitialized constant TokenFilter (NameError) >> >> Here is a snapshot of my code: >> ... >> require 'ferret' >> >> class MyFilter < Analysis::TokenFilter >> ... >> >> I works fine on my dev machine, but not a production server (shared >> host). >> Any idea? >> >> -- >> Posted via http://www.ruby-forum.com/. >> _______________________________________________ >> Ferret-talk mailing list >> Ferret-talk at rubyforge.org >> http://rubyforge.org/mailman/listinfo/ferret-talk >> From richard at livsey.org Fri Apr 28 18:33:40 2006 From: richard at livsey.org (Richard Livsey) Date: Sat, 29 Apr 2006 00:33:40 +0200 Subject: [Ferret-talk] Ferret failing to rebuild_index - occasionally unable to del Message-ID: I'm switching all my fulltext searching over to ferret using the acts_as_ferret plugin on windows (for development, will be deployed on linux). In my tests I have a setup method as follows to ensure the index is reset for each test: def setup Item.rebuild_index end This works fine for the initial few tests but then suddenly fails for the last few tests when it seems to be failing to delete a file. Loaded suite E:/rails/thatsprogress/test/unit/item_test Started ........EEEE Finished in 8.922 seconds. 1) Error: test_search_for_count(ItemTest): Errno::EACCES: Permission denied - E:/rails/thatsprogress/config/../index/test/Item/_j.cfs .... Full trace included later. It consistantly does it at this point, and looks like something is stopping _j.cfs from being deleted. Any ideas on what's going on and how to stop it? Thanks in advance. Full trace is as follows: Loaded suite E:/rails/thatsprogress/test/unit/item_test Started ........EEEE Finished in 8.922 seconds. 1) Error: test_search_for_count(ItemTest): Errno::EACCES: Permission denied - E:/rails/thatsprogress/config/../index/test/Item/_j.cfs D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:105:in `delete' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:105:in `refresh' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:104:in `each' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:123:in `each' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:123:in `each' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:104:in `refresh' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:101:in `synchronize' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:101:in `refresh' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:74:in `new' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:68:in `synchronize' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/store/fs_store.rb:68:in `new' D:/programming/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.1/lib/ferret/index/index.rb:133:in `initialize' E:/rails/thatsprogress/config/../vendor/plugins/acts_as_ferret/lib/acts_as_ferret.rb:223:in `new' E:/rails/thatsprogress/config/../vendor/plugins/acts_as_ferret/lib/acts_as_ferret.rb:223:in `rebuild_index' E:/rails/thatsprogress/test/unit/item_test.rb:15:in `setup_without_fixtures' E:/rails/thatsprogress/config/../vendor/rails/activerecord/lib/active_record/fixtures.rb:554:in `setup' E:/rails/thatsprogress/config/../vendor/rails/activerecord/lib/active_record/fixtures.rb:552:in `setup' -- R.Livsey http://livsey.org -- Posted via http://www.ruby-forum.com/. From carl at youngbloods.org Sat Apr 29 21:38:23 2006 From: carl at youngbloods.org (Carl Youngblood) Date: Sat, 29 Apr 2006 19:38:23 -0600 Subject: [Ferret-talk] Locale not set error when trying to use C version in a rails app Message-ID: I'm getting a locale not set error. Does anyone know how I should set my locale in my rails environment so that ferret knows what to do? Why isn't this a problem in the ruby version of ferret? Thanks, Carl