From robin at ohloh.net Thu Oct 4 15:51:02 2007 From: robin at ohloh.net (Robin Luckey) Date: Thu, 4 Oct 2007 12:51:02 -0700 Subject: [Rake-devel] Rule with multiple rule-based prereqs executes only first prereq Message-ID: If a rule-generated task has multiple prerequisites, only the first prerequisite may refer to a rule-generated task, otherwise the prerequisite task is not executed. The rakefile below demonstrates the bug: Rakefile: namespace :example do rule(/\ba$/) rule(/\bb$/) rule(/\bc$/ => ['a','b']) end Try it out: $ rake example:c --dry-run (in /Users/robin/projects/ohloh) ** Invoke example:c (first_time) ** Invoke a (first_time) ** Execute (dry run) a ** Execute (dry run) example:c I expect that b would also execute. A correction patch follows. Thanks, Robin Luckey Index: lib/rake.rb =================================================================== --- lib/rake.rb (revision 609) +++ lib/rake.rb (working copy) @@ -1715,7 +1715,7 @@ prereqs = sources.collect { |source| if File.exist?(source) || Rake::Task.task_defined?(source) source - elsif parent = enhance_with_matching_rule(sources.first, level+1) + elsif parent = enhance_with_matching_rule(source, level+1) parent.name else return nil From robin at ohloh.net Thu Oct 4 17:04:13 2007 From: robin at ohloh.net (Robin Luckey) Date: Thu, 4 Oct 2007 14:04:13 -0700 Subject: [Rake-devel] Feature request: expand all matching rules instead of first match only Message-ID: <52A52B63-198B-47EC-A272-9F32602BC935@ohloh.net> One of the nice things about rake is the ability to add additional actions or prerequisites to a task via multiple calls to "task". However, this cannot be done for rule-generated tasks, because the rule matching algorithm stops after the first match. It would be very nice if instead all matching rules were expanded into tasks. This would be helpful to our project because we would like to create a plug-in architecture that allows core rake tasks to be extended by a plug-in, even if those core tasks are created via rules. For example, we'd like this to be possible: namespace :example do task :foo rule(/\ba$/ => 'example:foo') task :bar rule(/\ba$/ => 'example:bar') end Executing 'a' should then execute 'foo' and 'bar'. It's a simple code change, but I'm not sure how this change might impact other users: Index: lib/rake.rb =================================================================== --- lib/rake.rb (revision 609) +++ lib/rake.rb (working copy) @@ -1626,13 +1626,13 @@ def enhance_with_matching_rule(task_name, level=0) fail Rake::RuleRecursionOverflowError, "Rule Recursion Too Deep" if level >= 16 + task = nil @rules.each do |pattern, extensions, block| if md = pattern.match(task_name) task = attempt_rule(task_name, extensions, block, level) - return task if task end end - nil + task rescue Rake::RuleRecursionOverflowError => ex ex.add_target(task_name) fail ex From bradphelan at xtargets.com Fri Oct 5 06:22:39 2007 From: bradphelan at xtargets.com (Brad Phelan) Date: Fri, 05 Oct 2007 12:22:39 +0200 Subject: [Rake-devel] Heirarchical Builds Message-ID: <4706106F.8000404@xtargets.com> Hi Rakers, Am new here to the list. Am joining because of a special requirement. I have been hacking the Rake source code without too much success to get the following requirements to work. ====================== (Req 1) When a 'rakefile' is executed it should be processed under the condition that the current directory is set to the directory that the rakefile is found in. This makes rakefiles agnostic as to where they are placed into the project heirarchy and can refer to local files using local paths. sub/rakefile.rb file "foo.o" => "foo.c" do |t| sh "#{gcc -c #{t.source} -o #{t}" end (Req 2) When executing build actions they should all execute from the project root directory. ie foo.c will be compiled in a shell with the following command. gcc -c sub/foo.c -o sub/foo.o ===================== As a first step I can create a simple wrapper called rake def rake file Dir.chdir File.dirname(file) do load File.basename(file) end end to change the current directory when executing sub rakes. This doesn't work because now the tasks/prerequisites are added to the build database without the full path to define them. Ie foo.o and foo.c are stored without the "sub" prefix. I can't for the life of me work out the exact place to patch this feature in to see if I can get it to work. One possible idea I have had is that we can extend the namespace concept to file based tasks. If the directory the task was defined in is automatically mapped to a namespace then perhaps the problem gets solved, though I haven't had a look at what it would require to do this. I'm also working on trying to add construction environments to Rake ala SCons so you can do something like Env.open do |env| env[:CCFLAGS] = "-DFoo" env.file 'main.out' => [ 'main.o' ] end Env.open do |env| env[:CCFLAGS] = "-DBar" env.file 'main2.out' => [ 'main2.o' ] end task :default => ['main.out', 'main2.out'] rule( '.o' => '.c' ) do |t| path = t[:CCPATH].collect { |p| " -I#{p}" } sh "cc #{t[:CCFLAGS]} #{path} #{t.source} -c -o #{t.name}" end rule '.out' => '.o' do |t| sh "cc -o #{t[:CCFLAGS]} #{t.name} #{t.prerequisites.join(' ')}" end And each file will be built with different flags. However I want to solve the hierarchical build problem first. Any pointers or suggestions would be appreciated. I would like to be able to convert over from using SCons in large hierarchical C/C++ projects but currently Rake lacks the above features. Regards Brad Phelan http://xtargets.com From zimbatm at oree.ch Fri Oct 5 07:58:49 2007 From: zimbatm at oree.ch (Jonas Pfenniger) Date: Fri, 5 Oct 2007 13:58:49 +0200 Subject: [Rake-devel] Heirarchical Builds In-Reply-To: <4706106F.8000404@xtargets.com> References: <4706106F.8000404@xtargets.com> Message-ID: Hi Brad, Welcome to Rake :) This is only a pointer, but what I would do is add some "path" and "env" variables to Task, which are set on Task#invoke. You could also use a mechanism like #desc to collect them. --- Cheers, zimbatm From bradphelan at xtargets.com Fri Oct 5 08:51:57 2007 From: bradphelan at xtargets.com (Brad Phelan) Date: Fri, 05 Oct 2007 14:51:57 +0200 Subject: [Rake-devel] Heirarchical Builds In-Reply-To: <4706106F.8000404@xtargets.com> References: <4706106F.8000404@xtargets.com> Message-ID: <4706336D.2060309@xtargets.com> Brad Phelan wrote: > Hi Rakers, > > Am new here to the list. Am joining because of a special requirement. I > have been hacking the Rake source code without too much success to get > the following requirements to work. > > It is always good to try and describe what you want to others. The comment I made about scoping path names for FileTask gave me a few ideas. I implemented the path as a named scope for FileTask and it seems to work ok. I've included a zip file with the directory. Unzip it and just do rake or rake clean and you will see the hierarchical build working. Also included is a half attempt at construction environments which doesn't quite yet work. My main rakefile is require 'cscanner' Rake::Task.env do |env| env[:CCFLAGS] = "-DFOO=2" file 'main.out' => [ 'main.o' ] end Rake::Task.env do |env| env[:CCFLAGS] = "-DBAR=5" file 'main2.out' => [ 'main2.o' ] end task :default => ['main.out', 'main2.out', 'sub/main3.out' ] task :clean do sh "rm *.o*" sh "rm sub/*.o*" end rake 'sub/sub' and the child rake file (sub/sub.rake) in the directory sub is require 'cscanner' Rake::Task.env do |env| env[:CCFLAGS] = "-DBAR=1 " env[:CCPATH] = ["."] file 'main3.out' end The output from the build is (in /home/phelan/ruby/rake) cc -DFOO=2 main.c -c -o main.o cc -o main.out main.o cc -DBAR=5 main2.c -c -o main2.o cc -o main2.out main2.o cc -DBAR=1 -I. sub/main3.c -c -o sub/main3.o cc -DBAR=1 -o sub/main3.out sub/main3.o As you can see the CCFLAGS are not propagated to all rules. Not sure why?????? The Rake::Task.env block just opens a new construction environment where you can set build specific flags. It is an orthogonal concept to the heirarchical build. Regards Brad http://xtargets.com From bradphelan at xtargets.com Thu Oct 11 04:18:58 2007 From: bradphelan at xtargets.com (Brad Phelan) Date: Thu, 11 Oct 2007 10:18:58 +0200 Subject: [Rake-devel] Weird Weird errors running rake tests Message-ID: <470DDC72.4090904@xtargets.com> All, I am making a patch to the rake source and running the tests to verify that all is ok. Without my changes the tests pass. With my changes I get a weird error. 42) Error: test_name_lookup_with_implicit_file_tasks(TestTaskManager): NoMethodError: undefined method `+@' for "/home/phelan/ruby/rake/trunk/trunk/test/data":String ./lib/rake.rb:682:in `rel_to_root' Obviously there is no method "+@" in my code. Has anybody seen this before? The offending line of code is r = File.dirname(r) looks harmless to me. I've also replaced it with r = r and I get the same result. The patch is below and if anybody can replicate this problem and understand it it would be great. No need to try and debug my more general code as that is probably not correct yet anyway. That is why I am trying to run the tests. Regards Brad $ svn diff -x -w lib/rake.rb Index: lib/rake.rb =================================================================== --- lib/rake.rb (revision 609) +++ lib/rake.rb (working copy) @@ -667,12 +667,40 @@ # ---------------------------------------------------------------- # Task class methods. # + # Task class methods. + # class << self # Apply the scope to the task name according to the rules for this kind # of task. File based tasks ignore the scope when creating the name. + def rel_to_root(*paths) + r = Application.root.to_s + paths.collect do |p| + p = p.to_s + n = 0 + while p.index(r) != 0 + n++ + r = File.dirname(r) + end + prefix = [".."]*n + prefix = File.join(*prefix) + File.join(prefix, p[r.length+1..-1]) + end + end def scope_name(scope, task_name) - task_name + + dir = File.expand_path File.join(Dir.pwd, task_name) + + # puts Application.root + # puts Dir.pwd + # puts task_name + dir = rel_to_root( dir )[0] + # puts dir + # puts "===" + return dir + end + + end end # class Rake::FileTask @@ -1756,6 +1784,10 @@ class Application include TaskManager + def self.root + @@root + end + # The name of the application (typically 'rake') attr_reader :name @@ -2108,6 +2140,9 @@ end here = Dir.pwd end + + @@root = here + puts "(in #{Dir.pwd})" unless options.silent $rakefile = @rakefile load File.expand_path(@rakefile) if @rakefile != '' From bradphelan at xtargets.com Thu Oct 11 04:40:51 2007 From: bradphelan at xtargets.com (Brad Phelan) Date: Thu, 11 Oct 2007 10:40:51 +0200 Subject: [Rake-devel] Weird Weird errors running rake tests In-Reply-To: <470DDC72.4090904@xtargets.com> References: <470DDC72.4090904@xtargets.com> Message-ID: <470DE193.9030103@xtargets.com> Brad Phelan wrote: > All, > > I am making a patch to the rake source and running the tests to verify > that all is ok. Without my changes the tests pass. With my changes I get > a weird error. > > 42) Error: > test_name_lookup_with_implicit_file_tasks(TestTaskManager): > NoMethodError: undefined method `+@' for > "/home/phelan/ruby/rake/trunk/trunk/test/data":String > ./lib/rake.rb:682:in `rel_to_root' > > Obviously there is no method "+@" in my code. Has anybody seen this > before? The offending line of code > is > > r = File.dirname(r) > > looks harmless to me. I've also replaced it with > > r = r > > and I get the same result. > > > The patch is below and if anybody can replicate this problem and > understand it it would be great. No need to try and debug my more > general code as that is probably not correct yet anyway. That is why I > am trying to run the tests. > > Regards > > Brad > > $ svn diff -x -w lib/rake.rb > Index: lib/rake.rb > =================================================================== > --- lib/rake.rb (revision 609) > +++ lib/rake.rb (working copy) > @@ -667,12 +667,40 @@ > # ---------------------------------------------------------------- > # Task class methods. > # > + # Task class methods. > + # > class << self > # Apply the scope to the task name according to the rules for > this kind > # of task. File based tasks ignore the scope when creating the name. > + def rel_to_root(*paths) > + r = Application.root.to_s > + paths.collect do |p| > + p = p.to_s > + n = 0 > + while p.index(r) != 0 > + n++ > Sorry. My fault. Been programming too much C. n++ is not valid Ruby and gets interpreted in a strange way. Changed the line to n+= and all is well -- Brad http://xtargets.com From zimbatm at oree.ch Thu Oct 11 07:31:19 2007 From: zimbatm at oree.ch (Jonas Pfenniger) Date: Thu, 11 Oct 2007 13:31:19 +0200 Subject: [Rake-devel] Weird Weird errors running rake tests In-Reply-To: <470DE193.9030103@xtargets.com> References: <470DDC72.4090904@xtargets.com> <470DE193.9030103@xtargets.com> Message-ID: > n++ Yes, +@ is the infix operator. In your case, it was evaluated as n + (+(r = File.dirname(r))) And String doesn't have that operator -- Cheers, zimbatm