From dave.duchene at airg.com Wed Jul 19 16:25:19 2006 From: dave.duchene at airg.com (Dave Duchene) Date: Wed, 19 Jul 2006 13:25:19 -0700 Subject: [Rant] Parallel processing of tasks Message-ID: <44BE952F.6010302@airg.com> Hello, all. Stefan mentioned a while back (http://rubyforge.org/pipermail/make-cafe/2006-February/000071.html) that parallel processing of tasks may be in the works. Is this still going to happen? Is there anything I could do to help make it happen? thanks, --Dave From xavier-list at noreality.net Sun Jul 23 06:31:58 2006 From: xavier-list at noreality.net (Xavier Shay) Date: Sun, 23 Jul 2006 20:31:58 +1000 Subject: [Rant] Rant tutorial Message-ID: <44C3501E.5050602@noreality.net> I've just posted a Rant tutorial at http://rhnh.net/articles/building-firefox-extensions As you can tell from the URL, it introduces Rant by creating a Rantfile to build Firefox extensions. I think it (or something similar) would be a valuable addition to the Rant docs, as there is currently no full examples (the examples directory is empty?) Cheers, Xavier http://rhnh.net/ From langstefan at gmx.at Mon Jul 24 15:45:46 2006 From: langstefan at gmx.at (Stefan Lang) Date: Mon, 24 Jul 2006 21:45:46 +0200 Subject: [Rant] Rant tutorial In-Reply-To: <44C3501E.5050602@noreality.net> References: <44C3501E.5050602@noreality.net> Message-ID: <200607242145.46308.langstefan@gmx.at> On Sunday 23 July 2006 12:31, Xavier Shay wrote: > I've just posted a Rant tutorial at > http://rhnh.net/articles/building-firefox-extensions > > As you can tell from the URL, it introduces Rant by creating a > Rantfile to build Firefox extensions. I think it (or something > similar) would be a valuable addition to the Rant docs, as there is > currently no full examples (the examples directory is empty?) Thank you for your effort! The examples directory shouldn't be empty. I've just checked all three rant-0.5.6 packages on RubyForge and they all contain examples under doc/examples as they should. With regards to your archive_rootdir_fix: I can implement that functionality for the Package::* tasks, since they link/copy all the files in the package under a new directory and then archive the new directory. I think simply including the final Rantfile from your tutorial in the Rant package wouldn't make much sense since the context is missing. I'll add a link to the tutorial on the Rant homepage. Regards, Stefan From langstefan at gmx.at Mon Jul 24 16:20:47 2006 From: langstefan at gmx.at (Stefan Lang) Date: Mon, 24 Jul 2006 22:20:47 +0200 Subject: [Rant] Parallel processing of tasks In-Reply-To: <44BE952F.6010302@airg.com> References: <44BE952F.6010302@airg.com> Message-ID: <200607242220.48016.langstefan@gmx.at> On Wednesday 19 July 2006 22:25, Dave Duchene wrote: > Hello, all. > > Stefan mentioned a while back > (http://rubyforge.org/pipermail/make-cafe/2006-February/000071.html >) that parallel processing of tasks may be in the works. Is this > still going to happen? Is there anything I could do to help make > it happen? I can't see it happen in the near future (~2 months), unless someone else makes it happen. There are some problems to overcome: Tasks that run a Ruby block might do anything, define new tasks manually invoke other tasks, change the process working directory. Especially the last one is very problematic. If the working directory was per thread, parallel task processing would certainly be simpler to implement. I don't know what exactly you are looking for. Rake has a very primitive parellel processing capability: multitask "foo" => ["bar", "baz"] task "bar" do 5.times { puts "bar"; sleep(1) } end task "baz" do 5.times { puts "baz"; sleep(1) } end When the task "foo" is invoked, Rake simply starts a thread for each dependency to invoke. Thus the "bar" and "baz" tasks run in parallel. There is no synchronization done. If one task changes a resource (might be Dir.pwd) on which the other one relies, the build will fail in a hard to predict and hard to debug way. I *can* add a generator to Rant that behaves like Rake's multitask if it suffices your current needs. Of course you can help make it happen. The best help would be if you can come up with a clean and powerful enough design for a new Rant core. Regards, Stefan From dave.duchene at airg.com Mon Jul 24 16:27:05 2006 From: dave.duchene at airg.com (Dave Duchene) Date: Mon, 24 Jul 2006 13:27:05 -0700 Subject: [Rant] Parallel processing of tasks In-Reply-To: <200607242220.48016.langstefan@gmx.at> References: <44BE952F.6010302@airg.com> <200607242220.48016.langstefan@gmx.at> Message-ID: <44C52D19.7070106@airg.com> Hi Stephan, thanks very much for your response. In the end, I went with rake and a variation on it's multi-task (since I'm using it to build C++, I only use N-way parallelism where N is the number of cores on the build box). The problems you mention are really nasty to solve in a general way, but they're trivial if you just make it clear to the developer that they need to watch out for them when using the tool;-). take care, --Dave Stefan Lang wrote: > On Wednesday 19 July 2006 22:25, Dave Duchene wrote: > >> Hello, all. >> >> Stefan mentioned a while back >> (http://rubyforge.org/pipermail/make-cafe/2006-February/000071.html >> ) that parallel processing of tasks may be in the works. Is this >> still going to happen? Is there anything I could do to help make >> it happen? >> > > I can't see it happen in the near future (~2 months), unless someone > else makes it happen. > > There are some problems to overcome: > Tasks that run a Ruby block might do anything, define new tasks > manually invoke other tasks, change the process working directory. > Especially the last one is very problematic. If the working > directory was per thread, parallel task processing would certainly > be simpler to implement. > > I don't know what exactly you are looking for. > Rake has a very primitive parellel processing capability: > > multitask "foo" => ["bar", "baz"] > > task "bar" do > 5.times { puts "bar"; sleep(1) } > end > > task "baz" do > 5.times { puts "baz"; sleep(1) } > end > > When the task "foo" is invoked, Rake simply starts a thread > for each dependency to invoke. Thus the "bar" and "baz" tasks > run in parallel. There is no synchronization done. If one > task changes a resource (might be Dir.pwd) on which the other > one relies, the build will fail in a hard to predict and hard > to debug way. > > I *can* add a generator to Rant that behaves like Rake's multitask > if it suffices your current needs. > > Of course you can help make it happen. The best help would be > if you can come up with a clean and powerful enough design > for a new Rant core. > > Regards, > Stefan > > _______________________________________________ > make-cafe mailing list > make-cafe at rubyforge.org > http://rubyforge.org/mailman/listinfo/make-cafe > -- Dave Duchene, MSc. AirG Suite 706, 1155 Robson Street Vancouver, B.C. Canada V6E 1B5 T. +1.604.408.2228 Ext. 116 F. +1.866.874.8136 E. dave.duchene at airg.com W: www.airg.com The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material communicated under NDA. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/make-cafe/attachments/20060724/90612321/attachment-0001.html