From nobody at rubyforge.org Mon Oct 1 04:24:27 2007 From: nobody at rubyforge.org (nobody at rubyforge.org) Date: Mon, 1 Oct 2007 04:24:27 -0400 (EDT) Subject: [Archipelago-submits] [327] trunk/archipelago/lib/archipelago/current.rb: fixed a bug in the mon_release override Message-ID: <20071001082427.AAA425240F01@rubyforge.org> Revision: 327 Author: zond Date: 2007-10-01 04:24:25 -0400 (Mon, 01 Oct 2007) Log Message: ----------- fixed a bug in the mon_release override Modified Paths: -------------- trunk/archipelago/lib/archipelago/current.rb Modified: trunk/archipelago/lib/archipelago/current.rb =================================================================== --- trunk/archipelago/lib/archipelago/current.rb 2007-09-30 18:58:52 UTC (rev 326) +++ trunk/archipelago/lib/archipelago/current.rb 2007-10-01 08:24:25 UTC (rev 327) @@ -32,8 +32,13 @@ module MonitorMixin def mon_release @mon_owner = nil - t = @mon_waiting_queue.shift - t = @mon_entering_queue.shift unless t + t = nil + until @mon_waiting_queue.empty? || (t && t.alive?) + t = @mon_waiting_queue.shift + end + until @mon_entering_queue.empty? || (t && t.alive?) + t = @mon_entering_queue.shift + end t.wakeup if t && t.alive? end end From nobody at rubyforge.org Mon Oct 1 15:00:49 2007 From: nobody at rubyforge.org (nobody at rubyforge.org) Date: Mon, 1 Oct 2007 15:00:49 -0400 (EDT) Subject: [Archipelago-submits] [328] trunk/archipelago/lib/archipelago/treasure.rb: made the dubloon class slightly simpler Message-ID: <20071001190049.D019D5240B56@rubyforge.org> Revision: 328 Author: zond Date: 2007-10-01 15:00:49 -0400 (Mon, 01 Oct 2007) Log Message: ----------- made the dubloon class slightly simpler Modified Paths: -------------- trunk/archipelago/lib/archipelago/treasure.rb Modified: trunk/archipelago/lib/archipelago/treasure.rb =================================================================== --- trunk/archipelago/lib/archipelago/treasure.rb 2007-10-01 08:24:25 UTC (rev 327) +++ trunk/archipelago/lib/archipelago/treasure.rb 2007-10-01 19:00:49 UTC (rev 328) @@ -167,14 +167,13 @@ end # # Initialize us with knowledge of our +chest+, the +key+ to our - # target in the +chest+, the known +public_methods+ of our target - # and any +transaction+ we are associated with. + # target in the +chest+ and any +transaction+ we are associated with. # - def initialize(key, chest, transaction, chest_id) + def initialize(key, chest, transaction) @key = key @chest = chest @transaction = transaction - @chest_id = chest_id + @chest_id = chest.service_id end # # A more or less normal dump of all our instance variables. @@ -205,7 +204,7 @@ # def join(transaction) @chest.join!(transaction) if transaction - return Dubloon.new(@key, @chest, transaction, @chest_id) + return Dubloon.new(@key, @chest, transaction) end # # Raises exception if the given +transaction+ @@ -462,7 +461,7 @@ if Dubloon.dubloon?(instance) return instance.join(transaction) else - return Dubloon.new(key, self, transaction, self.service_id) + return Dubloon.new(key, self, transaction) end end @@ -813,7 +812,7 @@ instance.with_transaction(transaction) do return_value = instance.send(method, *arguments, &block) end - return_value = Dubloon.new(key, self, transaction, service_id) if return_value == instance + return_value = Dubloon.new(key, self, transaction) if return_value == instance return return_value else return instance.send(method, *arguments, &block) @@ -840,7 +839,7 @@ begin return_value = instance.send(method, *arguments, &block) - return_value = Dubloon.new(key, self, nil, service_id) if return_value == instance + return_value = Dubloon.new(key, self, nil) if return_value == instance return return_value ensure @db.store_if_changed(key) @@ -918,7 +917,7 @@ # # Return the value if the value is in fact a proxy to what we are trying overwrite. # - return value if value.object_id == Dubloon.new(key, self, transaction, self.service_id).object_id + return value if value.object_id == Dubloon.new(key, self, transaction).object_id end if transaction @@ -945,7 +944,7 @@ return value if Dubloon.dubloon?(value) - return Dubloon.new(key, self, transaction, service_id) + return Dubloon.new(key, self, transaction) end # From nobody at rubyforge.org Mon Oct 1 15:17:24 2007 From: nobody at rubyforge.org (nobody at rubyforge.org) Date: Mon, 1 Oct 2007 15:17:24 -0400 (EDT) Subject: [Archipelago-submits] [329] trunk/archipelago/lib/archipelago/treasure.rb: more comment in treasure Message-ID: <20071001191724.16B165240B56@rubyforge.org> Revision: 329 Author: zond Date: 2007-10-01 15:17:23 -0400 (Mon, 01 Oct 2007) Log Message: ----------- more comment in treasure Modified Paths: -------------- trunk/archipelago/lib/archipelago/treasure.rb Modified: trunk/archipelago/lib/archipelago/treasure.rb =================================================================== --- trunk/archipelago/lib/archipelago/treasure.rb 2007-10-01 19:00:49 UTC (rev 328) +++ trunk/archipelago/lib/archipelago/treasure.rb 2007-10-01 19:17:23 UTC (rev 329) @@ -169,7 +169,10 @@ # Initialize us with knowledge of our +chest+, the +key+ to our # target in the +chest+ and any +transaction+ we are associated with. # - def initialize(key, chest, transaction) + # Optionally also the value we are pointing to, in order to enlighten + # it to its glorious status as Dubloon-proxied. + # + def initialize(key, chest, transaction, value = nil) @key = key @chest = chest @transaction = transaction From nobody at rubyforge.org Fri Oct 5 04:43:08 2007 From: nobody at rubyforge.org (nobody at rubyforge.org) Date: Fri, 5 Oct 2007 04:43:08 -0400 (EDT) Subject: [Archipelago-submits] [330] trunk/archipelago: made the DumpHashish slightly safer. Message-ID: <20071005084308.A7FF75240991@rubyforge.org> Revision: 330 Author: zond Date: 2007-10-05 04:43:07 -0400 (Fri, 05 Oct 2007) Log Message: ----------- made the DumpHashish slightly safer. added to the TODO. improved comments. Modified Paths: -------------- trunk/archipelago/TODO trunk/archipelago/lib/archipelago/hashish.rb trunk/archipelago/lib/archipelago/sanitation.rb Modified: trunk/archipelago/TODO =================================================================== --- trunk/archipelago/TODO 2007-10-01 19:17:23 UTC (rev 329) +++ trunk/archipelago/TODO 2007-10-05 08:43:07 UTC (rev 330) @@ -6,6 +6,9 @@ method that decide whether they are to be considered dirty, clean or auto-select. * This is now done. But it is not yet used to provide intelligence for the transaction mechanism. How should it compare dirty state before and after? + * Perhaps we can just do a @db.store_if_changed instead of @db[]= on commit? + That would only store objects that didnt look the exact same in the db or + had a dirty flag set. * Create a memcached-starter that publishes the address to the started memcached instance on the Disco network. @@ -21,3 +24,8 @@ * If the first chest is disconnected and then reappears. * If a new chest that takes responsibility for the Dubloon appears. + * Handle disconnects and connects on a bigger scale, ie net splits. As it stands + now the two nets would after the split (if a successful sync occurs) retain + complete copies of all data. After the join, it would always be the copy in the + 'other' network that became the real copy, since it would never be overwritten + by the original copy, but would try to overwrite it in case of redistribution. \ No newline at end of file Modified: trunk/archipelago/lib/archipelago/hashish.rb =================================================================== --- trunk/archipelago/lib/archipelago/hashish.rb 2007-10-01 19:17:23 UTC (rev 329) +++ trunk/archipelago/lib/archipelago/hashish.rb 2007-10-05 08:43:07 UTC (rev 330) @@ -293,6 +293,9 @@ # # An Archipelago::Dump network backed CachedHashish. # + # Compares hashes of the data to know if it has been changed since it was last read or written. + # Will assume it was changed if there is no memory of any hash. + # class DumpHashish include Archipelago::Hashish::CachedHashish # @@ -305,7 +308,9 @@ @hash_by_key = {} end def changed?(serialized_key, serialized_value) - return (old_hash = @hash_by_key[serialized_key]) && old_hash != Digest::SHA1.hexdigest(serialized_value) + old_hash = @hash_by_key[serialized_key] + return true unless old_hash + return old_hash != Digest::SHA1.hexdigest(serialized_value) end def do_get_timestamp_from_db(serialized_key) return @officer[officer_key(serialized_key, "timestamp")] Modified: trunk/archipelago/lib/archipelago/sanitation.rb =================================================================== --- trunk/archipelago/lib/archipelago/sanitation.rb 2007-10-01 19:17:23 UTC (rev 329) +++ trunk/archipelago/lib/archipelago/sanitation.rb 2007-10-05 08:43:07 UTC (rev 330) @@ -104,7 +104,7 @@ # # Write +key+ and +value+ into the site network with a good level of redundancy etc. # - # The key should must be a SHA1 hash. + # The key must be a SHA1 hash. # # Optionally the timestamp +t+ can be provided, but it defaults to now. # From nobody at rubyforge.org Sat Oct 27 07:48:19 2007 From: nobody at rubyforge.org (nobody at rubyforge.org) Date: Sat, 27 Oct 2007 07:48:19 -0400 (EDT) Subject: [Archipelago-submits] [331] trunk/archipelago/README: updated readme Message-ID: <20071027114819.EBCB918585CE@rubyforge.org> Revision: 331 Author: zond Date: 2007-10-27 07:48:19 -0400 (Sat, 27 Oct 2007) Log Message: ----------- updated readme Modified Paths: -------------- trunk/archipelago/README Modified: trunk/archipelago/README =================================================================== --- trunk/archipelago/README 2007-10-05 08:43:07 UTC (rev 330) +++ trunk/archipelago/README 2007-10-27 11:48:19 UTC (rev 331) @@ -5,14 +5,17 @@ == Dependencies: Archipelago::Hashish::BerkeleyHashishProvider:: ruby bdb: http://moulon.inra.fr/ruby/bdb.html Archipelago::Client::Base:: archipelago_rbtree, a patched and gemified version of the original at http://www.geocities.co.jp/SiliconValley-PaloAlto/3388/rbtree/README.html +Archipelago::Sanitation::Officer:: oneliner, an implementation of Online Codes: http://en.wikipedia.org/wiki/Online_codes, a rateless erasure code algorithm, used to achieve cheap generic redundance. == Sub packages: -Archipelago::Disco:: A UDP multicast discovery service useful to find services in your network with a minimum of configuration. -Archipelago::Tranny:: A distributed transaction manager inspired by the jini service mahalo. -Archipelago::Current:: A tiny concurrency toolkit used in various parts of Archipelago -Archipelago::Hashish:: A hash-like tool that provides transparent persistence. +Archipelago::Disco:: A UDP multicast discovery service inspired by the jini discovery service, useful for finding services in your network with a minimum of configuration. +Archipelago::Tranny:: A distributed transaction manager inspired by the jini service mahalo, useful for enabling transaction behaviour in other services. +Archipelago::Current:: A tiny concurrency toolkit used in various parts of Archipelago. +Archipelago::Hashish:: A hash-like tool that provides transparent persistence either on local disk or on distributed network db, used by Archipelago::Treasure. Archipelago::Treasure:: A distributed object database where the objects never leave the database, instead you do your operations upon references to the objects. It has support for serializably isolated transactions with optimistic concurrency control using Archipelago::Tranny or any transaction manager with similar semantics. -Archipelago::Pirate:: A client tool to allocate Archipelago::Treasure::Chests for different keys and act like an almost normal local Hash for providing distributed and scaleable object database facilities to any ruby application. +Archipelago::Pirate:: A client tool to allocate Archipelago::Treasure::Chests for different keys and act like an almost normal local Hash for providing distributed, concurrent and scaleable object database facilities for any ruby application. +Archipelago::Dump:: A distributed persistence layer for Archipelago::Hashish, used by Archipelago::Treasure to achieve generic redundance and automatic failover for the stored objects. +Archipelago::Sanitation:: A client tool to allocate Archipelago::Dump::Sites for different keys and act like an almost normal local Hash for providing distributed and redundant data storage to any ruby application. == Usage: