[Ruburple-development] [zond at troja.ath.cx: Re: lockups w/ ruburple?]

Mon Jun 25 07:24:13 EDT 2007

----- Forwarded message from Martin Kihlgren <zond at troja.ath.cx> -----

From: Martin Kihlgren <zond at troja.ath.cx>
To: Sanford Barr <sanford.barr at gmail.com>
Subject: Re: lockups w/ ruburple?

On Wed, May 23, 2007 at 12:28:16PM -0700, Sanford Barr wrote:
> Hi Martin,

Hey again Sanford!

> Here are my setup details:
> 
> libpurple 2.0.0 from the source distribution
> ruburple 0.0.8
> cpu is an intel dual core (thinkpad T60 laptop)
> Linux (ubuntu Fiesty)

Well, there shouldnt be any problem there...

> re: threads
> 
> Ok, how does the glib event queue get serviced?  I'm assuming it's
> from a different thread that then has to call back to ruby?
> 
> Are you single threading the callbacks to ruby or locking the
> structures on update?  (I'm wondering if glib can issue multiple
> events to ruby).  My apologies for not being familiar with glib.

Since libpurple is not threadsafe, I have to tell the main loop thread
in glib (which is started on Ruburple::init) to call functions in
libpurple. This is done by registering idle callbacks in the glib main
loop, with provided user data being a struct containing the function
to call and the parameters to it. Usually a mutex+conditional variable
is also used to wait until the glib main loop has actually executed
the provided function, thus allowing us to get return values from
libpurple as well.

And since ruby is not even remotely thread safe I cant let the
libpurple event hooks call directly into ruby code either, instead
they write other structs to a pipe that the a ruby thread (green, its
still the same kernel thread) selects, and then does pretty much the
same thing that the libpurple functions do in the other direction.

Its all far too complicated :/ 

(But less complicated than implementing the protocols of libpurple
myself, I hope :)

Anyway, possibly the mutex+conditional variable business could lock
your ruby thread forever, if the libpurple function it tries to call
doesnt return. That has never happened to me, however.

But if you want to to test for it, just check out and build the
ruburple sources from rubyforge and add some
printf("my debug statement\n");
fflush(NULL);
where you want to add traces.

regards,
//Martin Kihlgren

> On 5/20/07, Martin Kihlgren <zond at troja.ath.cx> wrote:
> >
> >Hm, here are my experiences with your test script:
> >
> >1) It never locked up the process. I wonder what the difference can
> >be? What version of libpurple are you running? What version of the
> >ruburple gem? What CPU? What OS? etc etc
> >
> >2) After I entered a couple accounts in it, it started sending
> >messages like crazy. After a while (30-60s) it stopped, but a ctrl-c
> >stopped it like it should.
> >
> >3) After that I had to wait a couple minutes for it to start sending
> >again, as if the services had decided that I had capped my ratio or
> >something.
> >
> >Are you perhaps trapping signals in your script? That can also make
> >the ruby interpeter not react to ctrl-c or basically any signal other
> >than 9. (I assume you meant below that it doesnt respond to SIGTERM,
> >rather than doesnt respond to SIGKILL, since -9 is SIGKILL :)
> >
> >About the library I can tell you that the ruby thread never calls a
> >function inside the library, everything goes through glib event
> >handlers added to the main glib event loop, so the ruby thread should
> >not (famous last words, I know :) be able to lock up due to errors in
> >the libpurple thread.
> >
> >I recommend that you try this with different versions, on different
> >machines. I ran my test on a Mac OSX with ruburple 0.0.8 and (I think,
> >I have experimented a bit with libpurple) pidgin 2.0.0.
> >
> >Another solution could be to run it through gdb, to see where it stops
> >and why. Look at
> >http://eigenclass.org/hiki.rb?ruby+live+process+introspection for some
> >hints on that :)
> >
> >//Martin
> >
> >On Sun, May 20, 2007 at 06:02:45AM -0700, Sanford Barr wrote:
> >> Hi Martin,
> >>
> >> Thanks for the response!
> >>
> >> The lockup is a hard lockup on the process.  No longer responds to input 
> >and
> >> no longer responds to SIGINT/KILL.  I have to do a kill -9 to stop the
> >> process.
> >>
> >> So somewhere in either the binding or the library there's an issue.  I 
> >can
> >> reproduce it consistently using the test program I sent you.  After about
> >> 4-5 messages, ping-ponging between my gtalk window and my Yahoo IM window
> >> the Ruby chatbot locks up.
> >>
> >> Thoughts on how to proceed?
> >>
> >> -San
> >>
> >>
> >>
> >> On 5/20/07, Martin Kihlgren <zond at troja.ath.cx> wrote:
> >> >
> >> >
> >> >How does it lock up? Like in crash, or like in 'doesnt seem to do
> >> >anything anymore'?
> >> >
> >> >If it is the former, then it sounds really mysterious, but debuggable
> >> >(I hope).
> >> >
> >> >If it is the latter, then it could just be that Yahoo or Google (or
> >> >both?) got their spam filters trigged and refuses to send any more
> >> >messages or you.
> >> >
> >> >I have in a few occasions noticed that things that work fine when I
> >> >occasionally test them manually stop working when I run stuff in tests
> >> >and programs. Perhaps due to this?
> >> >
> >> >Any thoughts?
> >> >
> >> >//Martin
> >> >
> >> >On Sat, May 19, 2007 at 09:47:11PM -0700, Sanford Barr wrote:
> >> >> Hi Martin,
> >> >>
> >> >> I'm just starting to work with ruburple and I'm running into a lockup
> >> >when
> >> >> talking between a google and yahoo acount.
> >> >>
> >> >> I'm making a simple echobot that just broadcasts between to different
> >> >chat
> >> >> accounts (google, yahoo).
> >> >>
> >> >> All works well, but eventually the bot locks up when alternating 
> >sending
> >> >> messages from
> >> >> the yahoo and google accounts (works fine if you only write from one
> >> >> account, but fails when you start ping-ponging messages
> >> >> from both accounts one after another).
> >> >>
> >> >> Any thoughts?  I'm not entirely sure I'm using the lib correctly.   
> >I'm
> >> >> including a quick hack of a test below.
> >> >>
> >> >> Thanks,
> >> >>
> >> >> -San
> >> >>
> >> >>
> >> >> require 'rubygems'
> >> >> require_gem 'ruburple'
> >> >> Ruburple::init
> >> >>
> >> >> $yacct = nil
> >> >> $gacct = nil
> >> >>
> >> >> def send_hello
> >> >>  $yacct.connection.send_im("yahoo_acct", "hello world!")
> >> >>  $gacct.connection.send_im("google_acct", "hello world!")
> >> >> end
> >> >>
> >> >> def broadcast(msg)
> >> >> end
> >> >>
> >> >> def receive_msg(a,b,c,d,e)
> >> >>  puts a.username
> >> >>  puts "rcv im: a:#{a}, b:#{b}, c:#{c}, d:#{d}, e:#{e}"
> >> >>  z = c.dup
> >> >>  # zap the '<body>' tag from gtalk
> >> >>  if c.index("<body>") == 0
> >> >>    z = c[6..-8]
> >> >>  end
> >> >>  username = b
> >> >>  full_message = "<b>[" + username + "]</b> " + z
> >> >>  $yacct.connection.send_im("yahoo_acct", full_message)
> >> >>  $gacct.connection.send_im("google_acct", full_message)
> >> >> end
> >> >>
> >> >>
> >> >> puts "Supported Protocols"
> >> >> Ruburple::protocols.each do |x| puts x.name + ": " + x.description end
> >> >> Ruburple::subscribe(:signed_on, (Proc.new { |a| puts "signed on: #{a}"
> >> >}))
> >> >> Ruburple::subscribe(:received_im_msg, (Proc.new { |a,b,c,d,e|
> >> >> receive_msg(a,b,c,d,e) }))
> >> >> yprot = Ruburple::get_protocol("Yahoo")
> >> >> gprot = Ruburple::get_protocol("XMPP")
> >> >> $yacct = yprot.get_account("yahoo", "pwd")
> >> >> $gacct = gprot.get_account("google", "pwd")
> >> >> $yacct.connect
> >> >> $gacct.connect
> >> >>
> >> >> sleep 5
> >> >> send_hello
> >> >> while 1
> >> >> # loop forever, assuming that ruby idle loop is wired to process msgs
> >> >>  sleep 1
> >> >> end
> >> >
> >> >--
> >> >###################################################################
> >> >I read the newspaper avidly.  It is my one form of continuous fiction.
> >> >                -- Aneurin Bevan
> >> >###################################################################
> >> >
> >>
> >>
> >>
> >> --
> >> Sanford Barr | Chief Stirrer, The STIRR Network | http://www.stirr.net/
> >> sanford at stirr.net
> >> 650/906-7883 (m)
> >
> >--
> >###################################################################
> >I read the newspaper avidly.  It is my one form of continuous fiction.
> >                -- Aneurin Bevan
> >###################################################################
> >
> 
> 
> -- 
> Sanford Barr | Chief Stirrer, The STIRR Network | http://www.stirr.net/
> sanford at stirr.net
> 650/906-7883 (m)

-- 
###################################################################
Newton's Law of Gravitation:
	What goes up must come down.  But don't expect it to come down where
	you can find it.  Murphy's Law applies to Newton's.
###################################################################

----- End forwarded message -----

-- 
###################################################################
"Whom are you?" said he, for he had been to night school.
		-- George Ade
###################################################################