From normalperson at yhbt.net  Thu Mar 10 18:29:05 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 10 Mar 2011 15:29:05 -0800
Subject: ATTN "sendfile" RubyGem users
Message-ID: <20110310232905.GA24873@dcvr.yhbt.net>

For anybody[1] using the sendfile 1.0.0 RubyGem with Rainbows!,
the next release of Rainbows! will require an upgrade to sendfile 1.1.0
if you want to continue using sendfile(2) without IO.copy_stream[2].

If you're currently running sendfile 1.0.0 with Rainbows 3.1.0, it's
entirely safe to start using sendfile 1.1.0 right away.

I just pushed this out to git://bogomips.org/rainbows.git :

>From cd8a874d18fe01e11bb57b91186b6c9f712a4b3f Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson at yhbt.net>
Date: Thu, 10 Mar 2011 15:06:10 -0800
Subject: [PATCH] switch from IO#sendfile_nonblock to IO#trysendfile

IO#trysendfile does not raise exceptions for common EAGAIN
errors, making it far less expensive to use with the following
concurrency models:

* Coolio
* CoolioFiberSpawn
* Revactor
* FiberSpawn
* FiberPool

This requires the new sendfile 1.1.0 RubyGem and removes support
for the sendfile 1.0.0.  All sendfile users must upgrade or be
left without sendfile(2) support.  IO#sendfile behaves the same
if you're using a multi-threaded concurrency option, but we
don't detect nor use it unless IO#trysendfile exists.
---
 TODO                                       |    2 +-
 lib/rainbows/coolio/client.rb              |   25 +++++++++++++------------
 lib/rainbows/epoll/client.rb               |   11 ++++++-----
 lib/rainbows/fiber/body.rb                 |   19 ++++++++++---------
 lib/rainbows/response.rb                   |   10 +++++-----
 lib/rainbows/revactor/client/methods.rb    |   17 +++++++++--------
 lib/rainbows/stream_file.rb                |    2 +-
 lib/rainbows/writer_thread_pool/client.rb  |    2 +-
 lib/rainbows/writer_thread_spawn/client.rb |    2 +-
 9 files changed, 47 insertions(+), 43 deletions(-)

diff --git a/TODO b/TODO
index a49e0c5..0929aa3 100644
--- a/TODO
+++ b/TODO
@@ -11,7 +11,7 @@ care about.
 * allow _OPTIONAL_ splice(2) with DevFdResponse under Linux
   (splice is very broken under some older kernels)
 
-* use IO#sendfile_nonblock for EventMachine/NeverBlock
+* use IO#trysendfile for EventMachine/NeverBlock
 
 * Open file cache Rack app/middleware (idea from nginx), since sendfile
   (and IO.copy_stream) allows pread(2)-style offsets
diff --git a/lib/rainbows/coolio/client.rb b/lib/rainbows/coolio/client.rb
index 9853321..2de421a 100644
--- a/lib/rainbows/coolio/client.rb
+++ b/lib/rainbows/coolio/client.rb
@@ -125,11 +125,8 @@ class Rainbows::Coolio::Client < Coolio::IO
     when true then return # #next! will clear this bit
     when nil # fall through
     else
-      begin
-        return stream_file_chunk(@deferred)
-      rescue EOFError # expected at file EOF
-        close_deferred # fall through
-      end
+      return if stream_file_chunk(@deferred)
+      close_deferred # EOF, fall through
     end
 
     case @state
@@ -179,7 +176,7 @@ class Rainbows::Coolio::Client < Coolio::IO
     KATO.delete(self)
   end
 
-  if IO.method_defined?(:sendfile_nonblock)
+  if IO.method_defined?(:trysendfile)
     def defer_file(status, headers, body, alive, io, st)
       if r = sendfile_range(status, headers)
         status, headers, range = r
@@ -192,11 +189,15 @@ class Rainbows::Coolio::Client < Coolio::IO
     end
 
     def stream_file_chunk(sf) # +sf+ is a Rainbows::StreamFile object
-      sf.offset += (n = @_io.sendfile_nonblock(sf, sf.offset, sf.count))
-      0 == (sf.count -= n) and raise EOFError
-      enable_write_watcher
-      rescue Errno::EAGAIN
-        enable_write_watcher
+      case n = @_io.trysendfile(sf, sf.offset, sf.count)
+      when Integer
+        sf.offset += n
+        return if 0 == (sf.count -= n)
+      when :wait_writable
+        return enable_write_watcher
+      else
+        return
+      end while true
     end
   else
     def defer_file(status, headers, body, alive, io, st)
@@ -205,7 +206,7 @@ class Rainbows::Coolio::Client < Coolio::IO
     end
 
     def stream_file_chunk(body)
-      write(body.to_io.sysread(0x4000))
+      buf = body.to_io.read(0x4000) and write(buf)
     end
   end
 
diff --git a/lib/rainbows/epoll/client.rb b/lib/rainbows/epoll/client.rb
index a243d5d..b7b0c9e 100644
--- a/lib/rainbows/epoll/client.rb
+++ b/lib/rainbows/epoll/client.rb
@@ -183,15 +183,16 @@ module Rainbows::Epoll::Client
 
   # returns +nil+ on EOF, :wait_writable if the client blocks
   def stream_file(sf) # +sf+ is a Rainbows::StreamFile object
-    begin
-      sf.offset += (n = sendfile_nonblock(sf, sf.offset, sf.count))
+    case n = trysendfile(sf, sf.offset, sf.count)
+    when Integer
+      sf.offset += n
       0 == (sf.count -= n) and return sf.close
-    rescue Errno::EAGAIN
-      return :wait_writable
+    else
+      return n # :wait_writable or nil
+    end while true
     rescue
       sf.close
       raise
-    end while true
   end
 
   def defer_file_stream(offset, count, io, body)
diff --git a/lib/rainbows/fiber/body.rb b/lib/rainbows/fiber/body.rb
index 872b1df..5b2c74b 100644
--- a/lib/rainbows/fiber/body.rb
+++ b/lib/rainbows/fiber/body.rb
@@ -5,19 +5,20 @@
 # this is meant to be included _after_ Rainbows::Response::Body
 module Rainbows::Fiber::Body # :nodoc:
 
-  # the sendfile 1.0.0+ gem includes IO#sendfile_nonblock
-  if IO.method_defined?(:sendfile_nonblock)
+  # the sendfile 1.1.0+ gem includes IO#trysendfile
+  if IO.method_defined?(:trysendfile)
     def write_body_file(body, range)
       sock, n, body = to_io, nil, body_to_io(body)
       offset, count = range ? range : [ 0, body.stat.size ]
-      begin
-        offset += (n = sock.sendfile_nonblock(body, offset, count))
-      rescue Errno::EAGAIN
+      case n = sock.trysendfile(body, offset, count)
+      when Integer
+        offset += n
+        return if 0 == (count -= n)
+      when :wait_writable
         kgio_wait_writable
-        retry
-      rescue EOFError
-        break
-      end while (count -= n) > 0
+      else # nil
+        return
+      end while true
       ensure
         close_if_private(body)
     end
diff --git a/lib/rainbows/response.rb b/lib/rainbows/response.rb
index 2c517f8..576ff8d 100644
--- a/lib/rainbows/response.rb
+++ b/lib/rainbows/response.rb
@@ -67,7 +67,7 @@ module Rainbows::Response
     end
 
     # generic response writer, used for most dynamically-generated responses
-    # and also when IO.copy_stream and/or IO#sendfile_nonblock is unavailable
+    # and also when IO.copy_stream and/or IO#trysendfile is unavailable
     def write_response(status, headers, body, alive)
       write_headers(status, headers, alive)
       write_body_each(body)
@@ -77,7 +77,7 @@ module Rainbows::Response
   end
   include Each
 
-  if IO.method_defined?(:sendfile_nonblock)
+  if IO.method_defined?(:trysendfile)
     module Sendfile
       def write_body_file(body, range)
         io = body_to_io(body)
@@ -90,7 +90,7 @@ module Rainbows::Response
   end
 
   if IO.respond_to?(:copy_stream)
-    unless IO.method_defined?(:sendfile_nonblock)
+    unless IO.method_defined?(:trysendfile)
       module CopyStream
         def write_body_file(body, range)
           range ? IO.copy_stream(body, self, range[1], range[0]) :
@@ -111,7 +111,7 @@ module Rainbows::Response
     alias write_body_stream write_body_each
   end  # ! IO.respond_to?(:copy_stream)
 
-  if IO.method_defined?(:sendfile_nonblock) || IO.respond_to?(:copy_stream)
+  if IO.method_defined?(:trysendfile) || IO.respond_to?(:copy_stream)
     HTTP_RANGE = 'HTTP_RANGE'
     Content_Range = 'Content-Range'.freeze
 
@@ -181,5 +181,5 @@ module Rainbows::Response
       end
     end
     include ToPath
-  end # IO.respond_to?(:copy_stream) || IO.method_defined?(:sendfile_nonblock)
+  end # IO.respond_to?(:copy_stream) || IO.method_defined?(:trysendfile)
 end
diff --git a/lib/rainbows/revactor/client/methods.rb b/lib/rainbows/revactor/client/methods.rb
index e9b39a3..b2e1847 100644
--- a/lib/rainbows/revactor/client/methods.rb
+++ b/lib/rainbows/revactor/client/methods.rb
@@ -1,7 +1,7 @@
 # -*- encoding: binary -*-
 # :enddoc:
 module Rainbows::Revactor::Client::Methods
-  if IO.method_defined?(:sendfile_nonblock)
+  if IO.method_defined?(:trysendfile)
     def write_body_file(body, range)
       body, client = body_to_io(body), @client
       sock = @client.instance_variable_get(:@_io)
@@ -9,9 +9,11 @@ module Rainbows::Revactor::Client::Methods
       write_complete = T[:"#{pfx}_write_complete", client]
       closed = T[:"#{pfx}_closed", client]
       offset, count = range ? range : [ 0, body.stat.size ]
-      begin
-        offset += (n = sock.sendfile_nonblock(body, offset, count))
-      rescue Errno::EAGAIN
+      case n = sock.trysendfile(body, offset, count)
+      when Integer
+        offset += n
+        return if 0 == (count -= n)
+      when :wait_writable
         # The @_write_buffer is empty at this point, trigger the
         # on_readable method which in turn triggers on_write_complete
         # even though nothing was written
@@ -21,10 +23,9 @@ module Rainbows::Revactor::Client::Methods
           filter.when(write_complete) {}
           filter.when(closed) { raise Errno::EPIPE }
         end
-        retry
-      rescue EOFError
-        break
-      end while (count -= n) > 0
+      else # nil
+        return
+      end while true
       ensure
         close_if_private(body)
     end
diff --git a/lib/rainbows/stream_file.rb b/lib/rainbows/stream_file.rb
index 11c84d4..4a77a2f 100644
--- a/lib/rainbows/stream_file.rb
+++ b/lib/rainbows/stream_file.rb
@@ -1,7 +1,7 @@
 # -*- encoding: binary -*-
 # :enddoc:
 
-# Used to keep track of file offsets in IO#sendfile_nonblock + evented
+# Used to keep track of file offsets in IO#trysendfile + evented
 # models.  We always maintain our own file offsets in userspace because
 # because sendfile() implementations offer pread()-like idempotency for
 # concurrency (multiple clients can read the same underlying file handle).
diff --git a/lib/rainbows/writer_thread_pool/client.rb b/lib/rainbows/writer_thread_pool/client.rb
index 526a623..f02826e 100644
--- a/lib/rainbows/writer_thread_pool/client.rb
+++ b/lib/rainbows/writer_thread_pool/client.rb
@@ -18,7 +18,7 @@ class Rainbows::WriterThreadPool::Client < Struct.new(:to_io, :q)
       }
     end
 
-    if IO.respond_to?(:copy_stream) || IO.method_defined?(:sendfile_nonblock)
+    if IO.respond_to?(:copy_stream) || IO.method_defined?(:trysendfile)
       def write_response(status, headers, body, alive)
         if body.respond_to?(:close)
           write_response_close(status, headers, body, alive)
diff --git a/lib/rainbows/writer_thread_spawn/client.rb b/lib/rainbows/writer_thread_spawn/client.rb
index b4166fa..3106253 100644
--- a/lib/rainbows/writer_thread_spawn/client.rb
+++ b/lib/rainbows/writer_thread_spawn/client.rb
@@ -21,7 +21,7 @@ class Rainbows::WriterThreadSpawn::Client < Struct.new(:to_io, :q, :thr)
       }
     end
 
-    if IO.respond_to?(:copy_stream) || IO.method_defined?(:sendfile_nonblock)
+    if IO.respond_to?(:copy_stream) || IO.method_defined?(:trysendfile)
       def write_response(status, headers, body, alive)
         self.q ||= queue_writer
         if body.respond_to?(:close)
-- 
Eric Wong

[1] - I'm still fairly certain nobody runs Rainbows! at all in
      production, so this message is addressed to imaginary users :D
[2] - Use Ruby 1.9.3dev for IO.copy_stream, 1.9.2 doesn't handle
      client disconnects properly

From fatimakotoka3 at att.net  Tue Mar 15 06:47:56 2011
From: fatimakotoka3 at att.net (Miss Fatima Kotoka)
Date: Tue, 15 Mar 2011 18:47:56 +0800
Subject: COMPLEMENTS OF THE DAY!!
Message-ID: <PC1FAD0CF51878d01ab6dafaa84cae8640109861cf831c@pc1fad0cf51878>

Hello Sir/Madam,

I am sorry for using this medium of communication in contacting you, but it is based on the urgent nature of this deal which I'm about to present to you for your consideration. I am Miss Fatima Kotoka, a Ghanaian national and I am 23 years old, the only daughter of Late General Emmanuel Kwasi Kotoka of Ghana Regiment, I am now in Isreal seeking for asylum, please I need your urgent assistance.

I am in position to claim the sum of $5.2M USD deposited by my father for onward transfer to his associate which I cannot reach at the moment. This money was intended for Hotel projects in that country. I am looking for a reliable person that will help me to secure this money as an associate of my father to enable us invest this money on any viable projects.

If you can represent my interest or if there is any other lucrative Business in your area of specialization to enable me us work together then I will not hesitate to detail you on your role in the transaction. As for your commission, I will let you know in the next mail.

Sincerely yours
Miss Fatima Kotoka 


From normalperson at yhbt.net  Tue Mar 15 08:48:00 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 15 Mar 2011 12:48:00 +0000
Subject: [ANN] Rainbows! 3.2.0 - trying to send files to slow clients
Message-ID: <20110315124800.GA12807@dcvr.yhbt.net>

Changes:

We now use IO#trysendfile in the sendfile 1.1.0 to reduce the
cost of generating backtraces for slow clients (from EAGAIN).
Nothing new for people not serving static files (but more
on the way).

Existing "sendfile" gem users must upgrade to 1.1.0
or risk being left without sendfile support at all:
 http://bogomips.org/rainbows.git/patch?id=cd8a874d

* http://rainbows.rubyforge.org/
* rainbows-talk at rubyforge.org
* git://bogomips.org/rainbows.git

-- 
Eric Wong

From normalperson at yhbt.net  Fri Mar 18 14:09:02 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 18 Mar 2011 11:09:02 -0700
Subject: [ANN] raindrops updates (mainly for Linux users)
Message-ID: <20110318180902.GA6076@dcvr.yhbt.net>

I figured I should cross post here.  raindrops (listen queue stats
package for Unicorn/Rainbows!) has gotten some largish updates over the
past few weeks:

raindrops 0.5.0 release announcement:
  http://mid.gmane.org/20110317033547.GA30653 at dcvr.yhbt.net

demo site changes (will probably be in 0.6.0 soon):
  http://mid.gmane.org/20110318094637.GA3526 at dcvr.yhbt.net

Feedback would be greatly appreciated (either here or
raindrops at librelist.com).  There are no backwards-incompatible changes
for existing users as far as I'm aware of.

Thanks for reading!

-- 
Eric Wong