From pic at superfluo.org Mon Jul 3 13:05:55 2006 From: pic at superfluo.org (Nicola Piccinini) Date: Mon, 03 Jul 2006 19:05:55 +0200 Subject: [Restful-rails-general] Accept header and cache In-Reply-To: References: <449C7A95.3070709@orcaware.com> <44A0D120.6030105@superfluo.org> <44A21B93.2070902@superfluo.org> Message-ID: <44A94E73.9080800@superfluo.org> Hi Dan, thanks very much for the detailed response. >> apropos of Accept header: >> [...] > > > I know that this type of thing happens more often than not, > but if you see a client behave in this manner it is most certainly > broken. > > In the original response the Vary header should be used to indicate > that the server used the Accept header to do content negotiation to > generate the response. On future requests, if the client notices > that its sending a different Accept header it should NOT attempt to > send the earlier response's ETag and Last-Modified header as the > If-Match and If-Unmodified-Since header respectively. > > Its only when a request matches a previous request *exactly* (as far > as the Vary headers go) that it should attempt to do a conditional > GET request. ok, so two questions about the restful-rails plugin expected behavior in the same situation (i.e. two consecutive requests with a different Accept header): 1. shouldn't the plugin automatically add "Accept" as a value in the Vary header? Code at line 101 in conditions.rb is there for this purpose? If so, based on a couple of experiments, it doesn't seem to work. If not so, should we manually add something like: response.vary << 'Accept' when necessary? 2. shouldn't the value of Etag header in the second response be different from the first one? Again the experiments show that this isn't the case. > [...] > Even when building RESTful systems sometimes its important > to be pragmatic and accept that browsers may need hacks > to allow them to work with those systems. sad but true. Best regards. -- Nicola Piccinini -- http://superfluo.org From dan.kubb at autopilotmarketing.com Mon Jul 3 15:12:55 2006 From: dan.kubb at autopilotmarketing.com (Dan Kubb) Date: Mon, 3 Jul 2006 12:12:55 -0700 Subject: [Restful-rails-general] Accept header and cache In-Reply-To: <44A94E73.9080800@superfluo.org> References: <449C7A95.3070709@orcaware.com> <44A0D120.6030105@superfluo.org> <44A21B93.2070902@superfluo.org> <44A94E73.9080800@superfluo.org> Message-ID: <34F1E1D3-5188-4227-813D-FC9193DC08CD@autopilotmarketing.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Nicola, > ok, so two questions about the restful-rails plugin expected > behavior in the same situation (i.e. two consecutive requests with > a different Accept header): > > 1. shouldn't the plugin automatically add "Accept" as a value in > the Vary header? Code at line 101 in conditions.rb is there for > this purpose? > If so, based on a couple of experiments, it doesn't seem to work. > If not so, should we manually add something like: > response.vary << 'Accept' > when necessary? No, RESTful Rails shouldn't be adding Accept to the Vary header on its own, since at no point does it look at the Accept header and send different content base on its value. If it was handling content negotiation (or conneg for short) then it would add the header. Conneg is the process of looking at the Accept header, and then rendering a different view to match the capabilities of the browser/user-agent. > 2. shouldn't the value of Etag header in the second response be > different from the first one? Again the experiments show that this > isn't the case. The ETag is currently generated based on the lock_version values of all the models that were used to render the view. It should probably include other information, like the template path and timestamp. Unfortunately at the point where you need to test the conditions you don't necessarily know which template is going to be used, or if one is going to be used at all. I'm not entirely sure how to best solve this though, but if anyone has ideas I'm open to them. - -- Thanks, Dan __________________________________________________________________ Dan Kubb Autopilot Marketing Inc. Email: dan.kubb at autopilotmarketing.com Phone: 1 (604) 820-0212 Web: http://autopilotmarketing.com/ vCard: http://autopilotmarketing.com/~dan.kubb/vcard __________________________________________________________________ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (Darwin) iD8DBQFEqWw44DfZD7OEWk0RAqenAKCBLfSCnZD7JDwKcHO9KqifPrpVtQCeNtwv gnn3xsav0+nKWPVTCjotgb8= =q20u -----END PGP SIGNATURE----- From pic at superfluo.org Sat Jul 8 01:21:39 2006 From: pic at superfluo.org (Nicola Piccinini) Date: Sat, 08 Jul 2006 07:21:39 +0200 Subject: [Restful-rails-general] Accept header and cache In-Reply-To: <34F1E1D3-5188-4227-813D-FC9193DC08CD@autopilotmarketing.com> References: <449C7A95.3070709@orcaware.com> <44A0D120.6030105@superfluo.org> <44A21B93.2070902@superfluo.org> <44A94E73.9080800@superfluo.org> <34F1E1D3-5188-4227-813D-FC9193DC08CD@autopilotmarketing.com> Message-ID: <44AF40E3.5090604@superfluo.org> > No, RESTful Rails shouldn't be adding Accept to the Vary header > on its own, since at no point does it look at the Accept header > and send different content base on its value. > > If it was handling content negotiation (or conneg for short) > then it would add the header. Conneg is the process of looking > at the Accept header, and then rendering a different view to > match the capabilities of the browser/user-agent. ok, so I think that the MimeResponds is responsible for this. This dirty hack: module ActionController::MimeResponds::InstanceMethods alias :_or_respond_to :respond_to def respond_to(*types, &block) _or_respond_to(*types, &block) response.vary << 'Accept' end end adds the 'Accept' value in the Vary header whenever the respond_to method is used. Imho, this makes sense because actually the respond_to method looks at the Accept header. >> 2. shouldn't the value of Etag header in the second response be >> different from the first one? Again the experiments show that this >> isn't the case. > > > The ETag is currently generated based on the lock_version > values of all the models that were used to render the view. > > It should probably include other information, like the template > path and timestamp. Unfortunately at the point where you need > to test the conditions you don't necessarily know which template > is going to be used, or if one is going to be used at all. exactly! > I'm not entirely sure how to best solve this though, but if anyone > has ideas I'm open to them. I haven't a solution but I like to discuss here about a ugly workaround to better understand the problem. 1. another dirty hack: class HTTP::Conditions def consider_for_etag @consider_for_etag ||= [] end end 2. in Conditions#test!: etag_values = [].concat(consider_for_etag).concat(lock_versions) response.etag = Digest::MD5.hexdigest(etag_values.join('|')) instead of response.etag = Digest::MD5.hexdigest(lock_versions.join('|')) 3. finally, in RESTful controllers, when appropriate: resource :by_id do |r| conditions.consider_for_etag << 'what you want' # ... In the case of content type, this could be: conditions.consider_for_etag << request.headers['Accept'] In this way the resource's Etag is surely different if the content type is different. Of course it's sub-optimal because it could be different also when the content type is the same ... . To make it optimal we should ask the Responders for the chosen template before it really uses it but this seems to me difficult. Moreover this instruction can't be added to the respond_to method because the Conditions#test! method is called before it. Any comment? Thanks, best regards. -- Nicola Piccinini -- http://superfluo.org From dan.kubb at autopilotmarketing.com Mon Jul 17 18:15:50 2006 From: dan.kubb at autopilotmarketing.com (Dan Kubb) Date: Mon, 17 Jul 2006 15:15:50 -0700 Subject: [Restful-rails-general] Accept header and cache In-Reply-To: <44AF40E3.5090604@superfluo.org> References: <449C7A95.3070709@orcaware.com> <44A0D120.6030105@superfluo.org> <44A21B93.2070902@superfluo.org> <44A94E73.9080800@superfluo.org> <34F1E1D3-5188-4227-813D-FC9193DC08CD@autopilotmarketing.com> <44AF40E3.5090604@superfluo.org> Message-ID: <3257C57F-32EB-4687-A015-B2BF50BDA9FE@autopilotmarketing.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Nicola, > ok, so I think that the MimeResponds is responsible for this. > > This dirty hack: > > ... snip ... > > adds the 'Accept' value in the Vary header whenever the respond_to > method is used. Imho, this makes sense because actually the > respond_to method looks at the Accept header. If Rails is not currently adding "Accept" to the Vary header when respond_to is called (I've not tested this) it should. You should submit a patch with your fix to the Rails core about this problem. Not putting Accept in the Vary header could cause problems with caching proxy servers getting two or more variants mixed up, thinking they are the same. > I haven't a solution but I like to discuss here about a ugly > workaround to better understand the problem. > > ... snip ... > > 3. finally, in RESTful controllers, when appropriate: > > resource :by_id do |r| > conditions.consider_for_etag << 'what you want' > # ... > > In the case of content type, this could be: > conditions.consider_for_etag << request.headers['Accept'] > > In this way the resource's Etag is surely different if the content > type is different. > Of course it's sub-optimal because it could be different also when > the content type is the same ... . To make it optimal we should ask > the Responders for the chosen template before it really uses it but > this seems to me difficult. > > Moreover this instruction can't be added to the respond_to method > because the Conditions#test! method is called before it. > > Any comment? I can see this as a temporary stop-gap measure. At least the responses would have different ETags based on the response variant. There would be cases where there are duplicates in the cache since the Accept header varies so much between different browsers. Its sort of a waste of space, but ok for now. I would accept a patch to RESTful Rails that can do this if you want to send one over. The long-term solution would be if in Rails there was a way of asking which template respond_to is going to use. It would have to use the Accept headers, plus see if the :format parameter was specified in the URL, since it overrides Accept in rails. From what I can tell there is no method you can use to find out the format of the response, its all done on the fly at render time. Does anyone know of a way to do this in Edge Rails now, or with minimal patching? - -- Thanks, Dan __________________________________________________________________ Dan Kubb Autopilot Marketing Inc. Email: dan.kubb at autopilotmarketing.com Phone: 1 (604) 820-0212 Web: http://autopilotmarketing.com/ vCard: http://autopilotmarketing.com/~dan.kubb/vcard __________________________________________________________________ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (Darwin) iD8DBQFEvAwX4DfZD7OEWk0RAiNOAKCXyE3G+2UrPDnDSrk4K3bd9lC0xgCePTTh c/1RIQS1wQ+o6Xktb+3M2nQ= =xlQH -----END PGP SIGNATURE----- From pic at superfluo.org Tue Jul 25 19:16:25 2006 From: pic at superfluo.org (Nicola Piccinini) Date: Wed, 26 Jul 2006 01:16:25 +0200 Subject: [Restful-rails-general] Accept header and cache In-Reply-To: <3257C57F-32EB-4687-A015-B2BF50BDA9FE@autopilotmarketing.com> References: <449C7A95.3070709@orcaware.com> <44A0D120.6030105@superfluo.org> <44A21B93.2070902@superfluo.org> <44A94E73.9080800@superfluo.org> <34F1E1D3-5188-4227-813D-FC9193DC08CD@autopilotmarketing.com> <44AF40E3.5090604@superfluo.org> <3257C57F-32EB-4687-A015-B2BF50BDA9FE@autopilotmarketing.com> Message-ID: <44C6A649.7000108@superfluo.org> >>adds the 'Accept' value in the Vary header whenever the respond_to >>method is used. Imho, this makes sense because actually the >>respond_to method looks at the Accept header. > > > If Rails is not currently adding "Accept" to the Vary > header when respond_to is called (I've not tested this) it > should. You should submit a patch with your fix to the > Rails core about this problem. my fix is not a minimal patch because depends on your code for multi values header and Vary header. I think that is better to alert the developers to the potential problem, referring to this thread in RESTful Rails mailing list for more information. I'm going to do this as soon http://dev.rubyonrails.org/ starts working again. > [...] > I can see this as a temporary stop-gap measure. yes, it is. > At least the > responses would have different ETags based on the response > variant. There would be cases where there are duplicates in > the cache since the Accept header varies so much between > different browsers. Its sort of a waste of space, but ok > for now. I would accept a patch to RESTful Rails that can > do this if you want to send one over. you find it in attachment. Let me know if you need something else. > The long-term solution would be if in Rails there was a way of asking > which template respond_to is going to use. It would have to use > the Accept headers, plus see if the :format parameter was specified > in the URL, since it overrides Accept in rails. > > From what I can tell there is no method you can use to find > out the format of the response, its all done on the fly at > render time. > > Does anyone know of a way to do this in Edge Rails now, or > with minimal patching? I've looked at Edge but it seems to me that the mime_responds.rb file isn't much different from the one in 1.1 version, hence it's still difficult to gather such information. Best regards. -- Nicola Piccinini -- http://superfluo.org -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: conditions_patch.txt Url: http://rubyforge.org/pipermail/restful-rails-general/attachments/20060726/9ca50098/attachment.txt From pic at superfluo.org Wed Jul 26 17:11:13 2006 From: pic at superfluo.org (Nicola Piccinini) Date: Wed, 26 Jul 2006 23:11:13 +0200 Subject: [Restful-rails-general] Accept header and cache In-Reply-To: <44C6A649.7000108@superfluo.org> References: <449C7A95.3070709@orcaware.com> <44A0D120.6030105@superfluo.org> <44A21B93.2070902@superfluo.org> <44A94E73.9080800@superfluo.org> <34F1E1D3-5188-4227-813D-FC9193DC08CD@autopilotmarketing.com> <44AF40E3.5090604@superfluo.org> <3257C57F-32EB-4687-A015-B2BF50BDA9FE@autopilotmarketing.com> <44C6A649.7000108@superfluo.org> Message-ID: <44C7DA71.2020302@superfluo.org> I've updated the patch (in attachment) with a short rdoc comment. http://dev.rubyonrails.org continues to be unusable. Best. -- Nicola Piccinini -- http://superfluo.org -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: conditions_patch.txt Url: http://rubyforge.org/pipermail/restful-rails-general/attachments/20060726/8cd9220e/attachment.txt