From ian at ianduggan.net  Mon Jun  1 18:32:08 2009
From: ian at ianduggan.net (Ian Duggan)
Date: Mon, 1 Jun 2009 15:32:08 -0700
Subject: [Mechanize-users] running unit tests
Message-ID: <222747e50906011532g52657ae6rf7b2513a519a15d9@mail.gmail.com>

I have a fresh checkout, but running "rake test" has 8 failures and 49 errors.

What needs to be setup to run unit tests?

I have an enhancement I'd like to provide, but would prefer to include
tests with it.

--Ian

From aaron.patterson at gmail.com  Mon Jun  1 18:47:24 2009
From: aaron.patterson at gmail.com (Aaron Patterson)
Date: Mon, 1 Jun 2009 15:47:24 -0700
Subject: [Mechanize-users] running unit tests
In-Reply-To: <222747e50906011532g52657ae6rf7b2513a519a15d9@mail.gmail.com>
References: <222747e50906011532g52657ae6rf7b2513a519a15d9@mail.gmail.com>
Message-ID: <6959e1680906011547j275c01d8n9365467cd4d17a7a@mail.gmail.com>

On Mon, Jun 1, 2009 at 3:32 PM, Ian Duggan <ian at ianduggan.net> wrote:
> I have a fresh checkout, but running "rake test" has 8 failures and 49 errors.
>
> What needs to be setup to run unit tests?
>
> I have an enhancement I'd like to provide, but would prefer to include
> tests with it.

I just pushed some stuff.  Try pulling and running the tests again.
If they fail, let me know which ones, along with the version of
nokogiri you have installed.

-- 
Aaron Patterson
http://tenderlovemaking.com/

From ian at ianduggan.net  Mon Jun  1 21:17:52 2009
From: ian at ianduggan.net (Ian Duggan)
Date: Mon, 1 Jun 2009 18:17:52 -0700
Subject: [Mechanize-users] running unit tests
In-Reply-To: <6959e1680906011547j275c01d8n9365467cd4d17a7a@mail.gmail.com>
References: <222747e50906011532g52657ae6rf7b2513a519a15d9@mail.gmail.com> 
	<6959e1680906011547j275c01d8n9365467cd4d17a7a@mail.gmail.com>
Message-ID: <222747e50906011817le01c7fbg1a20f67503ba00cf@mail.gmail.com>

On Mon, Jun 1, 2009 at 3:47 PM, Aaron Patterson
<aaron.patterson at gmail.com> wrote:
> On Mon, Jun 1, 2009 at 3:32 PM, Ian Duggan <ian at ianduggan.net> wrote:
>> I have a fresh checkout, but running "rake test" has 8 failures and 49 errors.
>>
>> What needs to be setup to run unit tests?
>>
>> I have an enhancement I'd like to provide, but would prefer to include
>> tests with it.
>
> I just pushed some stuff. ?Try pulling and running the tests again.
> If they fail, let me know which ones, along with the version of
> nokogiri you have installed.

Is there a server or anything that has to be running? I have a ton of
errors. Nokogiri is 1.2.3.

http://gist.github.com/121918

% git top
commit e16279dbb625b37886bdcef80913492685fe8a94
Merge: c1c92e3 cda821b
Author: Aaron Patterson <aaron.patterson at gmail.com>
Date:   Mon Jun 1 15:45:57 2009 -0700

    Merge branch 'master' of git at github.com:tenderlove/mechanize

From aaron at tenderlovemaking.com  Mon Jun  1 22:04:14 2009
From: aaron at tenderlovemaking.com (Aaron Patterson)
Date: Mon, 1 Jun 2009 19:04:14 -0700
Subject: [Mechanize-users] running unit tests
In-Reply-To: <222747e50906011817le01c7fbg1a20f67503ba00cf@mail.gmail.com>
References: <222747e50906011532g52657ae6rf7b2513a519a15d9@mail.gmail.com>
	<6959e1680906011547j275c01d8n9365467cd4d17a7a@mail.gmail.com>
	<222747e50906011817le01c7fbg1a20f67503ba00cf@mail.gmail.com>
Message-ID: <20090602014352.GA24165@Jordan.local>

On Mon, Jun 01, 2009 at 06:17:52PM -0700, Ian Duggan wrote:
> On Mon, Jun 1, 2009 at 3:47 PM, Aaron Patterson
> <aaron.patterson at gmail.com> wrote:
> > On Mon, Jun 1, 2009 at 3:32 PM, Ian Duggan <ian at ianduggan.net> wrote:
> >> I have a fresh checkout, but running "rake test" has 8 failures and 49 errors.
> >>
> >> What needs to be setup to run unit tests?
> >>
> >> I have an enhancement I'd like to provide, but would prefer to include
> >> tests with it.
> >
> > I just pushed some stuff. ?Try pulling and running the tests again.
> > If they fail, let me know which ones, along with the version of
> > nokogiri you have installed.
> 
> Is there a server or anything that has to be running? I have a ton of
> errors. Nokogiri is 1.2.3.

What does this say?:

  ruby -rubygems -e'require "nokogiri"; puts Nokogiri::LIBXML_VERSION'

-- 
Aaron Patterson
http://tenderlovemaking.com/

From ian at ianduggan.net  Tue Jun  2 13:36:46 2009
From: ian at ianduggan.net (Ian Duggan)
Date: Tue, 2 Jun 2009 10:36:46 -0700
Subject: [Mechanize-users] running unit tests
In-Reply-To: <20090602014352.GA24165@Jordan.local>
References: <222747e50906011532g52657ae6rf7b2513a519a15d9@mail.gmail.com> 
	<6959e1680906011547j275c01d8n9365467cd4d17a7a@mail.gmail.com> 
	<222747e50906011817le01c7fbg1a20f67503ba00cf@mail.gmail.com> 
	<20090602014352.GA24165@Jordan.local>
Message-ID: <222747e50906021036h1b46fb29m3b11d86c45b47151@mail.gmail.com>

On Mon, Jun 1, 2009 at 7:04 PM, Aaron Patterson
<aaron at tenderlovemaking.com> wrote:
> What does this say?:
>
> ?ruby -rubygems -e'require "nokogiri"; puts Nokogiri::LIBXML_VERSION'

% ruby -rubygems -e'require "nokogiri"; puts Nokogiri::LIBXML_VERSION'
2.6.32

From ian at ianduggan.net  Tue Jun  2 13:41:01 2009
From: ian at ianduggan.net (Ian Duggan)
Date: Tue, 2 Jun 2009 10:41:01 -0700
Subject: [Mechanize-users] running unit tests
In-Reply-To: <222747e50906021036h1b46fb29m3b11d86c45b47151@mail.gmail.com>
References: <222747e50906011532g52657ae6rf7b2513a519a15d9@mail.gmail.com> 
	<6959e1680906011547j275c01d8n9365467cd4d17a7a@mail.gmail.com> 
	<222747e50906011817le01c7fbg1a20f67503ba00cf@mail.gmail.com> 
	<20090602014352.GA24165@Jordan.local>
	<222747e50906021036h1b46fb29m3b11d86c45b47151@mail.gmail.com>
Message-ID: <222747e50906021041s27618d94y7e72ba3cfefe578d@mail.gmail.com>

On Tue, Jun 2, 2009 at 10:36 AM, Ian Duggan <ian at ianduggan.net> wrote:
> On Mon, Jun 1, 2009 at 7:04 PM, Aaron Patterson
> <aaron at tenderlovemaking.com> wrote:
>> What does this say?:
>>
>>  ruby -rubygems -e'require "nokogiri"; puts Nokogiri::LIBXML_VERSION'
>
> % ruby -rubygems -e'require "nokogiri"; puts Nokogiri::LIBXML_VERSION'
> 2.6.32
>

The error seems to come from this line:

Iconv.iconv(code, "UTF-8", s).join("")

--Ian

From astarr at wiredquote.com  Wed Jun 10 19:08:49 2009
From: astarr at wiredquote.com (Aaron Starr)
Date: Wed, 10 Jun 2009 16:08:49 -0700
Subject: [Mechanize-users] Mechanize consumes ~11 mb
Message-ID: <669cc1ca0906101608q38098dcfnc0e002d72d2fbcb2@mail.gmail.com>

I've got a little test script that just sleeps, calls `free -m` and sleeps
again for a while.

Adding "require 'mechanize'" to the top of the script increases the resident
memory (as reported by top) from about 5mb to about 16mb. That 11mb jump
seems a little crazy to me.

The initial 5mb is rubygems, which also seems a little crazy to me.

Am I way off track? Does this seem unremarkable to everyone else? Or, have I
inadvertently set the "use-a-lot-of-memory" switch?

Aaron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20090610/64a1c056/attachment.html>

From alex at blackkettle.org  Wed Jun 17 14:10:12 2009
From: alex at blackkettle.org (Alex Young)
Date: Wed, 17 Jun 2009 19:10:12 +0100
Subject: [Mechanize-users] Nokogiri encoding bug
Message-ID: <4A393184.6090303@blackkettle.org>

Hi there,

I spotted in the list archive someone having the same problem as I've 
just run into. I just thought I'd feed back some info in case it was 
useful for future googlers.  Here's a typical backtrace:

$ ruby -Itest -Ilib test/test_blank_form.rb
Loaded suite test/test_blank_form
Started
E
Finished in 0.003698 seconds.

   1) Error:
test_blank_form_query_string(BlankFormTest):
TypeError: can't convert nil into String
     /home/alex/gems/mechanize/lib/www/mechanize/util.rb:41:in `iconv'
     /home/alex/gems/mechanize/lib/www/mechanize/util.rb:41:in 
`from_native_charset'
     /home/alex/gems/mechanize/lib/www/mechanize/form.rb:153:in 
`from_native_charset'
     /home/alex/gems/mechanize/lib/www/mechanize/form.rb:144:in `proc_query'
     /home/alex/gems/mechanize/lib/www/mechanize/form.rb:143:in `map'
     /home/alex/gems/mechanize/lib/www/mechanize/form.rb:143:in `proc_query'
     /home/alex/gems/mechanize/lib/www/mechanize/form.rb:167:in 
`build_query'
     /home/alex/gems/mechanize/lib/www/mechanize/form.rb:166:in `each'
     /home/alex/gems/mechanize/lib/www/mechanize/form.rb:166:in 
`build_query'
     test/test_blank_form.rb:11:in `test_blank_form_query_string'

1 tests, 0 assertions, 0 failures, 1 errors

Versions are:

mechanize: 0.9.3.20090617085936, fresh from github
nokogiri: 1.2.3, gem installed
Nokogiri::LIBXML_VERSION: 2.6.32
OS: 64-bit Ubuntu Jaunty
ruby -v: ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]

I've tracked it back to the WWW::Mechanize::Page#parser method, 
specifically line 75:

   @parser = mech.html_parser.parse(html_body, nil, @encoding)

For a page I'm working on, @encoding is "utf-8" before this call (it's 
set in a Content-Type header), but @parser.encoding is nil immediately 
afterwards, and that's what eventually ends up being passed to iconv as 
the encoding value. I'm going to try following this upstream, so there 
might be a pull request coming in at some point if I get anywhere with 
it. That being said, a quick fix would be to simply ignore the value 
that comes back from the parser. Since we've already got the encoding, 
what more can the parser tell us? I don't understand that bit yet.

Thanks,
-- 
Alex

From alex at blackkettle.org  Wed Jun 17 15:43:13 2009
From: alex at blackkettle.org (Alex Young)
Date: Wed, 17 Jun 2009 20:43:13 +0100
Subject: [Mechanize-users] Nokogiri encoding bug
In-Reply-To: <4A393184.6090303@blackkettle.org>
References: <4A393184.6090303@blackkettle.org>
Message-ID: <4A394751.1070809@blackkettle.org>

Alex Young wrote:
> Hi there,
<snip>
> That being said, a quick fix would be to simply ignore the value 
> that comes back from the parser. Since we've already got the encoding, 
> what more can the parser tell us? I don't understand that bit yet.

Just a quick follow-up. The easiest patch to do this is:

diff --git a/lib/www/mechanize/page.rb b/lib/www/mechanize/page.rb
index 1f7d884..ac6909e 100644
--- a/lib/www/mechanize/page.rb
+++ b/lib/www/mechanize/page.rb
@@ -64,7 +64,7 @@ module WWW
        end

        def encoding
-        parser.respond_to?(:encoding) ? parser.encoding : nil
+        (parser.respond_to?(:encoding) ? parser.encoding : nil) || 
@encoding
        end

        def parser

That results in all the tests passing, except for the following:

   1) Failure:
test_another_mostly_broken_charset(TestPage) [./test/test_page.rb:32]:
<"UTF8"> expected but was
<nil>.

   2) Failure:
test_encoding_override_after_parser_was_initialized(TestPage) 
[./test/test_page.rb:58]:
<"ISO-8859-2"> expected but was
<"windows-1255">.

   3) Failure:
test_encoding_override_before_parser_initialized(TestPage) 
[./test/test_page.rb:47]:
<"ISO-8859-2"> expected but was
<"windows-1255">.

   4) Failure:
test_page_decoded_with_charset(TestPage) [./test/test_page.rb:99]:
<"EUC-JP"> expected but was
<nil>.

   5) Failure:
test_set_encoding(TestPage) [./test/test_page.rb:69]:
<"UTF-8"> expected but was
<nil>.

291 tests, 1502 assertions, 5 failures, 0 errors

There's clearly something screwy going on with libxml2's HTMLParser, but 
I don't have much more than that yet.

Hope this helps.

-- 
Alex

From jhuizingh at mindless.com  Wed Jun 17 22:55:48 2009
From: jhuizingh at mindless.com (Jonathan Huizingh)
Date: Wed, 17 Jun 2009 21:55:48 -0500
Subject: [Mechanize-users] Migrate from Hpricot to Nokogiri
Message-ID: <a7b4c39f0906171955t1f778154qa5a8e76d6a88961d@mail.gmail.com>

Hi,

I was using Mechanize with Hpricot in a rails app about 6 months ago.
I recently started to work on that app again, and was getting an
error:  "undefined method `text' for #<Array:0x7f8b0d349be0>".  It
took me a little bit of research, but I figured out that I was using a
newer version of the Mechanize gem on this machine and that it had
switched to using Nokogiri as its parser.

I would like to update my code to use Nokogiri since it is now the
default.  I have been looking around the documentation, but haven't
been able to find any examples of what I am trying to do.  Can
somebody give me an example of how I would update the following line?

unless logged_in_page.links.text(/link text to find/).length > 0
   ... more code here ...
end

I also saw that there were Hpricot decorators for Nokogiri objects.  I
assume this would allow me to use my Hpricot code as-is, but I
couldn't find an example of how to do that.  Could somebody give me an
example of that?

Thanks,

Jonathan

From aaron.patterson at gmail.com  Thu Jun 18 12:47:22 2009
From: aaron.patterson at gmail.com (Aaron Patterson)
Date: Thu, 18 Jun 2009 09:47:22 -0700
Subject: [Mechanize-users] Migrate from Hpricot to Nokogiri
In-Reply-To: <a7b4c39f0906171955t1f778154qa5a8e76d6a88961d@mail.gmail.com>
References: <a7b4c39f0906171955t1f778154qa5a8e76d6a88961d@mail.gmail.com>
Message-ID: <6959e1680906180947w75ea426dyd34bf1b9f3cd0347@mail.gmail.com>

On Wed, Jun 17, 2009 at 7:55 PM, Jonathan
Huizingh<jhuizingh at mindless.com> wrote:
> Hi,
>
> I was using Mechanize with Hpricot in a rails app about 6 months ago.
> I recently started to work on that app again, and was getting an
> error: ?"undefined method `text' for #<Array:0x7f8b0d349be0>". ?It
> took me a little bit of research, but I figured out that I was using a
> newer version of the Mechanize gem on this machine and that it had
> switched to using Nokogiri as its parser.
>

Yes!  Your problem isn't to do with hpricot vs nokogiri, but that the
links api has changed.  "links" returns a normal array now, so you
can't call "text" on it.

> I would like to update my code to use Nokogiri since it is now the
> default. ?I have been looking around the documentation, but haven't
> been able to find any examples of what I am trying to do. ?Can
> somebody give me an example of how I would update the following line?
>
> unless logged_in_page.links.text(/link text to find/).length > 0
> ? ... more code here ...
> end

try this:

  unless logged_in_page.links_with(:text => /link text to find/).length > 0
    ...
  end

> I also saw that there were Hpricot decorators for Nokogiri objects. ?I
> assume this would allow me to use my Hpricot code as-is, but I
> couldn't find an example of how to do that. ?Could somebody give me an
> example of that?

If you're only calling methods on mechanize, you shouldn't need to do that.

-- 
Aaron Patterson
http://tenderlovemaking.com/

From jhuizingh at mindless.com  Sun Jun 21 16:51:45 2009
From: jhuizingh at mindless.com (Jonathan Huizingh)
Date: Sun, 21 Jun 2009 15:51:45 -0500
Subject: [Mechanize-users] Migrate from Hpricot to Nokogiri
In-Reply-To: <6959e1680906180947w75ea426dyd34bf1b9f3cd0347@mail.gmail.com>
References: <a7b4c39f0906171955t1f778154qa5a8e76d6a88961d@mail.gmail.com> 
	<6959e1680906180947w75ea426dyd34bf1b9f3cd0347@mail.gmail.com>
Message-ID: <a7b4c39f0906211351n3e67139aj9632124e909c7578@mail.gmail.com>

Thanks.  Your suggestion worked like a charm.

I think what confused me is that this page
(http://mechanize.rubyforge.org/mechanize/EXAMPLES_rdoc.html) has
examples that show my way of doing it, even though they don't seem to
work anymore.

Thanks again for the help.

Jonathan


On Thu, Jun 18, 2009 at 11:47 AM, Aaron
Patterson<aaron.patterson at gmail.com> wrote:

>
> Yes! ?Your problem isn't to do with hpricot vs nokogiri, but that the
> links api has changed. ?"links" returns a normal array now, so you
> can't call "text" on it.
>
>> I would like to update my code to use Nokogiri since it is now the
>> default. ?I have been looking around the documentation, but haven't
>> been able to find any examples of what I am trying to do. ?Can
>> somebody give me an example of how I would update the following line?
>>
>> unless logged_in_page.links.text(/link text to find/).length > 0
>> ? ... more code here ...
>> end
>
> try this:
>
> ?unless logged_in_page.links_with(:text => /link text to find/).length > 0
> ? ?...
> ?end
>

From jhuizingh at mindless.com  Wed Jun 24 18:10:57 2009
From: jhuizingh at mindless.com (Jonathan Huizingh)
Date: Wed, 24 Jun 2009 17:10:57 -0500
Subject: [Mechanize-users] Get original html from document
Message-ID: <a7b4c39f0906241510l14d1033blcfd0e9d420715c9b@mail.gmail.com>

Is there a way to get the original html from a Nokogiri::HTML object?
I am trying to write an exception handler that saves the html that
Nokogiri parsed so that I can view it later and be sure that I'm
seeing exactly what Nokogiri saw and then I can figure out what went
wrong.

Also, how does Nokogiri::HTML handle it if there is invalid html
syntax in the page?

Thanks,

Jonathan Huizingh
Pronunciation:  HI-zing

From lee.hambley at gmail.com  Thu Jun 25 17:36:40 2009
From: lee.hambley at gmail.com (Lee Hambley)
Date: Thu, 25 Jun 2009 22:36:40 +0100
Subject: [Mechanize-users] Bad Form?
Message-ID: <e2172f210906251436w38c96e3eq11c56a5a875fee99@mail.gmail.com>

Hi,
This is my first post, I wanted to ask a quick question about "Bad Form" --
rather, am I doing something with Mechanise I shouldn't be doing?

Here's the long and short of it - I am using Mechanize with Cucumber and
RSpec to test a *ahem* Perl XML web service.. talk about technology soup,
but I have it working quite well.

I have an example project on GitHub that actually has a contrived Sinatra
application to replicate testing a 3rd party system that you can only test
the interface to.
https://github.com/leehambley/webrat-mechanize-xml-rpc<https://github.com/leehambley/webrat-mechanize-xml-rpc/tree>

The problem, or rather question is as such - I have, in my application's
test environment made use of PluggableParser, but overloaded it with the
.default, and .html calls to parse everything as myown XMLParser class
(copied almost verbatim from the CSVParser example in the RDoc)

I have effectively rendered it as not a browser, by forcing it to treat
anything that parses as XML as a
WWW::Mechanize::File<http://mechanize.rubyforge.org/mechanize/WWW/Mechanize/File.html>
rather
than a WWW::Mechanize::Page<http://mechanize.rubyforge.org/mechanize/WWW/Mechanize/Page.html>
-
which I'm not sure about as an implementation.

When the server sends a correct header (which isn't in all parts...) then I
do not need to set the .html parser to be XMLParser, it just recognizes the
text/xml content-type header, and parses it to a file correctly...

I guess, I have it all working, and we're testing happily the web service,
which is something our company does a lot of... so having Cucumber/Webrat
and Mechanize all working this way is just... outstanding, it's fantastic,
but I don't want to base my new suite of tests on a flawed principle!

Thoughts welcome, or just ignore me, and I'll sleep better knowing that I've
confessed my sins to the internet!

- Lee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20090625/2bb01e8e/attachment.html>

From lee.hambley at gmail.com  Fri Jun 26 06:36:19 2009
From: lee.hambley at gmail.com (Lee Hambley)
Date: Fri, 26 Jun 2009 11:36:19 +0100
Subject: [Mechanize-users] Bad Form?
In-Reply-To: <C304E18A-12F5-4D18-BC56-8AF3DFEBEC56@gmail.com>
References: <e2172f210906251436w38c96e3eq11c56a5a875fee99@mail.gmail.com>
	<C304E18A-12F5-4D18-BC56-8AF3DFEBEC56@gmail.com>
Message-ID: <e2172f210906260336g798ad79bqed3ae96d0aef2cc8@mail.gmail.com>

Cheers for the sanity check matt, it does treat anything with a content-type
text/xml header is processed correctly as a File, but text/html wasn't!
I've decided (with advice from the Cucumber people) that using MEchanize and
Webrat to test an XML webservice is mad overkill, so I'm refactoring!

- Lee

2009/6/25 Mat Schaffer <mat.schaffer at gmail.com>

> Sounds cool to me. I guess the only thing that confused me a little bit was
> the XML response to file object conversion which I though ive seen mechanize
> already do. Other than that what you're doing seems sound to me. Or at least
> not totally off the wall :-)
>
> -mat
>
> Sent from my iPhone
>
> On Jun 25, 2009, at 5:36 PM, Lee Hambley <lee.hambley at gmail.com> wrote:
>
> Hi,
> This is my first post, I wanted to ask a quick question about "Bad Form" --
> rather, am I doing something with Mechanise I shouldn't be doing?
>
> Here's the long and short of it - I am using Mechanize with Cucumber and
> RSpec to test a *ahem* Perl XML web service.. talk about technology soup,
> but I have it working quite well.
>
> I have an example project on GitHub that actually has a contrived Sinatra
> application to replicate testing a 3rd party system that you can only test
> the interface to. <https://github.com/leehambley/webrat-mechanize-xml-rpc/tree>
> https://github.com/leehambley/webrat-mechanize-xml-rpc
>
> The problem, or rather question is as such - I have, in my application's
> test environment made use of PluggableParser, but overloaded it with the
> .default, and .html calls to parse everything as myown XMLParser class
> (copied almost verbatim from the CSVParser example in the RDoc)
>
> I have effectively rendered it as not a browser, by forcing it to treat
> anything that parses as XML as a WWW::Mechanize::File<http://mechanize.rubyforge.org/mechanize/WWW/Mechanize/File.html> rather
> than a WWW::Mechanize::Page<http://mechanize.rubyforge.org/mechanize/WWW/Mechanize/Page.html> -
> which I'm not sure about as an implementation.
>
> When the server sends a correct header (which isn't in all parts...) then I
> do not need to set the .html parser to be XMLParser, it just recognizes the
> text/xml content-type header, and parses it to a file correctly...
>
> I guess, I have it all working, and we're testing happily the web service,
> which is something our company does a lot of... so having Cucumber/Webrat
> and Mechanize all working this way is just... outstanding, it's fantastic,
> but I don't want to base my new suite of tests on a flawed principle!
>
> Thoughts welcome, or just ignore me, and I'll sleep better knowing that
> I've confessed my sins to the internet!
>
> - Lee
>
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
>
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20090626/a8dbcc7d/attachment.html>

From lee.hambley at gmail.com  Sat Jun 27 09:05:44 2009
From: lee.hambley at gmail.com (Lee Hambley)
Date: Sat, 27 Jun 2009 14:05:44 +0100
Subject: [Mechanize-users] Bad Form?
In-Reply-To: <238FC4CB-C6F5-4AEB-BF71-E5A75652D6C4@gmail.com>
References: <e2172f210906251436w38c96e3eq11c56a5a875fee99@mail.gmail.com>
	<C304E18A-12F5-4D18-BC56-8AF3DFEBEC56@gmail.com>
	<e2172f210906260336g798ad79bqed3ae96d0aef2cc8@mail.gmail.com>
	<238FC4CB-C6F5-4AEB-BF71-E5A75652D6C4@gmail.com>
Message-ID: <e2172f210906270605k38bb1e66m2b1d83d96f8b699e@mail.gmail.com>

Mat,
The new problem is that under some circumstances (read: most of them) --
when calling something like has_xpath('/error/code') on the following XML, I
get an error because somewhere in the chain, something has injected some
HTML into the mix.

I documented what I thought mattered on GitHib's Gist service -
https://gist.github.com/2817bee77fd4eec1f28b - This is the same that the
#webrat channel on freenode convinced me neede some Net:HTTP love.. though I
see what you mean, even though I think we'll only need have_xpath() should()
and should_not() and maybe contain() -- trying to pull have_xpath() out of
the webrat matcher is a nightmare though, sooner or later you end up down in
the depths of the code at an if *session.selenium or session.webrat* and
it's back to the start!

Had limited success using the XPath in REXML, but that again isn't too
intuitive!

Yours Lost,

   - Lee


2009/6/26 Mat Schaffer <mat.schaffer at gmail.com>

> What's the new plan? I often end up at mechanize just because the base
> Net::HTTP stuff has me rolling too much of my own code. I'll agree that
> webrat is probably a bit much.
>
> Sent from my iPhone
>
> On Jun 26, 2009, at 6:36 AM, Lee Hambley <lee.hambley at gmail.com> wrote:
>
> Cheers for the sanity check matt, it does treat anything with a
> content-type text/xml header is processed correctly as a File, but text/html
> wasn't!
> I've decided (with advice from the Cucumber people) that using MEchanize
> and Webrat to test an XML webservice is mad overkill, so I'm refactoring!
>
> - Lee
>
> 2009/6/25 Mat Schaffer < <mat.schaffer at gmail.com>mat.schaffer at gmail.com>
>
>> Sounds cool to me. I guess the only thing that confused me a little bit
>> was the XML response to file object conversion which I though ive seen
>> mechanize already do. Other than that what you're doing seems sound to me.
>> Or at least not totally off the wall :-)
>>
>> -mat
>>
>> Sent from my iPhone
>>
>> On Jun 25, 2009, at 5:36 PM, Lee Hambley < <lee.hambley at gmail.com>
>> lee.hambley at gmail.com> wrote:
>>
>> Hi,
>> This is my first post, I wanted to ask a quick question about "Bad Form"
>> -- rather, am I doing something with Mechanise I shouldn't be doing?
>>
>> Here's the long and short of it - I am using Mechanize with Cucumber and
>> RSpec to test a *ahem* Perl XML web service.. talk about technology soup,
>> but I have it working quite well.
>>
>> I have an example project on GitHub that actually has a contrived Sinatra
>> application to replicate testing a 3rd party system that you can only test
>> the interface to. <https://github.com/leehambley/webrat-mechanize-xml-rpc/tree><https://github.com/leehambley/webrat-mechanize-xml-rpc>
>> https://github.com/leehambley/webrat-mechanize-xml-rpc
>>
>> The problem, or rather question is as such - I have, in my application's
>> test environment made use of PluggableParser, but overloaded it with the
>> .default, and .html calls to parse everything as myown XMLParser class
>> (copied almost verbatim from the CSVParser example in the RDoc)
>>
>> I have effectively rendered it as not a browser, by forcing it to treat
>> anything that parses as XML as a WWW::Mechanize::File<http://mechanize.rubyforge.org/mechanize/WWW/Mechanize/File.html> rather
>> than a WWW::Mechanize::Page<http://mechanize.rubyforge.org/mechanize/WWW/Mechanize/Page.html> -
>> which I'm not sure about as an implementation.
>>
>> When the server sends a correct header (which isn't in all parts...) then
>> I do not need to set the .html parser to be XMLParser, it just recognizes
>> the text/xml content-type header, and parses it to a file correctly...
>>
>> I guess, I have it all working, and we're testing happily the web service,
>> which is something our company does a lot of... so having Cucumber/Webrat
>> and Mechanize all working this way is just... outstanding, it's fantastic,
>> but I don't want to base my new suite of tests on a flawed principle!
>>
>> Thoughts welcome, or just ignore me, and I'll sleep better knowing that
>> I've confessed my sins to the internet!
>>
>> - Lee
>>
>> _______________________________________________
>> Mechanize-users mailing list
>> <Mechanize-users at rubyforge.org>Mechanize-users at rubyforge.org
>>  <http://rubyforge.org/mailman/listinfo/mechanize-users>
>> http://rubyforge.org/mailman/listinfo/mechanize-users
>>
>>
>> _______________________________________________
>> Mechanize-users mailing list
>>  <Mechanize-users at rubyforge.org>Mechanize-users at rubyforge.org
>>  <http://rubyforge.org/mailman/listinfo/mechanize-users>
>> http://rubyforge.org/mailman/listinfo/mechanize-users
>>
>
>  _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
>
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20090627/0e35c821/attachment.html>

From apoc at sixserv.org  Tue Jun 30 15:40:32 2009
From: apoc at sixserv.org (apoc)
Date: Tue, 30 Jun 2009 21:40:32 +0200
Subject: [Mechanize-users] ruby (mechanize , net/http)
In-Reply-To: <E1MLhhg-000PDT-Tw@ffe5.ukr.net>
References: <E1MLhhg-000PDT-Tw@ffe5.ukr.net>
Message-ID: <4A4A6A30.4000206@sixserv.org>

Khomich Vlad wrote:
> 
> Hi guys.
>  I`m using mechanize to get some images from the web application. I have
> 2 problems:
>  1-st : I can`t catch the links. The whole link looks like <a href =
> "https://.../reports/switch_accounts.html?..."> Full report</a> , i can see it in the page source i get with mech
> 
> And there`re couple of these that i try to catch with a singe regexp
> 
> i tried this :
> <code>
> final.links_with(:href => %r{/switch_accounts.html/} ).each do |link|
> puts link.href, link.text
> end
> </code>

The "%r{/switch_accounts.html/}" part seems wrong. Maybe this works?

> final.links_with(:href => /switch_accounts\.html/ ).each do |link|
(not tested)

From astarr at wiredquote.com  Tue Jun 30 15:45:25 2009
From: astarr at wiredquote.com (Aaron Starr)
Date: Tue, 30 Jun 2009 12:45:25 -0700
Subject: [Mechanize-users] ruby (mechanize , net/http)
In-Reply-To: <E1MLhhg-000PDT-Tw@ffe5.ukr.net>
References: <E1MLhhg-000PDT-Tw@ffe5.ukr.net>
Message-ID: <669cc1ca0906301245v29c81e59qc9db64cc5d79d364@mail.gmail.com>

Vlad,

This regular expression:

>
> %r{/switch_accounts.html/}
>

doesn't match this href:

>
> https://.../reports/switch_accounts.html?...
>

Because the regular expression has a trailing '/' after the
accounts.html. I think you could just change to this:

%r{/switch_accounts.html}

and get a match.

Also, something that I do which is useful for me, is to load a copy of
the page in irb, using something like this:

irb> page = Mechanize::Page.new(nil, {'content-type' => 'text/html'},
File.read('my.html'), 200)

(From memory, may have a detail wrong.)

Then, you can test your mechanize code interactively against the page,
and make sure it's working the way you expect, before you put it in
real code.

Aaron

From astarr at wiredquote.com  Tue Jun 30 15:48:06 2009
From: astarr at wiredquote.com (Aaron Starr)
Date: Tue, 30 Jun 2009 12:48:06 -0700
Subject: [Mechanize-users] ruby (mechanize , net/http)
In-Reply-To: <E1MLhhg-000PDT-Tw@ffe5.ukr.net>
References: <E1MLhhg-000PDT-Tw@ffe5.ukr.net>
Message-ID: <669cc1ca0906301248y562cb137vbec470777a67719b@mail.gmail.com>

Vlad,

> The second problem - how can i actually save the images to a selected
> folder(assuming i have the links for them)?

You don't need mechanize for this. Just use open-uri.

Aaron