All the Perl that's Practical to Extract and Report

headless gecko

slanning on 2008-02-20T15:27:09

Looks like someone is realizing one of my dreams with Crowbar: Crowbar is a web scraping environment based on the use of a server-side headless mozilla-based browser.

Javascript for WWW::Mechanize

jdavidb on 2008-02-20T15:52:17

FYI, Javascript support for WWW::Mechanize has recently begun to emerge. You'd have to do some Googling and installation of experimental versions to use it; IIRC it uses a plugins mechanism for WWW::Mechanize which exists only in an experimental version. I believe it can use a Perl-based Javascript engine or Perl bindings for Mozilla's Javascript engine.

Not sure if this is relevant to you or not, but maybe someone else will be glad to know. :)

Uh... been there, done that?

bart on 2008-02-20T18:12:26

I'm not sure why this problem has not already been solved with the modules Win32::IE::Mechanize (on Windows, uses the core engine of MS Internet Explorer) and its descendant Mozilla::Mechanize (uses the gecko core). I believe that both support execution of embedded Javascript — I know for sure that Win32::IE::Mechanize does.

Re:Uh... been there, done that?

runrig on 2008-02-20T19:33:00

Check out who the author of Moz::Mech is, and who the author of the post up above is...I'm sure he knows what the issues are. I'm guessing that M::M actually opens a browser window, and that he wants to not have to do that.

Re:Uh... been there, done that?

bart on 2008-02-20T20:27:00

Ah, right. I only have experience with Win32::IE::Mechanize which does not open any windows, but not with Mozilla::Mechanize. Odd that nobody I've ever talked to about this module ever mentioned that fact.