How to make the job easier....

petdance on 2004-02-05T19:18:34

I know it shouldn't bug me that not everyone knows about WWW::Mechanize at this point, but it does.

Linux Magazine article on WWW::Mech and webstuff

merlyn on 2004-02-05T19:31:44

See part one and part two of my Linux Magazine articles.

Re:Linux Magazine article on WWW::Mech and webstuf

jdavidb on 2004-02-06T14:12:46

Randal, thanks again for being wonderful! I was just going to post and say, "I'm aware of WWW::Mechanize, but haven't learned it yet; is there a good tutorial?" Somehow, you're always there when I need you. :)

One size fits all?

jordan on 2004-02-05T22:34:38

I'm using LWP to control a web app. This application is not developed in house and the pages can change without notice, although the developers _say_ they will give me a heads up. Some of the result tables are kind of messy, with lots of tables nested in tables. Improper operations of my automation could SERIOUSLY screw things up.

Given this scenario, I like to use HTTP::TreeBuilder to build a tree of the forms and the results I get and I check them over for consistent structure and format before acting on any of the information or submitting forms. Only when I know they haven't moved things around or nested things differently than I expect can I have sufficient confidence in operating the app this way.

Maybe I don't get it, but I don't see the point of using WWW:Mechanize in this application. Seems like, at best, I could use a few of the convenience functions but then still have to do all the processing I'm already doing.

Am I seriously off base?

Re:One size fits all?

petdance on 2004-02-05T23:21:01
I was just talking about the use of Mech for building a spider, which is what the original post pointed to.
In your case, it may or may not make sense to use Mech if you're doing a lot of custom form handling. However, there's no reason I can think of NOT to use Mech, since it's a proper subclass of LWP::UserAgent. I figure if you've got it on your machine, might as well use it.

Re:One size fits all?

jordan on 2004-02-05T23:44:55
Good points, but I couldn't see the job posting, the link was bad or something.

I don't have it on the machines though... We have a bunch of machines with the same Perl on them all and I have to request a new build and qualification to add modules. Or, I could support my own module paths for some things. I have done that, but that has drawbacks. Kind of a pain, actually.

I guess I could run my apps from something other than the standard configurations, but that has other problems.

I have been, and loving it.

hide on 2004-02-06T00:49:50

I began using LWP to try and create a spider to traverse a number of our webpages. When I started to collect all the data I was going to need to fill out forms, and parsing to follow links, I started to feel intimidated. I then remembered seeing WWW::Mechanize on CPAN. I quickly installed it. Within 30 minutes I was 4 pages into a complex website. Another 15 and I had completed an entire transaction.

In less than 100 lines of code I have a working spider that reads directions from an XML file. I immediately began talking about it to everyone who would listen. It's now made it's way into a couple other projects.

Great work, and thank you.

nice to see somebody in greece is clued up

TeeJay on 2004-02-06T15:06:02

whenever we go to greece I imagine being able to work there. Greece is ahead of the UK (and way ahead of the US) on mobile use and quality of service (I get better reception in remote valleys and even boating around unpopulated islands), as well as internet use.

If the current job fulfils its promise, maybe we can afford to buy a summer home in greece and I can work summers there and winter here.

Re:nice to see somebody in greece is clued up

lachoy on 2004-02-06T17:57:33
I had the same reaction when we went to Greece. Except that this job is in Athens, and Athens is not a place I'd like to live...

Re:nice to see somebody in greece is clued up

TeeJay on 2004-02-06T18:19:15
there are some very nice places close enough to greece. It would be worth it for long weekends just down the road. Failing that working from home in greece would be much nicer than working from home here.

The property we were looking at (or rather my in-laws looked at and we are interested in) is in a remote village on an island miles from athens.

Re:nice to see somebody in greece is clued up

lachoy on 2004-02-06T22:01:52
oh yeah, that would definitely work. Another benefit is that people aren't as work-centered as what I'm used to (in the US), and I'm pretty laid back.