Windows Perl as a first class citizen within our reach

Alias on 2009-08-17T03:21:35

I started Strawberry Perl (or more specifically, the original push for Vanilla Perl) because I wanted equality for Windows.

I spend most of my day on both Windows (desktop) and Linux (server) and I just wanted to not have to care about which one I happened to be on that moment.

While things have improved enormously, we're still not there yet. But I think we're now close to achieving something we can call "good-enough equality". I'll define this as the set of things that you would expect to work if Perl was written for Windows in the first place.

There are three things left to go to achieve this goal. They are (in order of importance)...

1. WWW::Mechanize

The continuing lack of a web scraping framework has the effect of disconnecting Windows users from the world of web services we've built. No online banking modules, no Flickr automation, or phpBB scripts, or ISBN queries, or Bugzilla queries, or Google/Yahoo/Baidu searching, or website testing, or any of the other hundreds of things that a normal internet citizen might want to do.

What makes it more annoying is that Mechanize itself actually works, it's just the test scripts that fail, because of HTTP::Server::Simple not working on Windows.

You can see the full sweep of CPAN out of bounds to Windows users here.

http://svn.ali.as/graph/HTTP-Server-Simple.svg

The good news is that Jesse Vincent is making progress on getting this working thanks to David Golden and up-and-coming Win32 build star KMX (who's also working on a new generation of bundled C libraries for Strawberry).

But the day that this starts working can't come soon enough, and I for one wouldn't mind if the mechanize tests would just skip the HTTP::Server::Simple things in the mean time.

2. BioPerl

Recently, we've seen a lot of traffic on #win32 from biologists trying to install BioPerl on Windows, but being unable to. One of the main reasons is due to the lack of DB_File, which that KMX has also helped us to fix (and hopefully will be available in Strawberry October).

3. SDL

Windows is the natural home of games, and the lack of SDL on Windows is a big deal. Any chance of providing an alternative to the PyGame behemoth (and it's attraction to younger developers) is moot until this works.

The new push to get the SDL back working again looks promising here, and I'm hopeful we'll see this resolved via some suitable Alien:: package in the next few months.

Once these three packages are working, I think we can say we have a Windows Perl that meets the standard for the types of tasks that most people do on Windows.

That will just leave crypto is our main deficit, but this isn't quite as obvious a failing as the others, and is not something we can fix in a single step.


HTTP::Server::Simple

jesse on 2009-08-17T05:07:22

I could really use more test reports from Win32 on the latest dev builds. KMX has something that works for him, but I neither use nor know a whole lot about Win32, so I really need the Win32 crowd to step up to help make this work.

(Ironically, it's not HTTP::Server::Simple that's not been happy on Win32, but its test suite.)

Re:HTTP::Server::Simple

Alias on 2009-08-17T06:04:40

I got Microsoft to build us a test farm specifically so you could have somewhere to do testing on :)

Re:HTTP::Server::Simple

Alias on 2009-08-17T06:42:47

The most recent HTTP::Server::Simple dev release installs correctly for me, and mechanize installs correctly on top of it.

Re:HTTP::Server::Simple

DiamondInTheRough on 2009-08-17T07:11:03

S_WORKSFORME on my laptop - just sent the positive report to CPANTesters.

1.60 is up

petdance on 2009-08-17T06:05:26

I just uploaded Mech 1.60 w/a skip_all for the HTTP::Server::Simple stuff.

If I may ..

systems on 2009-08-17T06:16:47

I would like to add few more packages to this list, these are some of the newer package that seem attractive

4. KiokuDB
5. Reaction

They are most likely not in popular use, but they are the most blogged about

Re:If I may ..

bart on 2009-08-17T11:11:45

but they are the most blogged about

Yet I'm not familiar with either, and I'm too lazy to look up what it's all about...

Can you introduce these packages shortly, and most of all, tell us why we should care?

Re:If I may ..

Arador on 2009-08-17T14:01:44

KiokuDB is a Moose based object database supporting multiple backend (filesystem, BerkeleyDB, DBI, etc...). You can read more about it here.

Reaction is a framework on top of Catalyst and Moose. It aims at making components more reusable. An introduction can be found here and here.

Both of them are examples of next-generation Perl projects.

Re:If I may ..

DiamondInTheRough on 2009-08-17T15:23:41

Those two look like they rely on stuff we mention in the first 3.

Reaction will be a lot closer to working now that WWW::Mechanize is now working. There isn't any obvious other bugs that I can see at first look, but I'm installing it to make sure.

As for KiokuDB, I'll have to check that one once we get DB_File working.

Re:If I may ..

DiamondInTheRough on 2009-08-17T16:04:22

MooseX::Types::DateTime fails tests, and signatures fails to build :( . I'm having to leave and let my machine find any others, however. Will check back later.

Re:If I may ..

DiamondInTheRough on 2009-08-18T02:57:03

4 more failures, including Test::WWW::Mechanize::Catalyse (which crashes), Email::Valid, and MUIR/Time-modules. (I can't recall what the third one was.

Work on the dependencies, and then the top level will work. :)

Re:If I may ..

kmx on 2009-08-18T07:58:18

It it due to the fork issue that was recently patched in 5.10.1RC1

Fork does not work together with namespace::clean on 5.10.0/win32 and there is no workaround for it.

Re:If I may ..

DiamondInTheRough on 2009-08-18T08:49:08

Forgot that detail. The one I pushed them to include, I take it?

Re:If I may ..

Alias on 2009-08-18T00:17:02

These may be novel and interesting, but they aren't important.

Re: BioPerl

cjfields on 2009-08-17T17:56:19

Funny thing on the Windows complaints. We haven't had any users popping up on the list about this problem, but that could be b/c it's a DB_File issue? Oh well. We've also had users complaining about lack of a PPM for AS Perl...

Just so you know, we're (by that, the bp devs) are planning on undertaking a major restructuring of BioPerl (been needed for years, really), into smaller, easier to maintain distributions similar to Moose. We'll probably tie it all together via a Task::BioPerl or similar for anyone wanting a larger distribution.

Re: BioPerl

Alias on 2009-08-18T00:16:26

You may not, but I've noticed a number of people showing up in #win32 trying to get it installed. We spent 2 hours a few weeks ago trying to get a cancer researcher up and running with some variant-discovery tool that didn't depend on much more than BioPerl.

Re: BioPerl

cjfields on 2009-08-19T17:48:42

You may not, but I've noticed a number of people showing up in #win32 trying to get it installed. We spent 2 hours a few weeks ago trying to get a cancer researcher up and running with some variant-discovery tool that didn't depend on much more than BioPerl.

Looks like one problem was temp file generation (and deletion) when running tests. KMX's proposed solution, though, leaves generated temp files on the user's computer. My suggested compromise seems to work on OS X just fine, though I have no access to WinXP/Vista/7. Maybe time for use to take advantage of your Sekrit?

Re: Windows Perl as a first class citizen

kid51 on 2009-08-17T20:18:38

A very good summary of the issues.

WWW::Mechanize - no problems with PPM

SuperCruncher on 2009-08-19T08:51:55

I've never had a problem using a PPM version of WWW::Mechanize on ActivePerl, so I think it's a bit misleading to say that Windows users haven't been able to use a web scraping framework. I've been using WWW::Mechanize on ActivePerl for years.

ActivePerl is far from perfect, but in some cases, I find PPM actually works better than installing directly off CPAN.

Re:WWW::Mechanize - no problems with PPM

Alias on 2009-08-19T10:24:34

The reason that PPM is able to deal with situations like this is that (so far as I can tell anyway, I could be wrong) for certain strategically important modules, they simply ignore failing tests and package the module anyway.

So yes, it works, if you work around the problem by using a third-party vendor who has packaged the module ignoring the failing tests.

These types of things are fine as temporary workarounds, but that doesn't mean that mechanize itself inherently "works".

SDL Perl

kthakore on 2009-08-23T02:15:35

It seems like there is a demand for SDL perl in windows. However I have yet to see any code for it. Ok that was mean but really we have been trying to make sdl perl with windows working for a while now. In the end we had to put a

croak "Help us!" if( $^O eq 'MSWin32');

for v2.2.1.8. If there really is an interest for SDL perl on windows come talk to us add sdl-devel@perl.org. Any help will be greatly appreciated. I will provide you will all help needed.

Don't forget Expect, please!

salva on 2009-09-01T07:41:03

Perl continues to be sys-admin best friend, and he needs Expect!

I know it is scary because Windows knows nothing about ttys and alike, but cygwin perl supports it, so it is doable.