Oslo Hackathon 2008 -- wrap-up writeup

dagolden on 2008-04-09T19:03:53

First, I'd like to thank The Perl Foundation for sponsoring my trip to the hackathon. I'd also like to thank Linpro for providing a wonderful venue as well as on-site food and refreshments throughout the hackathon. This is my wrap-up summary so they have a sense of what their contributions helped make possible.<\p>

Project 1 -- Test::Reporter transport plugins<\p>

I worked with rjbs on adding "transport" plugins to Test::Reporter to provide better options for how reports are sent to CPAN Testres. The existing transports (Mail::Send and Net::SMTP) were extracted into separate modules (Test::Reporter::Transport::Mail::Send, etc.). Plus we created three new transports:<\p>

  • Test::Reporter::Transport::HTTPGateway -- provides HTTP to email transport via a the new Test::Reporter::HTTPGateway module<\p><\li>

  • Test::Reporter::Transport::Net::SMTP::TLS -- provides TLS and authenticated SMTP (a long-standing wishlist item)<\p><\li>

  • Test::Reporter::Transport::File -- reports saved as files in a directory for offline CPAN Testing and manual submission <\p><\li>

Unfortuately, the HTTPGateway transport is only a temporary step towards a better CPAN Testers as the gateway still just relays to email. But people can set up their own gateways for now or perhaps a central server will be deployed somewhere.<\p>

Using transport plugins for all of these with a consistent API makes it easy for clients like CPAN::Reporter to switch between them based on configuration settings. E.g., in .cpanreporter/config.ini<\p>

transport=HTTPGateway http://gateway.example.com/http2email/

Project 2 -- CPAN Metabase for CPAN Testers 2.0<\p>

The big issue with CPAN Testers "1.0" (what we currently have) is that it uses email to cpan-testers@perl.org for submissions and the only archive for reports is effectively the perl.cpan.testers newsgroup. For a while, I and others were talking about creating CPAN Testers 2.0 to move to HTTP submission and a centralized, indexed, searchable database.<\p>

I'd hoped to interest people in Oslo in thinking through what it might require or even working on some pieces of it. What I found was that there are a lot of areas where people are looking to collect 'meta' information about CPAN distributions, each with their own approach and APIs for gathering, storing or publishing in a very application-specific way.<\p>

That got me and rjbs started designing a more general solution, which we've called a CPAN 'metabase'. This would work for CPAN Testers 2.0 or for other projects that want to store/publish distribution-level information.<\p>

After a few days of work, we achieved:<\p>

  • General design of a system (class hierarchy, architecture, etc.) to meet at least initial envisioned needs with the flexibility to expand over time<\p><\li>

  • A simple, working proof-of-concept -- with filesystem-based storage and indexing and the ability to store and retrieve a simple "fact" (a text string) via a web client.<\p><\li>

Next steps are to refine, standardize and document the classes and APIs; to add additional capabilities to the proof-of-concept; and to get a 0.01 release to CPAN for community feedback.<\p>

Other contributions<\p>

  • Moral support for Adam Kennedy's time-travelling efforts to build the April Strawberry Perl

  • Feedback on TAP diagnostic semantics and the implication for downstream consumers like CPAN::Reporter

  • Figured out a hack with the Berkeley DB libraries to help Tux build DB_File on Strawberry Perl

  • Participated in a best practice discussion/argument on environment variables, 'xt' directories, etc.<\p><\li>

Things that came up that I didn't get around to working on

  • CPAN::Reporter::Smoker -- a couple people (Gabor and someone else?) said it would be helpful if C::R::S could take a specific list for a work queue instead of trying to smoke only the latest things on CPAN. Now that someone asked for it, that gets bumped up on my priority list<\p><\li>

  • Salve brought up using boilerplating as a way of discussing and later disseminating best practices so I showed him a boilerplate tool in my repository (not yet released). He and rjbs thought it had promise given easy customization so I'm going to try to finish the documentation and get a 0.01 release out soon<\p><\li>

  • Schwern brought up the CPAN::PERL5INC taint bug in recent devel versions of CPAN.pm<\p><\li>

  • Various people were working on codifying and releasing modules for various parts of the PAUSE/CPAN tools. rjbs has a work project along these lines and brian d foy's work to index BackPAN is driving similar efforts. There was some discussion of making it a YAPC::NA hackathon project<\p><\li>

Other things I learned (not necessarily Perl)<\p>

  • git is awesome. I had been looking for a project to force me to live with it and get over the learning curve. Since Test::Reporter lives in git and rjbs knew it already, I got my chance. rjbs and I were routinely developing simultaneously on the same code base, merging back and forth several times a day and despite the huge flux in our early development code, it "just worked"<\p>

    The best part was that during the flight home, we continued hacking, using a git repository on a usb drive to exchange and merge our branches. We'd just walk it between our seats about every hour or so<\p><\li>

  • Developing Perl on a Ubuntu virtual machine on Windows is a vastly better, faster experience than native Perl on Windows. (It would be interesting for someone to figure out what's behind that)<\p><\li>

  • rjbs showed me the wonders of screen (and screen + irssi). I don't know how I could have gone so long without learning how cool it is.<\p><\li>

  • Norwegians are friendly. The organizers, Linpro staff and people on the streets were welcoming and helpful whenever we need it<\p><\li>

Other thanks<\p>

  • To Pobox (via rjbs) for a temporary mailbox for testing TSL and authenticated SMTP<\p><\li>

  • To the BBC for backstage.bbc.co.uk swag<\p><\li>

  • To jonasbn for being a sounding board as rjbs and I drew crazy metabase design diagrams

  • To rjbs for putting in a tremendous effort in a short period of time on projects that he wasn't even focusing on before the hackathon and for teaching me a lot of new tools

-- dagolden


Thanks

tsee on 2008-04-09T20:26:55

Hi David,

thanks for the write-up. You guys really did accomplish a lot of stuff in the short time.

Cheers,
Steffen

P.S.: Somewhen, I need to get off my arse (read: take some vacation and set aside the cash for travelling) and participate in some hackathon.