This post was orginally made as Portable Alien Library System on win32.perl.org. It is reposted here for wider commentary. Some additional design details, existing commentary and references can be found on that page.
Synopsis
Introduction
The Vanilla Perl Project provides a Win32 Perl and bundled compiler to build Perl modules from CPAN rather than relying on binary module package systems like PPM. Early feedback on the project demonstrate the many popular Perl modules that have dependencies on external libraries -- ones often assumed to exist on a Unix-based operating system.
This document describes the Portable Alien Library System (PALS) -- a framework for external library dependency specification and resolution. While its inspiration is drawn from Win32, its design is intended to provide benefits to module authors and users on all platforms.
Requirements
This list of functional objectives is adapted from the original Alien module and commentary in External Library Handling at win32.perl.org.
Dependency specification
The solution must provide a way for Perl module authors to specify a dependency on an external library. An important consideration is how a solution integrates with or diverges from existing CPAN methods and tools.
Library detection
Libraries may be pre-installed in well known locations (e.g. "/lib") or may be installed ad hoc. The solution will need to detect if a library (and associated headers) are available and identify the associated paths.
Library configuration
Given the wide variety of locations for libraries, the solution needs to provide an abstraction layer to other tools that ensures proper configuration. Examples include:
Binary library package generation
External library compilation from source can be substantially more time-consuming than typical XS compilation for Perl modules. Providing binaries of external libraries (whenever possible) from an online repository will provide a more user-friendly experience.
Binary library package installation
Whether compiled from source or downloaded as a binary, external modules will need to be installed such that they may be located later for library detection and configuration. The solution will need to address whether this should be platform independent or platform specific.
Design criteria
Portable
External library support is a potential problem all platforms -- not just Win32 -- and developers will be more inclined to use the solution to specify external dependencies if it addresses dependency problems portably. The solution should provide a standard interface for module developers and module users that abstracts platform and compiler details.
Modular
Library packagers may not wish to take on responsibility for creating a solution for all platforms or all compilers or may lack the necessary know-how. The solution should modularize functionality wherever possible to allow individual library packagers to work on separate parts of the solution for platforms of interest.
User-friendly
The solution should degrade gently. If library support is not available for a particular platform or compiler, the solution should fail cleanly with an explanation of the dependency problem prior to generating XS compilation failures during linking.
Maintainable
Ongoing availability or upgrades of an external library should not depend on a single individual or on tacit, undocumented packaging know-how. As with Perl::Dist::*, the published solution should contain an automated process to convert an external library from source (or binary package) to the binary package format used by the solution.
Design overview and commentary
The following design description will make more sense if you examine the Alien Ecosystem diagram.
Dependency specification
CPAN modules with external library dependencies will specify a module dependency on a corresponding Alien:: module, e.g. Alien::Foo for the "libfoo" module. Dependency management can thus be handled within the existing system using ExtUtils::MakeMaker, Module::Build or Module::Install prerequisite specifications.
Presence of an installed Alien::Foo should be considered evidence that libfoo is available. Alien::Foo must not install if libfoo cannot be found.
Library detection
Library searches will need to be done in platform specific ways. Perl Config values like "libpth" could be used as a starting point.
Library configuration
Binary library package generation
Alien::Foo::* should include the full procedure needed to generate a binary library package (a ".pal" file) for a particular platform. This should include downloading sources from a URL, any preparation work or patching, and generation of the binary.
This process could be done completely from the upstream library source, or could be a "repackaging" of another binary download, e.g. creating wrappers or definition files around a .dll on Win32.
At worst, this might just be a series of system() calls. The point is to ensure that the full process is available for future packagers to replicate, adapt or enhance.
Alien::Base should provide helper methods for URL downloading, PAL file generation, etc.
After the PAL file is generated, it can be uploaded to a repository for end-users to download for installation.
Binary library package installation
If not available already in platform-standard library directories, the library headers and binaries should be installed into the corresponding "auto" directory for the Alien module. E.g., "site/auto/Alien/Foo" for Alien::Foo. This location should be well-defined for all platforms, since it's the same as is used for XS modules.
Exact layout within this module is still open -- should there be "lib" and "include" subdirectories or should libraries and headers be combined in the top directory. Installation location of binary utilities included with libraries is still an open question.
The binary package may be either generated from source or downloaded from a URL (the default option).
Re:agreed
Alias on 2006-09-25T10:37:06
> This, indeed, was the original idea of the Alien:: namespace. I should know, I was in the meeting where this idea was discussed.
I find this odd though, because I've never seen any of this.
Based on what was uploaded to CPAN, I had come to see Alien:: as "We need this one specific thing really bad, so any evil thing you have to, just get it installed".
I never really saw it as something that co-ordinates with various platform-specific backends.
If so, I would have expected some to see some sort of Install::Binary::Driver::DebianApt type structure be implemented somewhere that the Alien:: module does a build_requires: dependency on... some combination of platform detection and platform-specific drivers.
Not this current situation where everything is lumped into the Alien:: module, which seems to be to appropriate for last-resort installation, but entirely inappropriate for large scale rollout.
So how did your original concept get turned so backwards?Re:agreed
jhi on 2006-09-27T05:26:52
> So how did your original concept get turned so backwards?
So how long have you been around this community, anyway?:-) Just kidding. Kind of.
That was the idea as I remember it. Maybe I didn't communicate it clearly enough. Maybe some other idea was thrown around that stuck more in people's minds. Maybe I should have followed up on the idea to produce actual implementations so that there would have been a precedent. I don't know.
But I still think it's a good idea. Instead of trying to be essentially a cross-platform "Configure", try to be a cross-platform installation tool. I know from experience that Configure is both a terribly good and a horribly delicate tool. It has taken more than ten years to build and it still only works in "UNIX enough" platforms. Trying to do something similar e.g. in Windows makes my stomach turn.
