Why is PHP so much easier for newbies?
Why does Java have the best IDE tools?
Why is Ruby prettier than Perl?
Why does Perl have the best package repository?
As I've played through Mass Effect 2 over the last few weeks, I see some interesting parallels.
In the Mass Effect universe, human technology is bootstrapped by the discovery of an ancient abandoned alien observation outpost on Mars, and the further discovery that the dwarf planet Charon is really an abandoned but active interstellar jump gate covered in ice.
Other similar species have done the same, resulting in a galactic community of around a dozen civilisations all based around the same basic technological underpinnings.
Despite these civilisations believing a recently (50,000 years) extinct civilisation built the gates, it turns out the technology is perhaps millions of years old.
Every 50,000 years, the synthetic AI race that built them returns from hiding in intergalactic space to wipe out all of the existing advanced species based on "their" technology, and reset the galaxy for the next set of civilisations to rise.
In a conversation between the game's protagonist and one of these old AIs, we are lambasted by the AI for taking the shortcut on their technology. The jump gates and other technology is left in place intentionally, so that each new generation of civilisations take a controlled and predictable development path, making it easier to destroy them.
The AI posits that it is the overcoming of adversity on your own that drives true technological advancement, and that easy routes make you (technologically) weak.
I think you can see something similar in the development of the different programming languages.
Java is long and wordy, taking a long time to type. The need to work around this limitation resulted in the proliferation of powerful IDEs, resulting in the annual 20 million line of code Eclipse release train.
PHP as a web language would have been stillborn if it didn't deal competently and quickly with the need to easily deploy code, the result of which is that you can effortlessly just change .html to .php, add a hello world tag, and upload via FTP as normal (something Perl still can't do well).
Python's need to gain mindshare against an entrenched Perl led to a huge focus on being easy to learn, to a simplification of the language, and to hugely popular things such as the PyGame library and game competitions.
Faced with the lack of truly great package repository, and with a web-heavy community, Ruby became the "prettiest" language. Creating an elegant website is both expected and required if you are going to gain mindshare for an idea.
And Perl's messy syntax and difficulties in the area of maintaining large codebases, combined with a pragmatic sysadmin-heavy community, resulted in an unmatched packaging system that allowed code to be maintained in small pieces, with enormous volumes of support infrastructure around it.
The ease of publishing and trend to smaller package that the CPAN allowed conversely means that the Perl community has never really had the need for pretty and elaborate websites, and the smaller package size means that we lack the giant headline libraries that make the payoff from website work better.
Our bias towards a pragmatic tech-savvy sysadmin userbase means we haven't really provided anything like the focus on learnability that has driven Python's gradual dominance in the mindshare of the young. It takes a certain rigour in your prioritisation to intentionally remove and dumb down existing powerful features so that the language is easier to learn.
Even for Strawberry, which focuses on the userbase with the lowest traditional knowledge, we intentionally have the smallest and most maintainable website possible and we don't even have the kind of introductory screencasts that we really really need (which should be easy but which I never seem to find the time to do).
If you throw a bunch of Perl coders against some PHP coders in a website competition, it is not unexpected that when both sides play to their strengths you will see something like http://geo2gov.com.au/html?location=e.g.+1+Oxford+Street from the Perl coders and something like http://www.hackdays.com/knowwhereyoulive/postcodes/view/2000 from the PHP coders.
The former required a massive amount of data extraction, transformation, aggregation, a gigabyte-sized PostGIS database, and deployment via a Linux virtual appliance to Amazon EC2 to allow for strategic load-shedding.
The latter required the ability to turn data into presentable and understandable information for real humans, and to make it pretty enough that they WANT to look at it.
Driving true technological progress, then, may often be about identifying weaknesses that are hard to solve but aren't completely impossible (and don't have any crippling long-term conceptual flaws at an economic or project-management level).
The three best projects I have driven - PPI, Strawberry, and (in part) Padre - all share this property. All three of these represent hard but not impossible problems, and require an awareness about which issues are intractable and which issues merely exist because there's been no need to solve them any better.
Padre in particular has suffered greatly from issues with Wx quality and threading. But given the low takeup of both threading and Wx it was reasonable to move forward on the basis that these would be fixed once there was something depending on them, and driving a need to fix them.
All of our early problems are gone now, and there is continued pressure to find ways to improve our use of (and the efficiency of) Perl's native ithreads.
Similarly, the creation of Strawberry required a lengthy year-long process of fixing Win32 bugs in all kinds of toolchain and low level modules, because we'd never had a proper working developer feedback loop before.
Similarly, Perl's current push for marketing and blogging and websites is directly resulting from Python's success in mindshare capture.
So my question for you to ponder this week is the following:
What can you see that Perl as a whole struggles to do well, for which a good solution is not impossible, and is only being held back by smaller problems which would go away if there was a working candidate solution put in place that needed those small problem solved.
The fact that you have to have adept-level Perl 5 experience to understand how to use Perl 5 (the language) in the fashion most useful for novices is a huge problem.
Re:Novice Accessibility
Alias on 2010-02-09T09:08:33
So what do you feel is the solution, and what are the small problems that would go away once we had a working candidate solution?
Re:Novice Accessibility
chromatic on 2010-02-09T19:27:02
As you suggest, a CPAN configuration usable by novices from a freshly-installed Perl 5 is important.
Moose installed by default is another piece.
The
strict,warnings, andfeaturepragmas on by default is the most controversial, but perhaps the most rewarding. I can hear the angry buzz of replies in the distance, but imagine if the only piece of little magic we had to explain to novices was "If you want to maintain this code for a long time, add ause 5.12;line to the top to identify which version of the Perl 5 syntax you used."All of a sudden, Perl itself would help the novice catch typos and avoid odd syntactic constructs that don't do as a new programmer might expect. We'd likely have to review the default error messages for clarity (as you've mentioned), but we may be able to halve the debugging time for the first real program a novice writes.
That's significant.
Re:Novice Accessibility
gabor on 2010-02-10T06:58:58
The only major problem with this is that most of the novices - at least those I encounter - need to use old versions of perl.So the usefulness of this would only be seen later down the road and only by some of the new users.
But one needs to step on the road to get there.
Re:Novice Accessibility
Aristotle on 2010-02-14T22:55:06
If we never get around to it because “it won’t help users now” then we’ll still be saying the same thing in 10 years. Sure, we’ll have to find interim solutions, but let’s not sacrifice the future for the more pressing present concerns.
I don't try Padre because (1) it's a pain in the ass to install on Snow Leopard via CPAN or
Re:Wx/threading?
tsee on 2010-02-09T08:27:14
What does your rant have to do with your subject line? Is the default OSX Perl compiled without threads? How does your rant address Adams post?
PS: I'm a vim user and will always be. I write C++ more or less all day. Yet when I have to refactor an icky bit of Perl, I switch editors because I'll take all the help I can get. Works fine for me. Doesn't have to work for you. Tough luck, get on.
How does your rant address Adams post?
educated_foo on 2010-02-10T03:39:28
"... and Padre changes the Perl community." I'll let you figure this one out for yourself.
I use Emacs, but am interested in editors, so I check them out from time to time. I tried Padre last month, and had to substitute my own Perl to get it to launch (I forget if it was a threading or 64-bit-only issue). I eventually ended up with a half-assed text editor with some non-compelling features.
In what way is Padre changing the Perl community?
gabor on 2010-02-10T07:12:21
You really don't have to explain why you are not trying Padre. The majority of the Perl community have not tried it (yet) and that's totally ok.IMHO "changing the Perl community" did not mean every Perl hacker now will have to use Padre.
I never thought Padre would catch any measurable part of the vim/emacs users to change their editor. Maybe once in a while, as tsee mentioned, they will use it and maybe they will recommend it to novices.
So the major change can come from:
- New people coming in to the Perl community who are not familiar with vim/emacs and still getting a reasonably good Perl aware editor.
- Existing Perl hackers (including vim/emacs users) understanding that novices might be better off with Padre than a text editor.
Padre, the Perl IDE is coming mainly from me I think. You might excuse me of being a fan boy of Padre.
Re:In what way is Padre changing the Perl communit
educated_foo on 2010-02-10T21:34:03
Existing Perl hackers (including vim/emacs users) understanding that novices might be better off with Padre than a text editor.
At least in the long term, they aren't. Someone adept at hacking FORTRAN, C, Perl, Java, LaTeX, and BASH is much more employable than someone who can only hack Perl.
Padre, the Perl IDE [perlide.org]
LOL. Even replying to a post criticizing your shameless pimping of some Perl module, you couldn't resist pimping it.
Re:Wx/threading?
Phred on 2010-02-09T17:41:56
I was able to install the snow leopard dmg successfully, and fire up padre without any problems.
However, Padre is still in it's infancy, and it is premature to say that it is changing the Perl community. It can't compete at all against my tricked out vim and emacs installations which have all sorts of perl ide helpers installed such as autocompletion, syntax checking, built in debugging, perl tidy, etc.
Saying that it *can* do everything that other perl ides do is actually different than being able to do those things.Re:Wx/threading?
Alias on 2010-02-10T01:23:07
I didn't say "is changing", merely "changes".
The concept/implementation of Padre taken over long time scales will change (and has started to, but only in tiny corners here and there) the nature of the Perl community.
I don't claim that this change has happened yet in any major tangible way.
But I see it coming...
Re:Wx/threading?
Phred on 2010-02-10T03:33:17
"I don't claim that this change has happened yet in any major tangible way."
I disagree - it's happened in a very tangible way. I should not have been overly critical; the fact that I could install a dmg and take it for a spin is huge. I'm downloading to my laptop right now ti spin it up again (installed on my desktop a couple weeks ago to test).
Re:Wx/threading?
Phred on 2010-02-10T03:35:33
Just installed it on my laptop - this is pretty badass actually!! Nice work!!
I tend to think of this along the lines of "making hard things easy and impossible things merely hard". With that in mind, the frontier has shifted since Perl was developing rapidly.
My gut feeling is that the "wow" factor has become harder for Perl to achieve because what wows people isn't insanely great string, dataset and administrative task manipulation. People want to see results quickly. That's not just about the learning curve -- it's the end-to-end process.
Part of the advantage of Java in an educational setting is that you get a decent GUI quickly which is more like "real programs" for anyone except die-hard CLI addicts. Ditto PHP, but for dynamic web pages on shared hosting.
Here are some areas where I think Perl is not as easy as it could or should be. A lot revolves around CPAN, since that is one of the strengths of Perl and we're not making it easy enough for people to take advantage of it:
Installing CPAN modules locally
CPAN(PLUS) should be designed for the case of a non-root user installing modules locally for use with the system Perl. local::lib has the right idea, but even bootstrapping it isn't the easiest things for someone new to Perl. What it does should have been built into CPAN(PLUS) ages ago. (I have a feature branch for CPAN.pm that provides this capability and will be looking to merge it after Perl 5.12 is released.)
Automatic, transparent CPAN configuration
The latest dev of CPAN.pm is much less annoying and verbose that it used to be. It does a pretty quiet auto-configuration and can even automatically locate mirror sites. But it could be better yet. A lot of the guts should be hidden from the ordinary user.
CPAN popularity metrics
CPAN has nearly 20,000 distributions. Even search.cpan.org gives dozens or more results on a search. Figuring out which of several similar modules is time consuming, even for experts. Combinations of CPAN Ratings, CPAN Testers results, etc. help. The top 100 list is another useful tool. But we need something better that either ties all these things together or replaces it with something else.
Shared hosting website deployment as easy as PHP
I can't do anything but draw the parallel. It makes the jump from HTML to a dynamic site really easy. No messing with frameworks, templating, HTTP engines. Doesn't HTML::Mason work more or less that way? It doesn't have a mod_php to go with it, but I wonder if something in between HTML and MVC would help people who want simple dynamic sites (backed by the power of CPAN.
Improve parallel processing and leveraging multi-core systems
Perl threads still suck. The best alternative is to fork off the entire interpreter and cobble together some way to collect results? (Parallel::Iterator, for example). There's got to be a better way to do this.
Smart defaults in core
Not just strict being on by default, but the core modules themselves should be a smart set of tools for modern programming tasks. I'm not saying the core should have the kitchen sink, but there should be some basics that come installed with any Perl without making users mess with CPAN. Things like HTTP, SSL, DBI (and probably SQLite) would be good places to start, I think. I'd like to see a standard OO included, but suspect that's too controversial to achieve.
How's that for starters?
-- dagolden
Re:making hard things easier
tsee on 2010-02-09T08:38:28
"Improve parallel processing and leveraging multi-core systems"
I agree about this. It's important going forward.
Also: Point me at a dynamic language that does this better. No. Not Python. They don't do concurrency. They do the same as Coro does and call it threads.
As for extending the core: We both know that's against the opinion of the powers to be. And the powers have the good reasons for their agenda. The correct way to do this is an extended core project, either via EPO or not. Have releases of that with every new perl, plus some. Then convince the distributions that they want a meta package for the extended core. It'll stand a small chance of catching on.
Re:making hard things easier
chromatic on 2010-02-09T08:42:46
The correct way to do this is an extended core project....
Exactly; that's the best way to do it well in any sense. If Strawberry has demonstrated anything, designing for the the utility of end users is inestimably important.
Re:making hard things easier
nicholas on 2010-02-09T09:59:45
As for extending the core: We both know that's against the opinion of the powers to be.
The "powers to be" aren't powers. We're just the people who do the work, rather than the talking.
If people feel that there should be a bigger core distribution, please branch http://perl5.git.perl.org/perl.git, add what you think needs adding, advertise it, get it established, demonstrate that you can support it on an ongoing basis, and if it's proven better than the current approach for the core distribution, then likely actually your version will be folded back as the official core distribution.
Actions speak louder than words, as Strawberry shows.
Re:making hard things easier
tsee on 2010-02-09T10:18:28
Nicholas, you took this wrong. But I'm also grumpy this morning, so maybe I just phrased it wrong.
"The "powers to be" aren't powers. We're just the people who do the work, rather than the talking."
They're powers for that very reason. I'm perfectly fine accepting the decisions of those who do the work. I think that's the only realistic way to run a large volunteer-only project. But you quoted only part of my comment. In the next sentence, I acknowledge that there's good reasons for this stance. These reasons are specifically related to:
"demonstrate that you can support it on an ongoing basis"
Which I understand to be the main idea behind making the very core as slim as possible. As it stands, it's all an act of balancing "ease of maintenance" vs. "out of the box user experience". Given the constraints on volunteer maintainers, erring on the side of maintainability is the only way to go. That and the fact that I do not have the time budget to fork the core doesn't mean I can't *wish* for a "batteries included" approach. Maybe, on a spare week-end, I'll try to come up with a mechanism that allows people to drop tarballs in some directory to have them build alongside the core. That would make the "so shut up and do it yourself, it's easy, just some work" argument more compelling.
"Actions speak louder than words, as Strawberry [strawberryperl.com] shows."
I'm not sure why you throw that at me. I've been involved in that to some extend by fixing win32 bugs in various modules when the whole thing was in its infancy. So thank you very much.
Re:making hard things easier
nicholas on 2010-02-09T10:33:51
I just phrased it wrong.
Yes, just that. Sorry for annoying you. I know that you contribute massively to keeping the dual life modules in shape. I'm not as aware of the other things that you do.
But I don't like the phrase. Particularly as it can be taken out of context, and likely will be some people who read it. It implies that the people referenced have the power to change that policy. Whereas the policy is totally defensive - we're already stretched beyond what a volunteer group can manage. If we tried to support more we'd soon collapse in failure. It's simply not an option. Hence we don't have "power".
Re:making hard things easier
mauzo on 2010-02-09T13:45:06
Maybe, on a spare week-end, I'll try to come up with a mechanism that allows people to drop tarballs in some directory to have them build alongside the core.
I thought that, at least in theory, you could just untar a distribution in ext/ and tell Configure to build it? I've no idea if this still works with the ext/ dist/ cpan/ rearrangement, though.
Re:making hard things easier
tsee on 2010-02-09T14:05:53
I thought that, at least in theory, you could just untar a
distribution in ext/ and tell Configure to build it? I've no
idea if this still works with the ext/ dist/ cpan/
rearrangement, though.Almost, but not quite. It shouldn't be hard if you have any idea how the perl build process works, but the point would be to enable doing this without any special knowledge at all. Off the top of my head, you will...
... have to make sure the build process of the module doesn't do any strange things that aren't compatible with a non-installed perl. ... have to decide where the module is to be installed: core lib, site lib, vendor lib? ... have to add things to MANIFEST. ... probably hunt down and torture anybody who tries to run /usr/bin/perl, /usr/bin/env perl or possibly just "perl" during the module build process. There was more I don't remember. But I hope it's not a *huge* deal.
Re:making hard things easier
Alias on 2010-02-09T14:48:38
This is only going to work for trivial cases.
Strawberry currently has 50 odd extra modules, mostly crypt, build/cpan upgrades, and database clients.
It's already bloody hard to maintain the list of dependencies by hand.
You really need to be able to bootstrap from core -> ( list of classes ) to maintain sanity.
Just installing Padre takes 100 packages and the dependency list changes (both up and down) on a weekly basis.
Re:making hard things easier
dagolden on 2010-02-09T14:23:38
I agree that it's not fair to blame "the powers" -- and, frankly, tsee and I are also among that crowd these days. We need more dialog on what is achievable and supportable and what not. But I think saying "fork perl and maintain it" is an overreaction.
I was the person that first built and released Strawberry Perl, so I have some experience on this topic. It was actually fairly easy. Build Perl, drop in a pre-configured CPAN::Config, and install extra stuff right from CPAN. The "hard" part was getting everything to work on Win32.
That sort of approach could potentially be done directly in the Perl core -- though probably building from bundled tarballs, not downloading directly from CPAN. I suspect a lot of dual-life modules outside the module toolchain could be packed in core as tarballs and built/tested by the toolchain rather than by the hybrid process that happens today. Maybe one day I'll have enough round tuits to make that happen.
Even without that, I think the (admittedly recent) trend of Perl's development is making it easier to coordinate dual-life modules in core. One of those trends was giving out commit bits to dual-life maintainers like me, so updates don't all fall on the pumpking and a handful of committers. But the more recent trend of time-boxed 5.11.X releases means that having the absolute latest/greatest version on CPAN also in core isn't as much of a release hurdle. I think we can do more to make the process smoother.
I'm not suggesting throwing everything and the kitchen sink into core. But I do think there needs to be a renewed discussion of what "out of the box" functionality should be in core. One that I feel adamantly about is that CPAN should be able to bootstrap itself via HTTP without requiring any external command line tools. That is my #1 project for Perl 5.13.
Hopefully, I have enough track record as a "doer" not just a "talker" to do a credible job of it and convince people that this will make life easier for users and maintainers.
-- dagolden
Re:making hard things easier
gravity on 2010-02-09T17:11:37
"Improve parallel processing and leveraging multi-core systems"
I agree about this. It's important going forward.
Also: Point me at a dynamic language that does this better.
Clojure is the only dynamic language that's done the multicore thing properly. To do that required a dramatic rethinking of how a language should be structured, leading to real innovations in how data structures are represented by the language internally as well as a software transactional memory system built on top of it. While I suppose it's possible to implement these things in perl, it'd become a radically different language if you did so.
Shared hosting CPAN module deployment
robinsmidsrod on 2010-02-09T09:10:07
You're idea about deployment as easy as PHP is spot on. The PHP concept of a web-based installer is really intuitive to work with for a novice.
I've been pondering this idea here:
http://github.com/robinsmidsrod/unnamed-perl-cms-project/blob/master/README#L82
If the novice could just download and install a web-based CPAN installer/updater to put on their shared hosting account and then just view that UI in their web-browser a lot would be done. To avoid the potential problem of a CPAN compiler fail (or lack of compiler) we could use binary PAR packages customized for the specific platform/distribution.
Obviously, we'd have to build some kind of automatic build system that compiles these binary PAR packages and publishes them in the public PAR repo. I'd suggest a stable/testing/unstable approach as Debian has to give the option of stability level.
The installer could also be configured automatically by pointing it at an URL with a configuration file specific for the application you're planning to use (maybe META.yml?).
If we have this we'd have an easy way of dealing with dependencies and the end user would just have to deal with the actual application installation procedure, not all the complications of using CPAN in a shell (which they are not familiar with, or might not even have access to).
Re:making hard things easier
nicholas on 2010-02-09T10:06:21
On parallel processing:
There's got to be a better way to do this.
No dynamic language has found it yet. Even 2 full time Google engineers have failed to find it for Python
CPython has a Global Interpreter Lock, single threads the interpreter, and hence single threads compute intensive task. [C]Ruby has "green threads" (i.e. it does the thread scheduling). There's no prior art. There's no proof that the problem even has a solution.
Re:making hard things easier
tsee on 2010-02-09T10:21:16
It's really hard to imagine a solution. Particularly in perl. In the face of automatic conversion of SVs on read access in new contexts, this seems utterly impossible without crazy locking.
Re:making hard things easier
gravity on 2010-02-09T17:13:18
As I noted above, Clojure is a dynamic language that provides a solution. There is prior art, it just doesn't look or feel like Perl/Python/Ruby.Re:making hard things easier
nicholas on 2010-02-09T17:18:50
Sorry, I wasn't clear enough. My fault. Prior art for a dynamic language implemented in C. Clojure is a dynamic programming language that targets the Java Virtual Machine. (To the best of my knowledge) Jython does threads just fine - it's CPython that can't, and even the folks at Google working on Unladen Swallow can't see how to take that Jython know-how and port it to the C implementation.
Re:making hard things easier
gravity on 2010-02-09T17:28:11
I don't think there's anything stopping Clojure from being implemented in C if they actually wanted it. The problem for CPython and Perl isn't that they're in C, but that the entire language is based on the idea of default mutability. Clojure was designed from the ground up with a different multicore-focused approach to state in mind, which is why it works. You'd have to fundamentally change Python and Perl as languages, no matter what their implementation, to get them to do handle threading well.Re:making hard things easier
nicholas on 2010-02-09T17:35:42
Good point.
Sadly, once again the punchline from the "Irishman giving directions" joke bites:
If I were you sir, I wouldn't start from here.
Re: get a decent GUI quickly
Eric Wilhelm on 2010-02-09T21:22:53
You mention this about Java's success, but many of the pain points you've elaborated on are details pertaining to the CPAN and deployment.
If Adam is right about bigger solutions being the impetus to solve more detailed problems, then ease of learning/setup/deployment for the CPAN should have been solved years ago. Or, maybe I'm mis-interpreting.
As for my "candidate solution", making "gimme a GUI" quick is something I've been trying to do with FreeTUIT. After years of developing with wxWidgets, I've gotten progressively fed-up with a series of its flaws and concluded that its oversized abstractions and cross-platform oddities make it unworkable. Now that PerlQt is finally available under an LGPL license (so you can legally take it to the office), I'm exploring that as a new foundation for FreeTUIT.
GUI programming does run up against a problem with threads in high-level languages. The simple example is non-blocking usage of a web API (which doesn't strictly require threads, but they could greatly simplify the usage.) More complicated is e.g. drawing into two widgets in parallel.
Re: get a decent GUI quickly
Alias on 2010-02-10T01:27:49
If you are going to move forwards with PerlQT, just make sure you solve (early) the problem of building and installing it on the three main desktop targets effortlessly (Win32, Mac, *NIX Gnome/KDE) because if you can't install a desktop program on the three main desktop environments, you lose a lot of traction immediately.
One thing that the Perl community starts to excel at is reflexivity - the religious attitude of Perl as the ideal and every critique as a heresy is subsiding and people start to understand the shortcomings - which is the first step towards overcoming them.
Answering your question - I concur robinsmisrod's comment on easy web deployment
Re:Advanced analog field
zby on 2010-02-09T11:33:43
Or rather I concur all the points by Dagolden above.
I recently thought of installing Padre on the most recent CentOS (which, in case you didn't know, is a community supported enterprise version of Red Hat Linux). The huge number of modules that need an upgrade is discouraging: it wants to upgrade 24 modules (and install 81 more), even after I already installed Wx.
The main problem I have with that is that I don't want to change anything in the system Perl, as this is contrary to the spirit of CentOS: don't upgrade anything that is stable, as the upgrade might break stuff.
What I want is to "upgrade" these modules only for Padre, i.e. locally.
Now what dagolden already mentioned would easily solve that problem: CPAN should be set up to install locally, i.e; not modifying the system libraries, by default, or at the very least, provide a painless option to do so.Having to mess with arguments for Makefile.PL with sometimes dubious, or at the very least, unpredictable results, is not the best imaginable solution.
p.s. I recently upgraded Padre to the newest version on Windows, and again, it wanted to upgrade tens of modules. Is that really necessary? It's so bad, eh, there are so many of them that CPAN almost chokes, that I almost skipped the upgrade.
Re:Installing or upgrading Padre is a pain
Alias on 2010-02-09T14:54:23
Often the new modules are needed.
There's a lot of work "on Padre" that actually goes into various dependencies, resulting in new releases and thus new dependencies.
The ORLite family alone has gone through 10-20 releases to add embedded database features that Padre needs.
0.56 was a bit more aggressive than most, since it bumps Module::Build and ExtUtils::MakeMaker to current, had some ORLite upgrades, and a few other minor things.
Some releases are worse than others, but it's our preference to push development forwards as fast as possible. We don't expect CPAN installers to be our primary userbase, we expect the binary package users to ultimately be that.
So we make some considerations for downstream packagers, but not for CPAN users.
download stand-alone binary version of Padre
gabor on 2010-02-10T10:08:57
We are aware of the fact that it is difficult and time consuming to install Padre from source hence we are encouraging the downstream distributors to include Padre and its major plugins.In order to allow you to use the latest (or almost latest) Padre on Linux we also have an experimental version of it that includes everything you need - even a threaded Perl.
See the Linux section on the download Padre page.
Re:download stand-alone binary version of Padre
bart on 2010-02-14T17:54:21
Interesting approach — if only it worked... It turns out not to be so easy.
I tried the "experimental" "Padre Stand Alone for Linux", but it immediately died, complaining about an incompatibility between Wx and
libstdc++.so("GLIBCXX_3.4.9 not found").Anyway... Do you see this approach as a solution for all big projects in Perl? Building all these binary distros seems like a lot of work to me. Plus, with several projects, you get a lot of separate file trees, with possibly a lot of overlap. That may be a bit of a shame, depending on how you look at it.
Re:download stand-alone binary version of Padre
gabor on 2010-02-14T18:03:03
I am not surprised it did not work - I tested it only a very limited number of platforms. (Namely 1) but still I am sorry to hear that.As I understand in order this to work we need to build a statically linked perl and wxwidgets and apparently this was not the case.
I think this can be one approach though I hope this could lead to a package similar to Strawberry Perl Professional but for Linux and similar OS-es which would mean other desktop application could use it as a platform.
I still think inclusion in the Linux distributions is another important approach, besides the source code distribution from CPAN.
I've used perl for:
Data munging
GUI's
Real-time, high availability, telecomms apps
One-off parsers many & various
Simulations & customer demos
Web-services & clients
Code-generation
Fun
& so on
Perl is admirably fit for any soft-dev task you have in mind, subject to performance constraints in certain situations. It may not be a *perfect* fit in all contexts but then again expectation to the contrary is probably not reasonable.
As for supposedly not being fit for large codebases - well the only empirical evidence I am aware of points the other way.
From the point of view of extra-community perception Perl's single biggest failing is that it didn't have Sun behind it in the early/mid 90's. The second biggest failing is that OO is not Perl's one true way & hence perl missed out when Comp Sci, Soft Engineering courses were designed in the mid-90's (See http://www.cse.wustl.edu/user/loui/praiseieee.html).
For my money Padre is the best way to tackle problems of perception. First impressions are important so a smart environment that:
Installs quickly and easily.
Is integrated with CPAN.
Comes bundled with usable examples covering a wide range of app domains.
Wizards to build dynamic web page generators etc.
Plays nicely with the web (e.g stream job ad feeds to a detachable console)
would all help to give first-timers a nice experience. The ability to customise the functionality & appearance would appeal to the more confident coder as well.
As far a way to influence academic hearts & minds then bio-informatics might be a way in. What extra could Padre offer the bio-perlist?
Re: Why Ruby is prettier and Padre changes the Per
Alias on 2010-02-09T14:56:09
The Strawberry team is going to bundle BioPerl in the default install of Strawberry Professional, which will also come with Padre pre-installed.
So we can offer than a zero-effort bootstrap into a BioPerl environment.
FYI
educated_foo on 2010-02-11T10:07:56
Speaking from experience in both academia and bioinformatics, BioPerl is the opposite of a selling point; it's over-engineered and half-implemented, almost always more trouble than it's worth. Perl is attractive for its text processing and system scripting. If you want to make Perl more attractive to biologists, make it easier to interface with C and R (e.g. via Inline::).
Re:FYI
mrStlee on 2010-02-11T11:01:05
Which Perl-wrapped C libraries would be interesting to biologists?Re:FYI
educated_foo on 2010-02-11T13:07:50
SVM-Light and the NCBI tools come to mind.
Re:FYI
mrStlee on 2010-02-11T15:58:57
There is a perl wrapper for SVM-Light:http://search.cpan.org/user/kwilliams/Algorithm-SVMLight/lib/Algorithm/SVMLight.p
m There are also perl & python wrappers for the NCBI tools:
http://www.bioinformatics.org/forums/forum.php?forum_id=6735
Given the extensible nature of Padre I'd imagine a customised version that ships these & presents dialogue boxes, tree views etc to interface them, wouldn't present an insurmountable challenge.
Re:FYI
educated_foo on 2010-02-11T17:13:49
That's completely useless for large datasets:
$s->add_instance
(attributes => {foo => 1, bar => 1, baz => 3},
label => 1);C is a good least common denominator, so it helps to make a scripting language's interface to C as painless as possible. XS is hardly "painless," so there's room for someone to create such an interface.
Re:FYI
mrStlee on 2010-02-11T17:29:12
Perhaps for large datasets you could load them from a file:read_instances($file)
?
What other features would you like to see in the interface? It may well be possible to wrap/sub-class Algorithm::SVMLight to tweak the interface.
If you have anything specific in mind I'd guess the author of the module would be happy to have a look.
Interface
educated_foo on 2010-02-11T17:44:52
A function to read data from a TSV/CSV file (in C, without going through Perl) would be extremely useful; ideally,
$filewould be a file handle rather than a name, so I could pipe it from standard input. A function to operate on adouble**generated by some other C library would also be useful.More generally, my point is that biologists need to be able to interface with many, many programs, and you can't expect canned interfaces to all of them to be available on CPAN. These programs are often UNIX commands, in which case writing a text file and calling them works. But if not, you can usually talk to them in C.
Re:Interface
mrStlee on 2010-02-13T11:20:02
I am not sure I get your point entirely. I think there are 2 cases:1) Where you'd like to interact with other unix processes via the pipe mechanism
2) Where you'd like a quick way to wrap C-libs which provide certain features.
1) Is something Perl excels at. I'd recommend you have a look at the Perl Cookbook, recipe 16.4 "Reading or Writing to Another Program". Or Gabor Szabo's Pipe module might do the trick:
http://search.cpan.org/user/szabgab/Pipe-0.03/lib/Pipe.pm
2) Is more problematic since to be able to wrap C-libs you have to have type conversion between the C-types & perl's types. That said if the types concerned are all standard char's, int's etc then the whole thing is much simplified. For example there is module called P5NCI which can be used to wrap libraries without XS.
I'd be very interested in looking at ways to make perl easier for biologists to use. I dress both ways as it happens - a qualified Marine Biologist who ended up in software development.
If you have any specific cases in mind I'd be happy to look at them and see if there is any chance to solve more general problems.
Re:Interface
educated_foo on 2010-02-21T02:58:57
My point is that a non-lousy interface to SVM-Light needs to handle large datasets. Algorithm::SVMLight was clearly written by someone who never used SVMLight on a decent-sized dataset. Such a dataset will almost always contain hundreds of megabytes of data, and come from either (1) a text file you download or (2) a C or FORTRAN function you call.
I don't think specific cases will help here. Here's the general problem: I have one million labeled data points generated by some program, and I want to use them to train a classifier. If I use the current lousy interface, I perform one million Perl function calls, and either run out of patience, or run out of RAM and crash. I don't want to do that.
Re:Interface
mrStlee on 2010-02-26T17:51:43
This is what the man page for Algorithm::SVMLight says:
"read_instances($file)
An alternative to calling add_instance_i() for each instance is to organize a collection of training data into SVMLight's standard "example_file" format, then call this read_instances() method to import the data. Under the hood, this calls SVMLight's read_documents() C function. When it's convenient for you to organize the data in this manner, you may see speed improvements. "
The way I read this is that assuming your data can be massaged into SVMLight format there is only the single perl call.
If so then the problem shifts to how to massage the data set into the SVM format.
So the workflow would look something like:
Generate data set
Translate to SVMLight form
Call "read_instances()"
It wouldn't be difficult to produce a sub-class of Algorithm::Light that recognises some known data formats and does the necessary translation before calling read_instances.
Would that help?
*facepalm*
educated_foo on 2010-03-01T03:05:02
How did I miss that? (Probably because the synopsis only used add_instance(), and I skimmed the rest too fast.) SVMLight format is pretty simple, so it's not too hard to dump your data in that format and then call read_instances(). So one minor suggestion -- adding instances in bulk, particularly for training, is far more common than adding them individually, so it should be in the synopsis.
FWIW, when I wrote an Octave binding to SVM-Light some years back, I used direct calls to the SVM-Light C interface (init_doc(), custom_kernel, etc.) to add a whole batch of instances. It was more work, but way more efficient (and flexible!) than serializing and going through the file system.
Re:*facepalm*
mrStlee on 2010-03-04T14:58:39
To be frank I'd have thought so too. Useful SVM's based on a few instances can't be all that common.
I've a theory that one reason the re-use revolution promised by the OO evangelists never happened is that the effort and ingenuity involved to work out what s/w you can re-use can make it risky effort-wise to even try. I wouldn't say this is anyone's fault in particular - there's just too much information to wade through.
If I understand you correctly you had Octave generate batches of instances in the required format and then pumped them directly into the SVM-Light engine? Something similar could be added to SVM::Light I'd think.
Re:*facepalm*
educated_foo on 2010-03-05T11:25:02
the effort and ingenuity involved to work out what s/w you can re-use can make it risky effort-wise to even try
I had much more hope for what people called "component-oriented programming" in the 90s -- large pieces of functionality with very simple interfaces. Small-grained objects are a symphony of fail.
If I understand you correctly you had Octave generate batches of instances in the required format and then pumped them directly into the SVM-Light engine?
Actually, I allowed 2-argument Octave functions as kernels, shoved the relevant data pointer into SVM-Light's data pointer, and used SVM-Light's
custom_kernel()interface to call back into Octave. I don't remember if I special-cased the standard 2-norm kernel, but that should not be too hard.Re: Why Ruby is prettier and Padre changes the Per
dearfrankg on 2010-02-09T15:13:03
We need to rethink the installation of perl programs. Waiting for cpan to do its thing is wrong! Compare installing a perl program with installing a program on WIndows or OSX.
Unless the install is as simple as these other methods perl will always have an image problem.
Perl installs remind me of the scrolling text when booting up the old unix box, but worse because it's interactive.
Sorry I don't have a fix for the problem.:) Re: Why Ruby is prettier and Padre changes the Per
Hercynium on 2010-02-09T22:36:58
For a while, I've had rattling around in my brain an idea for being able to do installations of Perl apps, from CPAN, with incredible flexibility.
I've yet to come up with a simple "pitch" for the idea, but let me try to describe...
First: Layouts
Different systems have different conventions for where applications, pieces of applications, and other related files et al. are to go. One may want to comply with the LSB, the FHS, Debian's policies, or whatever layout Windows or Solaris may dictate. Perhaps an admin wants to use a Stow-style layout, or even something they've come up with custom!
Next: Distributions
Typically one desires to obtain and install a particular distribution of software or data, or *something*. There are different types of distributions one would want to install: libraries, frameworks, CLI apps, GUI apps, web apps, servers, etc, etc...
Could it be possible to create an installation framework that can allow one to easily install any type of distribution with any type of layout and have the installed stuff Just Work(tm) while still allowing end-user customization of the process?
I've made past attempts at trying to envision the best way to go about it, but a combination of lack of tuits, and a seeming lack of demand has kept me from really trying to tackle it. One thing I'm certain of is that it would require creating Yet Another Build System. In addition, for things to work properly, distribution authors would probably be required to parameterize a lot more things, ie. a Lot More Work.
It's a dream, but hey, isn't that what code is made of?
:-) Re: Why Ruby is prettier and Padre changes the Per
Alias on 2010-02-10T01:29:49
Wrap a pretty gui over the scrolling text so people don't see it
A common thread in these use cases is that different module versions should be able to co-exist in an installed perl distribution. The most common cases are probably system perl, developer perl, and bundling an application's dependencies.
How to add this complexity without making it a burden? Allowing us to keep around all versions of modules? Have different levels of overlays beyond site_perl?
However implemented... It would be nice if the solution didn't require completely separate Perl installs or a revision control system.
Re:easier install/maintainance of Perl distributio
DiamondInTheRough on 2010-02-09T23:47:44
#5: in Strawberry's case, we provide the C/C++ compiler - so it's really not external, but it is provided for those things that need it. For most non-Windows operating systems, I do have to admit, Perl assumes that a C/C++ compiler is accessible for a lot of things.
:) And "Strawberry Professional" will have a lot more things prebuilt.
We "do" have a way to ship binaries, at least for one module at a time - it's called a
.par file. How good it is for your purposes, I don't know. PAR::Dist is the handler for those, and maybe it should be included in an extended-core bundle.
