The growth in the argument that the DarkPAN is somehow some form of boogieman, some wispy vaporous nothing that is unobservable and thus something able to be ignored astonishes me. And I'm not singling out one person here, I'm hearing it from a number of different people.
It's especially unnerving because a big chunk of it is sitting in plain view.
So let me repeat this again.
Until perhaps very recently, the DarkPAN was generally considered to be more or less all the Perl code that wasn't on the CPAN. Because if it wasn't on the CPAN, it was largly impossible to find. And even if you could find it, you couldn't do anything with it.
With the advent of PPI and Ohloh, the situation has now changed.
Now we know, for example, that just the small part of the DarkPAN that is visible is at least 4 times bigger than the CPAN. Because 4 times more code than CPAN is basically what Ohloh managed to uncover. Last time I looked, it was reporting about 97,000,000 lines of code, and we know that the CPAN is in the general range of 20-25,000,000 lines of code.
I've called this semi-visible part of the DarkPAN the GreyPAN, but that's just a way of categorising it. It's still a part of the DarkPAN. It's still code we can't feed into CPAN Testers or other automated systems to examine how a Perl change will impact it.
We also know that it's mostly not large codebases. Anecdotal reports from the Ohloh people themselves suggest it's made up of lots of different bits and pieces. So in fact, it serves as quite a useful analog for the consistency of the DarkPAN itself.
It's something we can suck in, process, and then run what-if scenarios for. At least at a document-level anyway.
Frankly, I'm sick and tired of this back and forth crap.
I'd like to see both sides of this argument shut up (me included) until we've actually built the thing that can suck in the visible parts of the DarkPAN and process it.
If you'd like to help, I've got most of the "Process it" part working now, but I'd dearly love some help on the "sucking it in" part.
If, after we've finished, we're able to establish that (for example) across all C code there is on average 1 line of utility Perl code for every 100 lines of C code then we've got something to work with.
Because we can take estimations about the total amount of C code in the world and extrapolate from that to make a guess at the total amount of Perl code in the world.
No competent project manager would agree to manage something unmeasurable.
My argument remains that attempting to do so with regard to the DarkPAN is futile.
Re:Agreed
rafael on 2009-07-05T21:46:56
Insistance is futile, you will not be assimilated?
Re:Agreed
chromatic on 2009-07-05T22:23:27
If you can't observe the DarkPAN, its contents, its needs, and the consequences of any change to Perl 5 are opaque. I believe that basing design decisions on the inobservable DarkPAN is a mistake.
Observing (at least part of) it gives the Perl community a better chance to evaluate proposed changes in documentation, training, best practices, idioms, workarounds, core library changes, and core language changes.
We may still disagree about the nature and scope of changes as well as their most appropriate or useful locations, but at least we'll have data to evaluate. That must be an improvement over intuition, instinct, and isolated personal experiences.
Re:Agreed
dearfrankg on 2009-07-06T09:19:41
Maybe the solution a two versions of perl.Re:Agreed
chromatic on 2009-07-06T14:54:55
That idea crossed my mind yesterday. Instead of enabling syntax features at runtime depending on the presence of a command-line flag or
use feature 5.10;, use#ifdefto compile in or compile out features. Build one binary without new syntax asmaintperlor something and the other asperl.This would increase the testing and smoking burden -- and that's a very real consideration for maintainability -- but it provides a backwards-compatible binary as well as a modern binary.
Re:Agreed
educated_foo on 2009-07-06T16:31:08
Or more. That seems to make sense when there are too many egos for one project. Linux does something like this, with the various branches (e.g. "-mm") coexisting with Linus's tree. Of course there's the usual drama, but they somehow manage to keep sharing patches.
Re:Odd to consider everything outside CPAN the sam
chromatic on 2009-07-08T07:33:50
It's not deliberate. It's that analyzing all code on the CPAN is so much easier than analyzing code anywhere else that the latter seems intractable.
Re:Odd to consider everything outside CPAN the sam
lini on 2009-07-08T14:33:16
With the analysis tools being made publicly available, input can be solicited from the community at large, and from a sampling of known users. As this is an old, long-standing problem, experiences from other development communities can be used. With the greater amount of interconnection today, it should be easier now than in the past to enable feedback and data to be collected.Re:Odd to consider everything outside CPAN the sam
chromatic on 2009-07-08T17:15:47
I agree and I welcome that.
