A wholly inadequate reply to an Anonymous Monk

pmichaud on 2010-04-22T21:47:27

[Reposted from . Normally I wouldn't repost but I'd like it to appear in my normal Perl 6 journal and rss feeds as well as on PerlMonks. --Pm]

An Anonymous Monk on perlmonks.org recently started a discussion on "The current state of Perl 6" in which he (she? they?) offers his views on various aspects of Perl 6 development. Personally, I find nearly all of Anonymous Monk's comments in the thread to be so devoid of logic and so far removed from reality/facts that it's hard to imagine providing an adequate reply to them.

Beyond that, I have other priorities in my life right now. Uppermost is that my family is going through a very difficult period at present -- I need to take care of them first. Rakudo and Perl 6 come in at a somewhat distant second to that. Responding to garbage being spouted by anonymous person(s) probably ought to not even be anywhere on my radar.

But at least one post in the thread bugs me sufficiently that I'd like to respond, even if it's an inadequate response. And I don't want my response buried in a thread somewhere, so it gets its own post.

Anonymous Monk writes:

Oh I'll tell you how you do that [write a grammar engine capable of supporting Perl 6]. It's very simple. You get people skilled for this exact task ! Those skills are acquired in universities(preferably good ones) where professors teach courses like "Formal Languages and Automata" or "Compiler theory". If you have a bunch of people who are open-source contributors but don't have the knowledge or haven't studied the right things ... [t]hey don't know it in theory and they're trying to put it in practice! (emphasis in original)

Who does Anonymous Monk think is working on Perl 6? Personally, I have a Ph.D. in Computer Science and taught university language courses for nearly 14 years. Damian Conway certainly also has a fairly strong university background (Monash University) and understands the theory and practice of language and compiler design. Jonathan Worthington has a degree from the University of Cambridge, where he specialized in compiler construction and formal languages. Larry Wall has a degree in Natural and Artificial Languages with two years of graduate work in Linguistics at U.C. Berkeley. Many of our other key contributors also have degrees and backgrounds in language theory and practice. I'd be willing to compare the academic credentials of this group against any other dynamic language team Anonymous Monk might care to postulate.

If you can't get real compiler people to use Perl6 and help with it, the average open-source rookie won't be able to deal with this.

Somehow I have trouble classifying Larry, Damian, Allison, etc. as being "not real compiler people". And they along with many other Perl 6 contributors have a lot of experience in nurturing open source developers and projects. If you feel Larry et al. are indeed unqualified to be working in the field of programming language design and implementation, you probably don't want to be using Perl at all, much less Perl 6.

People like Anonymous Monk spew lots of opinions about how long they think Perl 6 development should take, and then speculate on the reasons why it has taken longer than they estimate it should be taking. The speculations that crop up most often are things like "the people involved are clueless about language design/implementation" (see above), "the design process itself is flawed", "they're building on an inadequate toolset like Parrot", etc. For such people it's easy to toss out random thoughts about "the problems with Perl 6" without bothering to look at the obvious evidence to the contrary, as Anonymous Monk does above. Indeed, I'm often amused when people suggest that we should be doing things that we're already doing.

Returning to the topic of developing a grammar engine, which Anonymous Monk claims above as "very simple" and just needing "people skilled for this exact task", it's interesting to contrast his opinions with the actual development of the Perl 6 standard grammar (STD.pm6). I think STD.pm6 is also indicative of the challenges confronting all Perl 6 implementors. Consider:

  • Larry is the primary implementor for the standard grammar.
  • The standard grammar doesn't need a frozen specification to be implemented, because its implementation is considered part of the spec.
  • The implementation is built entirely on top of Perl 5 -- there are no "unproven virtual machines" involved or holding things back.

I think one could consider this to be an almost perfect situation for developing a new language implementation -- an experienced language designer/implementor working without significant external restrictions on top of an advanced programming platform like Perl 5. Yet it's been three years since Larry started working on this implementation of the standard grammar and parser, and while STD.pm6 is very powerful, it's also certainly not "finished", and has been through a number of significant refactors in its development. This says to me that the amount of time and effort involved is more likely due to the sheer audacity and ambition of what Perl 6 seeks to accomplish.

(Some will undoubtedly look at the above and knee-jerk respond with something like "If it's taken three years just to create the parser, then finishing a compiler will take far longer still and therefore Perl 6 will never see the light of day." This argument ignores the fact that other pieces are being implemented in parallel with Larry's work, that software development is not a strictly sequential process, and the obvious fact that there are already "working" Perl 6 implementations available.)

Anyone who thinks that Perl 6 is fundamentally based on traditional compiler construction techniques taught in universities frankly has no clue as to what a fundamental paradigm shift Perl 6 represents to language design and implementation. It's this fundamental change that ultimately gives Perl 6 its power, but it's also why Perl 6 development is not a trivial exercise that can be done by a few dedicated undergraduates. As TimToady on #perl6 says, "We already have lots of those kinds of languages."

Personally, I'm impressed and honored to be associated with the people who are working on Rakudo and Perl 6. I understand that people are frustrated (and even feel burned by) the long wait for something as cool as Perl 6; I share the frustration and like to think that I'm doing something constructive about it. But I also find the vast majority of suggestions, comments, and conclusions coming from Perl 6's anonymous detractors to be (1) things we've already done or are doing, (2) ungrounded in reality, (3) in direct contradiction to reasonably observable facts, or (4) attempts to discredit a product they have little interest in ever using themselves. And far too many of the comments, like the ones I've highlighted in this post, are so easily refuted with just a little bit of fact digging and common sense, it's often hard to believe anyone can be seriously making them in the first place. Yet there they are.

Returning to my original theme, I think my response here is inadequate because it leaves so many other of Anonymous Monk's claims in the thread unrefuted. I could undoubtedly spend many more hours analyzing and responding to the many fallacies and untruths in the thread, but frankly I don't believe that's the best use of my time. People such as Moritz Lenz, chromatic, Michael Schwern, and others are also writing extremely well-reasoned posts refuting the garbage, for which I'm grateful, but it's far easier for Anonymous Monk and his like to spin garbage than it is for a small number of us to clean up after it. And it does need to be cleaned up, otherwise it festers and results in even more public garbage that someone has to clean up.

I hope that this post will at least encourage more people in the Perl community to critically examine the things they hear and read regarding Perl 6, especially when coming from sources with no apparent standing or reputation within the community. And maybe a few more people will even assist in publicly refuting the garbage ("many hands make light work"), so that the sloppy thinking, analysis, and dialogue that people like Anonymous Monk post doesn't spread to infect all of Perl.

Pm

P.S.: Some may reasonably conclude from what I've written above that Perl 6 is somehow "aiming too high", that our goals should be scaled back to make something ready "right now". I have two responses to this: (1) we are making things ready 'right now', just grab any of the available packages and start working and reporting bugs, and (2) there are already 'scaled back' versions of Perl 6 appearing and being used, such as NQP or even the components that execute the standard grammar. Some of these other projects, like NQP, are being used for "real" programs and applications today; they're not simply theoretical exercises or research projects.

Others often claim that all this effort on Perl 6 would be better spent on improving Perl 5. In my experience, those of us working on Perl 6 have absolutely no qualms with seeing Perl 5 improve and continue to grow -- we welcome it. Indeed, many of the features appearing in Perl 5 today come directly from ideas and things first attempted in Perl 6, and we're genuinely happy to see that. But just because Perl 6 developers also like Perl 5 doesn't mean that doing Perl 5 core development is interesting to us, or that (in my case at least) we'd even be qualified to do Perl 5 core changes. We aren't commodity programmers that are interested in simply being unplugged from one project and into some other project that others think is more worthwhile. Personally, I'd prefer to see people who are really into Perl 5 improvements continue to work to make them happen, and that the surrounding ecosystem continue to evolve to enable that to happen more easily for more people. Indeed, this is my wish for all open-source projects, even the ones I don't find interesting or otherwise disagree with.


thanks, Patrick

rjbs on 2010-04-22T23:32:59

I am continually impressed with your calm, reasoned response to the onslaught of boorish armchair compiler designers. I don't know why people feel good about attacking other people, but it makes me feel good when I see responses like yours.

Re:thanks, Patrick

ank on 2010-06-14T19:53:04

Ricardo:

Years ago, I was talking with a few others in #perl about how cool perls features were, and you characterised that conversation as a "circle jerk." I wasn't aware of its meaning at that point, so I asked mugwump and he confirmed that it was a derogatory term.

When I see your response here, and Pmichaud's post, I have no option but to agree with your comment - this is indeed a "circle jerk."

It is not the critic who counts

schwigon on 2010-04-23T00:16:57

Thanks, Patrick, and also all those others for really doing the hard work. And Thank you for preparing that concise answer. It certainly will not fix the opinions of anonymous persons fearing they could experience something they not already know. Don't care too much for them.

If anything then their appearance just demonstrates the heavy impact you work on.

"It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood, who strives valiantly; who errs and comes short again and again; because there is not effort without error and shortcomings; but who does actually strive to do the deed; who knows the great enthusiasm, the great devotion, who spends himself in a worthy cause, who at the best knows in the end the triumph of high achievement and who at the worst, if he fails, at least he fails while daring greatly. So that his place shall never be with those cold and timid souls who know neither victory nor defeat."
-- Theodore Roosevelt

it is sad but it generates value

gabor on 2010-04-23T07:49:20

It is sad that you need to spend time, energy and feelings on answering silly or nasty accusations instead of spending those with your family or working on Rakudo but there is also a positive side to it.

You made some very good points (along with chromatic, moritz, Your Mother, petdance, CountZero and probably others) that people like me can use in other situations when talking about Perl 6 and Rakudo.

Thank you for your hard work and patience!

You rock!

A wholesome and shining reply

audreyt on 2010-04-23T09:02:25

Pm, that's wonderful on so many levels. I'd like to translate that into Chinese and repost with credit on my personal blog, if it's okay with you...

Also, reading your piece prompted me to add my own reply to chromatic++, as a practical exercise in sublimation of our collective memories.

All in all, thank you for carrying the torch forward, with a very sound design, and a much higher bus number by now. :-)

Re:A wholesome and shining reply

pmichaud on 2010-04-23T17:04:55

Audrey-

I'd love to have it translated and reposted, so please feel free to do so.

(The same goes for others who might wish to broadcast this farther and wider.)

Pm

Re:A wholesome and shining reply

audreyt on 2010-05-02T05:14:34

I've just posted a translation on http://bit.ly/awTinN - It's translated verbatim except for a final ":-)" suffix.

Thank you again for the wonderful post!

Hear, hear !

thickas on 2010-04-26T09:22:42

Read your Journal post on "Current State of P6" and wanted to express my
support for you/your work and to says "thanks" for taking on a very hard
job. I'm only an occasional Perl user - but have a friend who actively
contributes and maintains a useful segment of code in Nagios, so I get
pointed to interesting stuff like your post.
[I'm writing because I don't need to create another login out there in
the Web-verse. If there's anything useful in this, you have my
permission to edit and post as you see fit.]

There's a very impolite, but relevant response to critics in the
Computing World:
-"put up or shut up",
which translates to a nicer "Really? Show me how yours does it."

This was the stance within Bell Labs Computing Research (dept 1127).
And I'm guessing, something like this could've gone on at Xerox PARC.
It's not accidental that both these (small) groups profoundly changed
the world of computing.
Exactly as Perl, in all its manifestations, has done.

In 1995 as the Web was exploding, what were the alternatives to Perl for
CGI's and system admin?
Tcl? C? and some proprietary products.

Languages like  Java, Python, Ruby have all arrived, I'd argue, as a
direct response to perceived inadequacies of Perl 5.
I'm not sure where to put PHP :-)

But while they've garnered support/usage, do they have the same range of
use as the "Swiss Army Chainsaw"?
And how do they compare on software metrics that matter in production:
memory footprint, execution speed, robustness.

Is C++ a better, more useful C? Does it help with
Programming-in-the-large or Object Oriented programming?
I don't think so... Nor do Apple. OS/X, from the iPhone to Mac to
Servers, is written in Objective-C.
Nor do the bulk of FOSS projects: 'C' is still the lingua franca of the
Open Source world.

Perl (5) is like 'C'.
Many people think it 'could be better' and many have tried to create
replacements, but its use keeps expanding.
'C' is *really* hard to beat, it is (quoting PJ Plauger) "wallpaper -
you expect it everywhere" and has base of trained, competent/experienced
programmers.
'C' is not only 'sufficient' for the range of procedural language tasks,
it is excellent on many levels.
Like all professional tools, it is 'sharp'. You can hurt yourself badly
with high-power tools... Or handled well, produce incredible works. It's
all about the workman, not the tools.

For me, the overwhelming reason to use P5 is CPAN. This is the
definitive example of Software ReUse - which should be a cornerstone of
Software Engineering (and training in it). Somehow people end up wanting
to Invent the Wheel again themselves... Which doesn't take the
profession forward in any way. I love the small irony that CPAN is
actually a reused idea itself - based on CTAN for TeX.

The very success of CPAN, far eclipsing its progenitor, and that Tcl,
Python, Ruby etc *don't* have an equivalent illustrates two things about
the Perl Community:
- they are collaborative and 'give back' as a matter of course, and
- they are Software professionals who's interest is producing products
for others, and just amusing themselves or in the thrall of N.I.H.

It's what you do that counts, not say... "Ideas are Cheap", as was the
mantra at Xerox PARC.

Whichever of the Anonymous Monk posted the criticism of "it can't be all
that hard to write a parser" obviously hasn't tried to write a full
language. It isn't nearly as simple as a 3-4th year compiler project.
And all those "University Experts" - if they were really good at new
languages and compilers, then P5 and 'C' would be a thing of the past.
Proof by Existence :-)

Experience counts in programming:
Computing is a *performance* discipline.

Like Architecture, Surgery, Music, Art and Flying...
You need "Heads and Hands" - both knowledge/theory and experience/practice.

Professional Programming is different again:
- it is always done by a team: collaboration & co-operation is fundamental
- it's not a hobby, but paid work, where feeding your Ego isn't the
prime motivation
- it's about programming "for the next guy", who may well be yourself
in 6 months time
- it's about Programming-in-the-Large
- it's about delivering "Good Enough", which sometimes is It-just-runs,
and sometimes absolutely robust.
- and finally, its about *delivery*. Getting Things Done, not ideation
or criticism.
  Coulda, Shoulda, Woulda doesn't cut it in the Professional world.
- there's another thing to do with Professionals and Engineering:
  - Is there ever any excuse for a Professional to repeat, or allow
repetition of a know Error, Fault or Failure?
  - Engineering is about Safe, Reliable, Effective and Economical
solutions for others.

There's a very simple example to show the level of Professional Software
Engineering embodied in Perl:
cpan install.

Firstly, the source code, in 'C', is available if needed. and its
Version controlled, not lost like 40% of pre-Y2K source.
There's a well known, robust and simple process to get & install new
modules in an absolutely repeatable and predictable manner - with
over-rides when needed, not a constrained straight-jacket.
The stuff you're downloading has been testing on multiple platforms. It
comes quite trustworthy.
Everything that installs comes with its own set of *tests*.

And if it doesn't work for you, there are debuggers and more for your
programming pleasure :-)

Perl doesn't pay lip-service to good Software Engineering practices - it
just does them...
That's part of its success. It builds on strong professional
disciplines, and embeds them in its process.

The other part is that it leverages itself and work that's gone before.
It's more than CPAN and software re-use...

Finally, "Computing/I.T. is a Cognitive Amplifier".
It does for cognitive functions what machines do for muscles.

Alan Kay, inventor of Objects (in Smalltalk), is well know for saying
"Point of View is worth 80 IQ points".
Or, give ordinary people the right tools, and they can beat geniuses.

Perl has always performed this "cognitive amplifier" task.
That's worth more than Kay's 80 IQ points.
It turns a single Perl coder into a large team with best-in-the-world
solutions for many problems.

How would I have reacted to the Anonymous Monk?
- Show me what you've done that you think you have the knowledge, skill
and experience to criticise.
- If you have a point, do some work and commit some changes. Don't
waste my time and yours with hot air.

And what might Perl 6 achieve?
Coming from the 'pens' of highly competent & experienced folk, who've
already done this before, it's gotta be close to definitive solution to
the problem field it chooses to answer.

If P6 arrives with an IDE and good, integrated tool-chain, it should
scale to Programming-in-the-Large and 'real' programming projects. It
won't displace C or Objective-C, but will be a 'slam-dunk' solution to
general software practice.

But in typical Perl style, I expect the Revolution and Conquering the
World to be very low-key, even 'stealth mode'.
It'll probably pass with a whisper and people may ask, "isn't it obvious
to do this?".
Or not :-)

Many thanks for your years of hard work and contribution at a level I
could only wish for.

(From a former colleague of mine)