DSL Confusion

Ovid on 2007-08-06T13:20:08

chromatic wrote about a Ruby programmer who doesn't quite seem to "get" domain specific languages and uses his misconception to bash Perl. Aristotle points out that this programmer still doesn't get it.

Instead, I'd recommend that people read about DSLs and try to understand what that article is really talking about.

It's not talking about programming in such a way that you have meaningful subroutine or method names. As a counter-example, here's a snippet of a DSL I implemented for an implementation-agnostic query langauge:

[
  AND(
    name   => 'foo',
    l_name => 'something',
  ),
  OR( age => GT 3 ),
  OR(
    one__type  => LIKE 'email',
    fav_number => GE 42
  )
]

Now that looks an awfully lot like Perl code and, in fact, it's valid Perl. I implemented that in Perl and it worked, but you could also send it as a string and it still would work. However, as a string, I did not eval the string. Instead, the Perl code was comprised of subroutines which, when invoked, would return tokens. The string, when lexed, would return identical tokens. Then the token stream, whether from Perl or from a string, would be driven through the same parser and produce a query object which could then query anything you needed to, whether it be a database, CVS files, XML docs, whatever.

So our little DSL could be used to query anything we needed. We needed a DSL to ensure that we had implementation-agnostic searches. Got it? It was not eval'ing any code (there were block evals for catching exceptions, but that's not the same thing as a string eval, something Cosine is mistakenly using).

A DSL is a mini-language for clearly representing and working in a particular problem domain. In the example above, the mini-language also happens to be valid code in the primary programming language I'm writing, but that's almost a coincidence in this case. Serious DSLs generally require some method of transforming the strings into actual code. I used a proper lexer/parser (Parse::RecDescent wouldn't work because I need to decouple the lexing and parsing), it ran quickly, was safe to use, and allowed one to easily write queries without caring about what you were querying.


Internal and External DSLs

Adrian on 2007-08-06T14:35:29

Martin Fowler has a nice article where he uses the terms Internal and External DSLs to describe the difference between DSLs implemented within a language and those implemented in a separate language that is then parsed.

As somebody who played a great deal with Lisp I have a fondness for the whole code-as-data paradigm that leads to internal DSLs.

(Aside - feeling generally annoyed off with this utterly f**king pointless Ruby/Perl pissing match. We're turning into the Lisp community. Look how well they did. Sigh...)

Re:Internal and External DSLs

Ovid on 2007-08-06T15:25:48

OK, I can see how I have contributed to this. Sorry about that. I really like both languages (though I know Perl better, obviously).

And to be fair, I never really considered "internal" DSLs to be DSLs. That's just good programming. I realize this is just picking apart terms and it's probably unfair of me to disagree with what is apparently a common usage of the term, even though the comparison of internal and external DSLs is comparing cheese and Wednesday. They're radically different things with potentially different target audiences. However, if people think they're basically the same thing, I'll just shrug and move on.

Re:Internal and External DSLs

Adrian on 2007-08-06T15:49:00

OK, I can see how I have contributed to this. Sorry about that. I really like both languages (though I know Perl better, obviously).

My sigh wasn't aimed at you :-)

comparison of internal and external DSLs is comparing cheese and Wednesday

I don't think it is really - maybe my early Lisp experiences have totally warped me (my father's parenthesis and all that :-)

It's been my experience that the user of the DSL doesn't care whether it's internal or external. I've met testers using Watir that don't realise they've been writing Ruby. I've had clients write their customer facing tests in Perl - and not realise they're writing in Perl.

Re:Internal and External DSLs

chromatic on 2007-08-06T17:31:41

Aside - feeling generally annoyed off with this utterly f**king pointless Ruby/Perl pissing match.

Who made it Perl versus Ruby?

It just so happens that a portion of the Ruby community (probably mostly Rails fans) seems to get very excited every time one of them creates an API and jumps up and down yelling "I MADED A DEE ESS ELL!!! WHOOPAH! YUOU CANNOT DO THIS IN UR LANGAGES HEEHEH FOOLZORZ!" That could happen in any language where a silly idea propagated rapidly.

Meanwhile, I as a polyglot programmer look at that and say "Looks like an API I could have written in a any of half-dozen languages in a couple of hours. What makes it a DSL?" and can't get a straight answer. (Seriously, the defining feature of a DSL is that it's valid code you have to eval()?) If there's something to this DSL thing, programmers in general ought to be able to talk about it in ways that make sense!

The most honest answer I could expect to get it "I don't know, but everyone else seems to be writing valid Ruby in ways that exploit features of the parser, so I just did what they did. I don't really know what that term means either."

What The Article Is Really Talking About

chromatic on 2007-08-06T17:40:09

It's not talking about programming in such a way that you have meaningful subroutine or method names.

That's unfortunate. It should be.

When you write a DSL--a real DSL, with a parser and everything--you have the chance to choose your own syntax. That syntax can, and probably should, be appropriate to your domain. (You're writing a DSL because building a little language is the most appropriate way to solve domain problems, right?)

There's nothing wrong with that.

There's also nothing wrong with not writing a DSL. There's nothing wrong with writing a good API.

However, the concerns of writing a good API are different, in some ways, from the concerns of writing a good DSL. For one thing, you already have a pre-determined set of syntax you have to use. You have more constraints (at least unless you want to discuss LL or LALR, and I don't). I think the goals of each are different, too. With regard to an API, your goals should include clarity, correctness, and simplicity. The goals of writing a DSL include expressivity and succinctness, and perhaps learnability for domain experts who are not necessarily programmers.

Those goals are not always in alignment.

Reply

cosine on 2007-08-16T14:42:53

Ovid, I think this is an excellent post. I happen to disagree with you, of course, but your contribution to this debate is appreciated. I replied this morning: http://www.cosine.org/2007/08/16/languagepowered-domain-specific-language/