Re: Extensible syntax

[Chris Page <address@hidden>]

On Dec 11, 2003, at 20:20, Tom Locke wrote:

>>> There's Dylan, with its template based macro system, but Dylan 
>>> syntax is only extensible within a fairly strict framework. It 
>>> always looks like Dylan. Agree?
>>
>> Somewhat. You can't change the lexical grammar, and pairs like () {} 
>> [] must all be balanced. However, you can certainly define DSLs that 
>> look quite different from the standard Dylan statement macros.
>
> Can you define new operators for use in general Dylan statements? 
> Prefix/postfix/infix/mixfix?

No. That’s what I meant by “you can’t change the lexical grammar”. You 
can’t add lexemes, which most operators are. Within a macro body you 
can define “keywords” that conform to the lexical grammar for 
identifiers, which is pretty liberal compared to many languages, and 
you have some freedom to use the predefined operators for other 
purposes.

Going forward, I think it’s pretty clear that extending Dylan’s 
flexibility in this regard is desirable. In the meantime, Dylan’s 
pattern macro system is fairly elastic while making it very easy to 
write straightforward macros. I think the best macro system is probably 
one with two or three layers of detail, starting with basic patterns 
and including a separate layer for lexical extensions. I’d certainly 
prefer this to always having to write macros procedurally.

> I don't think you can because it doesn't fit with any of the ways that 
> Dylan notices 'aha - this is a macro'.

Macros support three basic patterns: definitions, functions, and 
statements, e.g.:

	define foo ... end			// definition macro call
	swap( ... )					// function macro call
	while ... end				// statement macro call

Each must start with some kind of identifier, like \foo, \swap, and 
\while. Function macros must contain parenthesis, which mark the 
start/end of the macro call body, and definition and statement macros 
must end with an “end”. Beyond that, however, there is quite a bit of 
flexibility for the syntax supported within the macro call body.

The dom-builder macro starts with “with-dom-builder”, for example, but 
the body of the macro call is quite unlike typical Dylan fragments. 
That is, the start/end of a macro call must be quite “Dylan-like”, but 
the macro body supports much more flexible DSLs.

As a random--and untested--example, I think you’d be able to define a 
DSL macro that supports executing Unix shell commands like so:

	shell-dsl
	  echo "Hello, world.";
	  ls | less;
	end;

Here, “echo”, “ls”, and “less” are valid Dylan identifiers, “"hello, 
world."” is a valid Dylan string, and the vertical bar “|” and 
semicolons “;” are valid Dylan tokens (the logical or operator and 
statement separator, respectively). In contrast, however, you couldn’t 
use the text “++” to define your own C-like pre- or postfix increment 
operator, since that is not a valid Dylan token.

But, as you can see, it’s (probably) flexible enough to define 
something very similar to a typical Unix shell language (which could be 
useful as an alternative to constructing strings and calling system() 
if you’re issuing a lot of shell commands).

-- 
Chris Page - Software Wrangler - palmOne, Inc.

   Dylan + You = Code
   <http://www.gwydiondylan.org/>
   <http://www.cafepress.com/chrispage>