Re: semantics of ?=

[Gabor Greif <address@hidden>]

Bruce Hoult wrote:
> 
> In article <200209151605.g8FG5xML008065@life.ai.mit.edu>,
>  Gabor Greif <gabor@mac.com> wrote:
> 
> > We had quite some discussion on IRC channel #dylan about what the
> > intensional name capture by ?= should accomplish.
> 
> Indeed.
> 
> > My idea is: ?=name in a macro expansion behaves exactly the same _as_if_
> > the users code had \name in that position.
> 
> Absolutely.  The question is: who is the user?

Every identifier that is mentioned by a macro template (or pattern default)
and that is not a toplevel identifier in the macro definitions environment is
hygienic.

All other identifiers mentioned by dylan code are unhygienic.
This includes normal macro patterns (e.g. ?name) that resolve to an
unhygienic name or an unhygienic pattern (e.g. ?=it) occurring in a
macro template (or pattern default).

I think this is the definition that we should stick to.


> 
> I believe that the user is the person who typed in the call to the
> macro, A, whose expansion contains a use of ?=.
> 
> Gabor seems to believe that the user is the person who types in a call
> of a macro C, which in its expansion uses macro B, which in it's
> expansion uses macro A, which in its expansion uses ?=.
> 
> In my opinion, Gabor's intepretation violates several basic tenets:
> 
> - the principle of least surprise.

Macro systems are surprising. Those of C and Lisp are especially surprising.
Hygienic macro systems are less surprising unless you _want_ the surprise.



> 
> - the principle that a macro that evaluates each of its arguments
> exactly once, in the appropriate order, has the same semantics as a
> function with the same body.


As long they refer to the same _bindings_ they do coincide. Since we have
a hygienic macro system, alpha-conversion is valid for macro definitions too.
Unless you use ?= anywhere, you can name your bindings in macros just like
you want.


> 
> [snipped examples of how Gwydion Dylan currently works, and indeed how
> it would *continue* to work under my interpretation]
> 
> > ------------------------------------------------------------
> >
[snipped stuff we do not differ on]

> 
> Now, it would be reasonable to use awhile() in a function, rather than
> at the top level:
> 
>   define method pig-out(where)
>     awhile(poll(where))
>       eat(it)
>     end
>   end;
> 
>   pig-out($fridge);
> 
> Works fine.  Try it.
> 
> It would also be reasonable to turn pig-out into an inline function, or
> a macro.  Let's try a macro:
> 
> ---------------------------------------
> module: capture
> 
> define macro awhile
>   {awhile (?exp:expression) ?:body end} =>
>     {for (?=it = ?exp then ?exp,
>           while: ?=it)
>        ?body
>      end}
> end macro;
> 
> define variable $fridge = as(<deque>, #("pizza", "steak", "yoghurt"));
> 
> define method poll(x)
>   ~x.empty? & x.pop
> end;
> 
> define method eat(x)
>   format-out("Eating %s\n", x);
> end;
> 
> define macro pig-out
>   {pig-out(?_where:expression)}
>     => {let where = ?_where;
>         awhile(poll(where))
>           eat(it)
>         end}
> end;
> 
> pig-out($fridge);
> ---------------------------------------
> In Top level form.:
> "capture.dylan", line 25, characters 15 through 16:
>               eat(it)
>                   ^^
>   Error: Undefined variable: it
> ---------------------------------------
> 
> Oops!  ?= working the way Gabor likes it has injected "it" directly into
> the top level and it is not visible inside the expansion of pig-out.

Nothing got injected. You defined a macro that mentions an identifier "it".
But a name "it" is not defined in the same toplevel environment as the macro
"pig-out". So "it" in macro "pig-out" becomes a hygienic name. This decisison
happens at macro-definition time, regardless of the knowledge whether \awhile
is a macro or a function.

You have the same situation with C++ templates (which can be regarded as a
hygienic macro system too when looked at from far enough).

template <typename BASE>
struct derived : BASE
{
  int i;
  derived() : i(Initially) {}
};

This will never compile (with a standard-compliant compiler) even if all BASE
candidates define some scalar Initially value in their scope.

You have to explicitly qualify Initially to be coming from BASE:

template <typename BASE>
struct derived : BASE
{
  int i;
  derived() : i(BASE::Initially) {}
};



> 
> I think it is surprising and wrong that perfectly reasonable code that
> uses awhile() suddenly stops working if you use it in the expansion of a
> macro.

Actually anaphoric macros are a surprise that they work at _all_. They appear
much of a hack to me, even though they are useful under some viewpoint.
However they are not the only way to get identical functionality.

You have other rules for macros than for functions. To make your example work,
you have to explicitly qualify "it" to be a non-hygienic reference:

define macro pig-out
  {pig-out(?_where:expression)}
    => {let where = ?_where;
        awhile(poll(where))
          eat(?=it)
        end}

Otherwise a reader seeing the definition of \pig-out _without_ seeing the definition of
\awhile would have a hard time to understand where "it" is supposed to come from.
This way he at least would suspect that some secret interworking between \pig-out and \awhile exists.

Consider the case where "it" _had_ a toplevel binding where \pig-out is defined
and the macro was defined thus:



define macro pig-out
  {pig-out(?_where:expression)}
    => {let where = ?_where;
        awhile(poll(where))
          it := ?=it;
          eat()
        end}

define variable it = #f;

define function eat()
  format-out("Eating %s\n", it);
end;


With your interpretation the assignment would be a no-op. And you would always
see #f output.

Or: The provider of the awhile macro chose to also unhygienically
inject "eat" into the ?body scope. You would automatically see that
binding. I'd say this would be a much greater surprise _at_runtime_
than a solvable compile error.



> 
> This behaviour will prevent a *huge* number of the macros in _On Lisp_
> from being converted into Dylan.

Please show us an example where this cannot be cured by using ?= in the outer
macros definition (and no special, i.e. thread-global variables are involved).

> 
> I believe that use of ?=it in awhile() should refer to a variable of
> that name in the textual environment in which awhile is mentioned.  It
> is the author of pig-out that is the "user" of awhile, NOT the person
> who wrote "pig-out($fridge)".

As explained above, no identifier that is introduced by a macro can be regarded as
a user (i.e. unhygienic) identifier. Unless marked specially, of course.

Just my humble opinion,


	Gabor


> 
> -- Bruce