And how expressive are they if we ignore speed?

[Jakub Travnik <address@hidden>]

And how expressive are they if we *ignore speed* ?


Image a language that is dynamically typed, have high order functions,
concise lambda syntax (aka blocks), and no macros.

Following purposes of macros were mentioned previously:
1, affect evaluation order
2, introduce literal data
3, introduce bindings

(compile time computation is skipped since we ignore speed)

Let me summarize solutions to them in such language:

1, evaluation order:
Blocks can be used to affect evaluation order. If they are enough
succinct this is ok.  Moreover this is non-issue in pure functional
languages.


2, Literal data:
They can be constructed with little overhead of explicit quoting.
Or as Avi Bryant shown expression to SQL query conversion in
Smalltalk, the structured data can be created by calling
functions/methods that will construct it for us.


3, bindings:
Block can do this too if we want local behavior of introduced
bindings.

Example:
Scott McKay shown us some transaction code.  When sane exceptions are
in language, things like transactions can be done naturally with
blocks.

in Ruby:
class Database
  ....
  def transaction
    success=false
    conn=DBConnection.new(self)
    yield conn   # call implicit block
    success=true
  ensure         # this is called 'finally' in java
    if success
      conn.commit
    else
      conn.rollback
    end
    conn.close
  end
end

calling:

 db.transaction {|tr|  # tr is the binding (actually local variable)
   tr.do_this
   tr.do_that
 }

Other example, in Ruby:
[1, 2, 3].each {|x|
  puts x
}
which does same as following code without use of macros, in lisp:
(dolist (x '(1 2 3))
  (format t "~A~%" x))



So is there something where macros are irreplaceable?
Yes.


We have omitted some use of macros, we have centered our view to
hygienic macros and missed a variable capture! Or in the other words,
using calling environment and/or wrapping new environment without
specifying used names.  This is the thing that language imagined at
the start does not support.

Yes. These macros are hideous and obscure, but still they are
convenient if they are well documented and understood.

Some languages have imitation of them by using global variables (since
they are really practical).

Example:  $_, $1, $. in Ruby (and Perl)

Why are they useful?

You usually don't want to use them, but occasionally you do. But
passing them all the time around is not good way.

imagine following user code for matching string against regular
expression (perl like style) in common lisp:

(match "12:34:56" "([0-9]+):([0-9]+):([0-9]+)"
  (format t "~A hours, ~A minutes and ~A seconds" $1 $2 $3))

notes:
1, $1 is not global so you can nest without problems
   (and call others code, they might match too!)
2, You don't have to pass $1 explicitly as block/lambda parameter.
   This is about the convenience.
   There may be much more of them $1start, $1end, $1length
   (they are easily computed but rarely used).
3, $4 is not defined since macro can count required number of them when
   regular expression argument is constant (usual case with r.e.).
   (safety)
4, We can use some other mechanism to transfer values, i.e. Ruby
   can match via both global variables and in OO way: the result of
   match is a object and you can ask for extracted parts, but almost
   no one is using this way, it is too verbose (succinctness is power).
5, (offtopic, but we live in real world)
   Effectiveness - compile time computation
   In pure functional language, partial execution might be viable
   solution to this to.
6, Lisp special variables with lambdas can probably do 1 and 2,
   but code would be too verbose.


But the imagined language can do this too if we have enough reflection
in run-time. Adding just caller-stack access with ability to read,
enumerate and modify variables and ability to inject variables to
block/lambda is sufficient to have more powerful system than the one
that macros provide. (oh yes, injecting variables to block/lambda
could make someone mad and will be considered harmful)

Inverse example: Ruby does not have good reflection of class
objects. You can add methods to class in run-time through syntax:

class SomeExistingClass
  def newmethod
    ...
  end
end

But there is no Class#addmethod method to add methods (via
block/lambda) to class (scoping rules would be probably problem here)
via program itself. Current solution is to use eval with string
containing the code above (if ruby have had macros it would be done
probably by macros).

When you will be designing your next language, give users ability to
do everything, I mean everything, that is normally done through syntax
to be done via reflexivity of runtime (if you don't care about speed
much).


So you may see me as being anti-macro. No, I'm not (yet).

I still use macros a lot in lisp code, my macros begin with
lightening syntax macros such as:

(defmacro fn (params &body body)
  "Shorthand for definig lambda functions"
  `(function (lambda ,params ,@body)))

(defmacro rfn (name params &body body)
  "Shorthand for defining recursive lambda functions, example:
   (funcall (rfn me (x) (if (< x 2) 1 (* (me (- x 1)) x))) 10) "
  `(labels ((,name ,params
	     ,@body))
    #',name))

(defmacro let1 (name value &body body)
  `(let ((,name ,value)) ,@body))

(Why wait for the Arc? We can have succinct lambda with macros.)

And here are some more complicated:

(defmacro unrolltemplate* (1stlevel 2ndlevel nameitemlist &body body)
  "Expand body multiple times with names replaced by respective elements of list
   in parallel way, set 1st level form start to 1stlevel and
   set 2nd level form start to 2ndlevel. If some item list is shorter, last item is reused.
   Names are substituted linearly, head entries may reference tail entries.
   If 2ndlevel is t, extra parenthesis are omitted.
   Example:
   (unrolltemplate* (if3 x) (- y) ((x (1 2 3)) (y (10 20))) (snafucate x y))
   --->
   (IF3 X (- Y (SNAFUCATE 1 10)) (- Y (SNAFUCATE 2 20)) (- Y (SNAFUCATE 3 20)))"
  (let ((result nil)
	(count (reduce (fn (prev ni) (max prev (length (second ni))))
		       nameitemlist
		       :initial-value 0)))
    (loop for i from (- count 1) downto 0
	  do (labels ((tree-replace (tree name item)
			(labels ((tree-replace (tree)
				   (cond ((consp tree) (cons (tree-replace (car tree))
							     (tree-replace (cdr tree))))
					 ((eq tree name) item)
					 (t tree))))
			  (tree-replace tree))))
	       (let1 partlist (reduce (fn (mybody ni)
					 (tree-replace mybody
						       (first ni)
						       (let1 itemlist (second ni)
							 (if (>= i (length itemlist))
							   (car (last itemlist))
							   (nth i itemlist)))))
				     nameitemlist
				     :initial-value body)
		 (if (eq 2ndlevel t)
		   (setf result (append partlist result))
		   (push (append 2ndlevel partlist) result)))))
    (append 1stlevel result)))

(defmacro unroll* (nameitemlist &body body)
  "Expand body multiple times with names replaced by respective elements of list
   in parallel way. If some item list is shorter, last item is reused.
   Names are substituted linearly, head entries may reference tail entries.
   Example:
   (unroll* ((x (1 2)) (y (3 4))) (snafucate x y))
   --->
   (PROGN (PROGN (SNAFUCATE 1 3)) (PROGN (SNAFUCATE 2 4)))"
  `(unrolltemplate* (progn) (progn) ,nameitemlist ,@body))

(defmacro unroll (name items &body body)
  "Expand body multiple times with name replaced by elements of items list"
  `(unroll* ((,name ,items)) ,@body))

They are powerful especially when nested.


Moreover, in the real world we do not ignore speed, so compile time
execution has sense.

Jakub Travnik
jabber://jtra@jabber.com

UNIX is like Sex: If you don't know it, you don't miss it. But if you
know it, you'll need it. -Anonymous