Re: Libraries and repositories

[Matthew Flatt <address@hidden>]

Quoting Adam Turoff:
> Committees aren't the problem.  Large standard libraries aren't
> the problem.  Ease of implementation isn't the problem.  They're
> all second-order effects of interoperability (or lack thereof).

Exactly.

> If however the Scheme committee or a single
> individual were to rally around a 20-page paper on a standardized
> module system (er, http://www.htus.org/Book/Staging/how-to-use-modules/),
> then things would be different.

I think many Scheme implementors agree with this, too. It's why I've
spent the last year developing on the above module system, and why I've
started talking to other implementors about it.

Even though we agree that Scheme needs a module system, there are real
problems to solve in balancing extensibility and interoperability. The
essential problems have to do with language-extending macros that are
themselves implemented in some extended language.

I'm optimisitic about our current design path for several reasons:

  1. We've directly addressed the problem of how to load code from a
     file.

     Strange as it may sound, the mundane task of loading code is one
     of the biggest stumbling blocks to distributing Scheme libraries
     right now. Sure, you can hand someone five files, but can they get
     them loaded into the interepreter in the right order from any
     directory? Or can they feed the files to the compiler so that the
     compiler can find the bits it needs to execute at compile time?
     This important problem hasn't been given a high priority in the
     past.

  2. We've also addressed the problem of separating compile time from
     run time.

     This point is closely related to the previous one; it's about how
     a compiler, for example, finds the code it must execute instead of
     compile. But making the distinction work with an expressive,
     mostly hygienic macro system is the main technical contribution of
     our new module system.

  3. Our new module system has already solved interoperability problems
     *within* PLT Scheme.

     PLT Scheme includes many Scheme dialects --- some for teaching,
     some for experiementing, and some for special domains --- and
     getting all tools to work with all of the language variants was a
     nightmare before. (Even I couldn't always figure out how to get a
     program that ran in the interpreter to run through the compiler.)
     Now, everything "just works".

The current module design is perhaps not streamlined enough for
immediate adoption by other implementors --- perhaps with more time and
experience. In any case, if there's a possibility of making Scheme
implementations more interoperable, many Schemers expect to find it
somewhere down the module road.

Matthew