A few common barriers to mainstream acceptance

[Eric Kidd <address@hidden>]

Since everybody's thinking about why good languages get adopted (or
rejected) by the mainstream, I'd like to point out a few common barriers
to acceptance I've seen along my way.

I program in Scheme and C++ (at my day job), and C, Python, Perl, Java,
Goo and Dylan (at previous jobs, and in my spare time).  I used to write
custom software for small businesses.

Here are some of my experiences.

* Network Effects Matter (Part I: Maintenance)

If you're delivering software to the average customer, you really need
to write it a popular language.  There are just so many companies and
small businesses who are not prepared to deal with something unusual. 
They need to be able find programmers on a short notice, figure out how
the code works, and fix things.  

Powerful new languages usually only make sense for early adopters.  And
many powerful new languages turn into dead, crufty backwaters, so a lot
of early adopters get burned.

Popularity snowballs.  You've got to get at least as far as Python was a
few years ago before the vast majority of organizations have any
business touching your tools.

* Network Effects Matter (Part II: Libraries)

Which would you rather do?  Write a dBase III/FoxPro file parser from
scratch in Common LISP, or use Perl's DBI::XBase?  The real world is
full of gross little problems that are better handled by a library.

Similarly, if you're programming for Gnome, KDE or Windows, important
parts of the system API probably haven't been wrapped for your favorite
language.  So you may save lots of time with your high-level language,
but you'll loose it all as soon as you need to start changing the fonts
in your text widgets (it's amazing how much time you can waste patching
PyGTK and making sure your users are all running the right version).

Fortunately, this problem isn't impossible to fix.  You could, for
example, add the basic C data types to your language (perhaps hiding
them in a library), and spend three weeks learning to parse C headers
(and grok C preprocessor macros).  Or you could link against Python and
allow programmers to call Python libraries from your language (which
takes less than a week).  Or you could decide to live with Microsoft's
"patent every parameter to the .NET API" approach, and target the CLR. 
Any of these approaches would probably work.

* Familiarity (Part I: Syntax)

Unless you're trying to support programmatic macros (a very worthy
goal), your syntax is basically arbitrary.  And if you sit down and make
a table, you'll find that the popular languages share a lot of syntax,
at least within a given user community.  Java, for example, is basically
a subset of LISP with a C++-ish syntax slapped on it.  The exact same
language using ML's syntax, or s-expressions, would have been far less
popular.

* Familiarity (Part II: Semantics)

If I haven't written any Prolog in a while, it takes me a *long* time to
get back up to speed, because Prolog has virtually no semantic overlap
with any other language I use--and because I've never been an especially
good Prolog programmer.

If I haven't written any LISP in a while, it's a lot easier, because
LISP supports pretty much any programming style I want to use (although
a new LISP user wouldn't necessarily know where to look).  Perl, Python,
C and Java support the "lowest common denominator" styles quite well,
too.

What's true for crazy programming polyglots is far more true for average
developers--switching to a new language is *hard* if there's no semantic
overlap with whatever they already know.

* Shared Libraries

This is a technical detail, but an important one.  Many otherwise nice
compiled languages don't allow you to patch a bug in a shared library,
recompile that library, ship it, and not break applications in the
field.

This is easy for scripting languages.  C, C++, Java and the CLR can all
handle it to some extent or another.  But Dylan (to pick on a language I
love dearly) has an almost unfixably bad shared library model.  And it's
not alone.

If your compiler produces a 5MB statically-linked "Hello, world"--or if
every build of your runtime library has a different ABI--then your
language is actually *inferior* to C++ in many common situations, no
matter how many nice features it has.

Summary:

  1) It's hard to justify using niche languages.  So don't design your
     language to be a permanent niche language; design it to be a
     mainstream language which you'd really enjoy programming in.
     (Python goes surprisingly far in this direction, with closures,
     list comprehensions, and generators.)

  2) Try to get as many libraries as humanly possible, even if you
     need to steal somebody else's.

  3) If at all humanly possible, choose a familiar, clean syntax.
     First impressions matter.

  4) Allow programmers to use the techniques they already know
     (in some form or another), and make it easy for them to get up
     the initial learning curve.  They can learn how to use
     advanced features later.

  5) Don't overlook unsexy--but really important--features like
     shared library support.

I guess I'm claiming that programmers are perfectly happy to have sexy
new language features, provided the basic language looks really easy to
learn.  And many cool languages (LISPs, Schemes, some strongly-typed
functional languages) could be packaged as Trojan horses and appeal to
enough people to get traction.

(Of course, it never hurts to have a killer feature or two--we chose PLT
Scheme (over Python) for our latest project because we really, really
needed some custom syntax, and we didn't want to write a whole language
from scratch.)

Cheers,
Eric