Concept Programming vs. Lisp

Concept programming is about finding good representations of concepts. One important consequence for tools is that you need extensible programming languages, because there is no fixed set of concepts. Discussions about the design of XL highlight this extensibility a lot, because this is what distinguishes XL from languages such as C++ or Ada. Several Lisp programmers pointed out that Lisp was already an extensible language, and concluded that I was merely reinventing the wheel. More precisely, I got the following comments, which I think deserve some better answer than the previous one I wrote:

Anyone writing a language today who isn't familiar with Scheme, Haskell, and ML may as well throw in the towel right now.

Reinventing the wheel is not a bad thing

To a large extent, designing a programming language and methodology today, like most of engineering, involves a lot of wheel reinvention. This is not a bad thing. Most people would be vastly unhappy if the designers of the Boeing 747 had said: "Let's not reinvent the wheel, we'll use the same old trusted design Romans used for their charriots".

A similar line of reasoning is "Lisp (or some other language) can do it, so why bother?" Well, assembly language can do everything. The reason we invent higher language designs is because we can't write large programs in assembly without making mistakes, or within a reasonable amount of time. The limit is not the language, it is the programmer. That is what concept programming focuses on.

Concept programming in Lisp

Concept programming is not restricted to a particular language. The first demonstration of a concept-programming tool was Moka, which is Java-based. Similarly, you can do concept programming in Lisp. Furthermore, Lisp already has all the attributes it takes to build concept representations at various levels of abstraction.

Of all existing programming languages, Lisp might be one of the closest to the ideal. In particular, Lisp programmers are used to not mixing levels of abstraction (like having pointers in the middle of a high-level algorithm), and are used to using multiple methods to extend the language (like modifying the parser, adding macros, doing program rewriting, etc). All of this is good for concept programming.

For instance, integrating new concepts such as objects in Lisp (CLOS) was much easier than in C (C++ or Objective-C). To me, Lisp is an important validation of the idea that language extensibility is a key to long-term survival. If past history is any indication, 20 years down the road, people are likely consider Java a language of the past, and to have replaced it with fancier languages. But they will probably still use something they call Lisp. However, that Lisp might be as remote from today's Lisp as Common Lisp is from a 1970's implementation. The point here is that Lisp is a language that evolves, whereas most other languages die a slow death.

Yet Lisp is far from perfect. At its most fundamental level, it is full of tiny concept casts. For instance, the names "car" and "cdr" (and I realize there are more modern names for the first element or tail of a list) are taken from the name of registers on some long forgotten computer model. Fundamentally, an addition in Lisp is represented as (+ 1 2); this is the output of the infix parsers. Similarly, Prolog extensions to Lisp such as Schelog or Poplog require the use of a Lispish syntax which only a mother or Lisp user could love :-)

A different approach to reflexion

For Lisp programmers, the main differences between XL and Lisp will not be extensibility. It will be day to day syntax and concepts. On the extensibility front, XL implements reflection very differently than Lisp.

Lisp has a very simple yet effective reflective model:

• Programs are represented by a single data type, lists.
• There is a direct, 1-to-1 relationship between internal program representation and standard program syntax: (+ 1 2) is the syntax for a program segment adding integers 1 and 2, as well as the syntax for a list representing that program.
• Lists are directly executable, without a programmer-visible translation step. Meta-programming is achieved by generating lists and then executing them immediately, so meta-programs are generally embedded in the program itself.

XL has a more convoluted, but more powerful and user-friendly meta-programming scheme:

• Programs are represented by an object-oriented parse-tree..
• The user-visible syntax hides the structure of the parse tree, and looks much like most Algol-derived languages.
• Meta-programming occurs as part of an explicit translation process. Meta-programs are distinct from the programs they manipulate, and behave as reusable language extensions.

The XL model is slightly more complicated conceptually, but has a few benefits:

• It doesn't constrain the "natural" syntax of the language. There are various parsers in Lisp to implement, for instance, infix notation, but it is generally seen as an unnatural form of Lisp.
• The internal representation has provisions to store additional compile-time data in the parse tree, for instance execution counts, data flow analysis information, or comments.
• Making meta-programming a separate process encourages the creation of reusable, general purpose language extensions.