Home | About | Partners | Contact Us
SourceForge Logo

Quick Links

Thoughts and Rants
SourceForge Info
Browse CVS
Mailing Lists

XL Links

XL Home
The XL Touch
Basic XL features
Why use XL?
Compiler Status
Frequently Asked Questions
XL Mailing list


Concept programming
Thin Tools
Active Libraries

More Info

Related Ideas
Some history
Other links
The GNU Project

Why XL?

The question "Why would I want to use XL?" was, by far, the most frequent and significant one in a recent discussion about XL on Slashdot. I had asked there what programmers cared about in programming languages. I expected mostly technical features, syntactic details. Instead, I got a different kind of answer: "we care about having a need for it." Other related questions and comments included:

  • Why create a new language? What's the point?
  • What problem are you trying to solve?
  • Why would anybody ever need it? How do you plan to convert developers?
  • What is XL better for than any other language? What new features do you bring?
  • This and that feature already exist in language X or Y
  • XL seems to be only about syntactic sugar, what is important is semantics...

I will try to address these issues on this page. As a side note, I deserved getting this kind of comments, because the initial presentation of XL really did not address them.

What is the Problem?

XL is indeed not about syntactic sugar, but about implementing a new programming approach, "Concept Programming", which I will define shortly. The first reason for creating XL was that no existing language came close to meet my needs for Concept Programming. I considered several functional, procedural and object-oriented languages, including LISP, Dylan, Objective-C, C++, SmallTalk, or even Prolog and Forth. None of them could not be easily extended to do what I wanted.

What is programming all about?

Before trying to define what Concept Programming is, we will start with a question. What is programming about? My personal answer is that programming is about turning abstract concepts into concrete implementations. All the history of progress in programming languages so far has been about increasing the level of abstraction, to be able to represent more and more complex concepts. I discuss that evolution on the Mozart page.

Yet, curiously, most programming languages have been creating this abstraction by reducing the infinite set of human concepts to at most a couple of key representations: structures, modules, objects, lists, lambda functions. This is a reasonable simplification, and most languages (except INTERCAL) are useful in some field of applicability where their main "view of the world" fits well.

The illusion of the "best paradigm"

But trying to use abstractions where they don't apply generally spells disaster. The problem is often not a limitations of any given representation, but rather the erroneous belief that it is inherently better than all others. Let's not forget that all of them without exception ultimately boil down to bits. You can program in an object-oriented way in C, it is simply less convenient. You can also add functional language features to C, if you are willing to spend the time and use awkward notations. Projects fail not because of language defects, but because people insist on using the wrong tools. Projects succeed when people use the right tools, not because of the features of these tools.

The belief in the superiority, universality and elegance of their preferred view of the world is particularly strong among users of "minority" languages, such as Lisp or Objective-C. A Slashdot comment, for instance, read something like: "People who don't know LISP are bound to reinvent it, badly." This kind of statement may be in reaction to the ignorance most people have of how well designed Lisp actually is. Lisp stood the test of time deservedly.

But no matter how well known the quote is, the objection is easily dismissed. Try and do Prolog-style logic programming in LISP, and you'll end up with a lot of useless effort (compared to using Prolog, of course.) Try and do numeric-intensive programming, and LISP is no good either, not because of its performance, but because mathematicians write 1 + 1 rather than (+ 1 1) Naturally, you could write an expression parser in LISP, but then in C++ and Fortran, you don't have to... Using Lisp for such projects is, in most cases, bigotry.

The paragraph above got me a lot of flak from the Lisp community. Please see a more complete discussion of this problem.

Multiple paradigms: the case of C++

One notable exception to the "one size fits all" methodology is C++. Bjarne Stroustrup consistently described C++ as embracing more than one paradigm at a time. The language indeed did embrace many of them over time. Unfortunately, it did so in a relatively organic and inconsistent way, making C++ extremely difficult to use well. Most C++ programmers only know and use a subset of C++. Still, I believe that C++ has earned its success by allowing you to do at the same time procedural and generic (template) programming, object-orientation and low-level memory management, etc.

So if I had decided to implement concept programming starting with any existing language, I would have had to start with C++. Another obvious reason is that C++ (and its cousins Java and C#) are the languages of choice for most programmers today. Most people don't use Lisp or Dylan or Objective-CaML for large and significant applications, irrespective of their qualities. So starting with those would not have bought me much...

Unfortunately, extending C++, even at the library level, is something that an army of people smarter than me can't seem to do right. I know that firsthand, I have been a (not very active) member of the C++ committee for two years. C++ is, in my humble opinion, slowly crumbling under its own weight. It may take some time, but C++ will need to be replaced or extended, and Bjarne Stroustrup himself certainly expects this to happen (although, obviously, he'd prefer the "extended" route :-)

An ecology of riches

Another frequent objection I received was: Why change tools? The ones I use are time tested. I even received a somewhat dreadful comment that read Find something that your language does better than all others, and I will consider using it.

This is ignoring a very specific property of computer science: Moore's law. Computers power has regularly doubled every 18 months or so. And so has the average complexity of programs. I learned programming on a calculator that could store 50 instructions and 8 floating-point values. How many programs today are less than 50 lines and use less than 8 variables? There are probably more Java-related acronyms than there were instructions on this machine. I don't envy programmers who learn programming today. At least, we old timers had BASIC...

Anyways, Moore's law is the reason we need new development tools all the time (or at least, until Moore's law stops ;-) Almost nobody in 1985 needed object-oriented programming. But you could not deal practically with the complexity of the GUI without it, so GUIs, OOP and C++ became mainstream together. In the same way, almost nobody in 1990 needed reflection or remote method invokation. But these were so useful with Internet and heterogeneous systems that the whole Java ecosystem grew around them. So Internet, Java and distributed programming became mainstream together. In both cases, there was a discontinuity, and the old way of doing thing, the old "paradigm" as we call it, became obsolete.

Paradigms in the making

I can't predict the future, but I don't take much risk in saying that some complexity ahead of us will require a new abstraction, a new concept representation. And I can also predict that neither C++ nor Java will easily "digest" this new discontinuity.

Some issues for which people are starting to develop new paradigms, and for which existing languages have to stretch to a point of rupture, include:

Concept Programming and XL are designed to be a flexible substrate which easily and seamlessly adapts to all these techniques, and more.

Now, what is this Concept Programming thing?

Concept Programming is a method where programming tools are designed to conveniently represent application concepts in their most useful form. In a Concept Programming environment, you should never have to choose between object orientation and functional programming. Better yet, you should be able to implement both (and others) from scratch, should the need arise. The intent, naturally, is that a Concept Programming system can digest discontinuities much better than older programming languages. XL is intended to be a valid substrate for implementing and integrating the next major programming technique. As a validation, it supports all current ones.

The key word in the above definition is conveniently. You can't claim Concept Programming capabilities for an existing programming language simply because you can stretch it unreasonably to accept some foreign paradigm, just as you can't claim that C is object oriented because you can write a C++ to C translator. What makes XL truly unique is how you can extend it to support a Prolog declarative style or an Ada tasking model, and how integrated these extensions become with the rest of the language.

Ultimately, the objective of representing concepts in their best format should include non-textual forms, such as graphical window representations in the Visual-Basic style, or equations manipulated in mathematical format. Some research is already done by Microsoft in this area. In this discussion, however, we are interested in what this means for textual programming languages in general, and for XL in particular.

Syntactic Sugar?

I believe I proved above than new languages don't appear because you can't do things with older languages. They appear because we, programmers, are lazy people, and we want to make our lives easier. Programming languages and other development tools have only one purpose: to simplify our lives, to give us more comfort, to improve our efficiency. It is specifically not the impossibility to do things using older languages. So I could feel justified to create a new programming language even if its only purpose was to be slightly more comfortable than others. Even without Concept Programming, I'd still have a justification for XL.

Don't bite the hand that feeds you

Now, while we are at inventing a new language, you can either choose to derive from an existing language, or create something new. The first path is more obvious, easier to walk. The problem is that, as I said, there would have been only one reasonable choice, C++. And I personally came to have a solid aversion for what I call C++ syntactic Tabasco with chunky glass bits. I know from experience that C++ compilers tend to bite their users. That's a bad thing, if you ask me.

So XL is full of syntactic sugar, not for the sake of creating a new language, but because as long as I create a new one, it might as well look nice.

Note: Readers who doubt me and believe C++ is not that hard and I could have just extended it are invited to answer these simple questions in less than one minute (let's make it one minute per question):

  • In the following declaration: int*(*x[10])(int); is x an array, a function or a pointer ?
  • What does bool x = f<g && h>(-3); mean in C++?
  • What is the template separation model? Cite one compiler which implements it?
Answers are in the next paragraphs, be patient :-)

But it looks like Pascal! Yuck!

Boy, did I get some flak for this! Everything from "Don't let your Ada background blind you" to "Perl is much more concise".

My first answer is: look how I care, it's my language. If you want a Concept Programming Perl, go and invent it yourself! I admit this is not too diplomatic, but there is some reality to it. Anyway, we can try to take a more convincing approach. Consider the declaration below:

array x[1..10] of (pointer to (function(integer y) return pointer to integer))

If you are honest, you will admit that, even if you know nothing about XL, you parsed this declaration a bit faster than the equivalent C++ declaration above (you know, the int*(*x[10])(int) thingie). People read code more than 10 times as often as they write it, so if I need to chose between readability and terseness, I choose readability any time! Now, you can answer the question about what x really is, without checking with the compiler or manual first (I know you did :-). If you are debugging code all day as real programmers do, you will appreciate the help this kind of syntax gives you.

XL is often more concise

Although the above XL code is significantly more verbose than C++, XL is not systematically more verbose than C++ either. In many cases, it is significantly shorter. Consider for instance the three following statements, which have the same purpose:

std::cout << "I=" << i << ", J=" << j << std::endl;
std::printf("I=%d, J=%d\n", i, j);
IO.WriteLn "I=", I, "J=", J

Based on my experiments porting STL and template-metaprogramming code to XL, generic code in particular tends to be both much simpler, less verbose, and overall shorter in XL than in C++, in large part due to all the template argument declarations that XL can omit thanks to the use of true generic types, and to significant simplifications in scoping rules.

XL is less ambiguous

Also, XL is a lot less ambiguous by construction. So you won't have the problem that you would have in C++ with the code I gave you. How many of you figured out that the bool x = f<g && h>(-3); thing might actually contain a template instantiation, as in:

template <int I> class f
    operator bool();
const int g = 1;
const int h = 2;

bool x = f<g && h>(-3);

So, what if I used bool, && and a confusing spacing? Do you believe that the compiler cares? And weren't you one of these many people who told me that making spacing / indentation significant as it is in XL was evil and a sure sign of big-brotherism? Ambiguity is generally a bad thing, both for the programmer and for the compiler. This is why I try to avoid it in the definition of the language.

XL is implementable

Don't feel ashamed if you were confused by the code above: so are many respected compilers. If your g++ rejects this as invalid, it means that it is too old, not that I am wrong. Other equally respected compilers will let this problem happen even if f is a function, although function templates with non-class template arguments are not allowed...

C++ is so complex that implementing correctly it is beyond the capabilities of even large corporations or open-source teams. Compilers today can still compete on the features that they implement correctly. Until recently, widely used compilers such as gcc or two commercial compilers from large vendors got it wrong for anything remotely difficult. As of today (May 1st 2001), no compiler in the world fully supports the template separation model, which is the possibility to mark templates as export and to instantiate them in a translation unit where the definition is not visible.

On the other hand, I kept the definition of XL simple enough that I believe I will be capable to implement the compiler on my own, hopefully within one year. The compiler already correctly parses any XL code (this is beyond what many C++ compilers can do for C++ :-) and has a significant portion of the semantics implemented, including lookup rules, expression reduction and some generic instantiation mechanisms.

A language that is simple to implement is also often simple to read. This is another nice benefit of simplicity.

XL is extensible

The main purpose of XL, however is to be easy to extend, and to support extentions gracefully. XL is intended to be the representation of choice for Mozart, both to implement it and to use it. So the syntax of XL must be unambiguous, to tolerate arbitrary semantic intrusions. Major extensions are assumed to be semantic, rather than syntactic, because this is where the action is. XL has a little less support for syntactic extensions.

Historically, extensions and changes cannot be added in C++, even by a committee of bright minds, without breaking something else, often user code. For instance, the introduction of namespace std was necessary to minimize the impact simple template names such as pair in the STL would have on user code (as I hope my second example above clarified.)

XL, on the other hand, is flexible enough that you can put in the library definitions that give comfortable semantic support to:

  • Data structures (array, list, hash table)
  • Control structures (multi-way if, loops)
  • Object orientation, with either strong and weak typing
  • Functional programming
  • Policy control ("Aspect oriented") programming
  • Prolog-style declarative programming (logic programming)
  • Ada-style multitasking
  • Assembly-level support
  • ... and almost anything you can think of

The objection that it is PL/1-style featuritis is easily answered. The whole purpose of XL is to allow multiple paradigms. Implementing them is simply a reality check that the approach is valid, not a language bloat (not any more than implementing X11 in C bloats the C language definition, even though X11 is a whole bloat in itself.) All of the above features are largely optional, and are in practice implemented mostly outside the XL compiler.

Naturally, implementing many of the above features require compiler support. But this support, to a large extent, builds on top of a single feature: XL pragmas. In XL, pragmas are an escape mechanism which is the major hook by which the language is extended beyond its base semantics.

Significant Examples

Three examples of how XL can be extended in different directions include: symbolic differentiation support, Ada-style tasking constructs, and Prolog-style declarative programming.

Symbolic Differentiation

I have already shown how simple semantic extensions can be added even to existing languages. Such extremely simple extensions are possible in existing languages mostly because they are extremely localized, and thus do not significantly disrupt the semantics of the hosting language. Naturally, a symbolic differentiation extension can also be used for XL, allowing the following code to become legal:

procedure Test() is
   with real T, Y
   for T in 0..50.0 step 0.01 loop
      Y := d(sin(2 * omega * t + theta) * exp(-decay * t))/dt
      IO.WriteLn "T=", T, " Y=", Y

Two major difference with implementation in other languages, however, are:

  1. The use of the standard escape mechanism (pragmas), illustrated above with the {use_differentiation} pragma. This allows the extension to spend much less time looking for the part where the language is extended, and may simplify its implementation. It also increases readbility of extended language variants.
  2. The integration of the extension mechanism in the language, allowing the extension itself to be part of the same source code.

The extension code for the differentiation, for instance, could be implemented by having a function implementing pragma {use_differentiation} in the source code:

import Coda=XL.Reflection
{reflective} procedure use_differentiation(Coda.Tree tree)

Thus, the main benefit of a real Concept Programming language for such simple extensions is integration and ease of use... which is what Concept Programming is all about.

Ada-style tasking

The tasking part of Ada is interesting because, for most people, this doesn't belong to the language but to libraries. In C and C++, for instance, one would typically use a library such as pthreads. Indeed, Ada-83 tasking had to be much revised in Ada-95, because it had proven to be too restrictive - a library would have been easier to fix.

What many people who did not use Ada ignore is that:

  1. Integration with the language allows the compiler to care about tasking safety when optimizing, and to respect it in the language semantics
  2. Integrated tasking is very convenient in practice, and allow for very elegant code compared to library tasking.

XL and Concept Programming solves the issue: tasking is implemented through a library (and is thus flexible). On the other hand, it appears as integrated in the language as it is in Ada.

Here is, for instance, the translation in XL of a classical Ada tasking example. I deliberately tried to keep the structure of the original examples. However, I reordered declaration to make them valid, and added guards on the use of Read and Write in the interface, where they are visible to the users of Buffer. In the example, we create static task and buffer objects, but they could naturally be created dynamically.

The trick is naturally that the TASK module implements various tasking-related pragmas such as {protected} and {entry}, and alters the behavior of the compiler accordingly. The modification doesn't need to be very complex. Typically, an {entry} procedure would simply have some code automatically inserted at the beginning to ensure only one task is entering it at the same time, and to copy arguments accross task stacks as necessary.

Similarly, the task type defines a more classical object, which simply initializes a task. The definition of the task type also involves reflection, however, to allow a value containing executable code to be used as the code to execute for the task. The expected behavior is that on creation of a task object, this code gets executed in a new task context. In C, you would have the code placed in a separate function, and a pointer to that function would be passed to pthread_create. The XL task object automates this process, and makes the intent much clearer.

Remember: the compiler is totally unaware of the existence of task module and does not treat it any specially. As a result, the tasking module needs not be built-in in the language, but can be offered by any third party. Real-time tasking implementations, implementations based on pthreads and many others can thus be supported concurrently in XL.

Prolog-style logic programming

Prolog is a programming language which is even more "special" than functional languages such as Lisp, because it is completely declarative. It therefore solves a very unique class of problems, where its elegance is absolutely unmatched. Being able to support this kind of programming in an integrated way was one of the most difficult problems I faced when designing XL, and the last one I solved. This is why I personnally find it interesting...

Here is the equivalent of a typical use of Prolog. As you can see, the two structure are extremely similar, even if the XL syntax is not fully optimized for this style of programming. Naturally, the implementation of the D.declaration object type is not the simplest thing in the world, but what really matter is that its use is indeed remarkably natural to Prolog users. Declarative code can be used from conventional procedural code either to find a complete solution, or as an iterator to control the exploration of various solutions.

A similar approach could be used to implement other kind of declarative languages used for different kind of problems, such as Alpha.

Copyright Christophe de Dinechin
First published Feb 17, 2000
Version 1.6 (updated 2004/01/20 06:16:14)