Why would anyone want to use LX?
Why would anyone want to use LX?
Christophe de Dinechin
Version 1.5 (updated 2002/01/23 16:15:22)
The question "Why would I want to use LX?" was, by far, the most frequent and significant one in a recent discussion about LX on Slashdot. I had asked there what programmers cared about in programming languages. I expected mostly technical features, syntactic details. Instead, I got a different kind of answer: "we care about having a need for it." Other related questions and comments included:
I will try to address these issues on this page. As a side note, I deserved getting this kind of comments, because the initial presentation of LX really did not address them.
LX is indeed not about syntactic sugar, but about implementing a new programming approach, "Concept Programming", which I will define shortly. The first reason for creating LX was that no existing language came close to meet my needs for Concept Programming. I considered several functional, procedural and object-oriented languages, including LISP, Dylan, Objective-C, C++, SmallTalk, or even Prolog and Forth. None of them could not be easily extended to do what I wanted.
Before trying to define what Concept Programming is, we will start with a question. What is programming about? My personal answer is that programming is about turning abstract concepts into concrete implementations. All the history of progress in programming languages so far has been about increasing the level of abstraction, to be able to represent more and more complex concepts. I discuss that evolution on the Mozart page.
Yet, curiously, most programming languages have been creating this abstraction by reducing the infinite set of human concepts to at most a couple of key representations: structures, modules, objects, lists, lambda functions. This is a reasonable simplification, and most languages (except INTERCAL) are useful in some field of applicability where their main "view of the world" fits well.
But trying to use abstractions where they don't apply generally spells disaster. The problem is often not a limitations of any given representation, but rather the erroneous belief that it is inherently better than all others. Let's not forget that all of them without exception ultimately boil down to bits. You can program in an object-oriented way in C, it is simply less convenient. You can also add functional language features to C, if you are willing to spend the time and use awkward notations. Projects fail not because of language defects, but because people insist on using the wrong tools. Projects succeed when people use the right tools, not because of the features of these tools.
The belief in the superiority, universality and elegance of their preferred view of the world is particularly strong among users of "minority" languages, such as Lisp or Objective-C. A Slashdot comment, for instance, read something like: "People who don't know LISP are bound to reinvent it, badly." This kind of statement may be in reaction to the ignorance most people have of how well designed Lisp actually is. Lisp stood the test of time deservedly.
But no matter how well known the quote is, the objection is easily
dismissed. Try and do Prolog-style logic programming in LISP, and you'll end
up with a lot of useless effort (compared to using Prolog, of course.)
Try and do numeric-intensive programming, and LISP is no good either,
not because of its performance, but because mathematicians write 1
+ 1 rather than (+ 1 1) Naturally, you could write an
expression parser in LISP, but then in C++ and Fortran, you don't have
to... Using Lisp for such projects is, in most cases, bigotry.
Multiple paradigms: the case of C++
One notable exception to the "one size fits all" methodology is C++. Bjarne Stroustrup consistently described C++ as embracing more than one paradigm at a time. The language indeed did embrace many of them over time. Unfortunately, it did so in a relatively organic and inconsistent way, making C++ extremely difficult to use well. Most C++ programmers only know and use a subset of C++. Still, I believe that C++ has earned its success by allowing you to do at the same time procedural and generic (template) programming, object-orientation and low-level memory management, etc.
So if I had decided to implement concept programming starting with any existing language, I would have had to start with C++. Another obvious reason is that C++ (and its cousins Java and C#) are the languages of choice for most programmers today. Most people don't use Lisp or Dylan or Objective-CaML for large and significant applications, irrespective of their qualities. So starting with those would not have bought me much...
Unfortunately, extending C++, even at the library level, is something that an army of people smarter than me can't seem to do right. I know that firsthand, I have been a (not very active) member of the C++ committee for two years. C++ is, in my humble opinion, slowly crumbling under its own weight. It may take some time, but C++ will need to be replaced or extended, and Bjarne Stroustrup himself certainly expects this to happen (although, obviously, he'd prefer the "extended" route :-)
Another frequent objection I received was: Why change tools? The ones I use are time tested. I even received a somewhat dreadful comment that read Find something that your language does better than all others, and I will consider using it.
This is ignoring a very specific property of computer science: Moore's law. Computers power has regularly doubled every 18 months or so. And so has the average complexity of programs. I learned programming on a calculator that could store 50 instructions and 8 floating-point values. How many programs today are less than 50 lines and use less than 8 variables? There are probably more Java-related acronyms than there were instructions on this machine. I don't envy programmers who learn programming today. At least, we old timers had BASIC...
Anyways, Moore's law is the reason we need new development tools all the time (or at least, until Moore's law stops ;-) Almost nobody in 1985 needed object-oriented programming. But you could not deal practically with the complexity of the GUI without it, so GUIs, OOP and C++ became mainstream together. In the same way, almost nobody in 1990 needed reflection or remote method invokation. But these were so useful with Internet and heterogeneous systems that the whole Java ecosystem grew around them. So Internet, Java and distributed programming became mainstream together. In both cases, there was a discontinuity, and the old way of doing thing, the old "paradigm" as we call it, became obsolete.
I can't predict the future, but I don't take much risk in saying that some complexity ahead of us will require a new abstraction, a new concept representation. And I can also predict that neither C++ nor Java will easily "digest" this new discontinuity.
Some issues for which people are starting to develop new paradigms, and for which existing languages have to stretch to a point of rupture, include:
Concept Programming and LX are designed to be a flexible substrate which easily and seamlessly adapts to all these techniques, and more.
Concept Programming is a method where programming tools are designed to conveniently represent application concepts in their most useful form. In a Concept Programming environment, you should never have to choose between object orientation and functional programming. Better yet, you should be able to implement both (and others) from scratch, should the need arise. The intent, naturally, is that a Concept Programming system can digest discontinuities much better than older programming languages. LX is intended to be a valid substrate for implementing and integrating the next major programming technique. As a validation, it supports all current ones.
The key word in the above definition is conveniently. You can't claim Concept Programming capabilities for an existing programming language simply because you can stretch it unreasonably to accept some foreign paradigm, just as you can't claim that C is object oriented because you can write a C++ to C translator. What makes LX truly unique is how you can extend it to support a Prolog declarative style or an Ada tasking model, and how integrated these extensions become with the rest of the language.
Ultimately, the objective of representing concepts in their best format should include non-textual forms, such as graphical window representations in the Visual-Basic style, or equations manipulated in mathematical format. Some research is already done by Microsoft in this area. In this discussion, however, we are interested in what this means for textual programming languages in general, and for LX in particular.
I believe I proved above than new languages don't appear because you can't do things with older languages. They appear because we, programmers, are lazy people, and we want to make our lives easier. Programming languages and other development tools have only one purpose: to simplify our lives, to give us more comfort, to improve our efficiency. It is specifically not the impossibility to do things using older languages. So I could feel justified to create a new programming language even if its only purpose was to be slightly more comfortable than others. Even without Concept Programming, I'd still have a justification for LX.
Now, while we are at inventing a new language, you can either choose to derive from an existing language, or create something new. The first path is more obvious, easier to walk. The problem is that, as I said, there would have been only one reasonable choice, C++. And I personally came to have a solid aversion for what I call C++ syntactic Tabasco with chunky glass bits. I know from experience that C++ compilers tend to bite their users. That's a bad thing, if you ask me.
So LX is full of syntactic sugar, not for the sake of creating a new language, but because as long as I create a new one, it might as well look nice.
Note: Readers who doubt me and believe C++ is not that hard and I could have just extended it are invited to answer these simple questions in less than one minute (let's make it one minute per question):
Answers are in the next paragraphs, be patient :-)
- In the following declaration: int*(*x[10])(int); is x an array, a function or a pointer ?
- What does bool x = f<g && h>(-3); mean in C++?
- What is the template separation model? Cite one compiler which implements it?
Boy, did I get some flak for this! Everything from "Don't let your Ada background blind you" to "Perl is much more concise".
My first answer is: look how I care, it's my language. If you want a Concept Programming Perl, go and invent it yourself! I admit this is not too diplomatic, but there is some reality to it. Anyway, we can try to take a more convincing approach. Consider the declaration below:
array x[1..10] of (pointer to (function(integer y) return pointer to integer))
If you are honest, you will admit that, even if you know nothing about LX, you parsed this declaration a bit faster than the equivalent C++ declaration above (you know, the int*(*x[10])(int) thingie). People read code more than 10 times as often as they write it, so if I need to chose between readability and terseness, I choose readability any time! Now, you can answer the question about what x really is, without checking with the compiler or manual first (I know you did :-). If you are debugging code all day as real programmers do, you will appreciate the help this kind of syntax gives you.
Although the above LX code is significantly more verbose than C++, LX is not systematically more verbose than C++ either. In many cases, it is significantly shorter. Consider for instance the three following statements, which have the same purpose:
std::cout << "I=" << i << ", J=" << j << std::endl;
std::printf("I=%d, J=%d\n", i, j);
IO.WriteLn "I=", I, "J=", J
Based on my experiments porting STL and template-metaprogramming
code to LX, generic code in particular tends to be both much simpler,
less verbose, and overall shorter in LX than in C++, in large part due
to all the template argument declarations that LX can omit thanks to the
use of true generic types, and to significant
simplifications in scoping rules.
LX is less ambiguous
Also, LX is a lot less ambiguous by construction. So you won't have the problem that you would have in C++ with the code I gave you. How many of you figured out that the bool x = f<g && h>(-3); thing might actually contain a template instantiation, as in:
template <int I> class f
{
public:
operator bool();
f(int);
};
const int g = 1;
const int h = 2;
bool x = f<g && h>(-3);
So, what if I used bool, && and a confusing spacing? Do you believe that the compiler cares? And weren't you one of these many people who told me that making spacing / indentation significant as it is in LX was evil and a sure sign of big-brotherism? Ambiguity is generally a bad thing, both for the programmer and for the compiler. This is why I try to avoid it in the definition of the language.
Don't feel ashamed if you were confused by the code above: so are many respected compilers. If your g++ rejects this as invalid, it means that it is too old, not that I am wrong. Other equally respected compilers will let this problem happen even if f is a function, although function templates with non-class template arguments are not allowed...
C++ is so complex that implementing correctly it is beyond the capabilities of even large corporations or open-source teams. Compilers today can still compete on the features that they implement correctly. Until recently, widely used compilers such as gcc or two commercial compilers from large vendors got it wrong for anything remotely difficult. As of today (May 1st 2001), no compiler in the world fully supports the template separation model, which is the possibility to mark templates as export and to instantiate them in a translation unit where the definition is not visible.
On the other hand, I kept the definition of LX simple enough that I believe I will be capable to implement the compiler on my own, hopefully within one year. The compiler already correctly parses any LX code (this is beyond what many C++ compilers can do for C++ :-) and has a significant portion of the semantics implemented, including lookup rules, expression reduction and some generic instantiation mechanisms.
A language that is simple to implement is also often simple to read. This is another nice benefit of simplicity.
The main purpose of LX, however is to be easy to extend, and to support extentions gracefully. LX is intended to be the representation of choice for Mozart, both to implement it and to use it. So the syntax of LX must be unambiguous, to tolerate arbitrary semantic intrusions. Major extensions are assumed to be semantic, rather than syntactic, because this is where the action is. LX has a little less support for syntactic extensions.
Historically, extensions and changes cannot be added in C++, even by a committee of bright minds, without breaking something else, often user code. For instance, the introduction of namespace std was necessary to minimize the impact simple template names such as pair in the STL would have on user code (as I hope my second example above clarified.)
LX, on the other hand, is flexible enough that you can put in the library definitions that give comfortable semantic support to:
The objection that it is PL/1-style featuritis is easily answered. The whole purpose of LX is to allow multiple paradigms. Implementing them is simply a reality check that the approach is valid, not a language bloat (not any more than implementing X11 in C bloats the C language definition, even though X11 is a whole bloat in itself.) All of the above features are largely optional, and are in practice implemented mostly outside the LX compiler.
Naturally, implementing many of the above features require compiler support. But this support, to a large extent, builds on top of a single feature: LX pragmas. In LX, pragmas are an escape mechanism which is the major hook by which the language is extended beyond its base semantics.
Three examples of how LX can be extended in different directions include: symbolic derivative support, Ada-style tasking constructs, and Prolog-style declarative programming.
I have already shown how simple semantic extensions can be added even to existing languages. Such extremely simple extensions are possible in existing languages mostly because they are extremely localized, and thus do not significantly disrupt the semantics of the hosting language. Naturally, a symbolic derivative extension can also be used for LX, allowing the following code to become legal:
{use_derivative}
procedure Test() is
with real T, Y
for T in 0..50.0 step 0.01 loop
Y := d(sin(2 * omega * t + theta) * exp(-decay * t))/dt
IO.WriteLn "T=", T, " Y=", Y
Two major difference with implementation in other languages, however, are:
The extension code for the derivative, for instance, could be implemented by having a function implementing pragma {use_derivative} in the source code:
import Coda=LX.Reflection
{reflective} procedure use_derivative(Coda.Tree tree)
Thus, the main benefit of a real Concept Programming language for such simple extensions is integration and ease of use... which is what Concept Programming is all about.
The tasking part of Ada is interesting because, for most people, this doesn't belong to the language but to libraries. In C and C++, for instance, one would typically use a library such as pthreads. Indeed, Ada-83 tasking had to be much revised in Ada-95, because it had proven to be too restrictive - a library would have been easier to fix.
What many people who did not use Ada ignore is that:
LX and Concept Programming solves the issue: tasking is implemented through a library (and is thus flexible). On the other hand, it appears as integrated in the language as it is in Ada.
Here is, for instance, the translation in LX of a classical Ada tasking example. I deliberately tried to keep the structure of the original examples. However, I reordered declaration to make them valid, and added guards on the use of Read and Write in the interface, where they are visible to the users of Buffer. In the example, we create static task and buffer objects, but they could naturally be created dynamically.
The trick is naturally that the TASK module implements various tasking-related pragmas such as {protected} and {entry}, and alters the behavior of the compiler accordingly. The modification doesn't need to be very complex. Typically, an {entry} procedure would simply have some code automatically inserted at the beginning to ensure only one task is entering it at the same time, and to copy arguments accross task stacks as necessary.
Similarly, the task type defines a more classical object, which simply initializes a task. The definition of the task type also involves reflection, however, to allow a value containing executable code to be used as the code to execute for the task. The expected behavior is that on creation of a task object, this code gets executed in a new task context. In C, you would have the code placed in a separate function, and a pointer to that function would be passed to pthread_create. The LX task object automates this process, and makes the intent much clearer.
Remember: the compiler is totally unaware of the existence of task module and does not treat it any specially. As a result, the tasking module needs not be built-in in the language, but can be offered by any third party. Real-time tasking implementations, implementations based on pthreads and many others can thus be supported concurrently in LX.
Prolog is a programming language which is even more "special" than functional languages such as Lisp, because it is completely declarative. It therefore solves a very unique class of problems, where its elegance is absolutely unmatched. Being able to support this kind of programming in an integrated way was one of the most difficult problems I faced when designing LX, and the last one I solved. This is why I personnally find it interesting...
Here is the equivalent of a typical use of Prolog. As you can see, the two structure are extremely similar, even if the LX syntax is not fully optimized for this style of programming. Naturally, the implementation of the D.declaration object type is not the simplest thing in the world, but what really matter is that its use is indeed remarkably natural to Prolog users. Declarative code can be used from conventional procedural code either to find a complete solution, or as an iterator to control the exploration of various solutions.
A similar approach could be used to implement other kind of declarative languages used for different kind of problems, such as Alpha.