|
The question "Why would I want to use XL?" was, by far,
the most frequent and significant one in a recent discussion
about XL on Slashdot. I had asked there what programmers cared
about in programming languages. I expected mostly technical features,
syntactic details. Instead, I got a different kind of answer: "we care
about having a need for it." Other related questions and comments
included:
- Why create a new language? What's the point?
- What problem are you trying to solve?
- Why would anybody ever need it? How do you plan to convert developers?
- What is XL better for than any other language? What new features do you bring?
- This and that feature already exist in language X or Y
- XL seems to be only about syntactic sugar, what is important is semantics...
I will try to address these issues on this page. As a side note, I
deserved getting this kind of comments, because the initial
presentation of XL really did not address them.
What is the Problem?
XL is indeed not about syntactic sugar, but about implementing a
new programming approach, "Concept Programming", which I will
define shortly. The first reason for creating XL was that no existing
language came close to meet my needs for Concept Programming. I
considered several functional, procedural and object-oriented
languages, including LISP, Dylan, Objective-C, C++, SmallTalk, or even
Prolog and Forth. None of them could not be easily extended to do what
I wanted.
What is programming all about?
Before trying to define what Concept Programming is, we will start
with a question. What is programming about? My personal answer
is that programming is about turning abstract concepts into concrete
implementations. All the history of progress in programming languages
so far has been about increasing the level of abstraction, to be able
to represent more and more complex concepts. I discuss that evolution
on the Mozart page.
Yet, curiously, most programming languages have been creating this
abstraction by reducing the infinite set of human concepts to at most
a couple of key representations: structures, modules, objects, lists,
lambda functions. This is a reasonable simplification, and most
languages
(except INTERCAL)
are useful in some field of applicability where their main "view of
the world" fits well.
The illusion of the "best paradigm"
But trying to use abstractions where they don't apply generally
spells disaster. The problem is often not a limitations of any given
representation, but rather the erroneous belief that it is inherently
better than all others. Let's not forget that all of them without
exception ultimately boil down to bits. You can program in an
object-oriented way in C, it is simply less convenient. You can also
add functional language features to C, if you are willing to spend the
time and use awkward notations. Projects fail not because of language
defects, but because people insist on using the wrong tools. Projects
succeed when people use the right tools, not because of the features
of these tools.
The belief in the superiority, universality and elegance of their
preferred view of the world is particularly strong among users of
"minority" languages, such as Lisp or Objective-C. A Slashdot comment,
for instance, read something like: "People who don't know LISP are
bound to reinvent it, badly." This kind of statement may be in
reaction to the ignorance most people have of how well designed Lisp
actually is. Lisp stood the test of time deservedly.
But no matter how well known the quote is, the objection is easily
dismissed. Try and do Prolog-style logic programming in LISP, and you'll end
up with a lot of useless effort (compared to using Prolog, of course.)
Try and do numeric-intensive programming, and LISP is no good either,
not because of its performance, but because mathematicians write 1
+ 1 rather than (+ 1 1) Naturally, you could write an
expression parser in LISP, but then in C++ and Fortran, you don't have
to... Using Lisp for such projects is, in most cases, bigotry.
The paragraph above got me a lot of flak from the Lisp community.
Please see a more complete discussion of
this problem.
Multiple paradigms: the case of C++
One notable exception to the "one size fits all" methodology is
C++. Bjarne Stroustrup consistently described C++ as embracing more
than one paradigm at a time. The language indeed did embrace many of
them over time. Unfortunately, it did so in a relatively organic and
inconsistent way, making C++ extremely difficult to use well. Most
C++ programmers only know and use a subset of C++. Still, I believe
that C++ has earned its success by allowing you to do at the same
time procedural and generic (template) programming,
object-orientation and low-level memory management, etc.
So if I had decided to implement concept programming starting with
any existing language, I would have had to start with C++. Another obvious
reason is that C++ (and its cousins Java and C#) are the languages of
choice for most programmers today. Most people don't use Lisp or Dylan
or Objective-CaML for large and significant applications, irrespective
of their qualities. So starting with those would not have bought me
much...
Unfortunately, extending C++, even at the library level, is
something that an army of people smarter than me can't seem to do
right. I know that firsthand, I have been a (not very active) member
of the C++ committee for two years. C++ is, in my humble opinion,
slowly crumbling under its own weight. It may take some time, but C++
will need to be replaced or extended, and Bjarne Stroustrup himself
certainly expects this to happen (although, obviously, he'd prefer the
"extended" route :-)
An ecology of riches
Another frequent objection I received was: Why change
tools? The ones I use are time tested. I even received a somewhat
dreadful comment that read Find something that your language does
better than all others, and I will consider using it.
This is ignoring a very specific property of computer science:
Moore's law. Computers power has regularly doubled every 18 months or
so. And so has the average complexity of programs. I learned
programming on a calculator that could store 50 instructions and 8
floating-point values. How many programs today are less than 50 lines
and use less than 8 variables? There are probably more Java-related
acronyms than there were instructions on this machine. I
don't envy programmers who learn programming today. At least, we old
timers had BASIC...
Anyways, Moore's law is the reason we need new development tools
all the time (or at least, until Moore's law stops ;-) Almost nobody in
1985 needed object-oriented programming. But you could not deal
practically with the complexity of the GUI without it, so GUIs, OOP
and C++ became mainstream together. In the same way, almost nobody in
1990 needed reflection or remote method invokation. But these were so
useful with Internet and heterogeneous systems that the whole Java
ecosystem grew around them. So Internet, Java and distributed
programming became mainstream together. In both cases, there was a
discontinuity, and the old way of doing thing, the old
"paradigm" as we call it, became obsolete.
Paradigms in the making
I can't predict the future, but I don't take much risk in saying
that some complexity ahead of us will require a new abstraction, a new
concept representation. And I can also predict that neither C++ nor Java
will easily "digest" this new discontinuity.
Some issues for which people are starting to develop new paradigms,
and for which existing languages have to stretch to a point of
rupture, include:
Concept Programming and XL are designed to be a flexible substrate
which easily and seamlessly adapts to all these techniques, and more.
Now, what is this Concept Programming thing?
Concept Programming is a method where programming tools are
designed to conveniently represent application concepts in
their most useful form. In a Concept Programming environment, you
should never have to choose between object orientation and functional
programming. Better yet, you should be able to implement both (and
others) from scratch, should the need arise. The intent, naturally, is
that a Concept Programming system can digest discontinuities much
better than older programming languages. XL is intended to be a valid
substrate for implementing and integrating the next major programming
technique. As a validation, it supports all current ones.
The key word in the above definition is conveniently. You
can't claim Concept Programming capabilities for an existing
programming language simply because you can stretch it unreasonably to
accept some foreign paradigm, just as you can't claim that C is object
oriented because you can write a C++ to C translator. What makes XL
truly unique is how you can extend it to support a
Prolog declarative style or an
Ada tasking model, and how integrated
these extensions become with the rest of the language.
Ultimately, the objective of representing concepts in their best
format should include non-textual forms, such as graphical window
representations in the Visual-Basic style, or equations manipulated in
mathematical format.
Some research is already
done by Microsoft in this area. In this discussion, however, we are
interested in what this means for textual programming languages in
general, and for XL in particular.
Syntactic Sugar?
I believe I proved above than new languages don't appear because
you can't do things with older languages. They appear because we,
programmers, are lazy people, and we want to make our lives
easier. Programming languages and other development tools have only
one purpose: to simplify our lives, to give us more comfort, to
improve our efficiency. It is specifically not the
impossibility to do things using older languages. So I could feel
justified to create a new programming language even if its only
purpose was to be slightly more comfortable than others. Even without
Concept Programming, I'd still have a justification for XL.
Don't bite the hand that feeds you
Now, while we are at inventing a new language, you can either
choose to derive from an existing language, or create something
new. The first path is more obvious, easier to walk. The problem is
that, as I said, there would have been only one
reasonable choice, C++. And I personally came to have a solid aversion
for what I call C++ syntactic Tabasco with chunky glass bits. I
know from
experience that C++ compilers tend to bite their users. That's a
bad thing, if you ask me.
So XL is full of syntactic sugar, not for the sake of creating a
new language, but because as long as I create a new one, it might as
well look nice.
Note: Readers who doubt me and believe C++ is not that
hard and I could have just extended it are invited to answer these
simple questions in less than one minute (let's make it one minute per
question):
- In the following declaration: int*(*x[10])(int); is
x an array, a function or a pointer ?
- What does bool x = f<g && h>(-3); mean in C++?
- What is the template separation model? Cite one compiler which implements it?
Answers are in the next paragraphs, be patient :-)
But it looks like Pascal! Yuck!
Boy, did I get some flak for this! Everything from "Don't let your Ada
background blind you" to "Perl is much more concise".
My first answer is: look how I care, it's my language. If you
want a Concept Programming Perl, go and invent it yourself! I admit
this is not too diplomatic, but there is some reality to it. Anyway,
we can try to take a more convincing approach. Consider the
declaration below:
array x[1..10] of (pointer to (function(integer y) return pointer to integer))
If you are honest, you will admit that, even if you know nothing
about XL, you parsed this declaration a bit faster than the equivalent
C++ declaration above (you know, the int*(*x[10])(int)
thingie). People read code more than 10 times as often as they write
it, so if I need to chose between readability and terseness, I choose
readability any time! Now, you can answer the question about what
x really is, without checking with the compiler or manual
first (I know you did :-). If you are debugging code all day as real
programmers do, you will appreciate the help this kind of syntax gives
you.
XL is often more concise
Although the above XL code is significantly more verbose than C++, XL is
not systematically more verbose than C++ either. In many cases, it is
significantly shorter. Consider for instance the three following
statements, which have the same purpose:
std::cout << "I=" << i << ", J=" << j << std::endl;
std::printf("I=%d, J=%d\n", i, j);
IO.WriteLn "I=", I, "J=", J
Based on my experiments porting STL and template-metaprogramming
code to XL, generic code in particular tends to be both much simpler,
less verbose, and overall shorter in XL than in C++, in large part due
to all the template argument declarations that XL can omit thanks to the
use of true generic types, and to significant
simplifications in scoping rules.
XL is less ambiguous
Also, XL is a lot less ambiguous by construction. So you won't have
the problem that you would have in C++ with the code I gave you. How
many of you figured out that the bool x = f<g &&
h>(-3); thing might actually contain a template instantiation,
as in:
template <int I> class f
{
public:
operator bool();
f(int);
};
const int g = 1;
const int h = 2;
bool x = f<g && h>(-3);
So, what if I used bool, && and a confusing
spacing? Do you believe that the compiler cares? And weren't you one
of these many people who told me that making spacing / indentation
significant as it is in XL was evil and a sure sign of big-brotherism?
Ambiguity is generally a bad thing, both for the programmer and for
the compiler. This is why I try to avoid it in the definition of the
language.
XL is implementable
Don't feel ashamed if you were confused by the code above: so are
many respected compilers. If your g++ rejects this as invalid, it
means that it is too old, not that I am wrong. Other equally respected
compilers will let this problem happen even if f is a
function, although function templates with non-class template
arguments are not allowed...
C++ is so complex that implementing correctly it is beyond the
capabilities of even large corporations or open-source
teams. Compilers today can still compete on the features that they
implement correctly. Until recently, widely used compilers such as gcc
or two commercial compilers from large vendors got it wrong for
anything remotely difficult. As of today (May 1st 2001), no
compiler in the world fully supports the template separation
model, which is the possibility to mark templates as export
and to instantiate them in a translation unit where the definition is
not visible.
On the other hand, I kept the definition of XL simple enough that I
believe I will be capable to implement the compiler on my own,
hopefully within one year. The compiler already correctly parses any
XL code (this is beyond what many C++ compilers can do for C++ :-) and
has a significant portion of the semantics implemented, including
lookup rules, expression reduction and some generic instantiation
mechanisms.
A language that is simple to implement is also often simple to
read. This is another nice benefit of simplicity.
XL is extensible
The main purpose of XL, however is to be easy to extend, and to
support extentions gracefully. XL is intended to be the representation
of choice for Mozart, both to implement it
and to use it. So the syntax of XL must be unambiguous, to
tolerate arbitrary semantic intrusions. Major extensions are assumed
to be semantic, rather than syntactic, because this is where the
action is. XL has a little less support for syntactic extensions.
Historically, extensions and changes cannot be added in C++, even
by a committee of bright minds, without breaking something
else, often user code. For instance, the introduction of
namespace std was necessary to minimize the impact
simple template names such as pair in the STL would have on
user code (as I hope my second example above clarified.)
XL, on the other hand, is flexible enough that you can put in
the library definitions that give comfortable semantic support to:
- Data structures (array, list, hash table)
- Control structures (multi-way if, loops)
- Object orientation, with either strong and weak typing
- Functional programming
- Policy control ("Aspect oriented") programming
- Prolog-style declarative programming (logic programming)
- Ada-style multitasking
- Assembly-level support
- ... and almost anything you can think of
The objection that it is PL/1-style featuritis is easily
answered. The whole purpose of XL is to allow multiple
paradigms. Implementing them is simply a reality check that the
approach is valid, not a language bloat (not any more than
implementing X11 in C bloats the C language definition, even though
X11 is a whole bloat in itself.) All of the above features are largely
optional, and are in practice implemented mostly outside the XL
compiler.
Naturally, implementing many of the above features require compiler
support. But this support, to a large extent, builds on top of a
single feature: XL pragmas. In XL, pragmas are an escape mechanism
which is the major hook by which the language is extended beyond its
base semantics.
Significant Examples
Three examples of how XL can be extended in different directions
include: symbolic differentiation support, Ada-style tasking constructs,
and Prolog-style declarative programming.
Symbolic Differentiation
I have already shown how simple semantic extensions can be added even
to existing languages. Such extremely simple extensions are possible
in existing languages mostly because they are extremely localized, and
thus do not significantly disrupt the semantics of the hosting
language. Naturally, a symbolic differentiation extension can also be used
for XL, allowing the following code to become legal:
{use_differentiation}
procedure Test() is
with real T, Y
for T in 0..50.0 step 0.01 loop
Y := d(sin(2 * omega * t + theta) * exp(-decay * t))/dt
IO.WriteLn "T=", T, " Y=", Y
Two major difference with implementation in other languages, however,
are:
- The use of the standard escape mechanism (pragmas), illustrated
above with the {use_differentiation} pragma. This allows the
extension to spend much less time looking for the part where the
language is extended, and may simplify its implementation. It
also increases readbility of extended language variants.
- The integration of the extension mechanism in the language,
allowing the extension itself to be part of the same source code.
The extension code for the differentiation, for instance, could be
implemented by having a function implementing pragma
{use_differentiation} in the source code:
import Coda=XL.Reflection
{reflective} procedure use_differentiation(Coda.Tree tree)
Thus, the main benefit of a real Concept Programming language for
such simple extensions is integration and ease of use... which is what
Concept Programming is all about.
Ada-style tasking
The tasking part of Ada is interesting because, for most people,
this doesn't belong to the language but to libraries. In C and C++,
for instance, one would typically use a library such as
pthreads. Indeed, Ada-83 tasking had to be much revised in
Ada-95, because it had proven to be too restrictive - a library would
have been easier to fix.
What many people who did not use Ada ignore is that:
- Integration with the language allows the compiler to care about
tasking safety when optimizing, and to respect it in the language
semantics
- Integrated tasking is very convenient in practice, and allow for
very elegant code compared to library tasking.
XL and Concept Programming solves the issue: tasking is implemented
through a library (and is thus flexible). On the other hand, it
appears as integrated in the language as it is in Ada.
Here is, for instance, the translation in
XL of a
classical
Ada tasking example. I deliberately tried to keep the structure of the
original examples. However, I reordered declaration to make them valid,
and added guards on the use of Read and Write in the
interface, where they are visible to the users of Buffer. In
the example, we create static task and buffer objects, but they could
naturally be created dynamically.
The trick is naturally that the TASK module implements
various tasking-related pragmas such as {protected} and
{entry}, and alters the behavior of the compiler
accordingly. The modification doesn't need to be very
complex. Typically, an {entry} procedure would simply have
some code automatically inserted at the beginning to ensure only one
task is entering it at the same time, and to copy arguments accross
task stacks as necessary.
Similarly, the task type defines a more classical object,
which simply initializes a task. The definition of the task
type also involves reflection, however, to allow a value containing
executable code to be used as the code to execute for the task. The
expected behavior is that on creation of a task object, this
code gets executed in a new task context. In C, you would have the
code placed in a separate function, and a pointer to that function
would be passed to pthread_create. The XL task
object automates this process, and makes the intent much clearer.
Remember: the compiler is totally unaware of the existence of task
module and does not treat it any specially. As a result, the tasking
module needs not be built-in in the language, but can be offered by
any third party. Real-time tasking implementations, implementations
based on pthreads and many others can thus be supported
concurrently in XL.
Prolog-style logic programming
Prolog is a programming language which is even more "special" than
functional languages such as Lisp, because it is completely
declarative. It therefore solves a very unique class of problems,
where its elegance is absolutely unmatched. Being able to support this
kind of programming in an integrated way was one of the most difficult
problems I faced when designing XL, and the last one I solved. This is
why I personnally find it interesting...
Here is the equivalent of a
typical use of Prolog. As you can see, the two structure are
extremely similar, even if the XL syntax is not fully optimized for
this style of programming. Naturally, the implementation of the
D.declaration object type is not the simplest thing in the
world, but what really matter is that its use is indeed remarkably
natural to Prolog users. Declarative code can be used from
conventional procedural code either to find a complete solution, or as
an iterator to control the exploration of various solutions.
A similar approach could be used to implement other kind of
declarative languages used for different kind of problems, such
as Alpha.
| 