XL Compiler Status

The prototype XL compiler, under development, is integrated in the Mozart distribution.

Core Language

We are slowly getting there. The following are the remaining important items to implement in the core language, to get a functionally complete compiler. With these in place, we should be able to start working on a library and on a compiler written in XL. These items are currently assigned to Christophe de Dinechin, but feel free to pick any if you feel like it... All dates are very tentative, and based roughly on being able to work on the compiler about 1 hour per evening max...

Recent slowdown The status on this page has remained largely unchanged between November 2002 and April 2003. This is related to a change in priorities for the main maintainer of the Mozart project, who had to concentrate on more urgent things (an international move with his family of 5). The project is only delayed, not cancelled. Stay tuned, progress should hopefully resume in a matter of months.

Completion	Description	Complexity
Oct-15-2002	Syntax for variants (case-when == unions)	Low
Oct-20-2002	Layout for variants (overlapping fields)	Low
1 day	Standard pointer types in the library (the code generation is already there)	Low
1 week	Code generation for bit fields	Low
1 week	Volatile access and pragma	Low
Nov 14, 2002	'any' lookup for expressions: any(x+y), similar to any.Write This turned out to be more difficult than expected, because it exposed problems with the way pending instantiations were dealt with.	Low
1 week	Variable-declaration-based lookup for expressions	Low
Oct 25, 2002	Global destructors.	Low
1 week	Code generation and library definition for arrays	Medium
1 week	'other' in type declarations (type distance is other real)	Medium
2 weeks	Constant folding and evaluation (required notably to know if generic arguments are constant)	Medium
3-6 weeks	Inliner ({inline} and {lazy} pragmas)	High
2-4 weeks	Generation of runtime type information	High
4-8 weeks	Dynamic dispatch ('any' arguments, similar to virtual functions)	High
4-8 weeks	Generics specialization	High

Once all this is done, the work on the library can really begin. But this is another story.

How to contribute

To get a feel for a new language, nothing beats looking at code. Various examples of test code which is known to compile can be found in the TESTS subdirectory. These small examples are often focusing on features that are specific to XL, or difficult to implement, or both. At this point, the objective is to test the compiler, not to demonstrate the language. However, there are a few more useful examples, such as the Julia set computation program.

CVS Access

In order to contribute, I recommend getting the latest source from CVS (either directly using CVS or by accessing the nightly-generated CVS tar balls.) If you don't know how to access projects hosted on SourceForge, please read the SourceForge documentation. From a Unix system, you would typically issue the following:

export CVS_RSH=ssh
export CVS_ROOT=:pserver:anonymous@cvs.mozart-dev.sourceforge.net:\
/cvsroot/mozart-dev
cvs -z9 checkout mozart

There will be a xl directory in the Mozart source tree, and that's where the compiler lives. To compile it, set the BUILDENV environment variable to your platform, for instance macosx for the configuration specified by Makefile.config.macosx. If BUILDENV is not set, then the Makefile tries to make a somewhat uneducated guess.

Which platforms are supported?

Mozart and XL are currently being developed on MacOS X, which is probably the best supported platforms. From time to time, I test the builds on Linux, which is therefore the second best supported platform (this include a 64-bit variant of Linux.) The compiler being used is GCC in both cases. The XL compiler generates C code that is also being tested with GCC.

Other platforms have been tested with various degrees of success. A while back, Attila Lendvai contributed a port to BeOS. Bernard Hurley and Ingo Bruss contributed a Windows port using CygWin. Solaris used to work, but has not been tested in a very long time. HP-UX works as long as you use gcc and not the HP ANSI C++ compiler (aCC), which has a bug preventing it from compiling the "perform" macro correctly (I filed a bug report, and hopefully this will be fixed soon.)

Syntax

The scanner is implemented in xl_scanner.cpp and transforms sequences of characters into tokens (such as an integer constant or an identifier.) The parser, implemented in xl_parser.cpp transforms these tokens into a valid Mozart tree. This is a recursive descent parser, not a yacc-based parser, for various reasons, notably support of the indentation-based syntax which proves difficult in yacc.

All parser functions are virtual. This is by design: it allows a XLParser-derived class to implements specific behaviors. This makes it easy to implement a slightly extended/modified syntax. It is unclear whether this is useful, since the focus on extensibility in XL is mostly with respect to semantics, using pragmas.

As far as I can tell, parser and scanner are complete. Minor adjustments do occur from time to time, as I use the language more.

Semantics

Most things now work almost correctly for positive test cases (that is, test cases which do not generate errors). Many negative test cases remain undetected, or detected through asserts or bus errors... Pending tasks and plan can be found in the TODO file for the xl directory.

The major remaining big-ticket item are the object-oriented features: inheritance, polymorphism, dynamic dispatch. These should not be very difficult, in particular compared to some features which have already been implemented. XL features data inheritance (aggregation of fields) and logical inheritance (is-a relationship in terms of functionality), and both need to be combined to work together. Also required are run-time type information, run-time support for dynamic dispatch, including multi-methods, and polymorphic objects (i.e. something like any shape S.)

Other less complicated tasks include:

• Enabling controlled interfaces in objects, which allows you to implement a data field using computed properties. The objective is that you'll never need to implement a getter or setter to get data encapsulation: a getter or setter is a pure implementation detail not visible from the interface.
• Generic specializations, both total and partial, and partial ordering of generics similar to partial template ordering in C++. XL adds a twist, because generic validation can rule out some candidates.
• Checking preconditions and postconditions. Currently, only assertions are processed. Also, assertions only work in code: object-related assertions (aka class invariants) do not work.
• XL-style variants are not supported. C-style unions would be trivial to implement with the current support. However, the objective is to really support tagged data types with validation of record fields with respect to various conditions, and flexible object layout.
• Inliner: call inlining is not implemented, although a basic shell is there.
• Invoking pragma plug-ins from shared libraries. Currently, the pragma implementation needs to be linked statically with the compiler, or be standalone pragma executable using thin-tools, which only work during the parse phase (this is how we implement the {derivation} pragma.
• Invokation of constructors and destructors for global objects.

It's likely that none of these features (contrary to objects support) would be absolutely required to implement a fairly decent level of library support that would make the compiler usable.

Code generation

The XL compiler currently generates C code (you can see the output using the -C option). I am not outputting real assembly at that stage because I don't want to go into the "back-end" business (register allocation, data flow analysis, optimizations). At least not until the front-end part is complete.

The C code being emitted looks ugly, but this is by design. Operations are decomposed to the extreme, as they would be with a real assembly language.

Library

The XL library is almost totally empty at that stage. The only things in there for now is a very simplified XL_BUILTINS module containing declarations for integer/real arithmetic, comparison, etc. There is no some elementary I/O, implemented in the XL.TEXT_IO and XL.UI.CONSOLE modules.

The first elements will be the construction of a rich set of data types: arrays, strings, lists, pointers, and all the related operations and algorithms. Coming next, various mathematical functions, and better I/O support.

Tips for debugging the compiler

While you are debugging (assuming you use GDB), you can print the value of a String S using print pstring(S), the value of a code tree (Coda * or derived, including XLSymboLTable) using print xltree(S), and the value of a XLGenericMap * using print xlgenmap(map) (beware, you most often need to take the address of a map in the compiler.)