Xtext is a DSL(Domain Specific Language) building system.
Setting up an Xtext Project
For a step-by-step guide to show how to setup an Xtext project see this page.
Xtext Concepts
Xtext allows us to compile (or, using xbase, interpret) to working code from a text file (source code) containing a user defined language (often known as a domain specific language DSL) together with an editor for writing and updating users code written in that DSL.
| The stages of a conventional compiler might look something like this: |  |
| Xtext works differently by converting to java source code, this eliminates the need for a lot of the low level stuff. It also adds a lot of convenience by automatically generating a code generator: |  |
The generation of the JVM bytecode is then done by the usual java compiler.
Xtext Structure
An Xtext project requires a large file structure in your workspace directory. However most of this file structure is built by the 'new Xtext project wizard' and the Xtext tools, so you don't necessarily need to understand the purpose of every file, you can just follow the process described on this page to create a new working project.
However, at some stage, it can be helpful to know your way round this file structure so I have drawn a diagram below. At this stage there is no need to know about all these files. In fact for simple projects you may only need to know about one file, the grammar file, the file with the '.xtext' extension.
Xtext functionality is built using fragments, these construct various parts of the file structure.
The fragments used are defined in the GeneratorEditor.mwe2 file.
The main functionality comes from the grammar (editor.text) and the generator (Generator.xtend). In many cases the remaining files can be left with their default contents.
- The contents of the grammar file are discussed on this page.
- The contents of the generator file are discussed on this page.
The Grammar
Once you have created the project then the most important, and possibly the only, file you need to edit is the grammar file (it has '.xtext' extension). For information about doing this see this page.
Depending on your requirements, it may not be necessary to write the whole grammar and code generator from scratch, Xbase provides rules and code for java-like expressions, this can greatly simplify the work if your language is not too unusual.
If you need to customise your language expressions more than Xbase allows but the language has a similar form to other languages then you could try a grammar generator that I am working on here.
User Interface
One of the many advantages of Xtext is that it automatically generates an Eclipse editor for your language. All you have to do is design the grammar and possibly a few other things and you get a powerful editor with syntax highlighting, auto complete, error messages and so on for free.
This is built on powerful technologies including:
- Antlr parser
- Eclipse model framework (EMF)
- Eclipse JFace text editor
The following diagram is intended to illustrate some of the mechanisms that do this. In many cases language designers don't need to know much about the internals as it happens automatically but it might be interesting to try to investigate what is going on behind the scenes.
Documentation
The official xtext documentation is here. It is good in places, for instance the 5 and 15 minute tutorials are a good start and some of the detailed stuff is good, but there are lots of gaps and more explanation needed in various topics. In my case, at least, I don't think this documentation provides sufficient explanation to fully learn xtext/xbase.
Some of these gaps are filled in by various blogs. The Itimis blog and also the various developers have their own blogs:
and other blogs about xtext:
Although I find searching peoples blogs a very inefficient way to get documentation, sometimes it is the only place where things are explained. Be careful when using these blogs as xtext (and especially xbase) is changing all the time and the information can get out of date.
and these sites are also useful:
- Xtext videos by XText Team
- Useful Videos by BorisBrodski
- xtext introduction video
- customise outline view
- fj-eclipse a small java implementation
Of course xtext/xbase is open source so you can always read the source code. However much of it is not commented, only what the Xtext team consider 'primary code' tends to be commented.
In addition to this information about xtext it is helpful (perhaps essential) to have a good understanding of:
- Java language, libraries and Java Virtual Machine (JVM)
- Principles of grammar (parser and lexer), EBNF and Antlr
- Developing using Eclipse
- Eclipse Modeling Framework (EMF) and ECore.
- Dependency Injection and Google Guice.
- Git, EGit and Github.
Other Pages on this Site
I have also tried to put useful information on this site when I come across anything useful. I find it helps me to have information all together in the same place:
- Creating an Xtext Project
- Grammar
- Specific Issues
- Generating Code
- Naming
- XBase
- Xtend
- XSemantics - helps with inferring types.
