interview: David Olofson -- the making of the scripting language EEL
David Olofson has been hacking audio code for some years. Here we chat to him about the making of his hard real time scripting language, EEL.
'EEL is a scripting and programming language, designed specifically for hard real time applications. The primary target areas of application are control engineering and audio synthesis, but EEL should also be suitable for game scripting and for adding scripting capabilities to real time multimedia applications.'
I'm really interested to know not only what Eel has to offer but also the actual mechanics of writing a scripting language and its interpreter.
Well, I'm pretty new to language development, actually. This is my first "real" one, and my first with a compiler + VM. Somehow, I feel pretty much at home with it anyway... Always been interested in this kind of stuff, and always wanted to "invent my own language", so I've been reading articles, papers and stuff on it whenever I ran into it. I actually got it to work, so I guess I picked up a few things along the way after all. ;-)
As to the mechanics, it's basically the standard approach:
1. Hack away to get the job done.
2. H*lluva' lot of testing and debugging.
3. Refactor stuff that didn't turn out right.
4. Some more testing and debugging.
5. Goto 1.
Next release (hacking furiously) will have an SDL/SDL_image/SDL_net binding + some little demos. Fun stuff, I hope. :-)
I think when you originally mentioned this project that games might be a use for EEL. The frame rates sound like it might not be the case. Is that so?
On the contrary, it seems like EEL might turn out to be rather well suited for game scripting, at least on the technical level. Don't have enough game scripting experience to tell if the EEL language is the best thing ever for that, but it's not too different from Lua, which is used in countless commercial and Free games.
Indeed, I've been worried that EEL might become too slow to compete with "normal" scripting languages in this domain, but it turns out that might have been unfounded. The difference in cycle times is just an order of magnitude, whereas instructions and operations deal with time frames several orders of magnitude smaller. Even on my old 933 MHz P-III, the VM executes over 11 MIPS on average, when not restricted by expensive object operations such as vector operations. Given a "scripting budget" of 10% of the total CPU time, that corresponds to some 11,000 VM instructions per frame in a game, or 1100 per buffer in an audio engine. The situation improves with faster CPUs.
Thus, most operations on the VM level concern so short time frames that throughput *is* the primary goal for all practical matters, as long as the optimizations aren't introducing virtually unbounded worst case execution times.
One of the few places where a real time scripting language differs significantly from a "normal" scripting language, is memory management. Regardless of the general method chosen, reclaiming memory frequently and aggressively enough without too much overhead can be problematic. Whereas a garbage collector might have trouble keeping up with code that generates lots of garbage without incurring too much overhead, a reference counting solution can get into serious trouble with code that occasionally destroys large hierarchies of objects.
Another sensitive area is major VM state changes, such as compiling code, loading and unloading of modules and registering classes. The EEL compiler is based on a recursive parser written in C, which means that it can't be called directly from the VM without completely stalling it until the compiler returns. This can be dealt with by running the compiler in a separate thread, and then transferring the compiled modules (EEL objects + VM code) to the VM that needs them.
However, even when compiling is done in another thread, the operation of loading, or rather linking, modules into the VM, registering any classes it exports etc, has to be specifically designed to be real time safe. Just doing it all on in one go is ok for small modules, but linking in a large module, that pulls in other modules, might register so many classes that internal structures have to be reallocated and so on; that just won't work, as there is no bound to how long it can take to complete.
Of course, there is the option of just saying "don't load modules on the fly", but that's not particularly helpful to certain types of applications.
As an example, my primary work related project uses EEL as part of the communication protocol, and thus needs to inject new code into the system on a regular basis - and this must be done without ever stalling the VM. There are other ways of doing this, but since the injected code is mostly signal generator functions, expression evaluators and the like, the simplest and most efficient way is to just insert new functions that are called directly.
You refer to audio synthesis as the prime use but what does this mean for the user? What can the user do with EEL that might not be otherwise possible with other things?
Actually, I'm trying not to specialize EEL more than what is required in order to solve problems in the "millisecond class real time" domain - which just happens to be where real time audio synthesis belongs. And of course, I happen to be rather interested in audio synthesis. :-)
EEL is intended to be like any other scripting language (high level, safe, etc), except with some "odd" design decisions that allow the programmer to focus on the actual problem, instead of working around inherently "real time hostile" features of the language.
No (normal) language can automatically guarantee that your code is real time safe, but a language (or rather, an implementation of one) *can* prevent you from ever creating real time safe code. EEL specifically tries to avoid the latter, while most other scripting languages don't even pretend to.
I wonder if we could look at some code examples relating to audio synthesis and possible games use. It would be interesting to see what it looks like.
I'd be interested in seeing that myself! ;-)
Unfortunately, I'm working around the clock on work related stuff, and getting EEL to do what I need for that is pretty much all I have time for. Not much time to play around with game or audio related stuff, that is. The EELBox (SDL binding) examples for the upcoming release (0.1.2) are about as close as it gets at this point.
Also, it should be noted that most of the code included with the distribution was written to test various language and VM features; not to demonstrate how to write proper EEL code. In some cases, I don't even know what the "right way" is yet. I want to see what works best in real code, rather than basing a detailed design purely on theoretical reasoning.
In the meantime what clues do people have to get started. There is eel_grammar.txt. Have I missed something else?...
That's about all there is in terms of documentation in currently available releases, and it's really more of a guide to the parser code than anything else.
There are the beginnings of language documentation in the upcoming 0.1.2, but I don't know how much time I'll be able to spend writing proper documentation in the near future. I'll have to write *some* no matter what; at least enough that the Windows GUI programmer can get his job done.
Thanks David.
Bookmark:
post to Delicious Digg Reddit Facebook StumbleUpon
Recent on Mstation: music: Vivian Girls, America's Cup, music: Too Young to Fall..., music: Pains of Being Pure At Heart, Berlin Lakes, music: Atarah Valentine, Travel - Copenhagen, House in the Desert
front page / music / software / games / hardware /wetware / guides / books / art / search / travel /rss / podcasts / contact us