Linux Multimedia - The Next Level of Performance
However, the Linux kernel has been lacking reliable low latency real time capabilities, required for quality multimedia playback, recording and editing. And there is no generally accepted standard for plugin technology, or integration of applications. Many applications use plugins, but even though these plugins are very similar in the way they interact with their host applications, they can only be used with the particular application for which they were written.
Some of these problems have already been taken care of, and the improvements are likely to be in the Linux 2.4 kernel, to be released around christmas this year. Other issues are being addressed, and solutions will be presented within a few months. Prototypes and beta test releases are already demonstrating performance similar to, or superior to that of dedicated multimedia operating systems. This on a system that can still perform equally well, or better, as a server...
First, what is this really about?
What is required for an operating system to be a viable multimedia platform? There are three fundamental capabilities that a system must have in order to handle multimedia, all of which are required to achieve good results:- Hard real time scheduling.
For high quality multimedia (ie without annoying dropouts or flickering), it is required that the system can provide data and processing power in a deterministic fashion. It's not acceptable allowing other, non-real time tasks to interfere with real time multimedia processing. Very few operating systems - apart from dedicated real time solutions (mostly found in embedded and industrial applications) - have any hard real time support at all. This fundamentally renders them useless for demanding high end multimedia, without the help of expensive, specialized hardware.
- Raw speed.
The system must be capable of handling high data transfer bandwidths, and allow applications to take advantage of the full processing power. For real time applications that require quick responses to user intervention and external events, this particularly means fast context switching and low overhead when moving data between the system and applications. This is a serious problem with most desktop operating systems, which makes them less appropriate for real time multimedia.
- Multimedia hardware support.
There must be support for the kind of hardware devices that are used by multimedia applications. Advanced applications require extended functionality, such at timestamping of data in drivers or hardware, and support for dedicated synchronization devices. Few systems have any means of accurately synchronizing multiple independent devices, even though some hardware has special features to support this. The result is that applications and drivers are forced to bypass the operating system in order to achieve accurate synchronization. Hardware without drivers supporting these features cannot be used in more demanding settings, just because of this.
- Concistent, flexible and easy-to-use APIs.
While throwing together a new API for every new feature seems to work for some OS vendors, it creates a nightmare for application developers, and a maintenance hell for the OS developers. It also makes it virtually impossible to port or reuse parts of the OS or the applications. The result is lots of work with no gain for anyone but those who can make money from solving these unnecessary problems. Why not spend the time on finetuning the OS and writing impressive quality applications instead?
- Transparently extensible APIs.
That is, APIs that allow enhancements of the system functionality without forcing all application developers to learn and use new API extensions. The point is that it shouldn't be necessary to upgrade everything to the latest version ever time some new features are added. Old applications and plugins should work with newer ones, without ugly workarounds, even though they will (usually) not be able to take advantage of the new improvements.
- Free/Open Source.
Imagine you are going to use a proprietary API in a dedicated or embeded system. This will allow you to use existing plugins and tools to shorten the development time, and perhaps also reuse code that you have previously written for other environments supported by that API. However, it also means that you may end up in a situation where the company that developed the API no longer exists, and the product is no longer supported... You have a serious problem. Sooner or later you will have to port all your code to a new environment, as you can't do anything about the fact that the old one doesn't support new hardware, new protocols and so on. This hurts, and costs time and money.
The Solution
It may sound overly optimistic suggesting that these six goals actually can be met, especially under a general purpose operating system such as GNU/Linux. But as a matter of fact, the really complicated technical problems are already solved; hard real time scheduling and raw speed. The kernel is already known for being very efficient, especially on high end workstations (two or four CPUs). Rock solid hard real time scheduling is going into a mainstream Linux kernel to be released soon. (Probably 2.4, around Christmas.)Also, large parts of the plugin and client/server APIs are already designed and partly implemented. It's all Free/Open Source, and even the actual design effort is open, and takes place on the linux-audio-dev mailing list. We welcome new participants, and would like to see more ideas and suggestions from multimedia developers and users.
Of course, as we develop this standard only to solve a real world problem (all right; we do enjoy working with this kind of thing!) - as opposed to making money from the product itself - we are trying to make everything as clean, re-usable and efficient as possible. Bad solutions are only accepted if there are no better alternatives. The design rules are those of good programming style; not good business.
What it means
The first and primary problem we set out to solve was the lack of a useful plugin standard for UN*X environments. Our solution is similar to the most popular plugin APIs used on MacOS and MS Windows; Steinberg's VST and Microsoft's DirectX, but is designed to handle other data formats, as well as audio. It's also designed to be usable when there are higher demands on performance, features and accuracy. It is designed from the ground up with hard real time processing in mind, and to fit well into embedded/dedicated systems, as well as computer workstations.The result: Almost any kind of real time multimedia processing can take advantage of the plugin API, and plugins are easy to port to various platforms.
The second problem is one that existing standards do not address in any useful way: Integration. Instead of leaving it to applications to cooperate and synchronize to each other when used in parallel (which usually results in poor performance, if at all possible), we will provide an interclient communication API that is similar to the plugin API. (The most significant difference being that plugins execute as callback functions, while clients run as separate processes.)
That is, multiple applications can run together with rock solid sample accurate sync, much like plugins inside a single engine. This without a whole additional API for developers to learn and understand - and it will also make it fairly easy to convert an application into a plugin or vice versa.
Any remaining problems?
There are two problems with multimedia on Linux, both of which are direct results of Linux not (yet) being a major multimedia OS; lack of drivers for high end hardware, and lack of multimedia applications.The first problem is already showing signs of going away. Some vendors have already released programming specifications for studio audio cards, and there seems to be an increasing interest among the rest. Hopefully, the late Linux developments on the technical side will increase this interest further.
The second problem is the usual "chicken or egg" dilemma. Developers don't want to write applications for platforms without users, and users can't use a platform that has no applications. Someone has to start the movement by releasing a useful and powerful application. That is, taking the risk of entering a new market. However, is it really that big a risk releasing an application that will perform orders of magnitude better than anything on one of the major platforms?
The current situation, and the future
There already are some applications (both Free/Open Source and proprietary), and most of these can already take advantage of the hard real time performance of the new Linux kernel. There are users who have been using these applications for years. And there are lots of users that would do anything to get rid of Microsoft Windows or MacOS in their setups. It's not for everyone to buy out of the reliability and performance problems by turning to dedicated recording and processing hardware. And some just don't want to, as these solutions too have their limitations; in flexibility, availability, compatibility and ease of use.The GNU/Linux platform can provide dedicated hardware performance with the flexibility and price/performance advantages of native processing. No other general purpose OS is capable of that. The Linux kernel is efficient and very reliable, which means that writing quality software for it actually pays off. The development efforts of the Free/Open Source community are driven by need and desire to create the best possible solutions to real life problems, and it is not a company that may be bought out and "terminated", or go bankrupt. Basing your solutions on the GNU/Linux platform is protecting your investment, as this platform is not going out of business until there is no longer a need for it.
David Olofson is working on a multimedia API (MuCoS) for Linux.
audiality@swipnet.se
|
|
