Contents |
Solving the real problems of Multimedia
Warning this text is meant as a bit of a controversial debate starter about
what should be the goals for something like the The Linux Foundation multimedia
working group. It attacks the issue of multimedia deficiencies under Linux from
a completely different direction than what has been done so far.
Be aware that I, Christian Schaller, the author of this text has been involved
with the development of GStreamer for the last seven years and is currently working
for Fluendo, a company doing various kinds of bussiness around the GStreamer framework.
So I am not pretending to be 'neutral' on this issue..
The discussion so far have either been oriented around sound systems or about
vague notions of shared API's and of course talks about standarisation.
Personally I feel that that debate is not the correct debate to have and
that we are instead ending up doing a lot of talk and work around solutions
of little interest. The current approach only lets us look backwards instead
of forwards. The focus I will advocate here is very different as it is
made on the assumption that the problem is not that we do not know what is
needed to 'solve' multimedia, cause I think we do, but it is simply a question
of resources and focus.
My basic claim is that application developers today want more than low level system
API's. Further clarifying the API's of ALSA, V4L2 or OSS do have value, but I honestly
do not think that this answers the needs of developers out there. Instead what
developers today do want is a feature complete and unified high level multimedia API,
like DirectShow and Quicktime.
How GStreamer address the true needs of developers
In the GStreamer community we are trying to address these issues every
day by adding support for more and more related technologies, with improved
RTP and DVB support being among the most recent. In addition to implementing
the functionality people report missing, we are working with the community to
implement a long range of applications ranging from music synthesizers, audio and
video playback applications, browser plugins, media center solutions, audio editors,
video editors, streaming servers and so on. All using GStreamer. Because in order to
meet the challenges of tomorrow we need a framework that can address all these types
of applications, not just deal with playback for instance.
In the GStreamer community we do continuous license vetting and checking
and work with both upstream and downstream developer on re-licensing, to
ensure that our stack stays LGPL licensed and that applications using GStreamer
can be shipped even with binary-only plugins or plugins which are open source,
but have non-GPL compatible licenses.
We also work with embedded systems makers trying to ensure that GStreamer stays
small and modular enough to fit their use cases, be that set-top boxes or mobile
devices like cell phones.
We have been working with the major Linux distributions for many years
now to ensure we package our software in a way that allow them to easily
include GStreamer without having to worry about patent and copyright
issues. This has produced the the current set of packages with GStreamer core,
base and good being things we feel should be safe from known patent and
licensing issues. While modules which have licensing or potential patent
issues are packaged in the ugly, bad and ffmpeg modules.
If we look at the Linux/Unix desktop market today I feel confident to say
that there are two kind of multimedia applications being currently made.
There are those that do 'everything' themselves and those that use GStreamer.
The few competitors GStreamer might have, are either much more narrowly targeted
(ie. playback only) or only used by a very few application and are thus highly
immature.
GStreamer is the only widely used framework out there which has made any serious
attempts at integrating with the linux low-level driver and API stack and also
tried to make it easier for the configuration of the system to be integrated with
that of the desktop. The now widely shared GConf keys under GNOME is a testament to
that.
While GStreamer is the best linux/unix solution out there today that doesn't mean
we are great for every usecase yet. The robustness of the framework in regards to
editing applications for instance still needs work, but we are confident that
our current effort in the area with the aid of the communities surrounding projects
like Jokosher and Pitivi will help us nail those down.
Our playback autoplugger needs to be even more flexible than it is today in order
to address situations like supporting output options requiring encoded data like
certain DSP decoder or the common usecase of S/PDIF ports.
We still need to design and implement a good way to automatically deal with
interlaced media in GStreamer. Currently the application developer has to manually
plug a deinterlacer into the pipeline.
In the context of interoperability we are writing a GStreamer backend for
Phonon to ensure that KDE4 can use GStreamer for its basic media playback
needs. This work has so far gone much slower than we hoped, but hopefully a first
working version can be released soon.
But these issues and more is not the problem of lack of standards, but the lack
of implementations.
So are there other issues that needs addressing outside GStreamer? Sure, there are a
few concrete problems in the sound driver layer for instance, like dmix being bad,
but once again this is a problem of implementation and coding and not of lack of
standards.
At Fluendo, the company I work for we try to solve the business issues
surrounding the use of multimedia on Linux by licensing and developing
codec plugins for GStreamer to ensure corporations that they have a
legally vetted solution available.
I attended the CE Linux forum some weeks ago and the question asked
there was 'what projects could we invest some money/developer time in
to make things progress further in regards to embedded linux multimedia'.
Which is the question I think the DTL Sound and Multimedia workgroup
need to start asking too, if it wants to make a difference.
If the DTL can get even more people and corporations to help out with
GStreamer development in one way or the other we can reach our shared
goal much quicker. The goal of having a competitive Linux multimedia stack.
A good example here is our recently much improved RTP/RTSP
support which have come about through an indirect collaboration between
Collabora, Fluendo, Intel, Nokia and Axis.
Recent Blog Posts
