kdbus details

gregkh's picture

Now that linux.conf.au is over, there has been a bunch of information
running around about the status of kdbus and the integration of
it with systemd. So, here’s a short summary of what’s going on
at the moment.

Lennart Poettering gave a talk about
kdbus at linux.conf.au. The talk can be viewed here,
and the slides are here. Go read the slides and watch the
talk, odds are, most of your questions will be answered there already.

For those who don’t want to take the time watching the talk,
lwn.net wrote up a great summary of the talk, and that article is
here. For those of you without a lwn.net
subscription, what are you waiting for? You’ll have to wait two weeks
before it comes out from behind the paid section of the website before
reading it, sorry.

There will be a systemd hack-fest a few days before
FOSDEM, where we should hopefully pound out the remaining
rough edges on the codebase and get it ready to be merged. Lennart will
also be giving his kdbus talk again at FOSDEM if
anyone wants to see it in person.

The kdbus code can be found in two places, both on google code,
and on github, depending on where you like to browse things. In a few
weeks we’ll probably be creating some patches and submitting it for
inclusion in the main kernel, but more testing with the latest
systemd code needs to be done first.

If you want more information about the kdbus interface, and how
it works, please see the kdbus.txt file for
details.

Binder vs. kdbus

A lot of people have asked about replacing Android’s binder
code with kdbus. I originally thought this could be done, but
as time has gone by, I’ve come to the conclusion that this will not
happen with the first version of kdbus, and possibly can never happen.

First off, go read that link describing binder that I pointed
to above, especially all of the links to different resources from that
page. That should give you more than you ever wanted to know about
binder.

Short answer

Binder is bound to the CPU, D-Bus (and hence kdbus), is bound to
RAM.

Long answer

Binder

Binder is an interface that Android uses to provide synchronous calling
(CPU) from one task to a thread of another task. There is no queueing
involved in these calls, other than the caller process is suspended
until the answering process returns. RAM is not interesting besides the
fact that it is used to share the data between the different callers.
The fact that the caller process gives up its CPU slice to the answering
process is key for how Android works with the binder library.

This is just like a syscall, and it behaves a lot like a mutex. The
communicating processes are directly connected to each other. There is
an upper limit of how many different processes can be using binder at
once, and I think it’s around 16 for most systems.

D-Bus

D-Bus is asynchronous, it queues (RAM) messages, keeps the messages in
order, and the receiver dequeues the messages. The CPU does not matter
at all other than it is used to do the asynchronous work of passing the
RAM around between the different processes.

This is a lot like network communication protocols. It is a very
“disconnected” communication method between processes. The upper limit
of message sizes and numbers is usually around 8Mb per connection and a
normal message is around 200-800 bytes.

Binder

The model of Binder was created for a microkernel-like device (side
note, go read this wonderful article about the history of
Danger written by one of the engineers at that company for a glimpse
into where the Android internals came from, binder included.) The model
of binder is very limited, inflexible in its use-cases, but very
powerful and extremely low-overhead and fast. Binder ensures that the
same CPU timeslice will go from the calling process into the called
process’s thread, and then come back into the caller when finished.
There is almost no scheduling involved, and is much like a syscall into
the kernel that does work for the calling process. This interface is
very well suited for cheap devices with almost no RAM and very low CPU
resources.

So, for systems like Android, binder makes total sense, especially given
the history of it and where it was designed to be used.

D-Bus

D-Bus is a create-store-forward, compose reply and then
create-store-forward messaging model which is more complex than binder,
but because of that, it is extremely flexible, versatile, network
transparent, much easier to manage, and very easy to let fully untrusted
peers take part of the communication model (hint, never let this happen
with binder, or bad things will happen…) D-Bus can scale up to huge
amounts of data, and with the implementation of kdbus it is
possible to pass gigabytes of buffers to every connection on the bus if
you really wanted to. CPU-wise, it is not as efficient as binder, but
is a much better general-purpose solution for general-purpose machines
and workloads.

CPU vs. RAM

Yes, it’s an over simplification of a different set of complex IPC
methods, but these 3 words should help you explain the differences
between binder and D-Bus and why kdbus isn’t going to be able
to easily replace binder anytime soon.

Never say never

Ok, before you start to object to the above statements, yes, we could
add functionality to kdbus to have some blocking ioctl calls
that implement something like:
write question -> block for reply and read reply one answer
for the request side, and then on the server side do:
write answer -> block in read
That would get kdbus a tiny bit closer to the binder model, by
queueing stuff in RAM instead of relying on a thread pool.

That might work, but would require a lot of work on the binder library
side in Android, and as a very limited number of people have write
access to that code (they all can be counted on one hand), and it’s a
non-trivial amount of work for a core function of Android that is
working very well today, I don’t know if it will ever happen.

But anything is possible, it’s just software you know…

Thanks

Many thanks to Kay Sievers who came up with the CPU vs. RAM description
of binder and D-Bus and whose email I pretty much just copied into this
post. Also thanks to Kay and Lennart for taking the time and energy to
put up with my silly statements about how kdbus could replace binder,
and totally proving me wrong, sorry for having you spend so much time on
this, but I now know you are right.

Also thanks to Daniel Mack and Kay for doing so much work on the kdbus
kernel code, that I don’t think any of my original implementation is
even present anymore, which is probably a good thing. Also thanks to
Tejun Heo for help with the memfd implementation and cgroups
help in kdbus.