Adam: a second birth to Android’s ddmlib
In this post, I’ll introduce you to adam: a kotlin-based replacement for developer-to-android-device interaction.
Most of the current tools for interaction with ADB server use ddmlib: a Java-based library that is supposed to help tool developers.
The following diagram illustrates the flow of interaction:
In case you’re executing something like adb shell ls your ADB client is the adb binary. Unless you specify that you’re using a non-default ADB server, it gets connected via TCP/IP to a daemon process of ADB server that is open by default at TCP/5037. This ADB server process acts as a multiplexor for all the different devices that can be connected via USB and also via TCP/IP (a special transport mode is also reserved by the locally running emulator).
ddmlib library is another frontend for the ADB server. We probably don’t want to start a new process for every new command on the device due to performance concerns.
ADB server API
All of the interaction with the ADB server happens via a TCP/IP socket. There is little to no documentation on how this should be done so I’ll try to explain this in a bit more detail here. First, you need to open a socket.
General request flow
Let’s start with a simple example: executing a shell command ls for a device with a emulator-5554 serial number. All the communication happens with the ISO/IEC 8859–1 encoding.
- Write
001Chost:transport:emulator-5554to select emulator-5554 as the target device for the next request on this socket. The first 4 bytes in the message indicate the total hex length of this message - Read 4 bytes from the socket. If everything is ok, then they will equal
OKAY. If that’s not the case then next 4 bytes will indicate the error message length in hex followed by the actual error message - Write
0008shell:lsto indicate that we want to use the shell service and execute thelscommand - Repeat step 2
- Read the actual output of the command from the socket
As you can see, the general structure of the request here starts with the request length.
Also worth pointing out is the flow of sending the request and then reading the status of the request processing followed by the actual response.
Types of requests
Requests to the ADB server could be divided into two separate groups:
- Single return request, e.g. executing
lsor listing currently connected devices - Continuous flow request, e.g. reading
logcatoutput or pushing a file to the device
Single return requests
Request target
We’ve used this request before in the form of 001Chost:transport:emulator-5554 . But what other values can we use here?
When executing request, you might ask for a response from ADB server, device with a specific serial number, a USB device or a local emulator instance.
The most explicit one is, of course, the serial number one. All the other requests assume that there is only one target. For example, host-usb: will fail if there are multiple devices connected via USB
The following is a skeleton of a request:
By default, each request is targeted at the ADB server and can be overridden. The methodhandshake is responsible for the execution of a request. Function createBaseRequest handles the creation of a serialized request and automatically appends the prefix for the message length.
To simplify the process of verifying that every single command returns the OKAY message I’ve written AndroidReadChannel :
Instead of reading the socket directly we now get back a TransportResponse which contains a boolean okay and an optional error message.
Executing a single return request
Every request that returns a single value should do the handshake and the implement the actual return of the value:
Since all of the processing logic happens inside the requests, and the threading is inherited from the scope of executing, the usage looks something like:
Shell-based requests
A lot of interaction with Android devices happens via the shell. Here is a partial list of commands that are commonly used:
- Listing device properties via
getprop - Package install via
pm install - Packages list via
pm list - Package uninstall via
pm uninstall
System requests
Other types of interactions are done using the services implemented in the ADB:
- Rebooting with
reboot:<|recovery|bootloader> - Port-forwarding
forward:SRC:TRG
Continuous requests
Shell commands
Some of the commands will continuously run until either the command will finish the execution or the command is cancelled. Notable examples are:
- Log output via
logcat - Running tests with
am instrument
Sync service
This is a file synchronization service, used to implement, for example:
- Pushing files
- Pulling files
These commands are continuos because they report back the status of the transfer. We all love our progress bars instead of a lifeless callback.
Adam
So why the heck would anyone work on a rewrite of an official library?
I’m the owner of Marathon: a cross-platform test runner. One of the platforms that it currently supports is Android, and we (the team behind this project) have faced a lot of limitations of the only presently available developer API for interaction with the ADB server.
Sub-optimal resources usage
As you’d expect from a test runner, most of the time, the runner is waiting for the tests to finish. This led us to the use of coroutines extensively throughout our codebase. Unfortunately, all of the ddmlib’s code is blocking. What’s worse is that most of its commands spawn a new thread for each request. When testing on hundreds of devices at the same time, this leads to large thread pools and uses a lot of RAM.
For example, executing 30 tests on three devices will require:
╔═══════════╦════════════════════╦═══════════════╗
║ ║ Threads ║ RAM ║
╠═══════════╬════════════════════╬═══════════════╣
║ Ddmlib ║ 31 ║ 815Mb ║
║ Adam ║ 28 ║ 436Mb ║
╚═══════════╩════════════════════╩═══════════════╝These numbers scale with the number of devices you add to the test run.
Code is not tested properly
This is one of the biggest concerns I have for ddmlib. It’s a library that is used extensively in a lot of tools: IntelliJ, IntelliJ Android plugin, fork, spoon and more. Nevertheless, I would expect more tests and coverage even though testing so many edge cases with new Thread() calls would be quite hard:
Limitations of ADB server are propagated to the user of ddmlib
We love using simple and clear API. With ddmlib even though most of the communication happens via sockets, the library itself doesn’t do a good job of propagating the most critical feature of each socket interaction — the timeout handling. Some of the commands have the configurable timeout as a parameter for the method like getScreenshot while others do not have this luxury. Even when the timeout can be specified, the implementation only respects the timeout for some part of the request, in case of getScreenshot its only one read method. If the write method gets stuck then you’re out of luck. This behaviour makes the usage of these API’s very hard and cumbersome.
General adam usage example
To use adam, you will need:
- A Kotlin-based project targeting JVM
- Coroutines support (inherited from adam as a dependency)
