Adam: a second birth to Android’s ddmlib

In this post, I’ll introduce you to adam: a kotlin-based replacement for developer-to-android-device interaction.

Most of the current tools for interaction with ADB server use ddmlib: a Java-based library that is supposed to help tool developers.

The following diagram illustrates the flow of interaction:

In case you’re executing something like your ADB client is the binary. Unless you specify that you’re using a non-default ADB server, it gets connected via TCP/IP to a daemon process of ADB server that is open by default at TCP/5037. This ADB server process acts as a multiplexor for all the different devices that can be connected via USB and also via TCP/IP (a special transport mode is also reserved by the locally running emulator).

ddmlib library is another frontend for the ADB server. We probably don’t want to start a new process for every new command on the device due to performance concerns.

ADB server API

All of the interaction with the ADB server happens via a TCP/IP socket. There is little to no documentation on how this should be done so I’ll try to explain this in a bit more detail here. First, you need to open a socket.

General request flow

Let’s start with a simple example: executing a shell command for a device with a serial number. All the communication happens with the ISO/IEC 8859–1 encoding.

  1. Write to select emulator-5554 as the target device for the next request on this socket. The first 4 bytes in the message indicate the total hex length of this message
  2. Read 4 bytes from the socket. If everything is ok, then they will equal . If that’s not the case then next 4 bytes will indicate the error message length in hex followed by the actual error message
  3. Write to indicate that we want to use the shell service and execute the command
  4. Repeat step 2
  5. Read the actual output of the command from the socket

As you can see, the general structure of the request here starts with the request length.

Also worth pointing out is the flow of sending the request and then reading the status of the request processing followed by the actual response.

Types of requests

Requests to the ADB server could be divided into two separate groups:

  1. Single return request, e.g. executing or listing currently connected devices
  2. Continuous flow request, e.g. reading output or pushing a file to the device

Single return requests

Request target

We’ve used this request before in the form of . But what other values can we use here?

When executing request, you might ask for a response from ADB server, device with a specific serial number, a USB device or a local emulator instance.

The most explicit one is, of course, the serial number one. All the other requests assume that there is only one target. For example, will fail if there are multiple devices connected via USB

The following is a skeleton of a request:

By default, each request is targeted at the ADB server and can be overridden. The method is responsible for the execution of a request. Function handles the creation of a serialized request and automatically appends the prefix for the message length.

To simplify the process of verifying that every single command returns the message I’ve written :

Instead of reading the socket directly we now get back a which contains a boolean and an optional error .

Executing a single return request

Every request that returns a single value should do the handshake and the implement the actual return of the value:

Since all of the processing logic happens inside the requests, and the threading is inherited from the scope of executing, the usage looks something like:

Shell-based requests

A lot of interaction with Android devices happens via the shell. Here is a partial list of commands that are commonly used:

  • Listing device properties via
  • Package install via
  • Packages list via
  • Package uninstall via

System requests

Other types of interactions are done using the services implemented in the ADB:

  • Rebooting with
  • Port-forwarding

Continuous requests

Shell commands

Some of the commands will continuously run until either the command will finish the execution or the command is cancelled. Notable examples are:

  • Log output via
  • Running tests with

Sync service

This is a file synchronization service, used to implement, for example:

  • Pushing files
  • Pulling files

These commands are continuos because they report back the status of the transfer. We all love our progress bars instead of a lifeless callback.

Adam

So why the heck would anyone work on a rewrite of an official library?

I’m the owner of Marathon: a cross-platform test runner. One of the platforms that it currently supports is Android, and we (the team behind this project) have faced a lot of limitations of the only presently available developer API for interaction with the ADB server.

Sub-optimal resources usage

As you’d expect from a test runner, most of the time, the runner is waiting for the tests to finish. This led us to the use of coroutines extensively throughout our codebase. Unfortunately, all of the ddmlib’s code is blocking. What’s worse is that most of its commands spawn a new thread for each request. When testing on hundreds of devices at the same time, this leads to large thread pools and uses a lot of RAM.

For example, executing 30 tests on three devices will require:

╔═══════════╦════════════════════╦═══════════════╗
║ ║ Threads ║ RAM ║
╠═══════════╬════════════════════╬═══════════════╣
║ Ddmlib ║ 31 ║ 815Mb ║
║ Adam ║ 28 ║ 436Mb ║
╚═══════════╩════════════════════╩═══════════════╝

These numbers scale with the number of devices you add to the test run.

Code is not tested properly

This is one of the biggest concerns I have for ddmlib. It’s a library that is used extensively in a lot of tools: IntelliJ, IntelliJ Android plugin, fork, spoon and more. Nevertheless, I would expect more tests and coverage even though testing so many edge cases with new Thread() calls would be quite hard:

Limitations of ADB server are propagated to the user of ddmlib

We love using simple and clear API. With ddmlib even though most of the communication happens via sockets, the library itself doesn’t do a good job of propagating the most critical feature of each socket interaction — the timeout handling. Some of the commands have the configurable timeout as a parameter for the method like getScreenshot while others do not have this luxury. Even when the timeout can be specified, the implementation only respects the timeout for some part of the request, in case of its only one method. If the method gets stuck then you’re out of luck. This behaviour makes the usage of these API’s very hard and cumbersome.

General adam usage example

To use adam, you will need:

  1. A Kotlin-based project targeting JVM
  2. Coroutines support (inherited from adam as a dependency)

Links

https://github.com/Malinskiy/adam

Software engineer & IT conference speaker; Landscape photographer + occasional portraits; Music teacher: piano guitar violin; Bike traveller, gymkhana