Analyzing and Extending Minicap for Dynamic Resolution and Rotation Handling

1. Introduction

Minicap is an open‑source component of the STF (Smartphone Test Farm) framework that provides screen capture functionality on Android devices. It is built with the Android NDK and serves as a low‑level dependency of STF.

Source code: https://github.com/openstf/minicap

When integrating Minicap into a real‑device cloud platform, two shortcomings were observed:

After device rotation, the captured image contains black regions and is incomplete.

The image resolution cannot be changed dynamically while Minicap is running; a restart is required for a different configuration.

To address these issues, the following sections analyze the original code and propose modifications.

2. Code Analysis

(1) Minicap Usage Overview

Command‑line start: Launch Minicap on the device and view process information via the terminal.

a. Verify Minicap can start

Run the command with the -P parameter (screen size) and check for an OK response.

b. Start Minicap

Execute the start command on the device.

c. View Minicap information

Use adb shell ps | grep minicap (illustrated in the original screenshots) to inspect the running service.

(2) Local Port Forwarding

Port forwarding is set up so that the host can receive the image stream from the device (see screenshot).

(3) Client Connection Creation

The client connects to the local forwarded port to receive images.

(4) Internal Execution Flow

(1) Parsing startup parameters

The -P option defines the size of the image generated by Minicap and the size transmitted to the client. Example: 1080x1920@360x640/0 means the device captures at 1080×1920, sends 360×640 to the client, and 0 indicates portrait orientation; rotation values 90, 180, 270 adjust for device rotation.

(2) Setting compression ratio

The -Q flag sets JPEG quality (0‑100).

(3) Initialising internal objects

Calls such as setRealInfo , setDesiredInfo , and setFrameAvailableListener configure display parameters and frame‑rate listeners, and the protocol header (24 bytes) is prepared.

void forceAspectRatio() {
    double aspect = static_cast<double>(realWidth) / static_cast<double>(realHeight);
    if (virtualHeight > (uint32_t) (virtualWidth / aspect)) {
        virtualHeight = static_cast<uint32_t>(round(virtualWidth / aspect));
    } else {
        virtualWidth = static_cast<uint32_t>(round(virtualHeight * aspect));
    }
}

(4) Listening for client connections (single client only)

int sfd = socket(AF_UNIX, SOCK_STREAM, 0);
if (sfd < 0) { return sfd; }
struct sockaddr_un addr;
memset(&addr, 0, sizeof(addr));
addr.sun_family = AF_UNIX;
strncpy(&addr.sun_path[1], sockname, strlen(sockname));
if (::bind(sfd, (struct sockaddr*) &addr, sizeof(sa_family_t) + strlen(sockname) + 1) < 0) {
    ::close(sfd);
    return -1;
}
::listen(sfd, 1);

(5) Server‑side socket communication

while (!gWaiter.isStopped() && (fd = server.accept()) > 0) {
    MCINFO("New client connection");
    if (pumps(fd, banner, BANNER_SIZE) < 0) {
        close(fd);
        MCINFO("pumps fail,return");
        continue;
    }
    ...
}

(6) Transmitting image data

Minicap sends the image size followed by the JPEG payload.

unsigned char* data = encoder.getEncodedData();
size_t size = encoder.getEncodedSize();
putUInt32LE(data, size);
if (pumps(fd, data, size+4) < 0) {
    MCINFO("pumps break");
    break;
}

3. Modifications to Minicap

(1) Real‑time rotation handling

Goal: After the device rotates, the captured image remains correct.

Changes:

Obtain the device’s actual screen resolution and orientation via minicap_try_get_display_info (fallback to try_get_framebuffer_display_info ).

Update desiredInfo.orientation with the retrieved orientation.

Re‑apply the configuration using setDesiredInfo and applyConfigChanges .

if (minicap_try_get_display_info(displayId, &info) != 0) {
    if (try_get_framebuffer_display_info(displayId, &info) != 0) {
        MCERROR("Unable to get display info");
        return EXIT_FAILURE;
    }
}

desiredInfo.orientation = info.orientation;

if (minicap->setDesiredInfo(desiredInfo) != 0) {
    MCERROR("Minicap did not accept desired display info");
    goto disaster;
}
if (minicap->applyConfigChanges() != 0) {
    MCERROR("Unable to start minicap with current config");
    goto disaster;
}

Result: Captured images are correctly oriented.

(2) Adding rotation metadata to the image stream

Goal: Clients can know the image’s rotation and display it without distortion.

Changes:

Increase the header size of the JPEG packet (from 4 bytes to 8 bytes).

Insert the rotation value before the image size.

unsigned char* data = encoder.getEncodedData() - 8;
size_t size = encoder.getEncodedSize();

putUInt32LE(data, rotation);
putUInt32LE(data + 4, size);

if (pumps(fd, data, size+8) < 0) {
    MCINFO("pumps break");
    break;
}

Result: The client receives both rotation and image data, enabling proper display.

(3) Dynamic resolution adjustment

Goal: Switch between low‑ and high‑definition streams based on network conditions.

Changes:

Retrieve the real screen size and orientation (same as in step 1).

Set desiredInfo.width and desiredInfo.height to the desired virtual dimensions.

Re‑apply the configuration.

desiredInfo.width = proj.virtualWidth;
desiredInfo.height = proj.virtualHeight;

if (minicap->setDesiredInfo(desiredInfo) != 0) {
    MCERROR("Minicap did not accept desired display info");
    goto disaster;
}
if (minicap->applyConfigChanges() != 0) {
    MCERROR("Unable to start minicap with current config");
    goto disaster;
}

Result: The transmitted image resolution can be changed on‑the‑fly, improving clarity when bandwidth permits.

(4) Handling special devices

Some devices fail to provide display info via the standard APIs. Alternative methods include:

Using Android’s WindowManager to query getInitialDisplaySize and getBaseDisplaySize .

Running ADB commands such as wm size , dumpsys window displays , or dumpsys display to obtain resolution.

Getting rotation via getWindowManager().getDefaultDisplay().getRotation() and mapping it to standard orientation constants, or via dumpsys input | grep SurfaceOrientation .

private int getScreenOrientation() {
    int rotation = getWindowManager().getDefaultDisplay().getRotation();
    DisplayMetrics dm = new DisplayMetrics();
    getWindowManager().getDefaultDisplay().getMetrics(dm);
    int width = dm.widthPixels;
    int height = dm.heightPixels;
    int orientation;
    if ((rotation == Surface.ROTATION_0 || rotation == Surface.ROTATION_180) && height > width ||
        (rotation == Surface.ROTATION_90 || rotation == Surface.ROTATION_270) && width > height) {
        switch(rotation) {
            case Surface.ROTATION_0: orientation = ActivityInfo.SCREEN_ORIENTATION_PORTRAIT; break;
            case Surface.ROTATION_90: orientation = ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE; break;
            case Surface.ROTATION_180: orientation = ActivityInfo.SCREEN_ORIENTATION_REVERSE_PORTRAIT; break;
            case Surface.ROTATION_270: orientation = ActivityInfo.SCREEN_ORIENTATION_REVERSE_LANDSCAPE; break;
            default: Log.e(TAG, "Unknown screen orientation. Defaulting to portrait."); orientation = ActivityInfo.SCREEN_ORIENTATION_PORTRAIT; break;
        }
    }
    return orientation;
}

4. Conclusion

Well‑designed code reduces long‑term maintenance costs. Before coding, design a structure that is easy to maintain and extend; keep the implementation concise and clear to minimize bugs.