DA

Personal project · Desktop application

Phone Master

Turn an Android phone into the controls for a Linux desktop

  • Python 3
  • websockets
  • xdotool
  • playerctl
  • x11vnc
  • systemd
  • Kotlin
  • Jetpack Compose
  • sshj
  • Android Keystore
  • OkHttp
  • RFB / VNC
Download View on GitLab

Two paired applications. A Python agent runs on the Linux machine as a systemd user service; a native Android client connects to it and turns the phone into a trackpad, a keyboard, a media remote, a dictation microphone and a viewer for the desktop screen.

Everything travels inside an SSH tunnel to the machine’s own sshd — no port is opened to the network and no separate account system exists. Authentication is a password or an SSH key that the app generates inside the Android Keystore, where the private half is non-exportable and every signature is produced by the phone’s secure hardware.

The control channel is a WebSocket carrying small JSON messages rather than one SSH command per event, because a mouse pointer cannot wait for a new shell each time it moves.

What it does

Control surfaces

  • Trackpad: relative pointer movement with adjustable sensitivity, tap for left click, double tap, long press for right click, and a scroll strip
  • Keyboard: free text plus the keys a touch keyboard has no room for — Ctrl, Alt, Tab, Esc, arrows and the function row, as modifiers and combinations
  • Media: play, pause, next and previous over MPRIS, with volume and mute through PipeWire
  • System commands: lock, suspend, log out, reboot and power off, plus any custom command defined on the desktop side
  • Dictation: on-device speech recognition in Serbian or English, typed straight into whatever has focus on the desktop

Screen mirroring

  • Mirrors the X11 session that is already running, rather than opening a second one — you see the desktop as it is, not an empty parallel login
  • A minimal RFB 3.8 client written for the project: VNC authentication, Raw, CopyRect and ZRLE encodings
  • ZRLE keeps the frames small enough to stay responsive through the tunnel
  • Multi-monitor aware: mirror everything, or clip to one screen
  • Touch drives the pointer as a trackpad rather than as absolute coordinates, which is what makes a 6-inch screen usable against a 27-inch desktop

Connection and security

  • The agent is reachable only through SSH port forwarding; nothing is exposed directly
  • The SSH private key is generated in the Android Keystore and never leaves the phone — signing happens inside the keystore, so there is no key file to copy or steal
  • Pairing from the phone: connect once with a password, then send the public key and the agent writes it into authorized_keys with the right permissions
  • Dangerous commands are marked on the desktop and ask for confirmation on the phone before they run
  • Features are individually switchable in the agent’s config, so the phone can be allowed the mouse but not the shell

Packaging and updates

  • The agent installs as a systemd user service tied to the graphical session, so it starts with the desktop and inherits the X11 display it needs to control
  • The Android build is signed with a stable release key, which is what makes updating in place possible at all
  • The app updates itself: on startup it reads a manifest from this site and offers the new version, with release notes and a progress bar
  • A downloaded APK is checked against its published SHA-256 before the installer is handed anything, so a truncated or substituted download cannot reach the install step

Download

Android APK

Android 14+

Signed build for sideloading. Android will ask you to allow installing apps from this source the first time. Later versions install themselves from inside the app.

Download

Source archive

Any platform

Both halves: the Python agent and the Android client, with their packaging scripts.

Download

Requirements

  • Phone: Android 14 or newer
  • Desktop: Linux running an X11 session (Wayland is not supported yet)
  • Desktop: openssh-server, plus xdotool; playerctl, wireplumber and x11vnc for the media, volume and screen features
  • Both devices on the same network — the phone reaches the desktop over SSH

Installing

Phone

Open the APK from the Downloads notification

Android asks once for permission to install from this app, then installs normally. From then on the app checks for new versions itself on startup and offers to install them, so this is the only manual install.

Desktop agent

cd desktop && ./packaging/install.sh

Creates a virtualenv under ~/.local/share/phone-master, installs a systemd user service and starts it. No root: it runs as you, because it needs your graphical session to control. --uninstall removes it again.

Pairing

Add the computer in the app → connect with your password → “Pošalji SSH ključ”

The app generates a key inside the Android Keystore and the agent installs the public half into ~/.ssh/authorized_keys. After that the password is no longer needed.

Source code

https://gitlab.com/dusan.atanackovic.pydev/phone-master

The repository is private. If you would like to look through the code, get in touch and I will grant access.