Home

DJI FPV Architecture

The DJI FPV system uses a shared architecture between the Goggles, Air Unit, and Vista.

The main Goggles, Air Unit, and Vista processor is a DJI P1 (seemingly codenamed Pigeon), a dual-core ARM processor with several coprocessors onboard. At least two Ceva XC4500 and one Ceva X1643 DSP are known so far, as well as a TrustZone with CryptoCell. Other processors in the family include E1 (Eagle) in the FPV Drone and S1 (Sparrow) in the V2 Remote Controller.

The basic FPV system runs Linux on one core and a custom real-time application on the second core.

On the Linux core, there's an Android kernel and base userland, not Android UI services like SurfaceFlinger or MediaServer.

For this reason, basic Android principles and services are all in play:

adb can be used for file transfer and debugging. The Android logger is employed, so adb logcat will give you interesting logs. setprop can be used to affect the Properties system and trigger actions in the running system.

DJI specific software is all stored in the /system partition.

DJI processes communicate with one another using an event bus and a message passing system called DUML.

The most important processes are in /system/bin , and libraries in /system/lib:

• dji_glasses (V1) or dji_gls_wm150 (V2) : The base DJI DIY FPV UI process. Has debugging symbols, so a great place to start.
• dji_glasses (V2) : The DJI Drone UI process. Whether to start dji_gls_wm150 or dji_glasses is selected by a script at boot time, and a full reboot is required to switch modes as the Drone also uses a different baseband/SDR RTOS image.
• dji_hdvt_gnd : The "Ground Station" HD Video Transmission service.

These services communicate over the DJI event bus using DUPC, a simple binary serialized / packed message bus format. dji_mb_ctrl and dji_mb_parser can be used to manipulate the message bus from userland. The messsage bus is also exposed over a USB-serial gadget, which can be communicated with using https://github.com/o-gs/dji-firmware-tools and comm_serialtalk.py , or online using https://b3yond.d3vl.com/duml/ .

In /system/lib:

• libtp1801_gui.so : The Embedded Wizard GUI framework
• libduml_*.so : The frameworks used by all shared DJI applications. FRWK and HAL contain hardware abstraction. OSAL contains OS Abstraction - basically RTOS primitives built on top of PThreads. Util contains... utilities.

UI is driven through a DirectFB mux - dji_glasses takes over DirectFB and something at a lower level using ICC and the "DMI devices" handles the mux between the overlay DirectFB and the raw video data from the baseband. An example of using DirectFB can be found at https://github.com/fpv-wtf/dfbdoom .

The base RF communication between the Goggles and Air Unit/Vista happens through a baseband OS called, rather simply, SDR, which runs on a second core on the same P1 processor, with shared address space. The second core is sometimes referred to as CP (Coprocessor), sometimes as SDR, and sometimes as rtos in various places in code.

Division between the RTOS and Linux seems to be through simple core affinity - Linux gets one core and the RF baseband gets the other core. Communication between the two systems happens through a combination of a proprietary ICC (Inter Core Communication) system, by virtual modem devices called "dmi" which are exposed in /dev by the pigeon_modem module (using the ICC framework under the covers), and by simply mapping buffers by opening /dev/mem since address space is shared.

cp_RTOS.bin is the RTOS image for the second core. This can be extracted, along with other Coprocessor data (like the DSP code and some data) from the file called cp.img in an update package, using https://github.com/o-gs/dji-firmware-tools.

There are a few interesting kernel modules, as well:

• pigeon_modem - this seems to provide the virtual networking interface which tunnels IP over the SDR radio link. It also provides /dev/dmi* devices which are used to, for example, stream video to and from the SDR baseband.
• icc_chnl_v2 - this is used by pigeon_modem to accomplish Inter Core Communication (ICC) with the SDR OS. It includes the usual queue, mailbox, and shared buffer primitives, seemingly with support for polling and interrupt driven queues and mailboxes.
• eagle-dsp - responsible for firmware upload and communication with a proprietary DSP coprocessor.
• linux_fusion - this is the DirectFB Fusion IPC driver. Fusion IPC seems to only be used for display tasks as far as I can tell, with DJI IPC mechanisms used in most places instead.
• prores-enc - it seems there's a hardware Prores encoder present (!)
• csa_dev - I think this is some kind of checksum offload accelerator.

Common things people might want to do:

• UI Customization: the UI is defined using XML in /system/gui/xml/ . The actions triggered by GUI elements are in libtp1801_gui.so. You can arrange menus, add/remove items, etc. simply by editing the XML files.

• MSP / Canvas Mode OSD / etc: There are a few ways this could be done. The Vista/Air Unit serial port is just /dev/ttyS0 on the Air Unit/Vista, and the goggles have an IP network link with the Vista/AU. So a trivial way to do this would be to just pipe /dev/ttyS0 over UDP back to the goggles, and then write a custom DirectFB renderer. This would require killing the goggles UI and messing with the air side to take control of the serial port, but there is no reason it wouldn't be possible. A more elegant approach would be to patch or expand the existing UI system. MSP messages are sent over the same DUML message bus, but unfortunately they are parsed on the air side and re-encoded into specific DUML/DUPC event responses, so the system would have to be heavily extended.

• Video In. The Apple input for iOS Goggles interface is managed in dji_hdvt_gnd and libduml_frwk. These would be good places to start. Or, a custom DirectFB application which opened the USB Bulk endpoints and directly accepted data could work as well - performance optimization would be interesting.