Ubuntu Frame – A picture is worth a thousand snaps
by Igor Ljubuncic on 3 December 2021
The development of graphical applications intended for use on IoT devices isn’t trivial. The complexity goes beyond the usual challenges that exist in the classic desktop and server domains. One, the IoT world is much less mature. Two, developers need to take into consideration various edge cases that do not apply to hands-on devices like laptops, for instance. Kiosks, industrial displays and digital signage devices require additional focus and rigor.
Ubuntu Frame is a solution designed to simplify and streamline the build and development of products that need graphical output. On a technical level, it is a fullscreen shell, based on Wayland, intended for interactive usage applications. On a product level, Ubuntu Frame bundles communication protocols, input protocols and security policies into a single kit, which can then be used in IoT devices. You can test it today.
Getting started
There are multiple ways you can try Ubuntu Frame. It is compatible with classical desktop applications as well as snaps, and you can run it on any standard Linux system that supports snaps, or Ubuntu Core. Ubuntu Frame integrates with toolkits such as Flutter, Qt, GTK, Electron, and SDL2, and can also be used for applications based on HTML5 and Java.
To get Ubuntu-Frame running, you need to install the snap, set the Wayland display environment variable, and then, in that same shell, launch any supported application, as we will demonstrate soon. First, make sure Ubuntu Frame is running:
snap install ubuntu-frame
export WAYLAND_DISPLAY=wayland-5
ubuntu-frame &
This set of commands will start Ubuntu Frame in the background. You should see a window on your display, with a somewhat confusing title Mir on X. This is the frame inside which graphical applications can run.
A handful of snaps ready to test and use
At this point, if you have any supported applications available out of the box, you can just run them. If you do not, Canonical’s Mir team has an IoT example on its GitHub page. You can clone the repository, and then build a snap for any one of the included applications by selecting the corresponding branch. At the moment, the repository includes working application examples for GTK3, Qt5 and SDL2. You can use these example snap recipes as a basis for snapping your own choice of application based on the same toolkits.
This is an excellent way to test how your application could or would behave once you deploy it on Ubuntu Core. You can then troubleshoot any potential problems and iron out usability issues that may arise in the development process.
You can further streamline the work by using several other tools:
- You can use the remote build feature to create snaps for architectures other than your own host system. Typically, people developing on the desktop will be using the x86 processors, whereas many IoT devices will use ARM processors.
- You can create virtual machines with graphical support using qemu-virgil. The virtual machine may also be used to install Ubuntu Core. You will then be able to SSH into the guest operating system, copy any snaps, and run necessary application tests.
Some rough edges …
You may encounter some problems, though. For example, if you run the Qt5-Bomber example, you will probably see the following errors on the command line:
snap run iot-example-graphical-snap
WARNING: wayland interface not connected! Please run: /snap/iot-example-graphical-snap/current/bin/setup.sh
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use
QT_QPA_PLATFORM=wayland to run on Wayland anyway.
In some cases, you will need to run application-specific setup scripts, which will help connect any interfaces that the software relies on, or set missing environment variables. You can also manually override any commands, and later implement the solution in the final product.
Ideally, testing the snap functionality locally, in virtual machines and Ubuntu Core will help you narrow down any potential problems. Then, you can request necessary assertions for automatic interface connections in the Snap Store, create application launchers that cover any missing bits, and give your end users a clean, smooth experience.
Summary
Ubuntu Frame is a convenient toolkit for IoT developers looking to get started with graphical applications on Ubuntu Core. It simplifies and streamlines the setup and testing, and the several examples available as part of the main repository should provide you with a good reference on how to snap your own applications. We’d appreciate feedback or suggestions, if you have any. So please join the Snapcraft forum, and let us know what you think.