How to troubleshoot application issues and crashes with snapd gdbserver

by Igor Ljubuncic on 13 May 2021

Sometimes, applications may not run well, or they could even crash. When such issues occur, it is useful to have a consistent, reproducible method of triggering the problem, so that developers can have a reliable way and sufficient data to troubleshoot the issues and produce a fix. In the software world, the GNU Debugger (gdb) is a powerful tool that allows developers to do just that.

With snaps, things are slightly more complicated. Snaps run as isolated, self-contained applications, with strong security confinement. They are managed and launched by the snapd service. This means that if you were to invoke gdb to troubleshoot snaps exhibiting startup or runtime issues, the actual application execution will be masked by the snapd processes that wrap it. To work around this phenomenon, and give developers the right tools for the job, the snap daemon also includes gdbserver, which allows users to inspect their applications in a manner that is very similar to the classic Linux system.

Invoke gdbserver

If you run a snap with the –gdb flag, gdb will launch and behave just as it would were it called against the same executable outside of the snap environment. Alternatively, you can run gdbserver, which also allows you to connect to gdb remotely, as well as offers the ability to run applications as a non-privileged (standard) user.

The syntax is as follows:

snap run --gdbserver “snap name”

This will start the snap, stop the execution at the entry point, instantiate gdb, and allow remote access to it via a random high port that will be printed on the command line.

snap run --gdbserver snapster

Welcome to "snap run --gdbserver".
You are right before your application is run.
Please open a different terminal and run:

gdb -ex="target remote :44626" -ex=continue -ex="signal SIGCONT"
(gdb) continue

or use your favorite gdb frontend and connect to :44626

Practical example

Let’s take a look at an application that crashes when invoked. If you’d like to follow along, please take a look at the following gist for details on the actual source code, the compilation flags, and the segfexample snap snapcraft.yaml contents.

snap run --gdbserver segfexample

Welcome to "snap run --gdbserver".
You are right before your application is run.
Please open a different terminal and run:

In a separate terminal window, running gdb -ex=”target remote :34621″ shows the following output:

gdb -ex="target remote :34621" -ex=continue -ex="signal SIGCONT"
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".

To resume execution, you need to type cont until you hit the issue – mind, like most issues that you can troubleshoot with gdb, these need to be time-independent, and reproducible:

Program received signal SIGCONT, Continued.
0x00007f1255a0918b in raise () from target:/lib/x86_64-linux-gnu/

Reading /lib/x86_64-linux-gnu/ from remote target…
Reading /lib/x86_64-linux-gnu/ from remote target…
Reading /lib/x86_64-linux-gnu/.debug/ from remote target…

Program received signal SIGSEGV, Segmentation fault.
0x0000561bbcd5e6be in main () at source.c:11
11            pointer[i]=i;

Once you know where the issue occurs, you can do troubleshooting like you normally would. You can set conditions, breakpoints, disassemble the code, and more. You will also need debug symbols for the specific versions of libraries that your applications uses to be able to decipher the functions where the bug occurs. If you use libraries available from your system archives, you can then install the matching debug packages. If you’re developing your own application, either compile it with symbols, or load the symbols into the debugger.


Troubleshooting problems is never easy. Sometimes, there can be complex issues in software, and it can take time and effort to resolve them. To that end, development ecosystems should be designed with as much foresight and flexibility as possible, to provide developers with a friendly and efficient setup that can help them fix code errors. With gdbserver, snapd offers snap publishers (and users) a convenient way of investigating and resolving repeatable bugs and crashes without compromising on the benefits of the snap security confinement.

If you have any suggestions on how to make application troubleshooting even more useful and resilient, please join our forum and share your thoughts.

Photo by christian buehner on Unsplash.

Newsletter Signup

Related posts

What has snapd ever done for other distros?

Snaps are self-contained applications designed with ease of use, security and portability in mind. They bundle their necessary dependencies, so they can work and run without modifications across numerous Linux distributions. How many? More than 40. Yet, often when technology is strongly associated with a particular product, in this case s […]

The magic behind snap interfaces

Snaps are confined, self-contained applications, designed with portability and security in mind. By default, strictly confined snaps run in isolation, with minimal access to system resources. For instance, they cannot access home, network, audio, or display. To make their snaps usable, developers and publishers can declare a set of interf […]

Hey snap, where’s my data?

Snaps are self-contained applications, isolated from the underlying operating systems by several layers of confinement. This mechanism introduces security benefits, but it may also create confusion for people who are less familiar with how snaps work and behave, especially when it comes to filesystem access. Today, we’d like to clarify so […]