from [email protected] to [email protected] on 17 May 14:59
https://feddit.org/post/12610534
Hi everyone!
I’m in the process of finally doing containers right in my NixOS installation. This is my ‘wishlist’:
- podman containers should be run by users with minimal permissions
- separate user per container
- containers managed by systemd services for easier management
My current work-in-progress setup looks like this:
For each service (called $name
), I have:
- a user and corresponding group (referred to as
$uid
in the following) - a directory
/srv/$name
owned by$uid
, in which mounted volumes are located
My containers are declared like this:
virtualisation.oci-containers.containers = { $name = { image = ...; ports = [ ... ]; volumes = [ "/srv/${name}/config:/config" ... ]; user = $uid:$gid; extraOptions = [ "--security-opt=no-new-privileges:true" ]; }; };
Now for the parts I don’t fully understand yet:
-
some images allow setting
environment.PUID
to specify a user. Does setting this option (and not settinguser=$uid
in the container declaration itself) mean that the container will be run as root, and the program inside will merely usePUID
when e.g. creating files? This would still allow a malicious container to run commands as root on the host, right? -
virtualisation.oci-containers.containers
creates a systemd service. Since this is not a user-service for my user$uid
, I need sudo to start/stop the container. Does that mean that the systemd service is run with root permissions, but it executes the command to spawn the container as$uid
? If whatever is running inside the container was malicious, is there a functional difference between the container being started ‘by root as$uid
’ and it being started by me (after logging in as$uid
)? -
Is it feasible to make these systemd services user-services owned by
$uid
instead? -
Are there further hardening steps I forgot about?
Thanks for your input!
#selfhosted
threaded - newest
What services are you running in your pods/containers? Are they local applications like libreoffice or are they network accessible in the more traditional style? What’s the advantage to running a podman container on your machine vs a Flatpak container?
Sorry for all the questions. This is an interesting setup and I’m just really curious.
These containers are running on various servers I have at home, not on a desktop machine. I use podman as an alternative to docker, because it’s fully libre and does not require running containers as root. To be honest, I’ve never thought about running flatpak containers for these kinds of services – do you have a setup like this that you want to share?
That makes sense. I’ve always thought of NixOS as a desktop distro, not as a server. Guess I need to expand my thinking!
I run Fedora Server with podman and docker side by side. I try to use podman whenever possible but sometimes it’s not worth the hassle so that’s when it becomes a docker container 😬
I’d say NixOS is great for servers, mostly. Only having to worry about certain things (secure boot with custom keys, FDE, partition layout, network, sshd, firejail, etc.) once, and then replicating the same setup on another machine is waaay more convenient than going “I wonder what I was thinking when setting up this machine” once in a while when looking at some machine again you haven’t touched in some time. When it comes to desktop usage, the whole thing does not feel as magical - configuring system options in e.g. KDE is still a lot of clicking around in a GUI. I still use it for my desktop machine, just so I don’t have to think about another distro.
PUID
is indeed handled inside the container itself, it'll run a container-provided script as whatever the container's UID 0 happens to be first which then drops to whatever$PUID
happens to be inside the container.user=
is enforced by Podman itself before the container starts, but Podman will still run as root in that setup. That means Podman is running "rootful", while if you started the container manually as $uid using the regular Podman CLI, it would be "rootless". That is a major difference in a lot of respects, including security, and you can find quite a bit of documentation on the differences between those operating modes online; it wouldn't fit in a comment. Rootless is generally considered the better mode, though there are some things that still require a rootful container.In the upcoming NixOS 25.05 or current unstable, there are some tools you can use to run containers rootless as another user more easily using a new
$name.podman.user = "";
setting. From what I understand they'll still be root-managed systemd system services that require sudo to operate, but that means privileges get dropped by systemd before running Podman, instead of dropped by Podman before running the container. This stuff is recent and I haven't used it, I just happen to know it exists, relevant nixpkgs commit if you wanna dig into it yourself: https://github.com/NixOS/nixpkgs/commit/7d443d378b07ad55686e9ba68faf16802c030025You just made my day, kind internet person! That’s exactly the holy grail setup I’ve been looking for for the last couple of months. Will try it out as soon as I can!