Programs that manipulate ptys break under nix-user-chroot

[rrbutani]

First off I just wanted to say thanks for this project; it's made it really easy to use nix as an unprivileged user and is an invaluable part of my setup.

---

This issue doesn't really detail a bug in nix-user-chroot so much as a use case it doesn't support. I don't really think there's anything nix-user-chroot can do about this but I'm also not very familiar with user namespaces; I'm posting this issue here in case I missed something/in case anyone has any other ideas. If this is deemed out of scope for nix-user-chroot by the maintainers, please feel free to close this issue.

---

While using nix-user-chroot I realized that I am unable to run applications that manipulate PTYs (I was trying to use the VSCode Remote Server but I'm fairly certain this affects anything that interacts with /dev/pts; i.e. running nix-user-chroot ~/.nix screen also fails for me). Poking around a bit with strace and friends revealed that calls to forkpty in glibc ultimately try to chgrp the pts that's created to the tty group which fails inside the chroot.

This is because there is no tty group inside the namespace that nix-user-chroot sets up, only the user's UID and GID are mapped in. As far as I can tell there is no way to map in additional groups without special permissions (we're locked out of calling setgroups and mapping in gids other than the users' requires the parent process of the namespace to have CAP_SETGID, I think).

Other approaches (including mounting in a new devpts in the namespace and patching glibc to not attempt the chgrp in this case) all either don't work or seem too intrusive.

Ultimately, I ended up writing a nix overlay that "wraps" binaries in a shell script that enters the chroot if called from outside of it. This, plus a little bit of glue to make it so that all the wrappers land in the same single directory so there's something symlink-free to add to $PATH, satisfies my particular use case (instead of running the remote server inside the chroot, I just have the binaries that it calls enter the chroot themselves as needed) but feels very unsatisfying.

Does anyone know if there's a more proper way to get around this?

[Mic92]

Another hack idea: Write a new /etc/group that maps tty to your own id. Since we have limited amount of users inside we maybe should not have the normal /etc/group , /etc/passwd anyway. Mounting a new pts might be also a good idea but than one would also need to implement pty forwarding code in nix-user-chroot...

[Mic92]

Sorry closed by accident.

[rrbutani]

@Mic92 Thank you so much for the extremely prompt response! Mounting in an /etc/group that maps tty to my user id indeed does fix the issue.

screen worked right away but the VSCode server was a little more finicky; it took me some time to realize that glibc's functions were shelling out to nscd (whose daemon runs outside the chroot) for user and group lookups.

---

I'm not sure what if any changes in nix-user-chroot are appropriate for this sort of thing (should we generate new /etc/passwd and /etc/group files and map them in? which groups should we make entries for? etc.).

I ended up adding functionality to map in absolute paths and paths in the current user's Nix profile and functionality that let's you exclude paths from the chroot while I was fiddling with /etc/group and friends. This was functionality I wanted for other reasons as well (to override /bin/bash, for example).

The changes are here and the config file format looks like this:

[excludes]
paths = [
    "/var/run/nscd",
]

[profile]
"/bin/sh"               = "/bin/sh"
"/bin/bash"             = "/bin/bash"
"/bin/python3"          = "/bin/python3"
"/bin/env"              = "/usr/bin/env"
"/etc/profile.d/nix.sh" = "/etc/profile"

[absolute]
"/some/disk/config/group"  = "/etc/group"
"/some/disk/config/passwd" = "/etc/passwd"

The implementation and some of the path wrangling logic are pretty rough as is but if this is something you'd be interesting in merging, I'm very happy to clean it up, add some tests, and open a PR.

[rrbutani]

I opened a PR for the changes described in the last comment: #75.

I think the original issue in this thread is solved so I'm going to close this issue; thanks for your help @Mic92!