# [SOLVED] unprivileged lxc container

## skunk

hi!

i'm trying to deploy unprivileged lxc containers by following this guide with this error:

```

mybp@rock ~ $ lxc-create -t download -n test -f ~/.config/lxc/mybp.conf -- -d ubuntu -r xenial -a amd64

WARN: could not reopen tty: Permission denied

unshare: Operation not permitted

read pipe: Success

lxc_container: lxccontainer.c: do_create_container_dir: 760 Failed to chown container dir

lxc_container: lxc_create.c: main: 271 Error creating container test

```

searching the web i've found out both /sys/fs/cgroup/cpuset/cgroup.clone_children and /proc/sys/kernel/unprivileged_userns_clone should be set to 1.

while the former was already set, the latter doesn't even exist nor does a kernel.unprivileged_userns_clone sysctl variable...

is there any kernel module i'm missing? which one?

thank youLast edited by skunk on Mon Feb 22, 2016 5:25 pm; edited 1 time in total

----------

## alinefr

As I understand, unprivileged does not mean that you could create it with an ordinary user. You still should build it with root privileges (and the examples in the wiki are executed by root).

----------

## skunk

i guessed it was a typo since the author even runs lxc-start as root...

however even creating the container as root fails:

```

rock ~ # lxc-create -t download -n mybp -f /home/mybp/.config/lxc/mybp.conf -- -d ubuntu -r xenial -a amd64

newuidmap: uid range [0-65536) -> [100000-165536) not allowed

error mapping child

setgid: Invalid argument

lxc_container: lxccontainer.c: create_run_template: 1125 container creation template for mybp failed

lxc_container: lxc_create.c: main: 271 Error creating container mybp

rock ~ # cat /home/mybp/.config/lxc/mybp.conf

lxc.network.type = veth

lxc.network.flags = up

lxc.network.link = eth0

lxc.network.name = eth0

lxc.network.ipv4 = 192.168.10.100/24

lxc.network.ipv4.gateway = 192.168.10.1

lxc.id_map = u 0 100000 65536

lxc.id_map = g 0 100000 65536

rock ~ # grep mybp /etc/sub* 2>/dev/null

/etc/subgid:mybp:100000:65536

/etc/subuid:mybp:100000:65536

```

as a side note: manually deployed privileged containers works fine...

----------

## skunk

it turns out hardened-sources is the culpit, so i've to figure out which of these options should be turned on/off:

```

CONFIG_GRACE_PERIOD=m

CONFIG_GRKERNSEC=y

CONFIG_GRKERNSEC_CONFIG_AUTO=y

# CONFIG_GRKERNSEC_CONFIG_CUSTOM is not set

CONFIG_GRKERNSEC_CONFIG_SERVER=y

# CONFIG_GRKERNSEC_CONFIG_DESKTOP is not set

CONFIG_GRKERNSEC_CONFIG_VIRT_NONE=y

# CONFIG_GRKERNSEC_CONFIG_VIRT_GUEST is not set

# CONFIG_GRKERNSEC_CONFIG_VIRT_HOST is not set

CONFIG_GRKERNSEC_CONFIG_PRIORITY_PERF=y

# CONFIG_GRKERNSEC_CONFIG_PRIORITY_SECURITY is not set

CONFIG_GRKERNSEC_PROC_GID=10

CONFIG_GRKERNSEC_SYMLINKOWN_GID=81

CONFIG_GRKERNSEC_KMEM=y

CONFIG_GRKERNSEC_IO=y

CONFIG_GRKERNSEC_BPF_HARDEN=y

CONFIG_GRKERNSEC_PERF_HARDEN=y

CONFIG_GRKERNSEC_RAND_THREADSTACK=y

CONFIG_GRKERNSEC_PROC_MEMMAP=y

CONFIG_GRKERNSEC_KSTACKOVERFLOW=y

CONFIG_GRKERNSEC_BRUTE=y

CONFIG_GRKERNSEC_MODHARDEN=y

CONFIG_GRKERNSEC_HIDESYM=y

CONFIG_GRKERNSEC_RANDSTRUCT=y

CONFIG_GRKERNSEC_RANDSTRUCT_PERFORMANCE=y

CONFIG_GRKERNSEC_KERN_LOCKOUT=y

# CONFIG_GRKERNSEC_NO_RBAC is not set

# CONFIG_GRKERNSEC_ACL_HIDEKERN is not set

CONFIG_GRKERNSEC_ACL_MAXTRIES=3

CONFIG_GRKERNSEC_ACL_TIMEOUT=30

# CONFIG_GRKERNSEC_PROC is not set

CONFIG_GRKERNSEC_LINK=y

# CONFIG_GRKERNSEC_SYMLINKOWN is not set

CONFIG_GRKERNSEC_FIFO=y

# CONFIG_GRKERNSEC_SYSFS_RESTRICT is not set

# CONFIG_GRKERNSEC_ROFS is not set

CONFIG_GRKERNSEC_DEVICE_SIDECHANNEL=y

CONFIG_GRKERNSEC_CHROOT=y

# CONFIG_GRKERNSEC_CHROOT_MOUNT is not set

# CONFIG_GRKERNSEC_CHROOT_DOUBLE is not set

# CONFIG_GRKERNSEC_CHROOT_PIVOT is not set

CONFIG_GRKERNSEC_CHROOT_CHDIR=y

# CONFIG_GRKERNSEC_CHROOT_CHMOD is not set

CONFIG_GRKERNSEC_CHROOT_FCHDIR=y

CONFIG_GRKERNSEC_CHROOT_MKNOD=y

CONFIG_GRKERNSEC_CHROOT_SHMAT=y

CONFIG_GRKERNSEC_CHROOT_UNIX=y

CONFIG_GRKERNSEC_CHROOT_FINDTASK=y

CONFIG_GRKERNSEC_CHROOT_NICE=y

CONFIG_GRKERNSEC_CHROOT_SYSCTL=y

CONFIG_GRKERNSEC_CHROOT_RENAME=y

# CONFIG_GRKERNSEC_CHROOT_CAPS is not set

# CONFIG_GRKERNSEC_AUDIT_GROUP is not set

# CONFIG_GRKERNSEC_EXECLOG is not set

CONFIG_GRKERNSEC_RESLOG=y

# CONFIG_GRKERNSEC_CHROOT_EXECLOG is not set

# CONFIG_GRKERNSEC_AUDIT_PTRACE is not set

# CONFIG_GRKERNSEC_AUDIT_CHDIR is not set

# CONFIG_GRKERNSEC_AUDIT_MOUNT is not set

CONFIG_GRKERNSEC_SIGNAL=y

# CONFIG_GRKERNSEC_FORKFAIL is not set

CONFIG_GRKERNSEC_TIME=y

CONFIG_GRKERNSEC_PROC_IPADDR=y

CONFIG_GRKERNSEC_RWXMAP_LOG=y

CONFIG_GRKERNSEC_DMESG=y

CONFIG_GRKERNSEC_HARDEN_PTRACE=y

CONFIG_GRKERNSEC_PTRACE_READEXEC=y

CONFIG_GRKERNSEC_SETXID=y

CONFIG_GRKERNSEC_HARDEN_IPC=y

# CONFIG_GRKERNSEC_TPE is not set

CONFIG_GRKERNSEC_BLACKHOLE=y

CONFIG_GRKERNSEC_NO_SIMULT_CONNECT=y

# CONFIG_GRKERNSEC_SOCKET is not set

CONFIG_GRKERNSEC_DENYUSB=y

# CONFIG_GRKERNSEC_DENYUSB_FORCE is not set

CONFIG_GRKERNSEC_SYSCTL=y

# CONFIG_GRKERNSEC_SYSCTL_DISTRO is not set

CONFIG_GRKERNSEC_SYSCTL_ON=y

CONFIG_GRKERNSEC_FLOODTIME=10

CONFIG_GRKERNSEC_FLOODBURST=6

```

in the mean time i've installed and booted gentoo-sources and now lxc-create works fine as user.

however after setting /sys/fs/cgroup/memory/memory.use_hierarchy, /sys/fs/cgroup/cpuset/cgroup.clone_children and changing permissions on /sys/fs/cgroup/*, i'm stuck again trying to start the container:

```

mybp@rock ~ $ lxc-start -n unpriv -f ~/.config/lxc/mybp.conf -l DEBUG -o mybp.log

newuidmap: write to uid_map failed: Invalid argument

lxc-start: start.c: lxc_spawn: 955 failed to set up id mapping

lxc-start: start.c: __lxc_start: 1080 failed to spawn 'unpriv'

lxc-start: lxc_start.c: main: 342 The container failed to start.

lxc-start: lxc_start.c: main: 346 Additional information can be obtained by setting the --logfile and --logpriority options.

```

follows the log:

```

      lxc-start 1455916894.089 INFO     lxc_utils - utils.c:get_rundir:409 - XDG_RUNTIME_DIR isn't set in the environment.

      lxc-start 1455916894.089 INFO     lxc_confile - confile.c:config_idmap:1325 - read uid map: type u nsid 0 hostid 100000 range 65536

      lxc-start 1455916894.089 INFO     lxc_confile - confile.c:config_idmap:1325 - read uid map: type g nsid 0 hostid 100000 range 65536

      lxc-start 1455916894.089 WARN     lxc_log - log.c:lxc_log_init:316 - lxc_log_init called with log already initialized

      lxc-start 1455916894.089 INFO     lxc_confile - confile.c:config_idmap:1325 - read uid map: type u nsid 0 hostid 100000 range 65536

      lxc-start 1455916894.089 INFO     lxc_confile - confile.c:config_idmap:1325 - read uid map: type g nsid 0 hostid 100000 range 65536

      lxc-start 1455916894.089 INFO     lxc_confile - confile.c:config_idmap:1325 - read uid map: type u nsid 0 hostid 100000 range 65536

      lxc-start 1455916894.089 INFO     lxc_confile - confile.c:config_idmap:1325 - read uid map: type g nsid 0 hostid 100000 range 65536

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpuset unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpu unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpuacct unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup blkio unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup memory unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup devices unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup freezer unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup net_cls unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup perf_event unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup net_prio unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup hugetlb unknown to /var/lxc/mybp/.local/share/lxc unpriv

      lxc-start 1455916894.090 INFO     lxc_utils - utils.c:get_rundir:409 - XDG_RUNTIME_DIR isn't set in the environment.

      lxc-start 1455916894.090 DEBUG    lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/2' (5/6)

      lxc-start 1455916894.090 DEBUG    lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/3' (7/8)

      lxc-start 1455916894.090 DEBUG    lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/4' (9/10)

      lxc-start 1455916894.090 DEBUG    lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/5' (11/12)

      lxc-start 1455916894.090 INFO     lxc_conf - conf.c:lxc_create_tty:3676 - tty's configured

      lxc-start 1455916894.090 DEBUG    lxc_start - start.c:setup_signal_fd:247 - sigchild handler set

      lxc-start 1455916894.090 DEBUG    lxc_console - console.c:lxc_console_peer_default:500 - opening /dev/tty for console peer

      lxc-start 1455916894.090 INFO     lxc_caps - caps.c:lxc_caps_up:101 - Last supported cap was 36

      lxc-start 1455916894.090 DEBUG    lxc_console - console.c:lxc_console_peer_default:506 - using '/dev/tty' as console

      lxc-start 1455916894.090 DEBUG    lxc_console - console.c:lxc_console_sigwinch_init:179 - 332 got SIGWINCH fd 17

      lxc-start 1455916894.090 DEBUG    lxc_console - console.c:lxc_console_winsz:88 - set winsz dstfd:14 cols:172 rows:45

      lxc-start 1455916894.487 INFO     lxc_start - start.c:lxc_init:443 - 'unpriv' is initialized

      lxc-start 1455916894.488 DEBUG    lxc_start - start.c:__lxc_start:1058 - Not dropping cap_sys_boot or watching utmp

      lxc-start 1455916894.488 INFO     lxc_start - start.c:lxc_spawn:802 - Cloning a new user namespace

      lxc-start 1455916894.488 INFO     lxc_cgroup - cgroup.c:cgroup_init:62 - cgroup driver cgroupfs initing for unpriv

      lxc-start 1455916894.492 ERROR    lxc_start - start.c:lxc_spawn:955 - failed to set up id mapping

      lxc-start 1455916894.492 INFO     lxc_utils - utils.c:get_rundir:409 - XDG_RUNTIME_DIR isn't set in the environment.

      lxc-start 1455916894.597 ERROR    lxc_start - start.c:__lxc_start:1080 - failed to spawn 'unpriv'

      lxc-start 1455916894.597 INFO     lxc_utils - utils.c:get_rundir:409 - XDG_RUNTIME_DIR isn't set in the environment.

      lxc-start 1455916894.597 INFO     lxc_utils - utils.c:get_rundir:409 - XDG_RUNTIME_DIR isn't set in the environment.

      lxc-start 1455916894.598 ERROR    lxc_start_ui - lxc_start.c:main:342 - The container failed to start.

      lxc-start 1455916894.598 ERROR    lxc_start_ui - lxc_start.c:main:346 - Additional information can be obtained by setting the --logfile and --logpriority options.

```

----------

## skunk

ok, got confused by the guide, i've left a note on the discussion tab...

----------

## mimosinnet

Skunk,

I have been using LXC for a while following a previous version of the wiki, and the Unprivileged Cointainers looks quite promising. Thanks a lot for the comments.

Cheers!

----------

## skunk

you're welcome, however i'm still wondering if this is enough for a true unprivileged container and why i get "newuidmap: write to uid_map failed: Invalid argument" if i set subuids/subgids to 100000-165536 regardless having SUB_U/GID_MIN and SUB_U/GID_MAX setted with 100000 and 600100000 into /etc/login.defs...

----------

