# emerge over chrooted nfs share?

## peter4

I've been wondering: is it possible to export one (slow) computer's whole filesystem via nfs, chroot into it from a more powerful computer and do the heavy lifting from there (emerge chromium)? 

I've tried setting it up, but had no success so far (it's the first time I'm dealing with nfs). I'm stuck at a point where I can mount the nfs share, but I can't access filesystems mounted there, like /proc or /sys. They appear as empty dirs. I tried "nohide" option, with no effect.

----------

## Hu

Why not use emerge --buildpkg on the fast computer and emerge --usepkgonly on the slow computer?

----------

## Jacekalex

Discc?

----------

## bbgermany

 *peter4 wrote:*   

> I've been wondering: is it possible to export one (slow) computer's whole filesystem via nfs, chroot into it from a more powerful computer and do the heavy lifting from there (emerge chromium)? 
> 
> I've tried setting it up, but had no success so far (it's the first time I'm dealing with nfs). I'm stuck at a point where I can mount the nfs share, but I can't access filesystems mounted there, like /proc or /sys. They appear as empty dirs. I tried "nohide" option, with no effect.

 

since you will need the /proc and /sys of the compiling system (the fast system), you should try mount via bind the local /sys and and /proc this way, before chroot into to nfs system:

```

mount -o bind /proc /mnt/nfs/proc

mount -o bind /sys /mnt/nfs/bind

```

in some cases you need the /dev as well. afterwards just do as stated in the handbook and everything should work.

bb

----------

## Chewi

I won't go into full detail just now but I do this with several machines using this script I wrote. Create a symlink to it like chroot-hostname.

```
#!/bin/sh

HOST=${0##*/}

HOST=${HOST#*-}

mkdir -p --mode=0755 /mnt/${HOST}

mount -t nfs -o rw,intr,noatime,actimeo=60,vers=4,fsc ${HOST}:/ /mnt/${HOST}

mount --bind /dev /mnt/${HOST}/dev

mount --bind /dev/shm /mnt/${HOST}/dev/shm

mount --bind /proc /mnt/${HOST}/proc

mount --bind /sys /mnt/${HOST}/sys

mount --bind /usr/portage /mnt/${HOST}/usr/portage

mount --bind /usr/local/portage /mnt/${HOST}/usr/local/portage

mount --bind /var/tmp/portage /mnt/${HOST}/var/tmp/portage

env -i - HOME="/root" TERM="$TERM" chroot /mnt/${HOST} /bin/bash -l

umount /mnt/${HOST}/dev/shm

umount /mnt/${HOST}/dev

umount /mnt/${HOST}/proc

umount /mnt/${HOST}/sys

umount /mnt/${HOST}/usr/portage

umount /mnt/${HOST}/usr/local/portage

umount /mnt/${HOST}/var/tmp/portage

umount /mnt/${HOST}
```

NFSv4 can be quite confusing, even if you've used NFSv3 before. I also sometimes get a very long delay before the bash prompt actually appears, even though no network traffic is passing during that time. I haven't been able to figure out why.

----------

## Hu

 *Chewi wrote:*   

> 
> 
> ```
> mount --bind /var/tmp/portage /mnt/${HOST}/var/tmp/portage
> ```
> ...

 Although I understand you do not want to send Portage temporary files over the network during compilation, binding your local /var/tmp/portage is a bit unsafe.  If you are merging the same package on multiple machines at once, they will all be sharing a work area, which could cause unwanted behavior.  You could avoid this by creating /var/tmp/portage/$HOST and mounting it into the chroot as /var/tmp/portage.

 *Chewi wrote:*   

> I also sometimes get a very long delay before the bash prompt actually appears, even though no network traffic is passing during that time. I haven't been able to figure out why.

 Have you checked which process is blocked during this time?  Specifically, is it stalled waiting for a mount to finish or is it stalled waiting for the chroot'd bash to initialize?  You can use set -x at the top of your script to print each line before it executes, so you can see how far it gets before the stall occurs.

----------

## Chewi

 *Hu wrote:*   

> Although I understand you do not want to send Portage temporary files over the network during compilation, binding your local /var/tmp/portage is a bit unsafe.  If you are merging the same package on multiple machines at once, they will all be sharing a work area, which could cause unwanted behavior.  You could avoid this by creating /var/tmp/portage/$HOST and mounting it into the chroot as /var/tmp/portage.

 

Good point, I hadn't thought of that. I don't usually build on more than one at once but as it happens, I was today. Chances of a conflict are low but yeah, it could happen and wouldn't be pretty.

 *Hu wrote:*   

> Have you checked which process is blocked during this time?  Specifically, is it stalled waiting for a mount to finish or is it stalled waiting for the chroot'd bash to initialize?  You can use set -x at the top of your script to print each line before it executes, so you can see how far it gets before the stall occurs.

 

It's not waiting for the NFS mount because if I do ls /mnt/hostname during the delay, it returns with the remote contents immediately. It's something to do with chroot and/or bash.

It's not the cause (it happened before) but I forgot to mention that I am using cachefilesd to help speed things up a bit though I'm not sure how effective it really is. I still get a lot of network traffic even for repetitive operations.

----------

## Hu

Since you know the specific hung command, you could modify your script to run that command under strace with output saved to a file.  Try strace -ff -tt -o /tmp/nfschroot.strace to get per-process traces, with timestamps.  Once the hang ends, you can examine the traces to see which process(es) were blocked and what system call they made right before blocking.

----------

## Joseph_sys

 *Chewi wrote:*   

> I won't go into full detail just now but I do this with several machines using this script I wrote. Create a symlink to it like chroot-hostname.
> 
> ```
> #!/bin/sh
> 
> ...

 

Can you explain me what these two line do in your script?

```
#!/bin/sh

HOST=${0##*/}

HOST=${HOST#*-}

... 
```

The second part of the script 

```
umount /mnt/${HOST}/dev/shm

umount /mnt/${HOST}/dev

umount /mnt/${HOST}/proc

umount /mnt/${HOST}/sys

umount /mnt/${HOST}/usr/portage

umount /mnt/${HOST}/usr/local/portage

umount /mnt/${HOST}/var/tmp/portage

umount /mnt/${HOST}
```

is run after you have done upgrade/compiling isn't it?

----------

## Joseph_sys

I'm trying to combine information to make bash script to boot strap via NFS Box_slow on Box_fast to upgrade slower boxes.

I've found this link:

http://www.wikigentoo.ksiezyc.pl/Slow_systems.htm

and this script:

```
#!/bin/sh

HOST=${0##*/}

HOST=${HOST#*-}

mkdir -p --mode=0755 /mnt/${HOST}

mount -t nfs -o rw,intr,noatime,actimeo=60,vers=4,fsc ${HOST}:/ /mnt/${HOST}

mount --bind /dev /mnt/${HOST}/dev

mount --bind /dev/shm /mnt/${HOST}/dev/shm

mount --bind /proc /mnt/${HOST}/proc

mount --bind /sys /mnt/${HOST}/sys

mount --bind /usr/portage /mnt/${HOST}/usr/portage

mount --bind /usr/local/portage /mnt/${HOST}/usr/local/portage

mount --bind /var/tmp/portage /mnt/${HOST}/var/tmp/portage

env -i - HOME="/root" TERM="$TERM" chroot /mnt/${HOST} /bin/bash -l

umount /mnt/${HOST}/dev/shm

umount /mnt/${HOST}/dev

umount /mnt/${HOST}/proc

umount /mnt/${HOST}/sys

umount /mnt/${HOST}/usr/portage

umount /mnt/${HOST}/usr/local/portage

umount /mnt/${HOST}/var/tmp/portage

umount /mnt/${HOST}
```

The part I is not clear to me in the above script is:

HOST=${0##*/}

HOST=${HOST#*-}

Shouldn't this bash script be in two parts: 

```
#!/bin/sh

HOST=${0##*/}

HOST=${HOST#*-}

mkdir -p --mode=0755 /mnt/${HOST}

mount -t nfs -o rw,intr,noatime,actimeo=60,vers=4,fsc ${HOST}:/ /mnt/${HOST}

mount --bind /dev /mnt/${HOST}/dev

mount --bind /dev/shm /mnt/${HOST}/dev/shm

mount --bind /proc /mnt/${HOST}/proc

mount --bind /sys /mnt/${HOST}/sys

mount --bind /usr/portage /mnt/${HOST}/usr/portage

mount --bind /usr/local/portage /mnt/${HOST}/usr/local/portage

mount --bind /var/tmp/portage /mnt/${HOST}/var/tmp/portage

env -i - HOME="/root" TERM="$TERM" chroot /mnt/${HOST} /bin/bash -l
```

```
#!/bin/sh

umount /mnt/${HOST}/dev/shm

umount /mnt/${HOST}/dev

umount /mnt/${HOST}/proc

umount /mnt/${HOST}/sys

umount /mnt/${HOST}/usr/portage

umount /mnt/${HOST}/usr/local/portage

umount /mnt/${HOST}/var/tmp/portage

umount /mnt/${HOST}
```

To start NFS do I need start "portmap"? 

```

bash_server # rc-update add portmap default

bash_server # rc-update add nfs default

bash_client # rc-update add portmap default

bash_client # rc-update add nfsmount default

bash_client # /etc/init.d/nfsmount start 
```

----------

## Chewi

 *Joseph_sys wrote:*   

> Can you explain me what these two line do in your script?
> 
> ```
> #!/bin/sh
> 
> ...

 

If you create a symlink to this script like chroot-foo then HOST will be set to foo and it will try to mount the share from that server.

 *Joseph_sys wrote:*   

> The second part of the script is run after you have done upgrade/compiling isn't it?

 

Yes but that script is old and there's a better way to do it now, which will only make the mounts visible inside the chroot and they will be more reliably unmounted when you exit.

```
#!/bin/bash

HOST=${0##*/}

HOST=${HOST#*-}

ROOT=/mnt/${HOST}

mkdir -p --mode=0755 "${ROOT}"

exec sudo unshare -m /bin/bash -c "

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '${HOST}:/' '${ROOT}'

mount --bind {,'${ROOT}'}/dev

mount --bind {,'${ROOT}'}/dev/pts

mount --bind {,'${ROOT}'}/dev/shm

mount --bind {,'${ROOT}'}/proc

mount --bind {,'${ROOT}'}/sys

mount --bind {,'${ROOT}'}/usr/local/portage

mount --bind {,'${ROOT}'}/usr/portage

mount --bind {,'${ROOT}'}/var/cache/edb/dep

mount --bind {,'${ROOT}'}/var/tmp/portage

exec chroot '${ROOT}' /bin/bash -l

"
```

Last edited by Chewi on Tue Jul 17, 2018 12:09 pm; edited 3 times in total

----------

## Joseph_sys

 *Chewi wrote:*   

> 
> 
> If you create a symlink to this script like chroot-foo then HOST will be set to foo and it will try to mount the share from that server.

 

Thank you for quick reply. 

I see, so I need to have a link file:

lrwxrwxrwx ......chroot-foo -> chroot.sh 

In this case do I need to have a NFS entry in fstab called "foo" to point to the correct host?

 *Joseph_sys wrote:*   

> The second part of the script is run after you have done upgrade/compiling isn't it?

 

 *Chewi wrote:*   

> Yes but that script is old and there's a better way to do it now, which will only make the mounts visible inside the chroot and they will be more reliably unmounted when you exit.
> 
> ```
> #!/bin/sh
> 
> ...

 

Do you still need to have the second part of the script "umount" after you exit/close the terminal?

I run onto this wiki link:

http://www.wikigentoo.ksiezyc.pl/Slow_systems.htm

```
# mount <IP of box B>:/ /mnt/slowmachine -o rsize=1024,wsize=1024,rw

(The -o rsize=1024,wsize=1024 prevents IP fragmentation, rw permits read-write operation on the NFS mount.)
```

Is the above entry useful? 

Also in the part NFS mount; Is starting "portmap" necessary? 

http://www.wikigentoo.ksiezyc.pl/Shared_Portage_via_NFS.htm

```
bash_server # rc-update add portmap default

bash_server # rc-update add nfs default

bash_client # rc-update add portmap default

bash_client # rc-update add nfsmount default

bash_client # /etc/init.d/nfsmount start 
```

----------

## Joseph_sys

I've tried to duplicate your script but got:

```
sh chroot-10.10.0.2 

env: ‘exec’: No such file or directory
```

10.10.0.2 is my local IP box running "NFS"

```
ll chroot*

lrwxrwxrwx 1 root root   9 Mar 28 18:31 chroot-10.10.0.2 -> chroot.sh

-rwxr--r-- 1 root root 634 Mar 28 18:32 chroot.sh
```

```
cat chroot.sh 

#!/bin/sh

HOST=${0##*/}

HOST=${HOST#*-}

ROOT=/mnt/${HOST}

mkdir -p --mode=0755 "${ROOT}"

env -i - HOME="/root" TERM="${TERM}" exec sudo unshare -m /bin/sh -c "

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '${HOST}:/' '${ROOT}'

mount --bind {,'${ROOT}'}/dev

mount --bind {,'${ROOT}'}/dev/pts

mount --bind {,'${ROOT}'}/dev/shm

mount --bind {,'${ROOT}'}/proc

mount --bind {,'${ROOT}'}/sys

mount --bind {,'${ROOT}'}/usr/local/portage

mount --bind {,'${ROOT}'}/usr/portage

mount --bind {,'${ROOT}'}/var/cache/edb/dep

mount --bind {,'${ROOT}'}/var/tmp/portage

exec chroot '${ROOT}' /bin/bash -l

"
```

----------

## krinn

 *Joseph_sys wrote:*   

> The part I is not clear to me in the above script is:
> 
> HOST=${0##*/}
> 
> HOST=${HOST#*-}

 

It's a little trick, it extract the name of the execute script and kick out the path from it (1st line)

2nd line, clear what is in front of -

Why doing that? because you can setup symlink with the hostname in it and it will find the hostname to use from the symlink

ie: ln -s thisscript thisscript-somehost

result in :

* /usr/bin/thiscript-somehost -> thiscript-somethost

* thisscript-somehost -> somehost

 *Quote:*   

> Shouldn't this bash script be in two parts:

 

Not really, the disturbing part is why mount --bind all these directories when you just unmount them just after?

Because in between, the script enter inside a chroot (the env -i - HOM...) and so it's only when you exit that chroot that the script continue with the umount commands

 *Quote:*   

> To start NFS do I need start "portmap"? 

 

No, the script create nfsv4 share, no need for portmap.

Anyway, when you want nfs openrc would start needed dep if need  :Wink: 

----------

## Joseph_sys

 *krinn wrote:*   

>  *Joseph_sys wrote:*   The part I is not clear to me in the above script is:
> 
> HOST=${0##*/}
> 
> HOST=${HOST#*-} 
> ...

 

Thank you for replying.  I have an updated form of script. 

```
#!/bin/sh

HOST=${0##*/}

HOST=${HOST#*-}

ROOT=/mnt/${HOST}

mkdir -p --mode=0755 "${ROOT}"

env -i - HOME="/root" TERM="${TERM}" exec sudo unshare -m /bin/sh -c "

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '${HOST}:/' '${ROOT}'

mount --bind {,'${ROOT}'}/dev

mount --bind {,'${ROOT}'}/dev/pts

mount --bind {,'${ROOT}'}/dev/shm

mount --bind {,'${ROOT}'}/proc

mount --bind {,'${ROOT}'}/sys

mount --bind {,'${ROOT}'}/usr/local/portage

mount --bind {,'${ROOT}'}/usr/portage

mount --bind {,'${ROOT}'}/var/cache/edb/dep

mount --bind {,'${ROOT}'}/var/tmp/portage

exec chroot '${ROOT}' /bin/bash -l

"
```

So I created:

```
lrwxrwxrwx 1 root root   9 Mar 28 18:31 chroot-10.10.0.2 -> chroot.sh

-rwxr--r-- 1 root root 634 Mar 28 18:32 chroot.sh
```

10.10.0.2 is my local IP box running "NFS"

When I run: "sh chroot-10.10.0.2" I get:

```
sh chroot-10.10.0.2

env: ‘exec’: No such file or directory
```

I think I'm doing something wrong.  

On a NFS-server I have: 

```
cat  /etc/exports

# /etc/exports: NFS file systems being exported.  See exports(5).

/ 10.10.0.5(rw,no_root_squash,sync,no_subtree_check)
```

----------

## Hu

 *Joseph_sys wrote:*   

> Do you still need to have the second part of the script "umount" after you exit/close the terminal?

 No.  Chewi switched to using a private mount namespace specifically to resolve that problem.  Since the mount commands run in an unshared mount namespace, they have no impact on processes outside the namespace.  When the last process in the namespace exits, the namespace ceases to exist and all mounts in it are automatically undone by the kernel.

----------

## krinn

 *Hu wrote:*   

> Have you checked which process is blocked during this time?  Specifically, is it stalled waiting for a mount to finish or is it stalled waiting for the chroot'd bash to initialize?

 

For me it's bash, because he is mounting the "host" /proc, /sys... inside the chroot, and they are invalid for the build host and refers to nothing.

ie: /proc/118 is a valid pid for the "host", but invalid for the build host, as 118 pid may not exists at all, and this goes for all files and entries

It's also dangerous, because the "host" values may clash with the "build host" and something nasty could happen!

ie: /dev/disk/by-id/totally_safe_to_clean point to /dev/sdb1 while /dev/sdb1 in the build host is not really something you can safely clean

While i get the idea why you do that Chewi, i have a bad feeling about it  :Smile: 

I would myself only mount the "host" /proc, /sys... only if an ebuild fails because it is seeking infos

----------

## pjp

 *Joseph_sys wrote:*   

> I'm trying to combine information to make bash script to boot strap via NFS Box_slow on Box_fast to upgrade slower boxes.
> 
> I've found 

  Not ideal, but merged this thread and its couple of posts as there was too much crossover.

----------

## Chewi

Apologies, I didn't test this script as much as I should have because I'd only given the "unshare" treatment to my local chroot script. I now remember that the env stuff isn't really necessary because sudo will effectively do that anyway. Change that line to:

```
exec sudo unshare -m /bin/sh -c "
```

You don't need to have an entry in fstab but you could adjust the script to work that way if you prefer. I see rsize and wsize options used a lot but the man page says that the largest supported value will always be used by default and I have found that to be the case with NFSv4.

 *krinn wrote:*   

> For me it's bash, because he is mounting the "host" /proc, /sys... inside the chroot, and they are invalid for the build host and refers to nothing.
> 
> ie: /proc/118 is a valid pid for the "host", but invalid for the build host, as 118 pid may not exists at all, and this goes for all files and entries
> 
> It's also dangerous, because the "host" values may clash with the "build host" and something nasty could happen!
> ...

 

I think you're confused. I have called unshare with -m, which only unshares the mount namespace. The PID namespace remains shared, just as it always has been when chrooting in the traditional manner. I've checked the Gentoo handbook and, true enough, it does tell you to mount a new /proc instance instead of bind mounting it. I'm not sure whether it said that back in 2002 when I first installed Gentoo but as far as I can tell, it doesn't make any difference unless you unshare the PID namespace as well. If I start a chroot and run ps, I can see all the processes from outside the chroot. Conversely, if I start nano in my chroot and run ps outside of it, I can see that nano instance in the listing and under /proc. Of course, you can unshare the other namespaces if you want to, which leads into things like LXC and Docker but I've never played with that. As for /dev, I don't think that is ever unshared. /dev/sda is /dev/sda regardless of which chroot you're in and things like /dev/disk/by-id are handled by the kernel.

----------

## krinn

 *Chewi wrote:*   

> I think you're confused.

 

Actually yes, but not by what you think, i mistake bind arguments for "dir dev" instead of "dev dir"  :Smile: 

----------

## Joseph_sys

 *Chewi wrote:*   

> Apologies, I didn't test this script as much as I should have because I'd only given the "unshare" treatment to my local chroot script. I now remember that the env stuff isn't really necessary because sudo will effectively do that anyway. Change that line to:
> 
> ```
> exec sudo unshare -m /bin/sh -c "
> ```
> ...

 

I think it worked. THANK YOU!

I've added "set -x" to the beginning of the script and run a link file:

"sh chroot-eden" --> chroot.sh

```
+ HOST=chroot-eden

+ HOST=eden

+ ROOT=/mnt/eden

+ mkdir -p --mode=0755 /mnt/eden

+ exec sudo unshare -m /bin/sh -c '

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '\''eden:/'\'' '\''/mnt/eden'\''

mount --bind {,'\''/mnt/eden'\''}/dev

mount --bind {,'\''/mnt/eden'\''}/dev/pts

mount --bind {,'\''/mnt/eden'\''}/dev/shm

mount --bind {,'\''/mnt/eden'\''}/proc

mount --bind {,'\''/mnt/eden'\''}/sys

mount --bind {,'\''/mnt/eden'\''}/usr/local/portage

mount --bind {,'\''/mnt/eden'\''}/usr/portage

mount --bind {,'\''/mnt/eden'\''}/var/cache/edb/dep

mount --bind {,'\''/mnt/eden'\''}/var/tmp/portage

exec chroot '\''/mnt/eden'\'' /bin/bash -l

'
```

eden - small (old) box running nfs server 

syscon3 - my 8-core AMD 

I was presented with root prompt "syscon3 /" and I know I'm in "chroot" as I created a file in / 1.txt and it appeared when I typed on "ls -al" 

However, when I logged from another shell to "syscon3" box the /mnt/eden/ dir is empty.  Where are the files mounted?

How do I change the prompt to be: "syscon3-eden"?

It would be less confusing and an indication that I'm in chroot environment.

----------

## Chewi

Great!

 *Joseph_sys wrote:*   

> I was presented with root prompt "syscon3 /" and I know I'm in "chroot" as I created a file in / 1.txt and it appeared when I typed on "ls -al" 
> 
> However, when I logged from another shell to "syscon3" box the /mnt/eden/ dir is empty.  Where are the files mounted?

 

Well that's the thing with unshared mount namespaces, you won't see them from another shell. If you just want another session, you can call the script again, or you could use tmux or similar. If you want to move files around then you could mount the NFS share separately in fstab and just leave the script to do the other mounts. If it's not available all the time, there is autofs but I've not tried that in years.

 *Joseph_sys wrote:*   

> How do I change the prompt to be: "syscon3-eden"?
> 
> It would be less confusing and an indication that I'm in chroot environment.

 

I've never tried it but you could unshare the UTS namespace as well, allowing an independent host and domain name. I guess you'd need to set it each time as part of the script. You could also manipulate the Bash prompt (PS1) based on some environment variable.

----------

## Joseph_sys

 *Chewi wrote:*   

> Great!
> 
>  *Joseph_sys wrote:*   I was presented with root prompt "syscon3 /" and I know I'm in "chroot" as I created a file in / 1.txt and it appeared when I typed on "ls -al" 
> 
> However, when I logged from another shell to "syscon3" box the /mnt/eden/ dir is empty.  Where are the files mounted? 
> ...

 

This is a live test. My old system has not been upgraded for about 250-days.  So I emerge --sync. and try to run in chroot evironment:

```
emerge --ask --oneshot -vq sys-devel/gcc 

...

libtool: install: warning: remember to run `libtool --finish /usr/lib/../lib'

make[4]: Nothing to be done for 'install-data-am'.

make[4]: Leaving directory '/var/tmp/portage/sys-devel/gcc-6.4.0-r1/work/build/i686-pc-linux-gnu/libatomic'

make[3]: Leaving directory '/var/tmp/portage/sys-devel/gcc-6.4.0-r1/work/build/i686-pc-linux-gnu/libatomic'

make[2]: Leaving directory '/var/tmp/portage/sys-devel/gcc-6.4.0-r1/work/build/i686-pc-linux-gnu/libatomic'

make[1]: Leaving directory '/var/tmp/portage/sys-devel/gcc-6.4.0-r1/work/build'

 * PT_PAX marking -r /var/tmp/portage/sys-devel/gcc-6.4.0-r1/image//usr/libexec/gcc/i686-pc-linux-gnu/6.4.0/cc1 with scanelf

 * XATTR_PAX marking -re /var/tmp/portage/sys-devel/gcc-6.4.0-r1/image//usr/libexec/gcc/i686-pc-linux-gnu/6.4.0/cc1 with setfattr

 * PT_PAX marking -r /var/tmp/portage/sys-devel/gcc-6.4.0-r1/image//usr/libexec/gcc/i686-pc-linux-gnu/6.4.0/cc1plus with scanelf

 * XATTR_PAX marking -re /var/tmp/portage/sys-devel/gcc-6.4.0-r1/image//usr/libexec/gcc/i686-pc-linux-gnu/6.4.0/cc1plus with setfattr

 * Final size of build directory: 1556216 KiB

 * Final size of installed tree: 151528 KiB

!!! Failed to copy extended attributes. In order to avoid this error,

!!! set FEATURES="-xattr" in make.conf.

!!! copy /var/tmp/portage/sys-devel/gcc-6.4.0-r1/image/usr/libexec/gcc/i686-pc-linux-gnu/6.4.0/cc1 -> /usr/libexec/gcc/i686-pc-linux-gnu/6.4.0/cc1 failed.

!!! Filesystem containing file '/usr/libexec/gcc/i686-pc-linux-gnu/6.4.0/cc1#new' does not support extended attribute 'user.pax.flags'

 * 

 * Please include /var/tmp/portage/sys-devel/gcc-6.4.0-r1/work/gcc-build-logs.tar.bz2 in your bug report.

 * 

 * 

 * The following package has failed to build, install, or execute postinst:

 * 

 *  (sys-devel/gcc-6.4.0-r1:6.4.0/6.4.0::gentoo, ebuild scheduled for merge), Log file:

 *   '/var/tmp/portage/sys-devel/gcc-6.4.0-r1/temp/build.log'

 * 
```

```
Portage 2.3.6 (python 3.4.5-final-0, default/linux/x86/13.0/desktop, gcc-4.9.4, glibc-2.23-r4, 4.9.72-gentoo x86_64)

=================================================================

System uname: Linux-4.9.72-gentoo-x86_64-AMD_Ryzen_5_1400_Quad-Core_Processor-with-gentoo-2.3

KiB Mem:    16432224 total,  12064148 free

KiB Swap:     524284 total,    524284 free

Timestamp of repository gentoo: Mon, 26 Mar 2018 21:00:01 +0000

sh bash 4.3_p48-r1

ld GNU ld (Gentoo 2.28 p1.2) 2.28

app-shells/bash:          4.3_p48-r1::gentoo

dev-java/java-config:     2.2.0-r3::gentoo

dev-lang/perl:            5.24.1-r2::gentoo

dev-lang/python:          2.7.12::gentoo, 3.4.5::gentoo

dev-util/cmake:           3.7.2::gentoo

dev-util/pkgconfig:       0.28-r2::gentoo

sys-apps/baselayout:      2.3::gentoo

sys-apps/openrc:          0.26.3::gentoo

sys-apps/sandbox:         2.10-r3::gentoo

sys-devel/autoconf:       2.13::gentoo, 2.69::gentoo

sys-devel/automake:       1.15-r2::gentoo

sys-devel/binutils:       2.28-r2::gentoo

sys-devel/gcc:            4.9.4::gentoo, 5.4.0-r3::gentoo

sys-devel/gcc-config:     1.7.3::gentoo

sys-devel/libtool:        2.4.6-r3::gentoo

sys-devel/make:           4.2.1::gentoo

sys-kernel/linux-headers: 4.4::gentoo (virtual/os-headers)

sys-libs/glibc:           2.23-r4::gentoo

Repositories:

gentoo

    location: /usr/portage

    sync-type: rsync

    sync-uri: rsync://10.0.0.103/gentoo-portage

    priority: -1000

x-portage

    location: /usr/local/portage

    masters: gentoo

    priority: 0

brother-overlay

    location: /var/lib/layman/brother-overlay

    masters: gentoo

    priority: 50

ACCEPT_KEYWORDS="x86"

ACCEPT_LICENSE="* -@EULA PUEL dlj-1.1"

CBUILD="i686-pc-linux-gnu"

CFLAGS="-O2 -march=i686 -pipe"

CHOST="i686-pc-linux-gnu"

CONFIG_PROTECT="/etc /usr/lib/fax /usr/share/easy-rsa /usr/share/gnupg/qualified.txt /usr/src/linux* /var/spool/fax/etc"

CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"

CXXFLAGS="-O2 -march=i686 -pipe"

DISTDIR="/usr/portage/distfiles"

EMERGE_DEFAULT_OPTS="--autounmask-write=y --keep-going --with-bdeps=y"

FCFLAGS="-O2 -march=i686 -pipe"

FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"

FFLAGS="-O2 -march=i686 -pipe"

GENTOO_MIRRORS="http://gentoo.llarian.net/ http://gentoo.mirrors.hoobly.com/ http://ftp.ucsb.edu/pub/mirrors/linux/gentoo/ http://gentoo.mirrors.easynews.com/linux/gentoo/"

LANG="en_US.UTF-8"

LC_ALL="en_US.UTF-8"

LDFLAGS="-Wl,-O1 -Wl,--as-needed"

MAKEOPTS="-j2"

PKGDIR="/usr/portage/packages"

PORTAGE_CONFIGROOT="/"

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"

PORTAGE_TMPDIR="/var/tmp"

USE="X a52 aac acpi alsa apache2 bluetooth branding bzip2 cairo cdda cdr cgi cli consolekit crypt cups cxx dbus dri dts dvd dvdr emboss encode exif fam flac fortran gdbm gif glamor gpm gtk iconv ipv6 jpeg lcms ldap libnotify mad mng modules mp3 mp4 mpeg ncurses nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppds qt3support qt5 readline scanner sdl seccomp spell ssl startup-notification svg tcpd tiff truetype type1 udev udisks unicode upower usb vorbis wxwidgets x264 x86 xattr xcb xml xv xvid zlib" ABI_X86="32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" L10N="en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5" RUBY_TARGETS="ruby22 ruby23" SANE_BACKENDS="fujitsu" USERLAND="GNU" VIDEO_CARDS="vga vesa fbdev via" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"

Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

```

Last edited by Joseph_sys on Fri Mar 30, 2018 4:24 am; edited 1 time in total

----------

## Chewi

So do what it says. NFS doesn't support extended attributes. This is not a hardened system so you could also set PAX_MARKINGS="none" in make.conf to stop it trying to apply them.

----------

## Joseph_sys

 *Chewi wrote:*   

> So do what it says. NFS doesn't support extended attributes. This is not a hardened system so you could also set PAX_MARKINGS="none" in make.conf to stop it trying to apply them.

 

That scrip worked very well! THANK YOU! 

I've "dusted-off" and upgraded my old system:

VIA Eden Processor 1200MHz  1GB or RAM (only)

The system wasn't upgraded in over 250-days so I upgraded to gcc-6.4.0-r1 and recompile 756-packages (-e @world) running on 

8-core AMD with 16GB or RAM

This scrip should be in the official Gentoo Documentation, may old systems are getting discarded as there are two small to compile newer/and larger packages.

The only think that puzzle me I don't get the the prompt "PS1=eden"

When I run: 

```
syscon3 /home/joseph # sh chroot-eden 

+ HOST=chroot-eden

+ HOST=eden

+ ROOT=/mnt/eden

+ PS1=eden

+ mkdir -p --mode=0755 /mnt/eden

+ exec sudo unshare -m /bin/sh -c '

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '\''eden:/'\'' '\''/mnt/eden'\''

mount --bind {,'\''/mnt/eden'\''}/dev

mount --bind {,'\''/mnt/eden'\''}/dev/pts

mount --bind {,'\''/mnt/eden'\''}/dev/shm

mount --bind {,'\''/mnt/eden'\''}/proc

mount --bind {,'\''/mnt/eden'\''}/sys

mount --bind {,'\''/mnt/eden'\''}/usr/local/portage

mount --bind {,'\''/mnt/eden'\''}/usr/portage

mount --bind {,'\''/mnt/eden'\''}/var/cache/edb/dep

mount --bind {,'\''/mnt/eden'\''}/var/tmp/portage

exec chroot '\''/mnt/eden'\'' /bin/bash -i

'

syscon3 / #
```

The "PS1=eden" when when the prompt shows up, I end-up with "syscon3 / #"

All the 756-packages recompile in about 24-hours.

gcc-6.4.0-r1 took only 1hr 39min

The packages that wouldn't compile is binary:

virtualbox-bin

----------

## Chewi

PS1 disappears because it gets stripped out by sudo for security reasons. Try changing the chroot line to this.

```
exec env PS1=eden chroot '${ROOT}' /bin/bash -l
```

----------

## Joseph_sys

 *Chewi wrote:*   

> PS1 disappears because it gets stripped out by sudo for security reasons. Try changing the chroot line to this.
> 
> ```
> exec env PS1=eden chroot '${ROOT}' /bin/bash -l
> ```
> ...

 

No, id didn't work:

```
syscon3 /home/thelma # sh chroot-eden 

+ HOST=chroot-eden

+ HOST=eden

+ ROOT=/mnt/eden

+ PS1=eden

+ mkdir -p --mode=0755 /mnt/eden

+ exec sudo unshare -m /bin/sh -c '

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '\''eden:/'\'' '\''/mnt/eden'\''

mount --bind {,'\''/mnt/eden'\''}/dev

mount --bind {,'\''/mnt/eden'\''}/dev/pts

mount --bind {,'\''/mnt/eden'\''}/dev/shm

mount --bind {,'\''/mnt/eden'\''}/proc

mount --bind {,'\''/mnt/eden'\''}/sys

mount --bind {,'\''/mnt/eden'\''}/usr/local/portage

mount --bind {,'\''/mnt/eden'\''}/usr/portage

mount --bind {,'\''/mnt/eden'\''}/var/cache/edb/dep

mount --bind {,'\''/mnt/eden'\''}/var/tmp/portage

#exec chroot '\''/mnt/eden'\'' /bin/bash -i

exec env PS1=eden chroot '\''/mnt/eden'\'' /bin/bash -i

'

syscon3 / #
```

----------

## Chewi

Oh of course, it'll be reset by ${ROOT}/etc/bash/bashrc. I guess you'll have to do something clever in there.

----------

## guitou

Hello.

I suppose PS1 is set... but out of chrooted env.

++

Gi)

Edit: replied too late!

----------

## Joseph_sys

 *Chewi wrote:*   

> Oh of course, it'll be reset by ${ROOT}/etc/bash/bashrc. I guess you'll have to do something clever in there.

 

I was trying to run your script on another remote network but I get:

"Illegal instruction"

It worked find on one of my network but not remote one.

```
#!/bin/sh

set -x

HOST=${0##*/}

HOST=${HOST#*-}

ROOT=/mnt/${HOST}

PS1="${HOST}"

mkdir -p --mode=0755 "${ROOT}"

#env -i - HOME="/root" TERM="${TERM}" exec sudo unshare -m /bin/sh -c "

exec sudo unshare -m /bin/sh -c "

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '${HOST}:/' '${ROOT}'

mount --bind {,'${ROOT}'}/dev

mount --bind {,'${ROOT}'}/dev/pts

mount --bind {,'${ROOT}'}/dev/shm

mount --bind {,'${ROOT}'}/proc

mount --bind {,'${ROOT}'}/sys

mount --bind {,'${ROOT}'}/usr/local/portage

mount --bind {,'${ROOT}'}/usr/portage

mount --bind {,'${ROOT}'}/var/cache/edb/dep

mount --bind {,'${ROOT}'}/var/tmp/portage

exec chroot '${ROOT}' /bin/bash -i

"
```

```
+ HOST=chroot-i5

+ HOST=i5

+ ROOT=/mnt/i5

+ PS1=i5

+ mkdir -p --mode=0755 /mnt/i5

+ exec sudo unshare -m /bin/sh -c '

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '\''i5:/'\'' '\''/mnt/i5'\''

mount --bind {,'\''/mnt/i5'\''}/dev

mount --bind {,'\''/mnt/i5'\''}/dev/pts

mount --bind {,'\''/mnt/i5'\''}/dev/shm

mount --bind {,'\''/mnt/i5'\''}/proc

mount --bind {,'\''/mnt/i5'\''}/sys

mount --bind {,'\''/mnt/i5'\''}/usr/local/portage

mount --bind {,'\''/mnt/i5'\''}/usr/portage

mount --bind {,'\''/mnt/i5'\''}/var/cache/edb/dep

mount --bind {,'\''/mnt/i5'\''}/var/tmp/portage

exec chroot '\''/mnt/i5'\'' /bin/bash -i

'

Illegal instruction
```

----------

## Chewi

Judging by the "i5" name, this is a Core i5 that has had its software built with CFLAGS that are not compatible with the processor you are now trying to run it on. Also be careful not to use -march=native or you might end up breaking the remote system.

----------

## Joseph_sys

 *Chewi wrote:*   

> Judging by the "i5" name, this is a Core i5 that has had its software built with CFLAGS that are not compatible with the processor you are now trying to run it on. Also be careful not to use -march=native or you might end up breaking the remote system.

 

The computer that would be doing the compiling is:

AMD FX(tm)-8350 Eight-Core Processor

CFLAGS="-march=native -O2 -pipe"

What should I use on the above computer?

The i5 (you are correct) is (chroot failed on this)

Intel(R) Core(TM) i5-4200U CPU @ 1.60GHz

CFLAGS="-march=native -O2 -pipe"

Though, I was able to run the chroot-script OK on another remote box (same network):

Intel(R) Atom(TM) CPU  330   @ 1.60GHz

CFLAGS="-march=core2 -O2 -pipe"

--------------

On my local network I recompile/upgraded my via chroot

VIA Eden Processor 1200MHz

CFLAGS="-O2 -march=i686 -pipe"

The computer that was doing compiling was:

AMD Ryzen 5 1400 Quad-Core Processor

CFLAGS="-march=native -O2 -pipe"

----------

## Chewi

Your Eden/Ryzen combo didn't break anything because only the Ryzen system had -march=native. If the Eden system had had that too, you would have found it broken following your upgrade. To be absolutely safe, stop using -march=native everywhere.

Your i5 system is a Haswell and your FX system is a Piledriver (bdver2). The gcc man page shows some slight differences between these and it only takes one instruction to break things. I think the most likely culprit is AVX2. For this to work, you would need to rebuild the i5 system or maybe even both with the lowest common denominator but it's hard to say what that would be. Usually this kind of problem is avoided because it is a much newer system doing the building. In your situation, you may want to consider distcc instead. In theory, the new stuff arriving in EAPI 7 will allow you to mount your remote system and build without chrooting but it is likely to break in other ways because this approach is mainly intended for cross-compiling.

----------

## pablocool

Hello Guys

I also wanted to use this cool method but failed. Please help in tshooting.

```
pablocool@wloski ~ $ cat chroot-10.0.0.100

#!/bin/sh

set -x

HOST=${0##*/}

HOST=${HOST#*-}

ROOT=/mnt/${HOST}

PS1="${HOST}"

mkdir -p --mode=0755 "${ROOT}"

#env -i - HOME="/root" TERM="${TERM}" exec sudo unshare -m /bin/sh -c "

exec sudo unshare -m /bin/sh -c "

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '${HOST}:/' '${ROOT}'

mount --bind {,'${ROOT}'}/dev

mount --bind {,'${ROOT}'}/dev/pts

mount --bind {,'${ROOT}'}/dev/shm

mount --bind {,'${ROOT}'}/proc

mount --bind {,'${ROOT}'}/sys

mount --bind {,'${ROOT}'}/usr/local/portage

mount --bind {,'${ROOT}'}/usr/portage

mount --bind {,'${ROOT}'}/var/cache/edb/dep

mount --bind {,'${ROOT}'}/var/tmp/portage

exec chroot '${ROOT}' /bin/bash -i

"

```

```
pablocool@wloski ~ $ sh chroot-10.0.0.100

+ HOST=chroot-10.0.0.100

+ HOST=10.0.0.100

+ ROOT=/mnt/10.0.0.100

+ PS1=10.0.0.100

+ mkdir -p --mode=0755 /mnt/10.0.0.100

+ exec sudo unshare -m /bin/sh -c

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '10.0.0.100:/' '/mnt/10.0.0.100'

mount --bind {,'/mnt/10.0.0.100'}/dev

mount --bind {,'/mnt/10.0.0.100'}/dev/pts

mount --bind {,'/mnt/10.0.0.100'}/dev/shm

mount --bind {,'/mnt/10.0.0.100'}/proc

mount --bind {,'/mnt/10.0.0.100'}/sys

mount --bind {,'/mnt/10.0.0.100'}/usr/local/portage

mount --bind {,'/mnt/10.0.0.100'}/usr/portage

mount --bind {,'/mnt/10.0.0.100'}/var/cache/edb/dep

mount --bind {,'/mnt/10.0.0.100'}/var/tmp/portage

exec chroot '/mnt/10.0.0.100' /bin/bash -i

mount: mount point {,/mnt/10.0.0.100}/dev does not exist
```

I cannot understand these {,' ... '} construction. Appreciate any explaination.

----------

## Chewi

Turns out brace expansion is a Bashism. You learn something every day. Replace #!/bin/sh with #!/bin/bash.

----------

## pablocool

Point for you even still it is not working.

Old PC is gentoo system it has:

lrwxrwxrwx 1 root root 4 07-14 21:04 /bin/sh -> bash

However only to test purposes as strong machine I used Debian VPS. It has:

lrwxrwxrwx 1 root root 4 lip 20  2016 /bin/sh -> dash

I am closer but still not working:

```
+ HOST=chroot-10.0.0.100

+ HOST=10.0.0.100

+ ROOT=/mnt/10.0.0.100

+ PS1=10.0.0.100

+ mkdir -p --mode=0755 /mnt/10.0.0.100

+ exec sudo unshare -m /bin/sh -c '

set -e

mount -t nfs -o rw,noatime,nocto,actimeo=60,lookupcache=positive,vers=4,fsc '\''10.0.0.100:/'\'' '\''/mnt/10.0.0.100'\''

mount --bind {,'\''/mnt/10.0.0.100'\''}/dev

mount --bind {,'\''/mnt/10.0.0.100'\''}/dev/pts

mount --bind {,'\''/mnt/10.0.0.100'\''}/dev/shm

mount --bind {,'\''/mnt/10.0.0.100'\''}/proc

mount --bind {,'\''/mnt/10.0.0.100'\''}/sys

mount --bind {,'\''/mnt/10.0.0.100'\''}/usr/local/portage

mount --bind {,'\''/mnt/10.0.0.100'\''}/usr/portage

mount --bind {,'\''/mnt/10.0.0.100'\''}/var/cache/edb/dep

mount --bind {,'\''/mnt/10.0.0.100'\''}/var/tmp/portage

exec chroot '\''/mnt/10.0.0.100'\'' /bin/bash -i

'

mount: mount point {,/mnt/10.0.0.100}/dev does not exist
```

Why do we need these brackets { } ?

EDIT:

Also this line needed update

+ exec sudo unshare -m /bin/sh -c '

----------

## Chewi

Oh, I missed the /bin/sh in the middle of the script. Replace that too.

It's just a short way of saying:

```
mount --bind /dev /mnt/10.0.0.100/dev 
```

----------

## pablocool

Thank you for help! IT is working.

To just information, its overkill of course but it is even working over internet over OpenVPN.

----------

