# distcc does not work in loopback mode

## Tender

Hello everybody,

distcc does not work in loopback mode, any advice?

distcc ~ # ifconfig

```

lo        Link encap:Local Loopback

          inet addr:127.0.0.1  Mask:255.0.0.0

          inet6 addr: ::1/128 Scope:Host

          UP LOOPBACK RUNNING  MTU:16436  Metric:1

          RX packets:13905 errors:0 dropped:0 overruns:0 frame:0

          TX packets:13905 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:129366653 (123.3 MiB)  TX bytes:129366653 (123.3 MiB)

wlan0     Link encap:Ethernet  HWaddr 00:11:95:d0:51:b0

          inet addr:192.168.5.102  Bcast:192.168.5.255  Mask:255.255.255.0

          inet6 addr: fe80::211:95ff:fed0:51b0/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:1492  Metric:1

          RX packets:2934 errors:0 dropped:0 overruns:0 frame:0

          TX packets:4570 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:1382146 (1.3 MiB)  TX bytes:511675 (499.6 KiB)

wmaster0  Link encap:UNSPEC  HWaddr 00-11-95-D0-51-B0-77-6C-00-00-00-00-00-00-00-00

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:0 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

```

distcc ~ # emerge -p distcc

```

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild   R   ] sys-devel/distcc-3.0-r4

```

distcc ~ # distcc -v

```

Using built-in specs.

Target: i586-pc-linux-gnu

Configured with: /var/tmp/portage/sys-devel/gcc-4.1.2/work/gcc-4.1.2/configure --prefix=/usr --bindir=/usr/i586-pc-linux-gnu/gcc-bin/4.1.2 --includedir=/usr/lib/gcc/i586-pc-linux-gnu/4.1.2/include --datadir=/usr/share/gcc-data/i586-pc-linux-gnu/4.1.2 --mandir=/usr/share/gcc-data/i586-pc-linux-gnu/4.1.2/man --infodir=/usr/share/gcc-data/i586-pc-linux-gnu/4.1.2/info --with-gxx-include-dir=/usr/lib/gcc/i586-pc-linux-gnu/4.1.2/include/g++-v4 --host=i586-pc-linux-gnu --build=i586-pc-linux-gnu --disable-altivec --enable-nls --without-included-gettext --with-system-zlib --disable-checking --disable-werror --enable-secureplt --disable-multilib --enable-libmudflap --disable-libssp --disable-libgcj --with-arch=i586 --enable-languages=c,c++,treelang,fortran --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu

Thread model: posix

gcc version 4.1.2 (Gentoo 4.1.2 p1.1)

```

lowpower ~ # distcc -j

```

4

```

nano /etc/conf.d/distccd

```

# /etc/conf.d/distccd: config file for /etc/init.d/distccd

DISTCCD_OPTS=""

# this is the distccd executable

DISTCCD_EXEC="/usr/bin/distccd"

# this is where distccd will store its pid file

DISTCCD_PIDFILE="/var/run/distccd/distccd.pid"

# set this option to run distccd with extra parameters

# Default port is 3632.  For most people the default is okay.

DISTCCD_OPTS="${DISTCCD_OPTS} --port 3632"

# Logging

# You can change some logging options here:

# --log-file FILE

# --log-level LEVEL  [critical,error,warning, notice, info, debug]

#

# Leaving --log-file blank will log to syslog

# example: --log-file /dev/null --log-level warning

# example: --log-level critical

DISTCCD_OPTS="${DISTCCD_OPTS} --log-level critical"

# SECURITY NOTICE:

# It is HIGHLY recomended that you use the --listen option

# for increased security. You can specify an IP to permit connections

# from or a CIDR mask

# --listen accepts only a single IP

# --allow is now mandatory as of distcc-2.18.

# example:  --allow 192.168.0.0/24

# example:  --allow 192.168.0.5 --allow 192.168.0.150

# example:  --listen 192.168.0.2

DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.5.0/24"

#DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.5.102"

# set this for niceness

# Default is 15

DISTCCD_OPTS="${DISTCCD_OPTS} -N 15"

```

nano /etc/distcc/hosts

```

# --- /etc/distcc/hosts -----------------------

# See the "Hosts Specification" section of

# "man distcc" for the format of this file.

#

# By default, just test that it works in loopback mode.

127.0.0.1

```

emerge wget

```

...

config.status: creating src/config.h

config.status: executing default commands

generating po/POTFILES from ./po/POTFILES.in

creating po/Makefile

cd src && make CC='i586-pc-linux-gnu-gcc' CPPFLAGS='' DEFS='-DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/usr/$cd doc && make CC='i586-pc-l$

i586-pc-linux-gnu-gcc -I. -I.   -DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/usr/share/locale\" -march=c3 -O2$cd po && make CC='i586-pc-li$

sed s/@/@@/g sample.wgetrc > sample.wgetrc.munged_for_texi_inclusion

i586-pc-linux-gnu-gcc -I. -I.   -DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/usr/share/locale\" -march=c3 -O2$make[1]: Entering directory $

file=./`echo be | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file be.po

makeinfo -I. ./wget.texi

distcc[4982] (dcc_pump_sendfile) ERROR: sendfile failed: Broken pipe

distcc[4982] (dcc_readx) ERROR: unexpected eof on fd5

distcc[4982] (dcc_r_token_int) ERROR: read failed while waiting for token "DONE"

distcc[4982] (dcc_r_result_header) ERROR: server provided no answer. Is the server configured to allow access from your IP addr$distcc[4982] Warning: failed$

file=./`echo bg | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file bg.po

distcc[4988] (dcc_writex) ERROR: failed to write: Connection reset by peer

distcc[4988] Warning: failed to distribute connect.c to 127.0.0.1, running locally instead

file=./`echo ca | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file ca.po

make[1]: Leaving directory `/var/tmp/portage/net-misc/wget-1.11.3/work/wget-1.11.3/doc'

cd tests && make CC='i586-pc-linux-gnu-gcc' CPPFLAGS='' DEFS='-DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/us$make[1]: Entering directory $

make[1]: Nothing to be done for `all'.

make[1]: Leaving directory `/var/tmp/portage/net-misc/wget-1.11.3/work/wget-1.11.3/tests'

cd util && make CC='i586-pc-linux-gnu-gcc' CPPFLAGS='' DEFS='-DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/usr$make[1]: Entering directory $

make[1]: Nothing to be done for `all'.

make[1]: Leaving directory `/var/tmp/portage/net-misc/wget-1.11.3/work/wget-1.11.3/util'

i586-pc-linux-gnu-gcc -I. -I.   -DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/usr/share/locale\" -march=c3 -O2$distcc[5020] (dcc_build_some$

cd windows && make CC='i586-pc-linux-gnu-gcc' CPPFLAGS='' DEFS='-DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/$make[1]: Entering directory $

make[1]: Nothing to be done for `all'.

make[1]: Leaving directory `/var/tmp/portage/net-misc/wget-1.11.3/work/wget-1.11.3/windows'

...

```

Tender

----------

## Peach

tried to use 

```
# distcc-config --set-hosts localhost
```

  :Question: 

I get problems passing the hosts in the config file, I always needed to set them by hand

----------

## Tender

I need 127.0.0.1 working because localhost does not go through distccd.

----------

## bunder

if the compile is running locally, why bother with distcc?  (just asking.  :Wink: )

----------

## cyrillic

Haha, I was scratching my head, trying to figure that one out too.

... but you just ran a successful test of distcc's security.

Since you don't have "--allow 127.0.0.1" in your DISTCCD_OPTS, compiles from that client were blocked.  :Smile: 

----------

## Tender

Thank you for your attention.

The reason why I use 127.0.0.1 is to test distccd using only one system, to avoid network problems.

Ok, the --allow 127.0.0.1 directive allow to compile without pump mode.

With pump mode (hosts : 127.0.0.1,cpp) the same compilation (randomly?) fails:

```

...

i586-pc-linux-gnu-gcc -I. -I.   -DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/usr/share/locale\" -march=c3 -O2 -pipe -fomit-frame-pointer -c

 convert.c

file=./`echo el | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file el.po

file=./`echo en_GB | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file en_GB.po

file=./`echo eo | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file eo.po

file=./`echo es | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file es.po

file=./`echo et | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file et.po

file=./`echo eu | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file eu.po

file=./`echo fi | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file fi.po

file=./`echo fr | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file fr.po

file=./`echo ga | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file ga.po

file=./`echo gl | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file gl.po

file=./`echo he | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file he.po

file=./`echo hr | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file hr.po

file=./`echo hu | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file hu.po

file=./`echo id | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file id.po

distcc[22907] (dcc_pump_sendfile) ERROR: sendfile failed: Broken pipe

distcc[22907] (dcc_writex) ERROR: failed to write: Broken pipe

distcc[22907] Warning: failed to distribute cmpt.c to 127.0.0.1,cpp, running locally instead

file=./`echo it | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file it.po

file=./`echo ja | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file ja.po

cd util && make CC='i586-pc-linux-gnu-gcc' CPPFLAGS='' DEFS='-DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/usr/share/locale\"' CFLAGS='-march=c3 -O2 -pipe -fomit-frame-pointer' LDFLAGS='' LIBS='-lssl -lcrypto  -ldl -lrt ' DESTDIR='' prefix='/usr' exec_prefix='/usr' bindir='/usr/bin' infodir='/usr/share/info' mandir='/usr/share/man' manext='1'

make[1]: Entering directory `/var/tmp/portage/net-misc/wget-1.11.3/work/wget-1.11.3/util'

make[1]: Nothing to be done for `all'.

make[1]: Leaving directory `/var/tmp/portage/net-misc/wget-1.11.3/work/wget-1.11.3/util'

file=./`echo nb | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file nb.po

i586-pc-linux-gnu-gcc -I. -I.   -DHAVE_CONFIG_H -DSYSTEM_WGETRC=\"/etc/wgetrc\" -DLOCALEDIR=\"/usr/share/locale\" -march=c3 -O2 -pipe -fomit-frame-pointer -c cookies.c

distcc[23057] (dcc_build_somewhere) Warning: failed to distribute, running locally instead

distcc[22925] (dcc_pump_sendfile) ERROR: sendfile failed: Broken pipe

distcc[22925] (dcc_writex) ERROR: failed to write: Broken pipe

distcc[22925] Warning: failed to distribute connect.c to 127.0.0.1,cpp, running locally instead

distcc[22950] (dcc_pump_sendfile) ERROR: sendfile failed: Broken pipe

distcc[22950] (dcc_writex) ERROR: failed to write: Broken pipe

distcc[22950] Warning: failed to distribute convert.c to 127.0.0.1,cpp, running locally instead

file=./`echo nl | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file nl.po

file=./`echo pl | sed 's,.*/,,'`.gmo \

          && rm -f $file && PATH=../src:$PATH /usr/bin/gmsgfmt -o $file pl.po

```

Problems of this type in a loopback configuration are very strange, what do you think?

Tender

----------

## gringo

pump mode doesn´t work very well for me either ( it´s just too unstable, in my experience at least ), problems are still to be expected at this development stage i suppose.

cheers

----------

## Tender

 *gringo wrote:*   

> pump mode doesn´t work very well for me either ( it´s just too unstable, in my experience at least ), problems are still to be expected at this development stage i suppose.
> 
> cheers

 

Yes, unstable also for me, why do they place it in the stable branch then?

Speaking of the non-pumped version, in my experience with a slow box (via c3 400 Mhz) and a fast box (amd athlon xp 2400+), it seems that precompilation on the slow box takes so much cpu time that the distributed compile load in the fast box is very very low : do you think this is a possible scenario or there is some more tuning to do (I have played with params a lot...)?

Tender

----------

## bunder

play with -j on the "smaller" machine.  another thing with distcc, is some packages don't use it...  if you set -j10 on a p1 for example, you'd probably crush it.   :Laughing: 

----------

## gringo

 *Quote:*   

> Yes, unstable also for me, why do they place it in the stable branch then? 

 

don´t know how the package is marked upstream but pump mode looks like sth. new and i would not be suprised if i hit problems.

In gentoo distcc-3 is clearly in the testing branch.

 *Quote:*   

> Speaking of the non-pumped version, in my experience with a slow box (via c3 400 Mhz) and a fast box (amd athlon xp 2400+), it seems that precompilation on the slow box takes so much cpu time that the distributed compile load in the fast box is very very low : do you think this is a possible scenario or there is some more tuning to do (I have played with params a lot...)? 

 

i´m not a distcc expert but i instruct distcc to NOT have localhost as one of the possible nodes if the machine running emerge is slow. It helps a bit but you still have the configure stage which in many cases takes longer than the compile itself. ccache may help a bit too and in case you are running gcc-4.3.x you can also try out -fdirectives-only, which should speed up the process a bit.

There are also several scripts around that adjust the -j parameter and the hosts in case they are available.

cheers

----------

## bunder

 *Quote:*   

> i´m not a distcc expert but i instruct distcc to NOT have localhost as one of the possible nodes if the machine running emerge is slow. 

 

++ 

i send all jobs on my network to a central distcc box.  wfm.   :Very Happy: 

----------

## krinn

you can not only set how much jobs distcc will do, you can also tell it how it will use them

cat /etc/distcc/hosts 

localhost/4 @hurin/2 @saruman/2 @faramir/2

as you see, localhost will take 4 jobs and ask 2 from others hosts, so might be a good idea to balance your job ratio between the fast and slow computer

edit: also forget to say, distcc respect the order of hosts file, it mean that hurin will always compile more than faramir, because it appears before faramir in the order.

----------

## Tender

Thank you everybody,

I did some test with distcc 3.1, it seems less unstable in pump mode than 3.0.

In my case the pump mode should load less the target system, right?

I am convinced that the target system is too slow to distribute compilations, also in pump mode.

Using top I see the helper system %cpu idle most of the time, with some sporadic active compilation.

The last and most performing /etc/distcc/hosts configuration tested:

```

192.168.5.102,cpp,lzo

```

If I understand well, in this way the target system should be free from compilation.

Anyway the difference between the local compilation and the distributed one is very low (the distributed one is however little faster), and the local configuration stage reduce the potential gain.

I will try distcc with a target system more powerful, and possibly in a Gbit network   :Wink:  .

Tender

----------

## gringo

as krinn suggested, try to push the nodes a bit. F.ex. i have a quadcore at home and i force distcc to distribute 12 jobs to it. 

Keep in mind that you have to increase the value of the -j switch accordingly in your MAKE_OPTS in order to get this to work properly and also keep in mind that distcc doesn´t work with all packages. Gcc f.ex forces only one job and in order to get the mozilla-* products and openoffice to compile with more than one thread you have to put WANT_MP=true in your make.conf IIRC, but keep in mind that this doesn´t work for everyone, specially openoffice is somewhat bitchy in my experience.

There is also the avahi useflag BTW, which automagically configures and distributes the jobs among the detected boxes but i didn´t like it, it just isn´t aggresive enough imo.

Also, as Bunder said, having sth. like -j24 in a slow box when building a package that doesn´t use distcc at all will actually kill the box  :Wink:  . There are several possibilities to tell portage to disable distcc for that packages.

cheers

----------

