# [HOWTO] distcc amd64 -> x86 with simple wrapper script

## all-inc.

Hi

Because of This Post I start this HowTo.

0. Introduction

Many people have bought an amd64 box and want it help their other x86 boxes compiling their stuff...

It's easier than most people think! Some guides advice to use crossdev. But that's not neccesairy! x86_64 gcc can compile fully x86-compatible 32-bit binaries with the -m32 option. That's it. But to make this thing working with distcc in a gentoo way, check this out.

1. Get the Stuff

Of course you need distcc at all boxes and the target's(I'll call the x86 machine target in this HowTo) gcc version installed on the host(s)(I'll call the amd64 helper(s) host(s))(Note: The target's gcc version hasn't to be the default on the hosts).

Get gcc-config and target's binutils version as well.

2. Let's wrap around

Now we install the x86 wrapper script on the host.

At first, you'll notice that gcc-config -B shows something like /usr/x86_64-pc-linux-gnu/gcc-bin/4.1.1

We create the pseudo gcc-bin dir with mkdir -p $(echo `gcc-config -B` | sed -e 's:x86_64:i686:').

Or, if your host's default gcc version isn't the one the target uses with mkdir -p $(echo `gcc-config -B x86_64-pc-linux-gnu-<target's gcc version>` | sed -e 's:x86_64:i686:').

Next, switch to this new dir: cd /usr/i686-pc-linux-gnu/gcc-bin/<t's version>

Here, create a file named i686-pseudo-wrapper.sh with the following content

```
#!/bin/bash

# wrapper script for the pseudo i686 compiler on amd64 machines

# by all-inc.

$(echo $(/usr/bin/which $0) | /usr/bin/sed -e 's:i686:x86_64:g') -m32 "$@"
```

At last, but not least, we have to create the symlinks for all compilers:

for c in {g++,gcc,c++,cpp}; do ln -s i686-pseudo-wrapper.sh ${c}; ln -s ${c} i686-pc-linux-gnu-${c}; done

(I think we can omit ln -s i686-pc-linux-gnu-gcc i686-pc-linux-gnu-gcc-<t's ver>)

3. Configure distccd(on the hosts!)

Go into /etc/conf.d and cp distccd distccd-x86

This is our special x86 compiler distccd configuration file. Change the red marked settings:

 *my /etc/conf.d/distccd-x86 without comments wrote:*   

> DISTCCD_OPTS="-j4"
> 
> DISTCCD_EXEC="/usr/bin/distccd"
> 
> DISTCCD_PIDFILE="/var/run/distccd/distccd-x86-<optional gcc version>.pid"
> ...

 

Now, cd /etc/init.d and do ln -s distccd distccd-x86 or cp distccd distccd-x86 if you prefer to not edit distccd(Note: The way we edit distccd won't affect the default amd64 distccd). I use the symlink because of flexibility. If you have three distccds, e.g. distccd-x86-3.4.4, distccd-x86-4.1.1 and distccd-amd64-3.4.4, you only have to create the configs in conf.d and the symlinks.

Edit distccd (or distccd-x86 if you don't like the symlink method) and change the red stuff(or, if you did't use the symlink method, only make static changes...so don't use variables like $_a_suffix etc.):

 *Quote:*   

> depend() {
> 
> 	need net
> 
> 	use ypbind
> ...

 

Now, your x86 distccd is finish! start it with /etc/init.d/distccd-x86

(Note: If the host's default gcc version isn't the target's one, please use /etc/conf.d,init.d/distccd-x86-<t's gcc version> so the right bin paths are extracted.)

4. Configure distcc(on the target!)

simply add host-ip:3633/3 or something similar that fits your needs to /etc/distcc/hosts

(Note: You don't need to set the port if you use the default port for distccd-x86)

And of course set MAKEOPTS="-j4" or similar in your make.conf.

Now test it!

I use this method for quiet a while and I already ran emerge -uD world on a x86 successfully, so I think it works  :Wink: 

I hope you liked this HowTo. If, or if not, please give me your feedback here. If you find errors or typos, PM me PLEASE

With best regards, Sebastian Stammler

----------

## Bigun

You may wanna change all the quotes to code.

----------

## all-inc.

Well, this would be more 'correct', but then I can't colorize some points. And I think it's much easier to see the points one has to change than find them oneself. Or am I missing something  :Question: 

----------

## Bigun

Nevermind, you fixed it... some of the code was coming out in emoticons.

----------

## ofornaro

Hi, Thank you for these scripts and notes.

I wish to use distcc, but in some cases stops with something similar to 

 *Quote:*   

> 
> 
> gcc -c -DHAVE_CONFIG_H -march=i686 -O2 -pipe -I. -I/var/tmp/portage/gcc-4.1.1/work/gcc-4.1.1/libiberty/../include  -W -Wall -pedantic -Wwrite-strings -Wstrict-prototypes /var/tmp/portage/gcc-4.1.1/work/gcc-4.1.1/libiberty/regex.c -o regex.o
> 
> /var/tmp/portage/gcc-4.1.1/work/gcc-4.1.1/libiberty/regex.c:1: error: el CPU que seleccionó no tiene soporte para el conjunto de instrucciones x86-64
> ...

 

Please note that this error only appears in some applications and many others compiles well.

Ther exist any workaround?

Osvaldo

----------

## all-inc.

you're welcome  :Wink: 

The errors you described are very easy to explain: Never use distcc with different compiler versions!

That means, never use distcc when you compile your new gcc. the reason is, that when you compile gcc, it compiles in 2 stages:

the new gcc gets compiled with the old compiler(no prob with distcc..same versions)

the new gcc gets again compiled with the previous compiled new gcc.(The versions have changed, distcc will break! and it should  :Wink:  ) why this? so the code for the new gcc is genereated from the new gcc  :Wink:  sounds weird, but if you think about it, you will understand that it's better. using gcc-4 compiled from a gcc-3 would be paradox. you want to use the optimized code from gcc-4, but the gcc itself is compiled from gcc-3...would be stupid.

there are some packages known to break when using distcc...most of them have "make -j1" in the src_compile() function in their ebuild so they won't use distcc(eg openoffice)... when some packages break with distcc, this has nothing to do with this method...it's just that it won't work with distcc on different archs at all(no matter if you use a real cross compiler or this wrapper script). if you experience distcc troubles with some packages, feel free to post at bugs.gentoo.org . of course, you should mention that your distcc helper boxes are amd64 and use gcc -m32.

i hope i could help you, sebastian

----------

## ofornaro

Thank You Sebastian, for an extensive explanation. I will to research a bit more. distcc is even a great tool.

Thanks again.

Osvaldo

----------

## javlinz

Hello.  I'm having trouble getting the distccd init script to work, and was hoping someone could help me troubleshoot it. My host computer has both GCC 3.4.6 and 4.1.1, with 4.1.1 being the default.  My target computer has GCC 3.4.6.

This is what I get when trying to start it with log level on debug.

 *Quote:*   

> 
> 
> # /etc/init.d/distccd-x86-3.4.6 start
> 
>  * Caching service dependencies ...                                                                                  [ ok ]
> ...

 

Here's my /etc/conf.d/distccd-x86-3.4.6

 *Quote:*   

> 
> 
> DISTCCD_OPTS="-j3"
> 
> DISTCCD_EXEC="/usr/bin/distccd"
> ...

 

Here's my /etc/init.d/distccd-x86-3.4.6, which is a symlink to /etc/init.d/distccd

 *Quote:*   

> 
> 
> #!/sbin/runscript
> 
> # $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.18/init,v 1.1 2004/10/12 17:21:43 lisa Exp $
> ...

 

Any help will be greatly appreciated.

----------

## all-inc.

hi, if I start my distccd with log-level debug, I also get this warning messages! But nevertheless it works perfectly, so I think the problems you experience haven't anything to do with this messages...the start-stop-daemon invocation fails, so try to substitute --quiet by --verbose in the /etc/init.d/distccd file... hopefully this will give us some useful information. dmesg | tail -n 20 could be interesitng, too. If all this doesn't help, please try to start your distccd manually with all options the start-stop-daemon would start it:

```
# PATH="/usr/i686-pc-linux-gnu/gcc-bin/3.4.6" /usr/bin/distccd --pid-file /var/run/distccd/distccd-x86-3.4.6.pid -N 10 --user distcc -j3 --port 3633 --log-level debug --listen 192.168.1.99 --allow 192.168.1.0/24
```

you could also extract some debug infos from the init script by inserting the following lines just before the TMPDIR="${TMPDIR}" \ line:

```
echo "_PATH is ${_PATH}"

echo "DISTCCD_OPTS are ${DISTCCD_OPTS}"
```

 and what else comes into your mind...

hope we'll find something

[EDIT] Please also add the output of your syslog that belongs to the distccd (/var/log/everything/current or /var/log/messages) or add --log-file /var/log/distccd-x86-3.4.6.log to the DISTCCD_OPTS (this option is recommend to use in the distccd manpage...see man distccd) and post its output. [/EDIT]

----------

## javlinz

Thank you for the reply.  With your help, I've been able to narrow down the problem.  I was able to start the init script when I removed "--listen 192.168.1.99" from DISTCCD_OPTS.

This is the output from /var/log/everything/current when I add  --listen to DISTCCD_OPTS

 *Quote:*   

> 
> 
> Oct  3 15:07:25 [distccd] (dcc_setup_daemon_path) daemon's PATH is /usr/i686-pc-linux-gnu/gcc-bin/3.4.6
> 
> Oct  3 15:07:25 [distccd] (dcc_should_be_inetd) stdin is a tty; assuming --daemon mode
> ...

 

This is the output from /var/log/everything/current when I remove  --listen from DISTCCD_OPTS

 *Quote:*   

> 
> 
> Oct  3 15:41:22 [distccd] (dcc_setup_daemon_path) daemon's PATH is /usr/i686-pc-linux-gnu/gcc-bin/3.4.6
> 
> Oct  3 15:41:22 [distccd] (dcc_should_be_inetd) stdin is a tty; assuming --daemon mode
> ...

 

Any idea of why I can't use --listen?

----------

## all-inc.

ok, good that we found the problem  :Wink:  hmm, your IP seems to be requested from a DHCP, right? perhaps your IP isn't always 192.168.1.99, what shows ifconfig ? Is it really your ip?

----------

## javlinz

Both my host's and target's IP addresses are static. My host's IP is 192.168.1.25, and target's IP is 192.168.1.99.  Will distcc work if I don't set the --listen option?

----------

## all-inc.

Ah, that's the point where you made an error! the --listen option specifies on which ip to listen on, that means, to which ip to bind! your distccd runs on the .25 machine, right? so you have to set --listen 192.168.1.25 because you bind to this address. this option doesn't mean listen for the given ip. --allow is for this. The one you called host is the machine with distccd running, I assume ^^

Of course distccd also works without --listen, it doesn't care, if it listens on all or only the given ip. It's just a security option. If this host is, eg., connected to the internet directly and you haven't set your firewall proper, some users could access the distccd service from the internet. OK, this doesn't happen every day, but who knows...  :Wink: 

----------

## javlinz

You're right. I had the --listen option mixed up   :Embarassed:  .  I've corrected the problem, and it's working great now. Thank you for your help, and for this HOWTO. Great work!

----------

## nickd

I'm a bit worried that I'm being stupid here, but the init.d script is failing. I think it's because ${SVCNAME} is undefined.

```
/etc/init.d/distccd-x86 start

 * Caching service dependencies ...                                                                        [ ok ]

 * Starting distccd/x86-default ...

/etc/init.d/distccd-x86: line 21:  : command not found

/etc/init.d/distccd-x86: line 23:  : command not found
```

```
14 start() {

15         [ -e "${DISTCCD_PIDFILE}" ] && rm -f ${DISTCCD_PIDFILE} &>/dev/null

16

17         #ebegin "Starting distccd"

18         ebegin "Starting distccd/${_a_suffix}-${_g_suffix:-"default"}"

19         chown distcc `dirname ${DISTCCD_PIDFILE}` &>/dev/null

20         _binpath="$(gcc-config -B ${_gcc_profile})"

21         [[ ${_a_suffix} == "x86" ]] && \

22         _PATH=$(echo ${_binpath} | \

23         sed -e 's:x86_64:i686:g') || \

24         _PATH=${_binpath}

25         TMPDIR="${TMPDIR}" \

26         #PATH="$(gcc-config --get-bin-path):${PATH}" \

27         PATH="${_PATH}" \
```

Any clues please.

TIA.

Nick

----------

## all-inc.

hi,

SVCNAME is provided by /sbin/runscript. (have a look at /sbin/runscript.sh)

in the lines 21 and 23 at your quotes i read: *Quote:*   

> 21         [[ ${_a_suffix} == "x86" ]] && \ 
> 
> 23         sed -e 's:x86_64:i686:g') || \

 

this commands can't show "/etc/init.d/distccd-x86: line 21:  : command not found" ... this doesn't make sense  :Wink: 

please show the correct lines that cause the error messages.[/glsa]

----------

## nickd

Thanks for the response.

Here's the whole file:

```
 1 #!/sbin/runscript

 2 # $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.18/init,v 1.1 200   4/10/12 17:21:43 lisa Exp $

 3 

 4 depend() {

 5         need net

 6         use ypbind

 7 }

 8 

 9 _a_suffix="$(echo ${SVCNAME} | cut -s -d "-" -f 2)" #extract arch from suffix    

10 [[ -z ${_a_suffix} ]] && _a_suffix="amd64" 

11 _g_suffix="$(echo ${SVCNAME} | cut -s -d "-" -f 3)" #extract gcc-version from    suffix 

12 [[ -z ${_g_suffix} ]] && _gcc_profile="" || _gcc_profile="x86_64-pc-linux-gnu   -${_g_suffix}"

13 

14 start() { 

15         [ -e "${DISTCCD_PIDFILE}" ] && rm -f ${DISTCCD_PIDFILE} &>/dev/null

16         

17         #ebegin "Starting distccd"

18         ebegin "Starting distccd/${_a_suffix}-${_g_suffix:-"default"}"

19         chown distcc `dirname ${DISTCCD_PIDFILE}` &>/dev/null

20         _binpath="$(gcc-config -B ${_gcc_profile})"

21         [[ ${_a_suffix} == "x86" ]] && \

22         _PATH=$(echo ${_binpath} | \ 

23         sed -e 's:x86_64:i686:g') || \

24         _PATH=${_binpath}

25         TMPDIR="${TMPDIR}" \

26         #PATH="$(gcc-config --get-bin-path):${PATH}" \

27         PATH="${_PATH}" \

28         /sbin/start-stop-daemon --start --quiet --startas ${DISTCCD_EXEC} \

29         --pidfile ${DISTCCD_PIDFILE} -- \

30         --pid-file ${DISTCCD_PIDFILE} -N ${DISTCCD_NICE} --user distcc \

31         ${DISTCCD_OPTS}

32 

33         eend $?

34 }

35 

36 stop() {

37         #ebegin "Stopping distccd"

38         ebegin "Stopping distccd/${_a_suffix}-${_g_suffix:-"default"}"

39         start-stop-daemon --stop --quiet --pidfile "${DISTCCD_PIDFILE}"

40         rm -f "${DISTCCD_PIDFILE}"

41         eend $?

42 }
```

Does that make any sense?

Ack the sunscript.sh - so why is mine not returning a SVCNAME value?  I see it there, and being exported.

N.

----------

## Fi

got all running,

 (except distcc bugs on distcc-config  --install, not creating the simlinks in /usr/lib/distcc/lib, which i did manualy)

but updating world stopped on initng-0.6.7

it uses cmake, if it may have anything to do with that, i get :

 error: CPU you selected does not support x86-64 instruction set

i've tried redefining " CC=i686-pc-linux-gnu-gcc emerge initng -u "

in case it was not pasing the correct cc version. thought i understand from this distcc-x86 script that it doesnt actually needs it (or does it?).

but it didnt work anyways.

Any ideas? Thanks

----------

## all-inc.

As I mentioned before, there are just some programs that stop compiling with errors, e.g. yours CPU you selected does not support x86-64 instruction set. Especially programs that act on a low level, like initng does. In that case just compile the package without distcc. if you experience such errors, please mail them at bugs.gentoo.org so they can change the ebuild to not use distccd in the future. but include some informations like the host and target machine archs and that you use this method here...

--edit-- ah, and you don't need CC=i686-pc-linux-gnu-gcc or similar --edit--

@nickd

why exactly do you think that SVCNAME isn't available?! please add some debug messages to your /etc/init.d/distccd-x86 file. for example, add

```
echo "svcname: ${SVCNAME}"

echo "_gcc_profile: ${_gcc_profile}; _a_suffix: ${_a_suffix}"
```

 and some other things that could be interesting... or add at line 14/15 echo "I call start()" and such stuff at other points...

cya all-inc.

----------

## Fi

so i investigated somewhat

and there is something not going well

if i use the wrapper script on my target computer 

so that /usr/lib/distcc/bin/gxx is mapped to /usr/lib/disticc/bin/i686-pc-linux-gnu-gxx

as the crossdev howto explains

then initng compiles with distcc

so i changed things around in the script changing

```
exec /usr/lib/distcc/bin/i686-pc-linux-gnu-g${0:$[-2]} "$@"

```

to

```
exec /usr/bin/distcc -m32  "$@"

```

and it works fine, it yields the same "CPU you selected does not support x86-64 instruction set" error if "-m32" is ommited

so i changed the line again to

```
exec /usr/lib/distcc/bin/x86_64-pc-linux-gnu-g${0:$[-2]} -m32 "$@"

```

and created the corresponding links of x86_64....-gxx to point to /usr/bin/distcc

but i'm getting problems here for some reason despite i removed localhost from /etc/distcc/hosts

it still does some compilation on localhost 

so i made a symlink from x86_64--- to /usr/bin/gxx on the target

with no succes for the moment (but trying this might be a stupid idea)

anyhow, there is something going wrong somewhere

i'd love to put logs of distcc but i can't find the log anywhere 

i wonder if distcc is actually using my configure options from /etc/conf.d/distcc....Last edited by Fi on Wed Nov 15, 2006 10:21 pm; edited 1 time in total

----------

## Fi

I was wondering if the problem is not comming from the host configuration

somehow the gxx command is not trapped by the wrapper script  on the host

probably because of a path problem

using a wrapper script on the target solves the problem in my case

what do you think?

----------

## Fi

so my little P3 has been updating things

and it blocked on opensp, but rather curiously it blocked at link time ??

i thought it should not have problems with linking

```
undefined reference to `__cxa_get_exception_ptr'
```

and ld exits with error status 1

i suppose i'll need to fill in a bug, 

thought i wonder if the problem is in distcc or the crosscompiling, i've searched google for __cxa---

and havnt found anything explicit

any ideas?

----------

## all-inc.

hi,

does this problem occur if you compile without distcc? please try for every problem if it occurs without distcc...

if you use ccache and updated your gcc, then clean your ccache database!

and pleeaassee add debug information/echo'ing to your script(s)!

thank you, all-inc.

----------

## Fi

sorry, i've of course tried without distcc and everything works fine,

i dont use ccache

i have the log level of distcc set to critical but havnt found any log (/var/log/critical??).

about debuging messages. i'm not sure what you would need

for the moment i've finished updating my system without distcc as i had too many problems

if nobody else is having problems, it migth be my install, perhaps some conflict between crossdev which i had installed before and your scripts. 

perhaps also network problems, when using distcc with my wireless network, (my amd64 box is used as master) the atheros drivers fails and i have to reinitialisze the network device, i never had this problem in any other situations !!!?!

so i switch to ieee1394 network link. but i still have problems with distcc.

i'll try to find some time to modify my scripts and retry compiling, and post more information

regards,

----------

## Fi

ok, /me stupid, i miss interpreted --log-level critical thing and thougth it meant log everything as critical

and not only log critical events 

changed it to debug and will post information later

----------

## Fi

fist, as i have done a depclean of my system, crossdev is gone, so it shouldnt be that

i've restroed my /usr/i686-pc-linux-gnu/gcc-bin/3.4.6/ directory

i've removed my wrapper on the target in /usr/lib/distcc/bin i had added to ensure -m32 argument was used

in debug of distcc on the host i get seg faults all the time :

for example

```

distccd[11819] (dcc_check_client) connection from 192.168.0.94:4711

distccd[11819] (dcc_check_address) match client 0x5e00a8c0, value 0xa8c0, mask 0xffff

distccd[11819] (dcc_r_token_int) got DIST00000001

distccd[11819] (dcc_r_token_int) got ARGC00000008

distccd[11819] (dcc_r_argv) reading 8 arguments from job submission

distccd[11819] (dcc_r_token_int) got ARGV00000015

distccd[11819] (dcc_r_argv) argv[0] = "i686-pc-linux-gnu-g++"

distccd[11819] (dcc_r_token_int) got ARGV0000000f

distccd[11819] (dcc_r_argv) argv[1] = "-march=pentium3"

distccd[11819] (dcc_r_token_int) got ARGV00000003

distccd[11819] (dcc_r_argv) argv[2] = "-O3"

distccd[11819] (dcc_r_token_int) got ARGV00000005

distccd[11819] (dcc_r_argv) argv[3] = "-pipe"

distccd[11819] (dcc_r_token_int) got ARGV00000002

distccd[11819] (dcc_r_argv) argv[4] = "-c"

distccd[11819] (dcc_r_token_int) got ARGV0000000d

distccd[11819] (dcc_r_argv) argv[5] = "ParserApp.cxx"

distccd[11819] (dcc_r_token_int) got ARGV00000002

distccd[11819] (dcc_r_argv) argv[6] = "-o"

distccd[11819] (dcc_r_token_int) got ARGV0000000b

distccd[11819] (dcc_r_argv) argv[7] = "ParserApp.o"

distccd[11819] (dcc_r_argv) got arguments: i686-pc-linux-gnu-g++ -march=pentium3 -O3 -pipe -c ParserApp.cxx -o ParserApp.o

distccd[11819] (dcc_scan_args) scanning arguments: i686-pc-linux-gnu-g++ -march=pentium3 -O3 -pipe -c ParserApp.cxx -o ParserApp.o

distccd[11819] (dcc_scan_args) found input file "ParserApp.cxx"

distccd[11819] (dcc_scan_args) found object/output file "ParserApp.o"

distccd[11819] compile from ParserApp.cxx to ParserApp.o

distccd[11819] (dcc_run_job) output file ParserApp.o

distccd[11819] (dcc_input_tmpnam) input file ParserApp.cxx

distccd[11819] (dcc_r_token_int) got DOTI00054f81

distccd[11819] (dcc_r_file) received 348033 bytes to file /tmp/distccd_eff4a1ee.ii

distccd[11819] (dcc_r_file_timed) 348033 bytes received in 0.140971s, rate 2411kB/s

distccd[11819] (dcc_set_input) changed input from "ParserApp.cxx" to "/tmp/distccd_eff4a1ee.ii"

distccd[11819] (dcc_set_input) command after: i686-pc-linux-gnu-g++ -march=pentium3 -O3 -pipe -c /tmp/distccd_eff4a1ee.ii -o ParserApp.o

distccd[11819] (dcc_set_output) changed output from "ParserApp.o" to "/tmp/distccd_efaca1ee.o"

distccd[11819] (dcc_set_output) command after: i686-pc-linux-gnu-g++ -march=pentium3 -O3 -pipe -c /tmp/distccd_eff4a1ee.ii -o /tmp/distccd_efaca1ee.o

distccd[11819] (dcc_spawn_child) forking to execute: i686-pc-linux-gnu-g++ -march=pentium3 -O3 -pipe -c /tmp/distccd_eff4a1ee.ii -o /tmp/distccd_efaca1ee.o

distccd[12002] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1

distccd[11819] (dcc_spawn_child) child started as pid12002

distccd[11819] (dcc_collect_child) cc child 12002 terminated with status 0xb

distccd[11819] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 82 minflt, 0 majflt

distccd[11819] (dcc_x_token_int) send DONE00000001

distccd[11819] (dcc_x_token_int) send STAT0000000b

distccd[11819] (dcc_x_file) send 99 byte file /tmp/distcc_e848a1ee.stderr with token SERR

distccd[11819] (dcc_x_token_int) send SERR00000063

distccd[11819] (dcc_x_file) send 0 byte file /tmp/distcc_e81ba1ee.stdout with token SOUT

distccd[11819] (dcc_x_token_int) send SOUT00000000

distccd[11819] (dcc_x_token_int) send DOTO00000000

distccd[11819] i686-pc-linux-gnu-g++ ParserApp.cxx on localhost:Segmentation fault 

distccd[11819] job complete

distccd[11819] (dcc_cleanup_tempfiles) deleted 4 temporary files

```

which should explain why compilation is sooo slow as it spends time using the network sending compilations that end up in seg fault

was wondering, would it be the -03 flag?

i'll try without

as for the echos in distcc-x86: it outputs

```

svcname: distccd-x86

_gcc_profile: ; _a_suffix: x86

```

i suppose it's ok, 

opensp did not end up with a compilation error this time

----------

## all-inc.

hmm, segfault shouldn't have something to do with the wrapper or the -O3 flags...this are "deeper" errors in the system  :Wink: 

just try out to compile a simple program like:

```
#include <iostream>

#include <string>

using namespace std;

int main() {

        string* s = new string("abcdefg");

        int c = 0;

        int idx = s->find('a');

        cout << s->substr((idx+=3), 3) << s->substr(idx, 3) << endl;

        delete s;

}
```

 by saving it in save.cc and then run /usr/lib/distcc/bin/i686-pc-linux-gnu-g++ save.cc. then, do file test to ensure that it's a 32 and not a 64 bit executable. if it segfaults, then just run a simple g++ save.cc so your regular 64-bit compiler runs. i hope we'll find out something...[/b]

----------

## Fi

Sorry, long time no loggin

indeed, i do suppose the segfault have to do with the system...somehow

thank very much for your suggestion, and involvement in trying to figure out what is going wrong with my setup. it is appreciated.

I tried to compiled as you said (from the target, my PIII i presumed ?)

it seems it compiles it on my localhost and hence outputs a 32bit exec

not understanding why, i re-emerged distcc on localhost...

i get this error during install,

```

 * Installing links to native compilers...

Creating /etc/env.d/02distcc...

Traceback (most recent call last):

  File "/usr/bin/distcc-config", line 140, in ?

    chost = re.compile('CHOST="(.*)"').search(makeconf).group(1)

AttributeError: 'NoneType' object has no attribute 'group'

 * Setting permissions on /var/run/distccd

```

 i had  corrected this by hand by creating the links in /usr/lib/distcc/bin

 but maybe i am doing something wrong here

or maybe if this install failed, something else is going wrong and didn't appear

note that the emerge of distcc does'nt echo any error after merge...

so i suppose all my problems come from my target system 

and i certainly should try to figure out what is going wrong with this emerge

(my system is up to date using ~x86, have done a revdep-rebuild )

something is going wrong with python, and i suppose i have to fil in a bug for distcc ebuild or something..

thought if you had any idea what could be causing this install bug, it'll help me much, as i have no clue at the moment

sincerely

Philippe Leroux

----------

## Kwirl

Hi,

i found this nice howto and it fits perfectly to my needs. But it is not working for my setup. The i686-Host sends its files to the amd64-machine and it tries to compile. But it ends up with a segfault. Here is the debug-aoutput from distccd:

```

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_check_client) connection from 192.168.0.1:56128

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_check_address) match client 0x11b6886, value 0x106886, mask 0xf0ffff

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got DIST00000001

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGC0000001a

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) reading 26 arguments from job submission

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000015

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[0] = "i686-pc-linux-gnu-g++"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000e

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[1] = "-Wno-long-long"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000007

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[2] = "-Wundef"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000005

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[3] = "-ansi"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000c

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[4] = "-Wcast-align"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000c

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[5] = "-Wconversion"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000011

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[6] = "-Wchar-subscripts"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000005

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[7] = "-Wall"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000002

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[8] = "-W"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000f

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[9] = "-Wpointer-arith"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000003

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[10] = "-O2"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000003

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[11] = "-O2"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000f

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[12] = "-march=pentium4"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000014

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[13] = "-fomit-frame-pointer"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000005

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[14] = "-pipe"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000011

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[15] = "-Wformat-security"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000001a

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[16] = "-Wmissing-format-attribute"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000015

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[17] = "-Wno-non-virtual-dtor"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000f

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[18] = "-fno-exceptions"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000e

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[19] = "-fno-check-new"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000b

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[20] = "-fno-common"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000002

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[21] = "-c"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000a

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[22] = "imdata.cpp"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000005

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[23] = "-fPIC"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV00000002

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[24] = "-o"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got ARGV0000000e

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) argv[25] = ".libs/imdata.o"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_argv) got arguments: i686-pc-linux-gnu-g++ -Wno-long-long -Wundef -ansi -Wcast-align -Wconversion -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -march=pentium4 -fomit-frame-pointer -pipe -Wformat-security -Wmissing-format-attribute -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common -c imdata.cpp -fPIC -o .libs/imdata.o

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_scan_args) scanning arguments: i686-pc-linux-gnu-g++ -Wno-long-long -Wundef -ansi -Wcast-align -Wconversion -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -march=pentium4 -fomit-frame-pointer -pipe -Wformat-security -Wmissing-format-attribute -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common -c imdata.cpp -fPIC -o .libs/imdata.o

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_scan_args) found input file "imdata.cpp"

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_scan_args) found object/output file ".libs/imdata.o"

Dec  4 15:24:10 ads2 distccd[4512]: compile from imdata.cpp to imdata.o

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_run_job) output file .libs/imdata.o

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_input_tmpnam) input file imdata.cpp

Dec  4 15:24:10 ads2 distccd[4512]: (dcc_r_token_int) got DOTI00045614

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_r_file) received 284180 bytes to file /tmp/distccd_1a882f8a.ii

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_r_file_timed) 284180 bytes received in 0.264960s, rate 1047kB/s

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_set_input) changed input from "imdata.cpp" to "/tmp/distccd_1a882f8a.ii"

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_set_input) command after: i686-pc-linux-gnu-g++ -Wno-long-long -Wundef -ansi -Wcast-align -Wconversion -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -march=pentium4 -fomit-frame-pointer -pipe -Wformat-security -Wmissing-format-attribute -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common -c /tmp/distccd_1a882f8a.ii -fPIC -o .libs/imdata.o

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_set_output) changed output from ".libs/imdata.o" to "/tmp/distccd_1aaa2f8a.o"

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_set_output) command after: i686-pc-linux-gnu-g++ -Wno-long-long -Wundef -ansi -Wcast-align -Wconversion -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -march=pentium4 -fomit-frame-pointer -pipe -Wformat-security -Wmissing-format-attribute -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common -c /tmp/distccd_1a882f8a.ii -fPIC -o /tmp/distccd_1aaa2f8a.o

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_spawn_child) forking to execute: i686-pc-linux-gnu-g++ -Wno-long-long -Wundef -ansi -Wcast-align -Wconversion -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -march=pentium4 -fomit-frame-pointer -pipe -Wformat-security -Wmissing-format-attribute -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common -c /tmp/distccd_1a882f8a.ii -fPIC -o /tmp/distccd_1aaa2f8a.o

Dec  4 15:24:11 ads2 distccd[4524]: (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1

Dec  4 15:24:11 ads2 distccd[4524]: segfault at 0000000000000005 rip 00002b4fb66cf35a rsp 00007ffff45344e0 error 4

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_spawn_child) child started as pid4524

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_collect_child) cc child 4524 terminated with status 0xb

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 104 minflt, 0 majflt

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_x_token_int) send DONE00000001

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_x_token_int) send STAT0000000b

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_x_file) send 98 byte file /tmp/distcc_111a2f8a.stderr with token SERR

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_x_token_int) send SERR00000062

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_x_file) send 0 byte file /tmp/distcc_12e42f8a.stdout with token SOUT

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_x_token_int) send SOUT00000000

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_x_token_int) send DOTO00000000

Dec  4 15:24:11 ads2 distccd[4512]: i686-pc-linux-gnu-g++ imdata.cpp on localhost:Segmentation fault

Dec  4 15:24:11 ads2 distccd[4512]: job complete

Dec  4 15:24:11 ads2 distccd[4512]: (dcc_cleanup_tempfiles) deleted 4 temporary files

```

Here is the output from "/etc/init.d/distccd-x86 start":

```

/etc/init.d/distccd-x86 start

 * Starting distccd/x86-default ...

distccd[5123] (dcc_should_be_inetd) stdin is a tty; assuming --daemon mode

distccd[5123] (dcc_discard_root) discarded root privileges, changed to uid=240 gid=2

```

Is this ok?

----------

## Kwirl

 *all-inc. wrote:*   

> by saving it in save.cc and then run /usr/lib/distcc/bin/i686-pc-linux-gnu-g++ save.cc. then, do file test to ensure that it's a 32 and not a 64 bit executable. if it segfaults, then just run a simple g++ save.cc so your regular 64-bit compiler runs. i hope we'll find out something...[/b]

 

I don't even have this programm /usr/lib/distcc/bin/i686-pc-linux-gnu-g++ on my 64bit-machine. There is just x86_64.... Is that correct?

----------

## tipp98

Awesome scripts. I only had one slight issue

```
/usr/i686-pc-linux-gnu/gcc-bin/4.1.2/gcc: line 5: /usr/bin/sed: No such file or directory

/usr/i686-pc-linux-gnu/gcc-bin/4.1.2/gcc: line 5: -m32: command not found

distcc[24353] ERROR: compile scripts/mod/empty.c on Falcon failed with exit code 127
```

after emerging sed, which didn't fix the problem, and trying to unmerge it with a "may damage your system" warning I realized it was already installed and simply changed /usr/bin/sed to /bin/sed.

Now my "make CC="distcc gcc" -j4" actively uses both systems. Although I didn't let the make finish to see if it works. 

A question however, is it possible to compile 64bit code on the 32bit machine in this manner, without using crossdev? I tried transposing the amd64's with the x86's and the x86_64's with the i686's in your script but when I try the kernel make I get 

```
arch/x86/kernel/process_64.c:1: error: code model 'kernel' not supported in the 32 bit mode

arch/x86/kernel/process_64.c:1: sorry, unimplemented: 64-bit mode not compiled in

distcc[19555] ERROR: compile arch/x86/kernel/process_64.c on Woody failed

make[1]: *** [arch/x86/kernel/process_64.o] Error 1

make: *** [arch/x86/kernel] Error 2

make: *** Waiting for unfinished jobs....
```

I see there is a multilib flag for gcc but it only referrs to "64bit systems". 

 *Quote:*   

> multilib
> 
> Global: On 64bit systems, if you want to be able to compile 32bit and 64bit binaries
> 
> 

 

Thats odd, multilib is not set on my 64bit system. Is this going to cause problems and if not what the heck is it for?

----------

## rada

Couple small fixes.

1. Wrapper script: /usr/bin/sed does not exist, its /bin/sed

2. Link i686-pc-linux-gnu-gcc -> i686-pseudo-wrapper.sh instead of i686-pc-linux-gnu-gcc -> gcc -> i686-pseudo-wrapper.sh

3. Add small reminder to do chmod +x on the script

 *all-inc. wrote:*   

> 
> 
> Next, switch to this new dir: cd /usr/i686-pc-linux-gnu/gcc-bin/<t's version>
> 
> Here, create a file named i686-pseudo-wrapper.sh with the following content
> ...

 

----------

## rada

 *tipp98 wrote:*   

> A question however, is it possible to compile 64bit code on the 32bit machine in this manner, without using crossdev?

 

No, x86 is a subset of x86_64.  x86_64 does not exist (natively) within x86.

 *tipp98 wrote:*   

> Thats odd, multilib is not set on my 64bit system. Is this going to cause problems and if not what the heck is it for?

 

To set the multilib useflag, switch to the 2008.0 profile: 'eselect profile list' then 'eselect profile set <number>'

----------

## Jormangeud

Thanks, Sebastian for the howto.

No need to edit or copy distcc init or configs on the x86_64 host.     

Lets just put the wrapper script in /usr/local/bin, it should be on the path for distccd.

/usr/local/bin/i686-pseudo-wrapper.bash:

```

#!/bin/bash

$(echo $(basename $0) | sed -e 's:i686:x86_64:g') -m32 "$@"

```

Then link it as the i686 compilers and make executable:

```
                                    

for i in {c++,cpp,g++,gcc}; do ln -s i686-pseudo-wrapper.bash /usr/local/bin/i686-pc-linux-gnu-$i; done

chmod a+x /usr/local/bin/i686-pseudo-wrapper.bash

```

The shorthands (c++, cpp, g++, gcc) probably need editing the distcc as on OP. However, ebuild environment should use the chost compiler and work fine.

----------

