# Really simple LAN distfiles sharing

## Ant P.

This isn't quite as effective as a full caching proxy or shared NFS distdir, but it takes seconds to set up and can reduce your external net traffic quite a lot.

On each machine you want to serve distfiles from, make a script in /etc/local.d/ (or your rc system's equivalent) like this, and start it up:

```
#!/bin/sh

set -e

wwwroot=/srv/www/distfiles-mirror    # feel free to change this

mkdir -p "$wwwroot"

ln -sT $(portageq envvar DISTDIR) "$wwwroot"/distfiles 2> /dev/null || true

busybox httpd -u nobody -p 8080 -h "$wwwroot"
```

On each client machine, edit /etc/portage/mirrors to list the hostnames or IPs you set up:

```
local http://foo:8080 http://bar.local:8080 http://192.168.1.100:8080
```

Hint: If some of the machines aren't on all the time, you may want to lower the fetch timeout (use `portageq envvar FETCHCOMMAND` to get the default, edit it and put it in make.conf).

----------

## NeddySeagoon

Ant P.,

```
emerge http-replicator
```

or an I missing something?

----------

## Buffoon

I have only one portage and only one distfiles on LAN, it is shared over NFS (every machine has its own packages, though). Why should I keep a copy of distfiles in every box?

----------

## szatox

 *Quote:*   

> but it takes seconds to set up

  I guess that's the reason.

And it's a really good tip. Well, it's a poor solution to this particular problem, but seriously.... Did you know busybox can act as a web server?  :Laughing: 

That little thing doesn't stop surprising me. It's awesome!

----------

## khayyam

 *szatox wrote:*   

> [...] but seriously.... Did you know busybox can act as a web server?

 

szatox ... I did, but never thought to put it to use in this way (thanks Ant). Also worth noting, busybox has a 'netcat' ("network swiss army knife").

```
CONFIG_NC_SERVER=y

CONFIG_NC_EXTRA=y

CONFIG_NC_110_COMPAT=y
```

best ... khay

----------

## Dr.Willy

Why

```
#!/bin/sh

set -e
```

over

```
#!/bin/sh -e
```

?

----------

## Ant P.

@Dr.Willy: it's just a convention I've ended up using over time. Maybe I'll stop doing that.

@everyone else: you all make good points, szatox is right that this is just a "quick and dirty" option. I was changing some things around on my own LAN and needed a usable fallback while I was putting the pieces back together, this seemed like a nice option because it's literally already there.

----------

## Syl20

 *Dr.Willy wrote:*   

> Why
> 
> ```
> #!/bin/sh
> 
> ...

 

For security reasons : you can ignore the shebang by running 

```
$ sh /path/script.sh
```

 instead of 

```
$ /path/script.sh
```

The "set ..." lines will always be executed.

----------

## chithanh

 *Buffoon wrote:*   

> I have only one portage and only one distfiles on LAN, it is shared over NFS (every machine has its own packages, though). Why should I keep a copy of distfiles in every box?

 If only trusted hosts have write access to the distfiles directory that is fine. But the checksumming / unpacking of portage is not atomic, so it may create a TOCTTOU race condition.

----------

## miket

 *chithanh wrote:*   

>  *Buffoon wrote:*   I have only one portage and only one distfiles on LAN, it is shared over NFS (every machine has its own packages, though). Why should I keep a copy of distfiles in every box? If only trusted hosts have write access to the distfiles directory that is fine. But the checksumming / unpacking of portage is not atomic, so it may create a TOCTTOU race condition.

 

I always operated with the suspicion that my sharing /usr/portage/ directory could make make me vulnerable to races on /usr/portage/distfiles/.  This led me to adopt the discipline of letting only one machine at a time run emerge.  When I went to a binhost solution, this problem went away--so long as stayed away from doing full package builds on the other hosts at the same time. (You still have to share /usr/portage/ to install packages built on the binhost.)

Right now I'm playing with using the binhost to build multiple sets of host configurations (e.g. desktop and laptop) with overlayfs overlays so as to maximize package reuse among flavors and avoid a lot of duplicate building.  Not only does this require a consistent set of compiler flags across all flavors (easy because all are some variety of Intel Core) but it also means that I do emerges for only one of these layers at a time.

----------

