# How to set up a FULL Mirror

## quakecon

Hey

I want to set up a FULL Gentoo Mirror in our company, so the server won't be public. I only found some instructions how to set up a local rsync mirror which is already running  :Smile:    I don't like the proxy solution, it would be great if the mirror could sync the whole distfile directory at night. 

What do I have to do? Is it just another entry in the rsyncd.conf file???

Thanks for your help

----------

## yaneurabeya

Uhm, in order to just serve distfiles, either add an rsync entry and then set the SYNC to be your rsync directory for your distfiles OR use an existing apache setup and share the directory, then add a SYNC reference in /etc/make.conf. Read man make.conf for more info.

----------

## quakecon

Thanks for your response...

Can you tell me how such an entry would look like?  

Tell me if I'm wrong:

This is my config:

```
# /etc/rsyncd.conf

# $Header: /var/cvsroot/gentoo-x86/net-misc/rsync/files/rsyncd.conf,v 1.6 2005/0

3/24 03:12:45 vapier Exp $

pid file = /var/run/rsyncd.pid

uid = nobody

gid = nobody

use chroot = yes

hosts allow=192.168.1.0/255.255.255.0

hosts deny=*

max connections = 5

motd file = /etc/rsync/rsyncd.motd

[gentoo-portage]

path = /usr/portage

comment = Gentoo Linux Portage tree mirror

exclude = distfiles/ packages/

```

I would just remove the "distfiles/" entry in the exclude line!? Is that all?? And with an emerge sync I'm also able to sync the distfiles?

This would be an easy solution, therefore I'm so confused if its really that easy  :Smile: 

----------

## slycordinator

Another way to do it is sharing through nfs.

http://gentoo-wiki.com/HOWTO_Using_a_shared_portage_via_NFS

http://gentoo-wiki.com/HOWTO_Share_Directories_via_NFS

----------

## quakecon

Thanks for your comment, but I'd like to set up a mirror that I only have to edit the make.conf file on all machines and enter my server as the source mirror.

--- EDIT ---

Ok I think I have an idea :

I just have to rsync the gentoo directory from a public mirror: that could be done with a single rsync command and a little cronjob, but the problem is i didn't found a public source server which has rsync enabled .?

Am I the first person who wants to set up his own local source mirror .....??????

----------

## slycordinator

 *quakecon wrote:*   

> Thanks for your comment, but I'd like to set up a mirror that I only have to edit the make.conf file on all machines and enter my server as the source mirror.

 

With NFS you just would need to edit fstab on the clients rather than make.conf

 *Quote:*   

> Am I the first person who wants to set up his own local source mirror .....??????

 

No.  But you don't need the entire source tree to set up a local mirror.  You set it up so that when you try to emerge something, the server downloads it if it doesn't have it already.  And I have no experience using rsync but this works in NFS.

----------

## Maedhros

Moved from Installing Gentoo to Networking & Security.

----------

## quakecon

Thanks for moving my post to the right position  :Smile: 

Yes it's also an approach to download the packages the first time from internet and cache them. But I want to cache all files  and sync them over the night, if nobody can help me ( I know maybe it's a wired idea  :Smile:  )  I'll use the proxy solution for caching  :Smile: 

----- EDIT  -----

I just tried the proxy out and i have to say it's absolutly not the solution which fits me  :Sad:    I hope somebody can help me to set up a full or distfile mirror...    :Smile: 

----------

## yaneurabeya

Hmmm.. I thought I posted a solution before, but my browser must have crashed or something. Odd.

Do this:

Mirror's rsync.conf

```
pid file = /var/run/rsyncd.pid

uid = nobody

gid = nobody

use chroot = yes

hosts allow=192.168.1.0/255.255.255.0

hosts deny=*

max connections = 5

motd file = /etc/rsync/rsyncd.motd

[distfiles]

path = /usr/portage/distfiles

comment = Distfiles

exclude = cvs-src

[gentoo-portage]

path = /usr/portage

comment = Gentoo Linux Portage tree mirror

exclude = distfiles/ packages/
```

Then run the following on all of your machines that sync (or edit if a GENTOO_MIRRORS var already exists in make.conf):

echo 'GENTOO_MIRRORS="rsync://server-ip-goes-here/"' >> /etc/make.conf

----------

## quakecon

Thanks yaneurabey for your help

I want to download all files from a public mirror to serve it in my LAN the solution you gave me is to serve the distfile directory of my server, so only the packages i downloaded on the server will be served, but i want ALL packages....

 but I think what i want is impossible/forbidden, or you have to sign up for a Public Gentoo Mirror.

Thanks for takin' time  :Smile: 

----------

## zeek

 *quakecon wrote:*   

> I want to download all files from a public mirror to serve it in my LAN the solution you gave me is to serve the distfile directory of my server, so only the packages i downloaded on the server will be served, but i want ALL packages....
> 
> 

 

What files?  The portage tree or the sources for the various ebuilds?

It would be a total waste of bandwidth to download all the sources, and you couldn't even do it if you wanted because there is no single location.

```
grep -r SRC_URI /usr/portage/
```

----------

## quakecon

AHHH ok  :Smile:   I thought all ebuilds are on all public servers, I didn't know that they are so widespreaded...

----------

## slycordinator

 *quakecon wrote:*   

> AHHH ok   I thought all ebuilds are on all public servers, I didn't know that they are so widespreaded...

 

You've misunderstood.  The ebuilds are on all the public rsync servers.  But the files that the ebuilds tell you to download are not.

If that doesn't explain it clearly, here's how it works:

1) You sync to a mirror that sends you all the ebuilds for the portage tree.

2) You want to install package X so you type in "emerge X"

3) Using the ebuild portage determines what files source files to download and where to get them.

And on this last part: Not all of those source code files will be on the mirrors.  In fact, ebuilds specify the default webpage so that if the file in question isn't on the gentoo mirrors it will download them from a default (like if you installed gaim, it would as a last resort try to download from one of the gaim mirrors).

----------

## yaneurabeya

Well, sly was mostly correct but I'll elaborate just a bit more. If your eyes gloss over a bit, just skip what I say  :Razz: .

When you type in 'emerge sync' rsync checks to see if a file is up to date with your syncing server. Then it subsequently goes through and checks all ebuilds to see if they are up to date in the individual branches of your portage tree, and updates if necessary.

When you type emerge [package], emerge checks the ebuild and sees what the dependencies are for the program and also checks to see if the program can be built with whatever ACCEPT_KEYWORDS you have in your /etc/make.conf, /etc/portage/package.keywords, or ACCEPT_KEYWORDS variable defined at runtime. The reasoning behind the dependency building is if you don't already have program X, then the program won't run or worse won't even compile. So if everything goes well, emerge assigns dependencies that are required for the program to run/compile, and checks to see if the files exist in your distfiles directory. If not, it checks to see if the file is fetchable via some source (some 'source files' are fetchable while others are not, take Maya for example where there's a fetch restriction). If they are fetchable, then emerge tries every SRC_URI given by the ebuild (that is if the maintainer specified "download from sourceforge as opposed to 'mirror X'!") or a mirror given by the GENTOO_MIRRORS, in linear order from the first to last. Then if the file's MD5 hash checks out with the one specified in the portage branch that belongs to the ebuild, emerge runs an 'extract operation', and finally proceeds to do anything required to install the program. There's more to it than that, but my explanation was somewhat in-depth I think.

So, rsync_mirrors and distfiles mirrors are two mutually exclusive things, which can live in tandem with each other, or be separate entities, meaning that one is not required for the other to run. I just suggest running both as rsync because it reduces the amount of services you need running, possibly security risks, and is an overall better solution possibly.

----------

## My_World

How-To create your own Gentoo Mirror

Have a look in the above mentioned post for a very detailed How-To, even how to setup your client machines, etc.

Have a carefull read of the above wiki, do not emerge the rsync server, just edit /etc/rsyncd.conf and edit your other boxes accordingly!

 :Razz: 

----------

