# rsyncing behind a firewall

## TheHaas

I have a Gentoo machine behind a firewall that doesn't let rsync through.  But I can ssh to a machine outside the firewall that  can.  Sounds like a solution for an SSH tunnel!

1. ) On the Gentoo machine, edit make.conf and change SYNC to:

```
SYNC="rsync://localhost:4040/gentoo-portage"
```

2.) Make an ssh tunnel from your Gentoo box to the remote box to go to rsync.gentoo.org, i.e.

```
ssh -L 4040:rsync.gentoo.org:873 remote.host.com
```

3.) Now rsync as usual from another prompt:

```
emerge rsync
```

----------

## ydleiF

Slick.

The way we do it here (I'm the admin) is that we have a proxy which allows connections to TCP port 873 which is what rsync uses.

----------

## meowsqueak

Well, I just tried this after coming to the same set of commands myself.

First I created the tunnel to my external rsync-permitted machine:

```
$ ssh -L 4040:rsync.gentoo.org:873 user@remote.host.com
```

Then I tried rsync directly:

```
$ rsync rsync://localhost:4040/gentoo-portage

Welcome to hawk.gentoo.org

 

Server Address : 134.68.220.97 

...etc
```

Then I tried emerge sync:

```
# SYNC="rsync://localhost:4040/gentoo-portage" emerge sync

>>> starting rsync with rsync://65.19.163.230/gentoo-portage...

>>> checking server timestamp ...

rsync: failed to connect to 65.19.163.230: Connection timed out

rsync error: error in socket IO (code 10) at clientserver.c(88)

>>> retry ...

```

Any idea why it's not retrieving the timestamp? I'd really like to get this working - emerge-webrsync really isn't ideal for me.

EDIT: Solved! You can't override SYNC from the command line. I'm not sure why I thought you could actually...

----------

## statmobile

Correct me if I'm wrong, but can't you just use the flag?

```
rsync --rsh=/usr/bin/ssh
```

----------

## meowsqueak

Only if you have ssh access to the final host you are trying to rsync to/from. In this case, we don't. This is about rsync tunnelled over ssh, not rsync using ssh. It's the inside-out problem...

----------

## monotux

a webrsync is the fastest (and easiest) way to do it, still  :Smile: 

----------

## meowsqueak

I disagree - I've been using emerge-webrsync for the last few weeks and it's hideously slow. It downloads the entire Portage tree every time. The first time I ran it, it took 2 hours! Using rsync properly is a far better solution - that's why Portage uses it.

----------

## skander

The port forwarding method on the OP solves my requisite for not running an rsync server on my home machine (and thus opening up another port on the firewall), but shouldn't there be a way to integrate that into the SYNC="..." setting in make.conf, making it do the equivalent of:

rsync -e ssh myuser@myownmirror:/src /local/dest

I tried setting the RSYNC_RSH environment variable, but that seems to get ignored (presumably because of the "rsync://" prefix in $SYNC.  Maybe I could play with setting $SYNC to something like:

SYNC="myuser@myhomecomputer.com:/usr/portage"

----------

## skander

Looks like this format for $SYNC is invalid:

SYNC="myuser@myhomecomputer.com:/usr/portage"

I looked through the emerge code and it only seems to accept "rsync://*" and "cvs://*" formats (and the metadata function).   Sigh.  I guess I'll be running an rsync server at home and just foward the port as outlined in the original posting.  That at least lets me get away with not opening up the rsync port.

----------

## monu

i get this error

>>> Starting retry 2 of 3 with rsync://rsync.gentoo.org/gentoo-portage

>>> checking server timestamp ...

rsync: getaddrinfo: rsync.gentoo.org 873: Temporary failure in name resolution

rsync error: error in socket IO (code 10) at clientserver.c(8 :Cool: 

>>> retry ...

Notice: (-3, 'Temporary failure in name resolution')

what should i do?

----------

## skander

Sounds like you don't have network access.

----------

## sekopaa

 *TheHaas wrote:*   

> I have a Gentoo machine behind a firewall that doesn't let rsync through.  But I can ssh to a machine outside the firewall that  can.  Sounds like a solution for an SSH tunnel!
> 
> 1. ) On the Gentoo machine, edit make.conf and change SYNC to:
> 
> ```
> ...

 

I'm in the same situation and would love to do this, but I have (probably an ssh config problem). My appologies, but I'm an ssh tunneling n00b.

Anyway steps 1 & 2 work fine, but when I "emerge sync" on my local machine I get:

```
>>> starting rsync with rsync://127.0.0.1:4040/gentoo-portage...

>>> checking server timestamp ...

rsync: connection unexpectedly closed (0 bytes received so far) [receiver]

rsync error: error in rsync protocol data stream (code 12) at io.c(434)

>>> retry ...
```

on my remote machine I get this error at the same time:

```
channel 2: open failed: connect failed: Connection refused
```

It seems that my ssh tunnel allows me to get to my remote machine, but won't forward me on to the rsync site. I can "emerge sync" from the remote machine, so I know the rsync port is open. My only thought is that it is a ssh config problem. I've looked through the sshd_config file and added "AllowTcpForwarding yes" which sounded like it might do the trick, but no... Any ideas?

-Thanks-

----------

## TheHaas

 *sekopaa wrote:*   

> 
> 
> I'm in the same situation and would love to do this, but I have (probably an ssh config problem). My appologies, but I'm an ssh tunneling n00b.
> 
> Anyway steps 1 & 2 work fine, but when I "emerge sync" on my local machine I get:
> ...

 

I think that a port is still being blocked someone, which is strange since you can connect via ssh.

Put this in your $HOME/.ssh/config file:

```
ForwardAgent yes
```

----------

## sekopaa

 *TheHaas wrote:*   

> I think that a port is still being blocked someone, which is strange since you can connect via ssh.
> 
> Put this in your $HOME/.ssh/config file:
> 
> ```
> ...

 

That did the trick!  :Very Happy: 

On a side note, you can set this flag system wide in /etc/ssh/ssh_config. I was only looking at /etc/ssh/sshd_config  :Embarassed: 

Thanks!

----------

## TheHaas

 *sekopaa wrote:*   

> 
> 
> That did the trick! 
> 
> On a side note, you can set this flag system wide in /etc/ssh/ssh_config. I was only looking at /etc/ssh/sshd_config 
> ...

 

I'm glad that it worked.

yeah -- ssh_config is for when you connect via client and sshd_config are options for your ssh server.  What's cool is that you can set your own settings in ~/.ssh/config and let your other users suffer with the defaults.  :Smile: 

----------

## dingfelder

I followed the instructions here and opened a tunnel, and now emerge works  :Smile: 

but emerge --sync gives:

 emerge --sync

>>> Starting rsync with rsync://127.0.0.1:4040/gentoo-portage...

>>> Checking server timestamp ...

rsync: failed to connect to 127.0.0.1: Connection refused (111)

rsync error: error in socket IO (code 10) at clientserver.c(104) [receiver=2.6.9]

>>> Retrying...

>>> Starting retry 1 of 3 with rsync://127.0.0.1:4040/gentoo-portage

>>> Checking server timestamp ...

rsync: failed to connect to 127.0.0.1: Connection refused (111)

rsync error: error in socket IO (code 10) at clientserver.c(104) [receiver=2.6.9]

>>> Retrying...

emerge --sync does work on the machine I am sshing to

thoughts?

----------

## dingfelder

bump...    nobody knows...

----------

## Fukai

Hi, this is not a solution but another solution ^_^ what I do is

```

ssh example.org -ND 1080

tsocks emerge  --sync

```

You don't need to change the SYNC var, all is done by tsocks it 'route' all traffic by the sock created by ssh

(you can use it with any app in a unifiqued fashion like tsocks wget, tsocks bla bla bla) you need to config the port

in /etc/socks/tsocks.conf

----------

## stanleysantos

I had to put the IP adress instead of rsync.gentoo.org ==>

```

ssh -l root -L 4040:134.68.220.73:873 ...

```

It works now!

----------

