# synchronize local and remote directory

## bpkri

Does anyone here have an idea how I can synchronize a remote directory to a local one by means of FTP? Are there any programs who can do that?

I have a gallery where sometimes I delete files and add new ones - adding is not a problem. gftp can see the files already existing and won't upload them again, but if I delete files I don't need anymore I also would want them to vanish from the ftp server. Any ideas?  :Question: 

My only means to access the remote dir other than reading is ftp.

And thanks a lot in advance.

----------

## Sven Vermeulen

Doesn't rsync support FTP?

----------

## compu-tom

 *Sven Vermeulen wrote:*   

> Doesn't rsync support FTP?

 

Not that I know.

For uploading to a remote side I would use "weex" or a similar tool.

For downloading "ncftpget" (emerge ncftp) comes into my mind. It has a recursive mode (-R).

If it is important to delete local files that were removed on the remote side, I would program an "expect"-based script (emerge expect) which controls an ftp client to get a recursive dir listing (ls -R1). Parse the output of "ls" and compare it with the local files to determine which files can be deleted. As an alternative to expect pipe the commands (login, ls, ...) to the ftp client.

----------

## bpkri

*sigh* okay - I feared there wouldn't be an easy solution to that  - at least for the local side. But weex is a great hint, I will try it out later. Thanks a lot  :Very Happy: 

The biggest problem with linux is not: What does a program do - but how do I solve a certain problem... there's a program for nearly everything  :Very Happy: 

..ummm... after doing a quick search at google and in the online package database: Where can I get weex? Google turns up a debian package and an empty sourceforge project....

Is it in portage - or where does it hide? (Website?)

----------

## compu-tom

AFAIK weex is no longer maintained and the website of the author is unavailable, but I have the latest version at home. I could send you...

There is a replacement for weex. I don't remember its name.

----------

## georgz

Look at http://freshmeat.net/search/?q=ftp%20AND%20mirror

 ftpmirror or ftpcopy, these could do what you want to have.

But as said before, I'd suggest using rsync  (or similar like Unison http://www.cis.upenn.edu/~bcpierce/unison/) for that.

----------

## compu-tom

I'd prefer rsync or cvsup, too. But is has no ftp support as required by bpkri.

----------

## bpkri

Additionally: I cannot install anything on the remote server - it's only from my webhoster, so I depend on a client solution. Will have a look at ftpcopy, ftpmirror - thanks  :Smile: 

----------

## bpkri

Well ftpcopy does fine (as does lftp) - sadly still only for the local site. Any ideas for the remote site?

Maybe I should see if I get a copy of weex somewhere and get to understand the code...  :Wink:  (Well maybe I better should learn c properly first)

----------

## amne

use lftp, "mirror -Re" should exactly do what you want.

(-R = sync to remote site, -e = delete files)

regards,

amne

----------

## slyzer

Hi,

I had the same problem and found an answer how to get it working with rsync. You can use lufs to mount the remote FTP directory local in your filesystem. So it is easy to use rsync and all other sys-apps  :Smile: 

cu

 slyzer

----------

## bpkri

DO you have mpore information on that? IT sounds really like a good thing  :Smile: 

----------

## slyzer

Here's the Sourceforge project page: http://lufs.sourceforge.net/lufs/ There is are some tips about that.

cu

 slyzer

----------

## bpkri

lufs is good. I like that idea very much. Sadly rsync does not work with dirs mounted by lufs. But the problem has thus been reduced to Synchronizing 2 "local" directories.    :Exclamation: 

----------

## honold

wow, this many replies and you all missed the boat  :Smile: 

http://www.cis.upenn.edu/~bcpierce/unison/

----------

## bpkri

 *honold wrote:*   

> wow, this many replies and you all missed the boat 
> 
> http://www.cis.upenn.edu/~bcpierce/unison/

 

Right - I didn't try unison yet. Though I took a look at its homepage... 

Well - okay I know why I didn't look at Unison first: FTP support is not mentioned. Now that I can mount ftp dirs as local dirs there is a chance it might work, but it uses the "rsync algorithm" to transfer files if I've seen that correctly, so chances are Unison will also have the same problem there.

Anyways - the problem now is much less complex - now I only need to make a "slave" directory only look like the "master" dir - timestamps and sizes don't matter, just filenames. I should be able to handle that - as soon as I have time.

----------

## honold

it can use ssh as a SECURE transport which can be automated with key-based authentication

----------

## bpkri

 *honold wrote:*   

> it can use ssh as a SECURE transport which can be automated with key-based authentication

 

That's nice, but I cannot access the ftp server by ssh - the only means of access are http (for reading) and ftp. And as such I have a problem there, I think. Thnx anyways  :Smile: 

----------

## cchapman

lftp works like a dream to remotely sync to ibiblio.  

I have a script if anyone wants it.

----------

## NineMinuteNap

 *honold wrote:*   

> it can use ssh as a SECURE transport which can be automated with key-based authentication

 

Could you point me in the direction of some documentation/examples on how to set this up?

I have a server at home and a server at the office (both running Gentoo) on which I would like to keep a directory synchronized.  Unfortunately, they cannot talk directly to one another as they are both behind firewalls.  However, I am able to access a third server from both machines and use Unison to sync the directory I need.  The last thing I need to do to get everything working is to set up cron jobs on both "client" servers, but I am using SSH and it prompts me for my password each time I run Unison.

I'd gladly RTFM, but so far, I haven't figured out which M to R.

----------

## bpkri

 *cchapman wrote:*   

> lftp works like a dream to remotely sync to ibiblio.  
> 
> I have a script if anyone wants it.

 

Yes, lftp can do that - synch a local dir to a remote dir - sadly I need the other way around  :Smile: 

And Reverse mirroring does not really work like I would expect it to do (copies too many files)

----------

## starbecks

 *cchapman wrote:*   

> lftp works like a dream to remotely sync to ibiblio.  
> 
> I have a script if anyone wants it.

 

I have to set up a process to download a load of files from an FTP everyday.  I'd be interested in your script!

----------

## socrat3

 *cchapman wrote:*   

> lftp works like a dream to remotely sync to ibiblio.  
> 
> I have a script if anyone wants it.

 

Can i have you script?

THX socrat3[at]linux.it

----------

