# synchronizing directories

## dju`

hi,

i'm wondering how can i synchronize 2 local dirs automatically. i mean, when a new file is created inside one directory, it's automatically copied to the second one and vice versa, in order to keep the 2 dirs synchronized. deleting, changing attributes and permissions would do the same. i also need to create replication chains and to make it work over nfs if possible. any solution please?

----------

## blake121666

Sounds like you want to setup a software RAID1 mirror.

EDIT:  never mind I just reread your post.  You want it to work over NFS?

----------

## dju`

yep over nfs. no i don't want raid because i only want to synchronize two specific directories among the filesystems, regardless of the fact they are root of a raid mirrored partition or not. Actually I want to synchronize automatically, say, /mnt/filer/mydocs/ (which is mounted over nfs from a nfs server) and /home/me/mydocs/. so every time i change something inside ~/mydocs/, the change would be replicated to /mnt/filer/mydocs/ (in that case, on the nfs server). and imagine the link goes down between the nfs server and my box, i would still have my version of ~/mydocs/, i would continue making changes to it and when the link will go up, the offline changes would be replicated (synchronized) to the remote directory. i presume this is basic replication, perhaps using a special filesystem type which would keep trace of every changes in a log before submitting it to other directories/some kind of server/dunno. i hope i'm clear enough, and if you can point me to the right direction it would be much appreciated.

----------

## dju`

i've just found something: would coda or intermezzo be good for me?

----------

## blake121666

I don't think linux itself has a capability for a file (or directiory) to signal that it has changed so that something could do an action on this.  IBM's JFS had this capability on AIX and OS/2 ... maybe it does in Linux as well somehow - I doubt it though.  You'd probably have to poll for changes in some way.  Or you could experiment with the RAID route (albeit in a non-standard and somewhat irregular way).  I think the md's can be somewhat opaque can't they?  I think I heard of someone running RAID over top of LVM (extremely dumb idea of course but I think I read that somewhere).  But that suggests to me that the RAID implementation is flexible and can be implemented in kooky ways.

----------

## blake121666

CODA is a GFS type thing AFAIK.  But I got thinking about how you could just mirror a LV in AIX just by essentially setting a "number of copies" parameter in the volume manager.  I looked into the Sistina LVM that everyone uses (myself included) in Linux and it appears that they haven't added this capability.  But I think Veritas is very similar to IBM's LVM and I see that they have ported to linux:

http://www.veritas.com/van/documentation/volumemanagerwin.html

Or maybe the IBM EVMS, or whatever they called it ... I haven't played much with that, has the capability you're looking for.

However you slice it, it looks to me like you want to mirror at the filesystem level and not a partition level (which I think the standard Linux RAID implementation kinda assumes) so you need to throw an abstraction layer in there somewhere.  Veritas or EVMS or Sistina's LVM is where I would direct my searching (I'm pretty sure sistina doesn't have this though).

I've never really found this approach to be too compelling though.  Maybe you just want to run rsync every once in a while one directory to the other.

----------

## dju`

 *blake121666 wrote:*   

> I've never really found this approach to be too compelling though.  Maybe you just want to run rsync every once in a while one directory to the other.

 

probably. also, i can't remember the name of a synchronization software used by laptop users in disconnected mode, i heard about it some time ago...

thanks anyway for your information.

----------

## Ox-

Maybe app-admin/fam combined with rsync or net-misc/unison?

----------

## dju`

unison... thank you.

----------

## dju`

after having read the documentation, it appears that unison is what i need. i'm going to install and test it. after that, i'll see for the process automation, maybe with fam. thanks all!

----------

## dju`

yeah, unison works great!  :Smile: 

----------

## blake121666

Hey, you might want to look at fuse as well (sys-fs/fuse).  Its example program fusexmp.c can be used as a template for all of the functions you'd want.  I figure you could redefine any things that would change a file to do the same in another directory (such as .rename, .write, ... etc).  Should be fairly easy to setup.

I'm messing around with it right now to get rsync to access metadata for its decisions instead of the actual files in the backup directory of my backup server and it's pretty neat IMO.

----------

## killercow

dJu,

Did you get unison working? and could you tell me how you use it?

Im looking for a file system for my application webserver cluster, which should all read their files from the same source, but would also be able to perform write operations against their local "cache" which would then be forwarded to the main file pool..

I don't want to use NFS because its to slow. (and doesn't scale) NFS is also very scary, since it has not faiover support/redundancy/local cache towork from if the file server dies.

I don't want to use rsync because the indexing is far to cpu intensive.

I don't want to use openAFS, since i can't seem to understand it (have read the manual twice)

Coda looks nice, but i need to setup some flaky server->client->server setup to make it update its files to the main file pool.

Samba/cifs will have the same problems as NFS, but might a bit faster. (but still, its samba, and thus MS, and thus in the spotlight for crackers/scriptkiddies.)

I think unison will do great, bu i'd like to hear some experiences before installing it.

greets 

Jan

----------

## dju`

i have unison working here. why don't you give it a try? basically, after having set up the "roots" on each client (that is, the 2 paths (one local and the other local or remote) that have to be in sync), you issue the "unison" command and the roots get synchronized. you may have to resolve conflicts but if you have ever worked with things like cvs, it shouldn't be a problem.

----------

