[nzlug] Replication

[Martin Bähr]

On Fri, May 11, 2007 at 04:37:43PM +1200, Jonathon Sim wrote:
> In my experience with this kind of thing, multi-master file 
> synchronisation with that much data is pretty much futile (one you 
> haven't listed there is csync2, but it also cant handle that much data 
> either).

(not that this helps wit hthe current problem, but)
i don't see why.
can't in most cases the synchronization happen one file at a time?
adding to that handling for new, removed and renamed files should be all
that's needed.

there should be no need to keep information for 500GB of data in memory.
especially if renames/moves are not handled. but even with those, the
task should be possible if you store the necesary information in a disk
cache. sure, it will be slower, but futile?

with the continuous growth of diskspace the need to be able to
synchronize terabytes of data will soon be rather common, so i do expect
that these tools will eventually adapt to handle that...

or am i missing something fundamental here?

greetings, martin.
-- 
cooperative communication with sTeam      -     caudium, pike, roxen and unix
offering: programming, training and administration   -  anywhere in the world
--
pike programmer   working in new zealand        open-steam.org|webhaven.co.nz
unix system-      bahai.or.at                        iaeste.(tuwien.ac|or).at
administrator     (caudium|gotpike).org                          is.schon.org
Martin Bähr       http://www.iaeste.or.at/~mbaehr/