@madduck is there a way to make rsync behave like that? I'm backing up my media library and sometimes only one song's tag changes, it has to checksum all files and as mentioned in the other article read all 80 gigs of my library to do so. On a USB drive through SSH. Yeah. Any ideas?

I also think 10% is kinda heavy. Your idea is nice.

hi..............

Laptop Deals

There are a number of problems with your script. For example, it does break --backup. Since you're deleting/overwriting the target file with the brute-force cp first, it will be lost and cannot be backupped anymore. Emulating this is next to impossible. Also, in the --dry-run line, adding slashes to src arguments alters rsync behaviour (these could easily be taken away). Also if the source or target are actually files (as opposed to dirs) the script breaks, while rsync alone does not.

Nice idea, but better don't rely on this script and take a code dive instead :-)

http://advogato.org/article/964.html

start at the beginning: stop after the first section. there is core technology required which will allow a much easier transition away from the client-server model.

Note that the theme used by Branchable is available in ikiwiki for use anywhere (actiontabs theme).

One thing that hit me immediately about branchable is it is more visually appealing than the basic ikiwiki theme. I realise the basic ikiwiki theme is basic on purpose - but, one thing I've put off discussing re ikiwiki is a more appealing theme for ikiwiki.info at least, and perhaps a more detailed basewiki theme too. I've put it off because I expect resistance :) -- Jon

Regarding -W, the man page says it's the default for local paths. Since rsync is in my experience still cpu-bound on local paths, I think -W must not be disabling all the checksums. Probably rsync is still doing the md5sum that it uses as a whole-file consistency check. -W may disable the rolling checksum only.

A code-dive is in order..

Liw: I think -W might be the magic option I was looking for! Hidden amoung the hundred or so other magic options. :)

Madduck: Actually, I've been doing all my testing on a N2100. Although disk writes have been going to a USB disk. Still, rsync with checksumming is much much slower than just blasting the bits.

Rsync always checksums. Please see the man page, if you don't believe me:

              Note that rsync always verifies that each transferred  file  was
              correctly  reconstructed  on  the  receiving  side by checking a
              whole-file checksum that is generated  as  the  file  is  trans‐
              ferred

That checksum is a MD5sum. There is also a second, rolling checksum used by the rsync algorithm. Apparently it does both.

And, there is a third one that has to be enabled with --checksum to better detect if an existing file has been changed. But that one is not really relevant.

Does the rsync -W option do the same thing as brute-forcing cp?

Oh, are you saying that it's faster on slow CPUs to brute-force cp the whole thing, rather than to bother finding the differences? If so, then I am sure this could be patched into rsync. Otoh, I find that even on my NAS hardware, IO is the bottleneck, not the CPU. This is especially true for the Thecus N2100…

Unless you pass --checksum, rsync looks at files' size+mtime to determine if they need to be updated, and only then does checksumming to transfer the differences. I am having difficulties seeing how your stdio/shell/1-cp-process-spawned-per-file solution should be any faster.

I'll probably use that option next time I can't stand a 10G file any longer

I am in the moot to brainstorm what would be needed.

-a DHT where all the freedomboxen publish their IPs (Retroshare and Emule use those)

-an dead easy way to generate/exchange GPG public keys with friends (not existant AFAIK)

-Diaspora (or something that does what they say it will do)

-a distributed file system that back ups all the (encrypted) stuff from the box to my friends (maybe http://tahoe-lafs.org)

-silent updates

-silent upgrade to new major releases (a lot harder)

-IMAP mail server with spam filter (not as resource hungry and crappy as SA)

-a good AJAXy web mail frontend (http://roundcube.net/)

-VOIP stuff

-encrypt everything

What else? (putting TOR in the first default install is not wise IMO) Not easy, but still not something the FOSS or Debian community couldn't put together. The only system administration people should be doing themselves is backing up their private key .. that might be too hard for many, but there is no way around that, or is there?

I played the Osmos demo and I can actually take my Workrave-mandated typing breaks without dying. So my carpal tunnels say I should spend the $10.

Yup. I found your page because I googled "git tube map" just to see if anyone else had thought that!

Hi,

Just found flashybrid a couple of days ago :)

Was installing Debian onto my QNAP TS-239 NAS, booting from an USB-key (didn't want to wipe to factory flash). To limit the writes on the USB media I was (apt-cache ..) searching for something flash-related and found flashybrid.

Much appreciated.

Mark

As a belated followup, I eventually discovered what was making apache crash when I configured it to fork a lot of processes -- I had a ulimit 40 stuck in its startup script to work around a different issue earlier. Oops.

(Also, to the best of my knowledge, the privacy issue still exists in the Palm Pre.)

Don't buy from them. I am having trouble getting service.