{
"attachments": [],
"comments_archived": true,
"date": "2003-07-07T18:45:21.000Z",
"layout": "post",
"title": "Backups with my eyes closed?",
"wordpress_id": 439,
"wordpress_slug": "backup-question",
"wordpress_url": "http://www.decafbad.com/blog/?p=439",
"year": "2003",
"month": "07",
"day": "07",
"isDir": false,
"slug": "backup-question",
"type": "entry",
"postName": "2003-07-07-backup-question",
"html": "Okay, so at my new job I'm the Guy if it has a transistor in it. I'm\ndeveloper, sysadmin, and hardware jockey all in one. This is fun\nto a certain extent, since it tests pretty much everything I know from\nA through Z. And so far, I'm doing okay. Every now and then, though,\nI get a bit stumped.
\nMy most recent adventure involves developing a backup routine for the\noffice. I just got tape backup working on a Linux box for a big\nSamba-shared directory that we all work out of. I'm currently winging\nit with star and cpio in =CRON=-scheduled scripts that manage a\n6-tape rotation for me.
\nFull backups on alternating tapes on Fridays,\nwith incrementals inbetween on tapes labeled by the day. I even have\nthe server eject the tape and IM me a few times until I go change to\nthe day's tape. Tested recovery, and though it could be smoother, it\nis at least possible at the moment. I figure this is pretty good\nfor my first personal encounter with managing serious backup. I plan\nto keep researching and to upgrade software at some point soon.
\nSo, now my boss asks me: \"Hey, can you backup this other folder for me?\nI don't want to share it, though, and I don't want you to be able to\nread the files.\" This folder contains some important yet sensitive\nthings like salary information and other things to which I have no\nbusiness having access.
\nMy stumper then, is this: How do I grab (or cause to be uploaded) a\nfolder of files for backup, say as large as 2GB, from a WinXP machine,\nwithout having any access myself to read the file contents. I'll be\nable to install whatever I need on the WinXP machine, but the idea is\nthat, when the bits leave that machine for the Linux backup server,\nthere should be no way for me to read their contents. But, I must be\nable to usefully backup and, in conjunction with the owner of the\nfiles, restore in case of disaster.
\nOh yeah, and I have no budget for software. So, I'm trying to work\nthis out using only free tools.
\nSo, my first though is some sort of encryption on the WinXP machine.\nEncrypt with GPG or something, leaving my boss with the secret key\non a floppy and the passphrase in his head. Upload these files\nto a special folder on our shared drive, and it all gets backed up\nlike everything else.
\nOr, since I don't even really want to know the names or number of\nfiles in this sensitive folder, can I somehow ZIP up the whole\nshebang and encrypt that before uploading?
\nUnder Linux, none of this would be much of a problem to me. But,\nunder WinXP, my knowledge of available tools and means of automation\nfail me.
\nAny hints from out there?
\n\n\n\nshortname=backup_question
\n\n\n\n",
"body": "Okay, so at my new job I'm the Guy if it has a transistor in it. I'm\r\ndeveloper, sysadmin, and hardware jockey all in one. This is fun\r\nto a certain extent, since it tests pretty much everything I know from\r\nA through Z. And so far, I'm doing okay. Every now and then, though,\r\nI get a bit stumped.\r\n
\r\nMy most recent adventure involves developing a backup routine for the\r\noffice. I just got tape backup working on a Linux box for a big\r\nSamba-shared directory that we all work out of. I'm currently winging\r\nit with star and cpio in =CRON=-scheduled scripts that manage a\r\n6-tape rotation for me.\r\n
\r\nFull backups on alternating tapes on Fridays,\r\nwith incrementals inbetween on tapes labeled by the day. I even have\r\nthe server eject the tape and IM me a few times until I go change to\r\nthe day's tape. Tested recovery, and though it could be smoother, it\r\nis at least possible at the moment. I figure this is pretty good\r\nfor my first personal encounter with managing serious backup. I plan\r\nto keep researching and to upgrade software at some point soon.\r\n
\r\nSo, now my boss asks me: \"Hey, can you backup this other folder for me?\r\nI don't want to share it, though, and I don't want you to be able to\r\nread the files.\" This folder contains some important yet sensitive\r\nthings like salary information and other things to which I have no\r\nbusiness having access.\r\n
\r\nMy stumper then, is this: How do I grab (or cause to be uploaded) a\r\nfolder of files for backup, say as large as 2GB, from a WinXP machine,\r\nwithout having any access myself to read the file contents. I'll be\r\nable to install whatever I need on the WinXP machine, but the idea is\r\nthat, when the bits leave that machine for the Linux backup server,\r\nthere should be no way for me to read their contents. But, I must be\r\nable to usefully backup and, in conjunction with the owner of the\r\nfiles, restore in case of disaster.\r\n
\r\nOh yeah, and I have no budget for software. So, I'm trying to work\r\nthis out using only free tools.\r\n
\r\nSo, my first though is some sort of encryption on the WinXP machine.\r\nEncrypt with GPG or something, leaving my boss with the secret key\r\non a floppy and the passphrase in his head. Upload these files\r\nto a special folder on our shared drive, and it all gets backed up\r\nlike everything else.\r\n
\r\nOr, since I don't even really want to know the names or number of\r\nfiles in this sensitive folder, can I somehow ZIP up the whole\r\nshebang and encrypt that before uploading?\r\n
\r\nUnder Linux, none of this would be much of a problem to me. But,\r\nunder WinXP, my knowledge of available tools and means of automation\r\nfail me.\r\n
\r\nAny hints from out there?\r\n\r\nshortname=backup_question\r\n\r\n\r\n ",
"parentPath": "./content/posts/archives/2003",
"path": "2003/07/07/backup-question",
"summary": "Okay, so at my new job I'm the Guy if it has a transistor in it. I'm\ndeveloper, sysadmin, and hardware jockey all in one. This is fun\nto a certain extent, since it tests pretty much everything I know from\nA through Z. And so far, I'm doing okay. Every now and then, though,\nI get a bit stumped.\nMy most recent adventure involves developing a backup routine for the\noffice. I just got tape backup working on a Linux box for a big\nSamba-shared directory that we all work out of. I'm currently winging\nit with star and cpio in =CRON=-scheduled scripts that manage a\n6-tape rotation for me.\nFull backups on alternating tapes on Fridays,\nwith incrementals inbetween on tapes labeled by the day. I even have\nthe server eject the tape and IM me a few times until I go change to\nthe day's tape. Tested recovery, and though it could be smoother, it\nis at least possible at the moment. I figure this is pretty good\nfor my first personal encounter with managing serious backup. I plan\nto keep researching and to upgrade software at some point soon.\nSo, now my boss asks me: \"Hey, can you backup this other folder for me?\nI don't want to share it, though, and I don't want you to be able to\nread the files.\" This folder contains some important yet sensitive\nthings like salary information and other things to which I have no\nbusiness having access.\nMy stumper then, is this: How do I grab (or cause to be uploaded) a\nfolder of files for backup, say as large as 2GB, from a WinXP machine,\nwithout having any access myself to read the file contents. I'll be\nable to install whatever I need on the WinXP machine, but the idea is\nthat, when the bits leave that machine for the Linux backup server,\nthere should be no way for me to read their contents. But, I must be\nable to usefully backup and, in conjunction with the owner of the\nfiles, restore in case of disaster.\nOh yeah, and I have no budget for software. So, I'm trying to work\nthis out using only free tools.\nSo, my first though is some sort of encryption on the WinXP machine.\nEncrypt with GPG or something, leaving my boss with the secret key\non a floppy and the passphrase in his head. Upload these files\nto a special folder on our shared drive, and it all gets backed up\nlike everything else.\nOr, since I don't even really want to know the names or number of\nfiles in this sensitive folder, can I somehow ZIP up the whole\nshebang and encrypt that before uploading?\nUnder Linux, none of this would be much of a problem to me. But,\nunder WinXP, my knowledge of available tools and means of automation\nfail me.\nAny hints from out there?",
"needsBuild": true,
"prevPostPath": "2003/07/07/syndications-formats",
"prevPostTitle": "Is the magic in RSS, or in Syndication?",
"nextPostPath": "2003/07/11/ultra-liberal-feed-parser",
"nextPostTitle": "Ultra-liberal feed parser"
} ![](\"http://www.gravatar.com/avatar.php?gravatar_id=468c6b5110ed4bea9c148cf736744c43&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n Jim\r\n I believe the latest versions of winzip and pkzip actually have decent encryption (none of this trivially breakable stuff.\r\n\r\nHowever, I was under the impression GPG worked on Windows, and you can install cygwin to get all the other things like tar.\r\n \r\n \r\n \r\n ![](\"http://www.gravatar.com/avatar.php?gravatar_id=8bcf295bb2e23d6ee0d80710023f2b66&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n Anonymous\r\n Install cygwin on the XP box, then something like:\r\ntar cf - secretdir | gpg -c > destdir/mysecretfiles.tar.gpg\r\n \r\n \r\n \r\n ![](\"http://www.gravatar.com/avatar.php?gravatar_id=588bdfdda82be46c638d6956c55ebc38&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n Gnomon\r\n My Win32 survival kit necessarily contains UnxUtils (a large cross-section of the GNU fileutils ported to run natively, available at http://unxutils.sf.net) which means that you need only bother installing those tools which you'll actually use.\r\n\r\nIn this case, I'd recommend tar and bzip2, and perhaps GPG on top. Sure, it might not be as pretty as Winzip, but it's a heck of a lot more scriptable and there won't be any nasty encryption incompatabilities (see http://slashdot.org/articles/03/06/10/1542216.shtml?tid=185 for some background about this). Don't forget that the command-line FTP client in Win32 is scriptable and that there are dozens of Windows cron implementations.\r\n\r\n[trackback: http://24.102.209.201/weblogs/ben/internet/www/weblogs/posts/posted-elsewhere/DECAFBAD/WinXP-hidden-backup-solution.writeback]\r\n \r\n \r\n \r\n ![](\"http://www.gravatar.com/avatar.php?gravatar_id=cdb8499967998aa056bc65bf1b415608&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n Joshua Prismon\r\n (See URL for well formed post)\r\nFirst of all, for Linux backups, let me Highly recommend Amanda for Linux backups. Scaling other technologies gets too difficult, but we have a several hundred server network that Amanda tackles without problem. In addition, Amanda software is now being ported to win32, giving you a way to backup directly to tape (or rather to a Amanda spool). Here is One Such Effort. I had a project a few years back moving encrypted bank transactions across a network. Here is how I implemented it: Two folders are needed: \r\nThe Unencrypted directory that has all of the important stuff. (A) \r\nA encrypted directory that is automatically written to. (B) Only B should be shared, and no unencrypted data should be present. There is a slight mathematical risk to zipping and then encrypting files (since the header of the ZIP is a well known structure), but the purpose of this excercise is to keep noisy employees away, not be 100% mathematically secure. Your proccess then works as follows: \r\nHave your boss schedule a script that takes every file in A is zipped into a single file. (Winzip) \r\nHave your boss schedule a script that encrypts that zip. (Use GPGP, but remind your boss that he must keep the key secure. \r\nHave your boss schedule a script that moves only the encrypted file to B. \r\nUse NCFTP or SAMBA to automatically transfer B over to a backup server. (Use SMBFS or smclient if using samba). If he does not want to share it, transfer it using NCFTP. WinZip works great for automatically compressing things. I would not use it's encryption or password features tho. Use GPGP with a secure key to encrypt, or roll your own using .NET's awesome encryption support. (I recommend staying with GPGP).\r\n \r\n \r\n \r\n ![](\"http://www.gravatar.com/avatar.php?gravatar_id=07f3adb78c0d76857f03edadf68617fe&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n Robert Barksdale\r\n Curious? Since the files are top secret, why not put the onus on your boss? XP has a couple of built-in tools that will make the job easy.\r\nFirst, provided NTFS is utilized, he can take advantage of XP's Encrypting File System. This will allow him to encrypt and compress his \"important data\".\r\n\r\nNext, assuming he has access to a dir on the \"back-up\" server, he can use XP's Backup, to schedule the backup to run once, Weekly, monthly, at system start-up or logon.\r\n\r\nAgain, this puts him in-control of his own data, and should eliminate the need of \"touching\" his important files. Here are a couple of links for additional information.\r\n\r\nEncrypting Sile System\r\nXP's Backup\r\n\r\nI hope this helps.\r\n \r\n \r\n \r\n ![](\"http://www.gravatar.com/avatar.php?gravatar_id=c7a3895376100fa2a20cabab927bf35b&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n David\r\n He should just give you access to the contents: if you can't trust your sysadmin, you can't trust anyone. I'm sure you're mature enough not to open the files.\r\n\r\nJust because you can read something, doesn't mean you will.\r\n \r\n \r\n \r\n ![](\"http://www.gravatar.com/avatar.php?gravatar_id=b53911af792174097e8acb282414b44c&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n Bill Kearney\r\n If it's on an Active Directory enabled network the backup tools should be able to handle it. The idea being with use of domain certificates it's possible to backup that which you cannot normally access.\r\n\r\nThe other quick and dirty solution is to just zip the files into a password protected archive. Then put that archive on a folder that gets backed up. Winzip's command-line features should make this trivial. My XP box is out for service so I can't check and see what the XP native zip functions would offer here. But zipped archives are probably the simplest way to approach this. Write a vbscript, set it up as a scheduled job and let it do the grunt work of creating the archive, adding the files and then copying it to the approapriate backup location.\r\n \r\n \r\n \r\n ![](\"http://www.gravatar.com/avatar.php?gravatar_id=43f54f4e781c8374414d45c287474cac&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n mike\r\n I would highly sugest not relying on ntfs' encryption : http://www.annoyances.org/exec/forum/winxp/t1037834762 . Instead go grab a copy the cygwin project. A windows install simply isn't complete without it. tar & gpg, not to mention about 1001 needful utils, come stock dependin on which packages you install. Heck I'd prob have some sort of fit if it weren't for it's OpenSSH & SFree ports.\r\n \r\n \r\n \r\n
Archived Comments
\n\n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=468c6b5110ed4bea9c148cf736744c43&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nJim\n \n\n\n \n\n\nI believe the latest versions of winzip and pkzip actually have decent encryption (none of this trivially breakable stuff.\nHowever, I was under the impression GPG worked on Windows, and you can install cygwin to get all the other things like tar. \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=8bcf295bb2e23d6ee0d80710023f2b66&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nAnonymous\n \n\n\n \n\n\nInstall cygwin on the XP box, then something like:\ntar cf - secretdir | gpg -c > destdir/mysecretfiles.tar.gpg \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=588bdfdda82be46c638d6956c55ebc38&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nGnomon\n \n\n\n \n\n\nMy Win32 survival kit necessarily contains UnxUtils (a large cross-section of the GNU fileutils ported to run natively, available at http://unxutils.sf.net) which means that you need only bother installing those tools which you'll actually use.\nIn this case, I'd recommend tar and bzip2, and perhaps GPG on top. Sure, it might not be as pretty as Winzip, but it's a heck of a lot more scriptable and there won't be any nasty encryption incompatabilities (see http://slashdot.org/articles/03/06/10/1542216.shtml?tid=185 for some background about this). Don't forget that the command-line FTP client in Win32 is scriptable and that there are dozens of Windows cron implementations.\n[trackback: http://24.102.209.201/weblogs/ben/internet/www/weblogs/posts/posted-elsewhere/DECAFBAD/WinXP-hidden-backup-solution.writeback] \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=cdb8499967998aa056bc65bf1b415608&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nJoshua Prismon\n \n\n\n \n\n\n(See URL for well formed post)\nFirst of all, for Linux backups, let me Highly recommend Amanda for Linux backups. Scaling other technologies gets too difficult, but we have a several hundred server network that Amanda tackles without problem. In addition, Amanda software is now being ported to win32, giving you a way to backup directly to tape (or rather to a Amanda spool). Here is One Such Effort. I had a project a few years back moving encrypted bank transactions across a network. Here is how I implemented it: Two folders are needed: \nThe Unencrypted directory that has all of the important stuff. (A) \nA encrypted directory that is automatically written to. (B) Only B should be shared, and no unencrypted data should be present. There is a slight mathematical risk to zipping and then encrypting files (since the header of the ZIP is a well known structure), but the purpose of this excercise is to keep noisy employees away, not be 100% mathematically secure. Your proccess then works as follows: \nHave your boss schedule a script that takes every file in A is zipped into a single file. (Winzip) \nHave your boss schedule a script that encrypts that zip. (Use GPGP, but remind your boss that he must keep the key secure. \nHave your boss schedule a script that moves only the encrypted file to B. \nUse NCFTP or SAMBA to automatically transfer B over to a backup server. (Use SMBFS or smclient if using samba). If he does not want to share it, transfer it using NCFTP. WinZip works great for automatically compressing things. I would not use it's encryption or password features tho. Use GPGP with a secure key to encrypt, or roll your own using .NET's awesome encryption support. (I recommend staying with GPGP). \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=cdb8499967998aa056bc65bf1b415608&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nJoshua Prismon\n \n\n\n \n\n\n(See URL for well formed post)\nFirst of all, for Linux backups, let me Highly recommend Amanda for Linux backups. Scaling other technologies gets too difficult, but we have a several hundred server network that Amanda tackles without problem. In addition, Amanda software is now being ported to win32, giving you a way to backup directly to tape (or rather to a Amanda spool). Here is One Such Effort. I had a project a few years back moving encrypted bank transactions across a network. Here is how I implemented it: Two folders are needed: \nThe Unencrypted directory that has all of the important stuff. (A) \nA encrypted directory that is automatically written to. (B) Only B should be shared, and no unencrypted data should be present. There is a slight mathematical risk to zipping and then encrypting files (since the header of the ZIP is a well known structure), but the purpose of this excercise is to keep noisy employees away, not be 100% mathematically secure. Your proccess then works as follows: \nHave your boss schedule a script that takes every file in A is zipped into a single file. (Winzip) \nHave your boss schedule a script that encrypts that zip. (Use GPGP, but remind your boss that he must keep the key secure. \nHave your boss schedule a script that moves only the encrypted file to B. \nUse NCFTP or SAMBA to automatically transfer B over to a backup server. (Use SMBFS or smclient if using samba). If he does not want to share it, transfer it using NCFTP. WinZip works great for automatically compressing things. I would not use it's encryption or password features tho. Use GPGP with a secure key to encrypt, or roll your own using .NET's awesome encryption support. (I recommend staying with GPGP). \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=07f3adb78c0d76857f03edadf68617fe&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nRobert Barksdale\n \n\n\n \n\n\nCurious? Since the files are top secret, why not put the onus on your boss? XP has a couple of built-in tools that will make the job easy.\nFirst, provided NTFS is utilized, he can take advantage of XP's Encrypting File System. This will allow him to encrypt and compress his \"important data\".\nNext, assuming he has access to a dir on the \"back-up\" server, he can use XP's Backup, to schedule the backup to run once, Weekly, monthly, at system start-up or logon.\nAgain, this puts him in-control of his own data, and should eliminate the need of \"touching\" his important files. Here are a couple of links for additional information.\nEncrypting Sile System\nXP's Backup\nI hope this helps. \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=c7a3895376100fa2a20cabab927bf35b&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nDavid\n \n\n\n \n\n\nHe should just give you access to the contents: if you can't trust your sysadmin, you can't trust anyone. I'm sure you're mature enough not to open the files.\nJust because you can read something, doesn't mean you will. \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=b53911af792174097e8acb282414b44c&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nBill Kearney\n \n\n\n \n\n\nIf it's on an Active Directory enabled network the backup tools should be able to handle it. The idea being with use of domain certificates it's possible to backup that which you cannot normally access.\nThe other quick and dirty solution is to just zip the files into a password protected archive. Then put that archive on a folder that gets backed up. Winzip's command-line features should make this trivial. My XP box is out for service so I can't check and see what the XP native zip functions would offer here. But zipped archives are probably the simplest way to approach this. Write a vbscript, set it up as a scheduled job and let it do the grunt work of creating the archive, adding the files and then copying it to the approapriate backup location. \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=43f54f4e781c8374414d45c287474cac&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nmike\n \n\n\n \n\n\nI would highly sugest not relying on ntfs' encryption : http://www.annoyances.org/exec/forum/winxp/t1037834762 . Instead go grab a copy the cygwin project. A windows install simply isn't complete without it. tar & gpg, not to mention about 1001 needful utils, come stock dependin on which packages you install. Heck I'd prob have some sort of fit if it weren't for it's OpenSSH & SFree ports. \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=43f54f4e781c8374414d45c287474cac&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nmike\n \n\n\n \n\n\nI would highly sugest not relying on ntfs' encryption : http://www.annoyances.org/exec/forum/winxp/t1037834762 . Instead go grab a copy the cygwin project. A windows install simply isn't complete without it. tar & gpg, not to mention about 1001 needful utils, come stock dependin on which packages you install. Heck I'd prob have some sort of fit if it weren't for it's OpenSSH & XFree ports. \n\n\n \n
\n\n\n\n
\n
\n
\n
\n
\n
\n
\n
\n
\n