the itjerk

my adventures with technology

Tag Archives: data

my data and the cloud

Working from home gives me a lot of time in front of my computer. Being who I am – an itjerk – I’ve decided to streamline all my cloud services, and clean up my data wherever and everywhere it may be.

First, let’s talk a little about data. Data is ubiquitous. Folders, files, drives, cloud, it just piles up. Keeping it organized though is the key to keeping it safe. Specific categories of data require specific solutions. Here’s mine:

  1. Documents. These are almost exclusively on my PC’s home folder, or in Google Drive. I’ll use OneDrive to keep them backed up, until they are archived (see below).
  2. Photos. Camera photos are in Google Photos (Android Phone user) and then eventually backed up to my RAID1 and external drives, while other photos are in Pictures folders, or shared with Cloud services.
  3. Music. This resides on a RAID1 on my Linux computer, and is backed up to an external drive.
  4. Archived Data. IMPORTANT! Every year I create a folder on my Desktop for all the digital ephemera I collect. It gets backed up to my RAID1 and external drives. I also clean/clear out my home folders, or at least I endeavor to.
  5. Backups. Yep, that’s a class of data. I have a backup of my home folders on my RAID1 and external drives.
  6. The zillion other random photos and files. They are everywhere! In the cloud, in my Downloads folder, on flash drives and backup disks; this is the thing to organize and clean up!

Cloud services are great, but it’s pretty easy for things to get out of hand if you have too many. Here’s a quick overview of my Cloud Services:

Google Backup & Sync. Google let me down when they removed Google Photos from their sync services. I had my photos syncing down to a D: drive on my PC, which was easy enough to backup to external drives. Maybe too easy? No more – it’s a manual download process. Fuckers. Anyway it can do exactly what it says – backup your computer to the cloud, and the cloud down to your computer – but I’m moving on. Google knows enough about me.

Google Drive. This does have exceptional value to me, especially when I was writing my book, as a “work-in-progress” repository for documents. However, like all cloud services, it’s also a wasteland for random bits of data – photos, saved files, wip documents that never finish, transfers – all of which needs cleaning up much more so than backing up. I’ll use it for working in the cloud, but not as a repository for data.

Mega. Anyone remember Kim Dotcom? Whatever happened to him? Anyway, I use Mega to backup my book files. It’s a task-specific solution that serves my need, and is free.

OneDrive. I’m a newly converted fan. Why? Well, I use it extensively at work, but also because of my Surface Go: I used my Microsoft account (Live? 365?) to initially create my user account, and since the Go has only a 64GB hard drive, OneDrive’s is a must (but to be honest, I don’t store any data on the Go). But here’s another essential feature of OneDrive – it can automatically backup your Desktop, Documents and/or Pictures folder. The free plan comes with 5GB of storage, which is good for two of those folders, and 100GB plan only costs $2/month. It does have a Personal Vault (password protected folder), but it only holds three (3) files! LOL! Well integrated into File Explorer, and works with a Mac.

Dropbox. The original cloud, I think, it just ended up being a whole lot of meaningless files for me. I’ll keep the account but only for sharing with others.

Adobe Creative Cloud. I used CC apps for my book, but as this is provided by my work, it’s not a personal solution for me.

iCloud. I’m not an Apple user, but if I were, I would probably use this instead of OneDrive.

Box. Another service I use at work, but redundant with OneDrive.

Amazon Drive. If you have a Prime account, it offers unlimited photo storage. That’s tempting, but Amazon knows enough about me already. Plus, I want my precious digital photos in my physical possession.

So what’s the plan? Use Google Drive as a work space, and OneDrive as my primary cloud. As for the rest of my data, I’ve got a spare D: drive on my PC that I’m using as a staging point as I clean out my various folders and drives. Time I have, OCD I’m great at, so let’s execute the plan!

raid, again, and backup

I bought two 3TB drives to replace my 1TB RAID. Easy enough, because with hardware RAID 1 the disks are identical: you can pull one drive out, plug it into a USB enclosure, and provided your computer is compatible with the drive’s format — ext4 in my case — you will have two functioning backup drives to copy over to the new RAID (and eventually erase/shred).
sudo shred -v -nX -z /dev/sdX

A thing about buying a hard drive. I notice that the marketing has now shifted to “intended use” of the drive – Desktop, NAS, Surveillance, etc. Guess what, I call b.s. — they’re all the same drives, probably just the more expensive ones were tested more (to justify price/warranty). Please, let me know otherwise if you think I am wrong.

Maybe I should have got 4TB discs? I don’t know. Going from 70% used to 70% available is a jump. I do want to rip more of my CD library to my computer, because digital music is here to stay (and when I say “more” of my library, I mean a “curated” more of my library). I rip to FLAC, which is all the quality I require (considering the source is 16bit/44khz), so how much will I need? Figuring FLAC at level 5 is about 300-350 MB per CD, 2TB will hold a LOT of CDs.

2000000 MB / 330 MB = 6060 CDs

I should probably also mention that RAID 1 is not a backup. It’s a safety copy in case one drive fails. You want backup? You gotta have two of everything. One here, and one over there. More later.

The bigger question then is what do we do with all the data we have. If I really think about it, I need to back up my photos, SOME of my music, my data (book, website, etc), and what else? Do I really need all the crap on my computer? All the files on those backup drives and old hard disks? Probably not, because I don’t even know what I have most of the time and … p0rn should never be downloaded! 😉

My next project will be to organize all my “digitalia,” and what a project that will be!

nas-ty of backups

My Seagate NAS 110 took a dive this past week. After I couldn’t print, I then noticed the NAS kept on rebooting itself. Fortunately the disk was fine and I had all my data, the problem laying somewhere in the hardware of the device. Trucked down to Microcenter where I bought a new one, this time a Buffalo product.

All of this brings up backups in general, and the power of two. When you think of all the things you need to backup, there really isn’t that many. Photos and Videos are the most obvious, because the digital era allows us to take a gazillion of them, and who prints them out anymore? What else needs to be backed up? I wrote a book, so there’s a far amount of digital goodness with that. I’m sure there’s other things that I want backed up, in fact, I usually just put them on a backup drive and forget what I even have there!

Here’s where the power of two comes in: At a minimum, you need two of everything. Your backup will fail, just as my NAS did. Your hard drive may fail, as I’ve had happen before. Those flash drives? Fail. Hell, I even accidentally deleted some folders once. Recently. Ouch.

I like a NAS drive because all my computers can connect to it, so it centralizes the location of my data. That’s good and bad, good when it works, bad when it fails. I also use Dropbox for some things (current writings), DVDs for completed projects, and a portable HD for things I don’t want others to see.

Just remember. Have at a minimum of two copies of everything. One will vanish, when you least expect it.