My Desktop Storage Needs Help

Back when I purchased this 2013 Mac Pro, I wrote about needing a desktop RAID system for storage. The Mac Pro at the time came with a base configuration of a 256GB SSD, and peaked with a 1TB model that was about $1000 more. I decided to stick with the base storage configuration and then I purchased a Pegasus2 desktop RAID box and put in four 3TB drives in a RAID 5 setup.

As time went on, the drives started to die, and I replaced them with 4TB models, so I currently have about 12TB of usable storage on my desktop, with about 2/3rds of it occupied. With respect to capacity, it’s doing just fine, though it’s starting to fill up quicker since I switched to recording video in 4K with my iPhone and 5D Mark IV.

While capacity has been okay, recently I’m running into a limitation of this setup with respect to speed. I’m blaming SSDs. SSDs are very common now in desktops and ubiquitous in laptops because of their performance benefits. The problem is apps are starting to be coded with the assumption they are being run on flash storage. This is especially problematic with syncing apps like Dropbox.

Let me explain. I currently have a two-Mac working environment. I use my desktop most of the time, but I also have a 13″ MacBook Pro that I’ll work on while on the road. I use syncing tools like Dropbox and Resilio to keep files up-to-date between the two devices.

One app I use frequently is Lightroom. Its important to me that I have my photos available on both Macs, but a Lightroom library has to be stored locally. So I put my Lightroom library on Dropbox, and the RAW photo masters are in a separate directory on my desktop Mac. If I’m going to be editing photos on my laptop, I’ll generate Smart Previews for those albums on my desktop and be able to develop them on the MacBook Pro. This works really well.

The problem I’m running into recently is during Lightroom imports. Whenever I import photos into Lightroom, Dropbox detects the changes in the library and kicks off indexing/syncing. This puts more I/O load on the disk that is already busy from the import, and things grind to a halt. The sync/import activity even keeps me from playing a single 1080p HD movie from the library…on a current Mac Pro…with RAID storage. The disk just can’t keep up. It’s even worse if something like a Time Machine backup happens to be running, or if Backblaze is trying to push files offsite.

Clearly something needs to change. The problem is that I would like something faster but still with enough capacity to grow. I like the Pegasus box a lot, so it’s tempting to just buy 4 SSDs to use in that. However, SSDs are still pretty cost prohibitive in larger capacities. I certainly couldn’t afford 4TB models to retain my current storage capacity. Even four 2TB SSDs would be a good chunk of change. I could buy some 1TB models and have 3TB of usable space in a RAID 5, but that would require a complete restructuring of where I store data. I would have to keep only my recent data on the desktop, and archive bigger data like photos and videos to a NAS with more storage. Then backups become an issue, etc.

With the new Mac Pro just around the corner (rumored for 2019 release), I’ll be upgrading to a new Mac somewhat soon, but not immediately. So I haven’t decided if I should do something about storage now or wait. But every couple of days I have plenty of extra time to plan my next steps while waiting for my disks to stop thrashing.