Homemade Hybrid Disk Drives Part-1
It's sometimes better to bake it yourself. This is the introduction to the problem and the beginning of the solution.
We would like our drive space to be fast, inexpensive, and good (plentiful/reliable). It’s another version of the fast/cheap/good engineering trade-offs that haunt all of our decisions about technology. I won’t waste ink discussing whether any storage function can be considered ‘good’ if it’s not reliable, or there isn’t enough of it. To qualify as storage, when we get back to it, it has to all be there just the way we left it.
What we all love about SSD’s is that they’re fast, with low access times and fast transfer rates. They are expensive compared to HDD’s on a capacity-per-dollar scale, and they’re good in their reliability … about the same as HDD’s although with different failure modes … but not so good in providing plentiful storage.
A Few Reference SSD Products
Remember the early 32GB SSD’s? To explain ‘plentiful’, there are many choices of 8TB HDD’s available today for the enterprise, project studio, and home market. There are just a few 8TB SSD’s in the world, and they’re clearly targeted at the enterprise market. For example, 7.68TB Hitachi Ultrastar SS200 connects to a 12Gbits/sec HBA, and costs a cool $7000 because of its massive 1 drive write per day (DWPD) endurance, 5 year warranty and 10^17 Uncorrectable Error Rate; about 100 times better than conventional drives. The endurance calculates to over 13600 TBW (TeraBytes Written). That's five times more endurance than a Western Digital Gold Enterprise spinning HDD.
For availability in 2018, Samsung has announced the PM16434, a 30TB SSD, and Numbus Data has announced a 100TB SSD in its ExaDrive DC Series, but both of these products are in sample quantities to special testing enterprises right now. No pricing information has been released.
Rating SSD's
With dance contest scoring, the three judges would give today’s SSD’s a {7.6 ~ 6.7 ~ 3.3} score. A ‘7.6’ for speed, a ‘6.7’ for cost and a ‘3.3’ for capacity/reliability.
Here’s a quick explanation of my scoring:
Speed:
-
L2 CPU cache is a 10
-
RAM is a 9 1/10th RAM speed is an 8 1/100th RAM speed is a 7
Cost:
-
RAM cost per GB is a 5
-
1/10th RAM cost/GB is a 6 1/100th RAM cost/GB is a 7
-
1/1000th RAM cost/GB is an 8 1/10000th cost/GB is a 9
Capacity:
-
1GB is a 3 10GB is a 4 100GB is a 5
-
1TB is a 6 10TB is a 7 100TB is an 8 1PB is a 9
A Little SSD History
I did everything in orders of magnitude because that’s the range of things we have to deal with in the computing world. In June of 1980 the first announcement of an available 1GB hard drive appeared as the IBM 3380. It cost about $40,000. Let that sink in for a moment. Yeah, it took another 12 years, until 1992, to see the familiar drives we know to today in the 1GB capacity. I remember where I was standing in a parking lot in New Jersey when I first heard of the 1GB HDD for personal computers. I was stunned and mystified by the possibilities. The $800+ price stunned me too. Check out this cool chart denoting the progress of HDD cost per gigabyte over the decades.
[ http://www.mkomo.com/cost-per-gigabyte ]
Jumping back into SSD's, in 2009, the 32GB Intel X25-E cost $12.88 per GB, and a Samsung 32GB SLC SSD was $8.29 per GB, while the 80GB Intel X25-M did a lot better at $4.29 per GB.
This 2009 AnandTech article is worth reading for many reasons concerning cost and SSD performance issues.
[ https://www.anandtech.com/show/2738/2 ]
After a few years of progress to 2012, the status of off-the-shelf SSD tech at that time would have pushed the judges to offer a {7.2 ~ 6.0 ~ 2.8} score. The middle value, the cost per gigabyte for SSD storage has moved dramatically. As an example, in February 2012, Cnet provided an article about comparative SSD performance.
[ https://www.cnet.com/products/intel-ssd-520-series/review/2/ ]
In that article, they noted the ‘street’ pricing of popular SSD’s at that time. The price per GByte ranged from $2.21 to a low of $1.45. Given that a typical Crucial 500GB SSD sells today for about $0.23 per GB, the middle score of the triad has progressed to be nearly two orders of magnitude better than those early days of PATA SSD's.
HDD Progress
My first SSD back in 2011 was a 120GB Intel Series 320 SSD. It was announced with a 36 TBW figure. It could read at 270 MBytes/sec and write at 130 MBytes/sec. Now, my slowest HDD (the Western Digital Black 2.5-inch 1TB) can write just as fast, and at 235 MBytes/sec for read/write, the WD Gold Enterprise Class drives do just nearly everything (but not 4k small read/write) as well as the vintage-2011 SSD while offering about 50x the data capacity. That’s progress.
The Need for Terabytes
Today, I’m usually thinking about my drive space in terabyte increments because of my video editing ambitions. While each of my operating systems (macOS Sierra, Win-10 Pro, Win-7 Pro) has their own ~500GB SSD, the nature of video editing is that the media pool starts out with nearly 150GB of video from a typical concert, and grows from there as editing progresses.
I don't keep any media on the boot drives. A 1-terabyte media drive needs constant attention to keep it from overflowing with concurrent projects, and a simple save-as-duplicate-project mistake in Final Cut Pro X can cause a flash flood of gigabytes that can quickly overwhelm a 1TB SSD. Don’t ask me how I know this.
The Need for Bandwidth
Fortunately, from what I can tell, there isn’t any compelling reason to chase the gold-medal rewards of NMVe SSD drive with their 2 GBytes/sec and 3 GBytes/sec transfer rates. When editing video, as long as your multi-cam views can smoothly display without dropped frames, timeline scrubbing can proceed without dropped frames, and thumbnails pop up quickly enough, then it’s a good day with the video NLE. Import and copy speeds just have to be fast enough for the impatient user, and there’s little to be gained on rendering and export that can be helped with massively faster disk I/O.
There is plenty of disk bandwidth that can be consumed by 4K camera output, but if it’s not RAW footage, then an SSD RAID-0 pair will take you quite a long way, even with a 6-camera multi-cam. The Blackmagic disk testing utility in macOS does a nice job of showing the drive capability. I’ve noted the test results in the top chart that shows all write XXXw and read XXXr values for every SSD and HDD device in the Mac Pro.
RAM Bandwidth Okay with Low Scores
In addition, although I haven’t made a RAMdisk for direct testing, the Mac Performance Guide says my RAM can do about 15 GBytes/sec. The ATTO Benchmark gave figures higher than 11 GBytes/sec with the L1 cache engaged, and its other measurements of 'real' disks correspond nicely with the Blackmagic utility.
[ https://macperformanceguide.com/Reviews-MacProWestmere-MemoryBandwidth.html ]
I suspect that two things are at play here. Firstly, the RAM in the Mac Pro is displayed as under-clocked to half-speed by default when running Windows. I don't know if this is just a 'cosmetic' bug or not. It seems to be verified by the slightly lower Geekbench 4 scores; about 24K in macOS and about 21.5K in Windows. Secondly, I think that there’s no way that a disk I/O benchmark could achieve the same throughput as a RAM benchmark.
The Fast Enough Doctrine and HHDD's
No matter what, the ‘Fast-Enough’ doctrine applies here too. In computing, if any element is fast enough to prevent queuing or starvation from occurring for its bandwidth demands, then it is ‘fast enough’. I have seen all 24 logical cores of my 3.33Ghz Mac Pro pinned at 97+ percent usage when doing some stuff in my video editors. That’s part of why I’m confident that the RAM is fast enough.
How close can I come to the ideal of a {9.0 ~ 9.0 ~ 9.0} balanced solution for my multi-terabyte ambitions? There has to be a cool strategy in play. I have one. It revolves around Homemade Hybrid Disk Drives. HHDD. The implementation begins in Part-2.
- Ted Gary of TedLand
May 17, 2018