Failure-Tolerant HDDs

Summary

This page describes failure-tolerant disk storage products called parallel-transfer Hard Disk Drives (pHDDs).  pHDDs have much higher transfer rates than conventional HDDs and a variable level of failure-tolerance.  pHDDs contain a number of Head-Disk Assemblies (HDAs) and a custom controller chip as components.

All of the HDAs in a pHDD are written and read simultaneously.  The disks within all of the HDAs are synchronized so that a pHDD appears to be a conventional HDD but with much higher performance and failure-tolerance which is impossible in a conventional HDD.

To summarize, 

• pHDDs could significantly increase revenues and profits at disk drive companies because fewer types of components need to be manufactured, fewer employees are needed, and a higher percentage of manufactured media can be used in pHDD products than in conventional HDDs because of the increased power of the ECC system used in pHDDs.

• pHDDs have much higher transfer rates and a much higher overall performance than conventional HDDs.

• pHDDs have automatic, built-in backup eliminating the expense, nuisance and need for RAID controller boards, extra drives and backup software.

• pHDDs will have higher performance, consume less space and less power than equivalent capacity conventional HDDs.

• pHDDs will enable disk drive companies to make many unique types of pHDD models while only manufacturing a few types of HDAs increasing economies of scale and driving down the cost of HDAs.

• pHDDs allow the use of media that is unusable in conventional HDDs because the pHDD ECC system operates on a two dimensional array of data instead of a one dimensional array as is done in conventional HDDs.

• pHDDs have features that are highly desirable and commercially valuable but impossible with conventional HDDs.

• pHDDs have more commercial value than conventional HDDs and can be priced higher and offer more profit potential than conventional HDDs.

Reasons for Developing pHDD Products that use Tiny HDAs

There are a number of reasons for developing pHDD products that use tiny HDAs as components even though, initially, they may be higher priced than conventional HDDs of the same capacity.

In 1987, a study was conducted by Control Data Corporation on HDD trends.  At that time, 3.5 inch HDDs were just beginning to appear so the study was based on HDDs with diameters of 14”, 8”, 5.25” and 3.5”.  Even though the study is old, the findings are still valid.  The study found that as HDD physical sizes decreased, the amount of data that could be stored in a cubic volume (GigaBytes per Cubic Inch) was increasing exponentially as illustrated below.

Another finding of the study was that, as the HDDs were made smaller, the number of watts per unit of data (Watts per GigaByte) consumed by the drives was decreasing exponentially as shown.

The reason the power consumed by an HDD varies so dramatically with the diameter of the disks is that the majority of the power consumed by an HDD is consumed by the motor in spinning the disks.  Rotating disks are like a fan, there is a lot of aerodynamic drag, and the inertia of a disk varies as the fourth power of radius.  In order to maintain the same degree of controllability, the power consumed by the motor also increases as the fourth power of radius!

The disk drive industry does not immediately rush to develop smaller and smaller drives for a couple of reasons.  First, when newer and smaller diameter drives are introduced into the market, they are always priced substantially higher in terms of Dollars per GigaByte than older, larger diameter drives because the Dollars per GigaByte prices decrease only with manufacturing maturity and manufacturing volume which takes time as illustrated below.

The curves shown above are generally true and accurate, but are not meant to be exact or precise.  The curves illustrate the fact that the introductory Dollars per GigaByte prices of all smaller diameter drives are always much higher than the existing Dollars per GigaByte prices of larger diameter drives, that prices will fall with time as a result of manufacturing maturity and large manufacturing volumes and that there are price crossover points.  Whenever there is a price crossover point, the larger diameter drives will most likely become extinct – 14 inch drives are an example.

Initially, every new, smaller drive product cannot compete on a Dollar per GigaByte basis with older, mature products that are being manufactured in high volumes.  Therefore, there is naturally a strong reluctance in any existing disk drive company to start to develop smaller drives because they may not be attractive to customers initially, and they will not initially generate the profit that older drives do.

When going to smaller drives, the capacity per drive decreases substantially and, to implement large capacity disk databases, more smaller drives (or components) are needed than if larger capacity drives were used.  This increase in the number of components in disk databases decreases the mean time between failures (MTBF) of the database systems because the MTBF of a system is inversely proportional to the number of components in the system.

The answer to the MTBF problem is to implement an error-correction system that can automatically correct for component failures, and that is precisely what a parallelized and pipelined RS error correction system does.

The following paragaph is based on 1987 numbers.

It is sometimes difficult to overcome the initial unattractiveness of smaller drives due to their higher Dollar per GigaByte cost and price.  However, in the case of HDAs to be used as components in pHDD products, it is important to anticipate manufacturing volumes and to anticipate when there may be another price crossover point.  Assume that, on the average, each pHDD contains 16 HDA components.  According to a past CNET news article, 254 million conventional HDDs were sold in 2003, 278 million were forecast to be sold in 2004 and 358 million in 2007.  Since a pHDD is roughly equivalent to an equal capacity conventional HDD, manufacturing volumes of HDAs would be 16x the volumes of conventional HDDs so that, if 10% of the conventional HDDs had been replaced with pHDDs, that would have resulted in approximately 30 million pHDDs being sold in 2005 and 36 million pHDDs sold in 2007.  Those numbers translate into 480 million HDAs being sold in 2005 and 576 million HDAs being sold in 2007.  It’s easy to see that HDA manufacturing volumes could quickly exceed more than 1 billion units per year which would put the HDA component into the same category as a DRAM memory chip as far as manufacturing volumes are concerned since billions of DRAM chips are sold each year.  Without a doubt, manufacturing more than a billion HDAs per year would cause an unprecedented drop in HDA costs and prices.  Never before in the history of the disk drive industry has the manufacturing volume of a single HDA even come close to approaching the billion units per year level. 

It is expected that the trends identified in the 1987 Control Data HDD Study have continued and that pHDDs developed today using tiny disk HDAs as components will, most likely, be physically smaller, consume less power and have higher performance than conventional HDDs of the same capacity.

Current tiny drives are being threatened by Flash memory.  If Flash is successful, there may be an excess number of tiny drives and the market for tiny HDDs may dry up.  Excess tiny drive HDAs could be used in pHDD products.

Performance Reasons for Developing pHDD Products

Today's hard disk drives are the only components in a modern computer that write and read data one bit at a time!  Processors in modern PCs can operate on 32‑bit or 64‑bit chunks of data at a rate of up to 3 GigaHertz, yet conventional HDDs read and write data one bit at a time!

In order for the data transfer rate of an HDD to match the data transfer rates of other parallel-transfer components such as microprocessors or main memories, a parallel-transfer HDD or a pHDD must be developed.

Any conventional HDD of capacity C can be divided into K, HDA components of capacity C/K which will result in the creation of a pHDD with the same capacity but with K times the transfer rate.

The RPM of the component HDAs can be the same or higher than the RPM of a conventional HDD and the HDA disks can be synchronized to create pHDDs that will always outperform conventional HDDs of the same capacity.  The pHDD will be able to read and write data K times faster than an equivalent capacity, conventional HDD and the seek times and rotational latencies should be equivalent or better than those of a conventional HDD.

pHDDs can be made failure-tolerant with an arbitrary level of built-in failure-tolerance by using a parallelized and pipelined RS ECC system as shown here.

The performance and level of failure tolerance in all RAID storage systems could be dramatically improved by unplugging the existing, conventional HDDs and plugging in pHDDs.

Even if pHDDs cost more than conventional HDDs of the same capacity, they will, for most applications, still have a much higher performance/price ratio than conventional HDDs.  That is, they will be able to serve more information in a unit of time than a standard HDD with the same capacity so they are more valuable than conventional drives in the same way that a waiter in a restaurant that can serve 100 customers in an hour is more valuable than a waiter that can only serve 10 customers in an hour.

In the past, a common misconception existed in the disk drive industry that a pHDD must cost less than a conventional HDD of the same capacity in order for it to be an economically viable product, but that belief is not true.  Even if pHDDs cost more than conventional HDDs of the same capacity, they still can be economically viable products as long as the extra performance and failure-tolerance benefits gained by using a pHDD are perceived by potential buyers as being worth at least as much as the price difference.

Many buyers of PC systems, for example, do not hesitate to pay several hundred dollars more for a PC with the fastest possible processor.  Intel charges less than $100 for their slowest processors and over $600 for their fastest processors.  Buyers have proven that they believe the extra performance provided by Intel’s fastest processors is worth a “premium” of several hundred dollars.  It is reasonable to believe that buyers of pHDD products will also believe the extra performance and failure tolerance provided by a pHDD is worth a few hundred dollars premium.

Performance Reasons to Develop pHDD Arrays

The performance of an HDD array compared to the performance of a pHDD array is shown below. 

Given any HDD array with N HDDs, the performance of the HDD array is illustrated in the upper part of Figure 6.  The lines labeled with “PH” represent the average time it takes to position the heads and the lines labeled with “IO” represent the average time it takes to do an input or output data transfer to one of the drives.

Given any HDD of capacity C, a pHDD of the same capacity that uses K HDAs can always be developed, and the transfer rate of the equivalent capacity pHDD will be K times the transfer rate of the HDD while the average time to position the heads will remain the same.  Given the performance of any HDD array with N HDDs as shown in the upper part of the drawing below, a pHDD array can be developed with N equal capacity pHDDs that will clearly outperform the HDD array by a wide margin as illustrated in the lower part of the drawing.

FailureTolerance Reasons for Developing pHDD Products

With the use of a parallel Reed-Solomon encoder and decoder within each pHDD and with R redundant drives in each pHDD, up to R drives can fail with no loss of data or performance.  In addition, the raw error rates of each component HDA can be much higher than conventional HDD raw error rates because the PRS system can correct any random errors that the conventional ECC does not correct.

In most cases, a pHDD with one PRS encoder and decoder will be used with N conventional RS encoders and decoders.  The conventional ECC will correct most of the errors due to defects and random errors in each HDA.  The PRS decoder will correct errors that the conventional RS decoders cannot possibly correct – like errors caused by HDA failures, head crashes, etc.

Reasons for Developing pHDD Products Related to Manufacturing

Manufacturing pHDDs instead of HDDs could dramatically reduce the number of types of drives that a disk drive manufacturer would have to manufacture.  Today, the major disk drive vendors manufacture 40-60 different HDD models.

There are some common components in current manufacturers’ HDDs, such as common heads and common disks, but it is important to understand that each HDA has its own very precise mechanical characteristics and that those characteristics, such as the resonance frequencies, can change with only slight differences in structure.  Because of that, two different HDAs – even though they use the same heads and same disks – may have completely different vibration characteristics and may require two different servo systems.  With a pHDD, all of the HDAs would be identical and all of the servo systems would be identical.

If the major disk drive vendors only manufactured pHDDs, they would only have to manufacture a few types of HDAs (possibly only one type) yet they could still create literally hundreds of different pHDD models by using a different number of data and/or redundant drives in each model.  Each pHDD model would then have its own level of performance and failure tolerance.

The manufacturing volume of HDAs would be dramatically increased from millions per year to hundreds of millions per year.  The HDA components would be assembled into pHDDs in the same way as any other component would be.

A large disk drive company could potentially save millions of dollars quarterly if it gradually switched from designing and manufacturing HDDs to designing and manufacturing pHDDs.

Reasons for Developing pHDD Products Based on More Effective Use of Media and More Profit Potential

ECC Tek has proposed that pHDD products be developed that could use media that has been rejected as “unusable” in conventional HDDs because of too many errors and/or defects.  Selling pHDDs which use highly defective media at normal HDD price levels instead of deeply discounted prices could enable all disk drive companies to make more effective use of manufactured media and to realize millions of dollars in revenue that otherwise would not exist.

pHDD products which use media that is unusable in conventional HDDs will, most likely, be much more reliable than standard HDDs because a “lateral”, “horizontal”   or “parallel” ECC coding scheme will be used in combination with the conventional, “longitudinal”, “vertical” or “serial” ECC scheme that is used in conventional HDDs.  The horizontal ECC literally opens up an entirely “new dimension” to the problem of correcting errors because now ECC can be applied to a two‑dimensional array of data items instead of just a one‑dimensional array of data items as is currently being done in conventional HDDs.  No one will be able to legitimately claim that pHDD products are in any way inferior to standard HDDs.  In fact, pHDD products will, most likely, be highly superior in performance and reliability to conventional HDDs.

With a number of HDAs ganged together, the new parallel RS decoder and the improved serial RS decoders will have information from multiple, statistically independent sources and the resulting random distribution of errors will be ideally suited for correction using RS codes.

The law of large numbers (from statistics) applies to pHDDs, and the result is that the statistical or probabilistic behavior of the system will closely follow probabilistic models so the behavior of pHDD systems will be very predictable.

Reasons for Developing pHDD Products Related to Users’ Experience

Users of desktop and laptop computer systems will be willing to pay a premium for a PC or Mac if it has an ultra-fast, failure-tolerant hard drive in it.  If a system already costs over a thousand dollars, many users will be willing to pay an additional premium to get a super-duper, ultra-high-performance, failure tolerant hard drive in the same way that users are willing to pay a premium to get Intel's fastest and most powerful processors.

Once users use a Pentium PC operating at 3 GHz, they will never be willing to go back to a Pentium PC operating at 250 MHz!  There is a market for faster processors because users like them, get used to using them and don't want anything slower once they use them.

The same thing will be true for systems which contain pHDDs in place of conventional HDDs.  Users will buy them because they will only cost a little more than systems that contain conventional HDDs, but they will be a lot faster and will be failure-tolerant.  In a PC, pHDDs will accelerate boot up, accelerate all OS disk operations and accelerate application programs.  Once users get used to using systems which contain pHDDs, they will never go back to systems that use conventional HDDs.

One major impact to users of PCs which contain a pHDD instead of a conventional HDD would be that, on power up, the operating system would load into memory almost instantaneously – like an "instant on" computer.  Most PC users would love that.

Another impact on computer users would be the increased level of comfort and security a user would feel knowing that, if one or more HDA components fail, no data will be lost.  All the user has to do is remove the failing drive component and replace it with a new one – just like replacing a light bulb.

Any application programs that do a lot of disk I/O would also see a significant performance boost – such as programs that download music, movies or pictures.

pHDD Pricing Strategy

The price of a pHDD product should be based on its features and those features should be priced on a feature by feature basis which justifies the fact that a pHDD will probably initially cost more than a conventional HDD.

For example, suppose you went to buy a new PC and were told about two PCs that had exactly the same performance but that one of the PCs took 1.5 minutes to boot-up and the other one took only 10 seconds to boot-up.  What would the "fast boot-up feature" be worth?  Most people would gladly pay $100 to $150 more to get the fast boot-up feature.

The fast boot-up PC should also be noticeably faster at running application programs – especially ones that require large file transfers like transferring pictures or video.  A certain amount of money could be added to a price of a PC if it has a "fast file transfer feature".

Then there is the "automatic, built-in backup feature".  That feature should be priced based upon the amount of backup that is built-in.  For example, a pHDD that allows one HDA to fail may add an additional $200 to the price, one that allows two HDAs to fail might be priced at $300 more and one that tolerates three HDA failures might be priced at $400 more.

Each feature should be priced individually.  Pricing each feature individually makes good sense and is reasonable.  That's what is done with automobiles.  When you buy a car, you pay for each accessory or feature individually.  When you buy a PC with a conventional HDD, you do not have the option of adding those features, but when you buy a PC with a pHDD you do.

It would be easy to justify a pHDD price that is substantially higher than an equivalent capacity HDD price.  The extra price is due to the added features.

This is a perfectly reasonable, rational, logical and fair pricing strategy.

Most people are alike, and they would gladly pay a significant premium to get the added feature.