Russian Home-Built 90 TB File Server

[Omega]

The following question was recently posed on the TWL staff email:

http://gizmodo.com/5669358/homemade-...lion-songssooo
who could listen to 24 million songs ever!

with one other respondent quipping:

A DJ DJ-ing for a few million parties a once?

90 TB is only ~2.8 million songs in flac[1], which is only ~228,000 hours of music[1]. At eight hours per day of listening, that'd take about 28,500 days, or 78 years. While the 2007 US life expectancy at birth was 77.9 years overall[2], if you're a white male in the US that makes it to 65 years old, you were expected to live another 17.1 years (up to 82) as of 2006 [3]. So, while it's within the realm of possibility to listen to 89.28 TiB of music, the realistic life expectancy of the components in that system is considerably lower[4] .

Humor aside, this got me to thinking about the technicalities of the situation, such as drive redundancy, and the hardware he used to connect it all together.

I'm a bit disappointed that gizmodo didn't do more legwork to actually provide some useful information about how it was done, but thankfully it was possible to trace back a few layers to the source[5], which explains a bit more of the configuration. While maybe none of you were curious about it, he's using a SuperMicro X7DWN+ motherboard, a 16-port HighPoint RocketRAID 3540, two 8-port RocketRaid 3520s, an 8-port RocketRAID 2320, two RocketRAID 2220s, and the on-board 6-port Intel ESB2 (6321ESB, similar to ICH9) to present nine logical volumes to the OS, "mostly raid5." If he built all of the arrays with 2TB drives, he'd only be using 56 drives (plus one or two for the ~150 GB system drive/array, but still at least six drives short of the 64 he said were used), so it's not immediately apparent how some of the smaller arrays are comprised.

All 89.28 TiB appear to be physically present on this one system, so I'm also not entirely sure what the second desktop-class motherboard with a beefy-looking two-port NIC also pictured[6] is for.

My technological curiosity aside, I'm definitely a bit jealous of such capacity (though, not of that exact ghetto setup, nor his choice of configuration). That said, the minimum of $5000 just for the drives, $450 for the motherboard, $650 for a pair of Xeon E5420s, $150 for the RAM, $600 for the 16-port controller, $800 for the pair of RocketRAID 3520s, $250 for the 2320, and $500 for the pair of 2220s comes to a starting point of about $8500 before talking about power supplies, cabling, the second mystery system, or shipping, which I'm content with *not* spending on this rig. Plus, if you're going to do it, you should do it right: http://blog.backblaze.com/2009/09/01...cloud-storage/ .


[1] These numbers calculated from average file size--32.8 MiB--and duration--4.79 minutes--of the 991 flac files on my laptop.
[2] http://www.cdc.gov/nchs/fastats/lifexpec.htm
[3] ftp://ftp.cdc.gov/pub/Health_Statist...s/table026.xls
[4] While the WD Green drives don't list MTBF, its big brother the RE4 has 1.2 million hours MTBF ( http://www.wdc.com/en/products/Products.asp?DriveID=732 ), which is obviously supposed to be higher than the consumer drives (and is rated for 100% more load/unload cycles than the green drives). With 64 drives in the system as he reports, he should statistically see a failure every 2.1 years with RE4 drives. Given that RE4 drives go through burn-in testing not mentioned for greens, it can be inferred that "infant mortality" will adversely affect MTBF on the greens further, in addition to differences in manufacturing and firmware/operation.
[5] http://basanovich.livejournal.com/163813.html
[6] http://basik.ru/maxx/bas/gravicapa/11.jpg

[Dan]

I think that the real question here is why on earth would someone actually need a 90TB file server at their home. I have a domain and hosting for a personal web page that I pay $2.99/mo for. I have unlimited bandwidth and storage. Sure...its not in my physical possession, and could be hacked much easier, but as for videos, pictures, and files that I store on a regular basis, there is no need for something soo expensive.

As for important data, or stuff I want to keep hidden, go get a couple of Western Digital 3TB drives and throw them in RAID 5 and encrypt them...that should be more than enough for the vast majority of people...

[Frank]

90TB..... ummm can you say the ultimate mirrored web site host?

[TinaTwilight]

Wonderful.......it's constructed from plywood and holds around 90TB of data.

checking software free

[Denae84]

Thank you for the good information..Thanks for sharing...