Future Storage Systems: A pause in workflow

Since I started this article series, I’ve had the awesome opportunity to have my ideas (well, some of the early articles at least) reviewed by person(s) who deal with the actual infrastructure of storage systems day in, day out.  The benefit of such peer review is that you get to learn at the symbolic “feet” of the masters and discover flaws, omissions, and understated features that need to be understood and incorporated.  This post is dedicated to some of those discussions and, where applicable, my understanding of how the FSS either incorporates or misses the boat.

To start things off, a very esteemed engineer (John Walton) in our Symmetrix group got to read Part 1 of the FSS story. He provided the following feedback:

Hey Dave, in general most fault tolerant or highly available systems maintain a minimum of shared state space between members of separate fault domains. this reduces the work required to formally prove your system is resilient to faults. cache coherency protocols with distributed state are often difficult to do this with. in the past CC versions of HT have not been usable for this reason.

Regarding the coherency statement. thanks for bringing that up. as an overview, I was trying to get at the concept of coherency without getting too deep into the mechanics. Also to consider is the platform’s ability to facilitate node coherency. I know with the design of Socket L1-SP and dual HT links (one cHT and one nHT), AMD was promoting the ability to have coherency across a single 8 bit HT1.x channel while using the non-coherent link for sideband and/or random access. This actually IMPROVES somewhat with the HT3.x gen channels on Shanghai.  This would be due to increased bandwidth, low latency, and processing optimizations.

The next question comes from a very good friend of mine who has graciously taken the time out of his schedule to read this series.

What does (the FSS node architecture) do to the (AMD) 4P (83xx) offerings? Doesn’t it kind of negate the premium to AMD/Intel for the over 2P market?

There are a couple of different ways of looking at this question.  In most storage systems, node to node communication is made either over PCI(e or x) or using GigE. The upside to this is legacy bus usage (i.e. no real need to “think outside of the box”) and common platform accessibility. It also means that you would not need to use n-way processors for most applications, thus driving down platform and development costs.  The  big downside to this legacy bus based approach is scaling and bandwidth. Simple two node storage arrays using PCIe are limited to that connection which does nothing but handle simple I/O and bus chatter. Lose that link, you’re going to trespass, etc.  Re-establishing that link can be disruptive (in certain products) and generally can be a pain to handle.

The FSS uses AMD Opteron 8300 series processors for several reasons.  The first reason relates to using Hypertransport as a connectivity medium.  The 8300 series processors provide (in addition to the system HT links), HT links (8 bit) to other processors(and co-processors…remember, Torrenza is important!) within a given “system.”  The 2300 series does NOT provide this same level of HT capability.  Secondly, a trade-off had to be made with scaling vs. static nodes.  In a world where scaling storage processing nodes has become increasingly the best approach to handling accessory functions within the system OS, the FSS needs to be able to add significant processing and memory bandwidth capabilities that can only be provided by the 8300 series.  In an ironic twist, it’s more expensive for the processors (8300 vs 2300) but cheaper than designing a brand new interconnect schema for Intel Dunnington processors.

Closing Thoughts:

We’ll see if any other comments come in while this FSS article series is still in development.  As they do, I’ll keep updating.

Related articles by Zemanta

• EMC finally puts Symmetrix line on thin provisioning diet
• EMC does flash drives on big storage
• IBM’s XIV storage lacks high-end features, analyst says
• NetApp soups up its filers
• Battle of the SSD strategies: Sun vs. EMC
• EMC execs: We’ll drive SSD cost down ‘as fast as we can’
• IBM’s XIV storage lacks high-end features, analyst says