Well-designed and highly cost-effective storage
Written: Nov 12 '04 (Updated Jul 30 '05)
|
Product Rating:
|
|
|
Pros: Inexpensive hardware and support, cross-platform, industrial design
Cons: Non-redundant RAID controllers, split personality, no SNMP performance monitoring
The Bottom Line: The XServe RAID offers nearly enterprise-class storage management for a very low price and ongoing cost of ownership. It essentially offers many midrange features for a low-end price.
|
|
|
| majid's Full Review: Apple Xserve RAID (M8668LL/A) Fibre Channel Drive ... |
My company is upgrading its ageing Network Appliance filers (see my Epinions review of them) for a trio of Apple's XServe RAID. We first ran a pilot project using one XServe RAID in a non-critical development environment, and after 2 months of service, decided to upgrade our production servers as well.
Our primary consideration was to find a cost-effective but reliable storage solution for a moderately intensive 24x7 on-line transaction processing (OLTP) Oracle database environment. The support contracts on our obsolete NetApp filers were running at $14,000 per year. For that price, we could buy new storage altogether. We paid $300K for 3 filers in 2000, with 600GB of effective storage, and a yearly maintenance cost of $12K. Our new setup cost us $36K, has a capacity of 5.2TB, is about 7 to 10 times faster on our disk benchmarks, and has 3 years of prepaid support included in the price. This improvement is much faster than Moore's law.
The Apple XServe RAID is a low-cost RAID controller. It achieves its low cost by eschewing expensive enterprise-class SCSI or Fibre Channel (FC) hard drives, opting instead for commodity IDE drives, which are much cheaper because of economies of scale driven by their intended market, desktop PCs.
It is important to understand the trade-offs: IDE drives have much lower profit margins, and are not designed for 24x7 duty cycles. They usually have cheaper motors and mechanical components, and are not as durable as the more expensive drives. The IDE interface also does allows only one disk operation to be active at any given time, whereas the SCSI or FC drives have command queuing, which allows several disk operations to be supplied simultaneously to the disk's built-in controller, allowing it to optimize data fetches and writes as the drive's arm moves across the disk. IDE drives are thus less efficient at multitasking workloads that do a lot of random access to data. It should be noted, however, that Serial ATA (SATA) drives, the next incarnation of the ATA standard, support native command queuing. On the plus side, the lower cost of IDE drives means you can buy more spindles for the same budget, and thus get better performance overall, and IDE drives have much larger densities than the more conservative SCSI and FC ones, as of this writing 400GB vs. 73GB or so.
There is a big debate in the industry between proponents of Storage Area Networks (SAN) and Network-Attached Storage (NAS). EMC is the leader for SAN, Netapp for NAS. In a SAN, the disk array is essentially viewed as one huge logical hard drive, and data is accessed a block at a time via a server's OS or a database like Oracle. The underlying technology is Fibre Channel, which is hideously expensive (specially the switches), but offers high performance and low latency. In NAS, the disk array behaves like a file server using common protocols like NFS (Unix), SMB/CIFS (Windows) or even as a web server. The underlying network is good ol' Ethernet, albeit Gigabit Ethernet. NAS are traditionally used more for unstructured data like a high-end replacement for NT server for users' home directories, but you can also use Oracle on one, with some caveats. SANs are more common in high-end, mainframe-like data center environments. NAS are more popular with ISPs and workgroups.
I compared the XServe RAID to our old NetApp filers. The comparison is not strictly fair, as the service offered is slightly different: the NetApps are NAS units, the XServe RAID are SAN units. The Netapps also offer some high-end manageability features such as snapshots (the ability to take a logical snapshot of your data that is "frozen" in time, usually for backup purposes), remote mirroring (the ability to have one RAID unit constantly updating another one over a long-distance data link, useful for disaster recovery) and clustering (two units share disk shelves, and either one can take over for the other if it fails). On the XServe RAID, you would have to provide the same features at the operating system level on the server that is connected to the it.
The XServe RAID uses IDE drives internally, but it offers two 2Gbps Fibre Channel (FC) interfaces to connect to the servers that will be using it. This is an interesting choice - FC is a high-end technology used mostly in mainframe-class data centers. Most competing RAID units offer a SCSI interface instead. One product that has a similar architecture is the Dell-EMC AX100, which sells like hot cakes even though it is twice as costly. This shows how aggressive Apple's pricing is, and how seriously they are trying to break into the enterprise market.
Apple's use of FC offers an upgrade path to networked storage, although the high price of FC switches probably precludes this use for primary markets of small business and video editing, but Apple also has a thriving business selling machines for supercomputer clusters and biotechnology companies (the CEO of Genentech sits on Apple's board), who would benefit from this capability. They can have a farm of XServe G5 servers connected to a farm of XServe RAID storage via a FC SAN, without being constrained to a 1-1 connection between XServe G5 and XServe RAID, thus yielding more efficient use of hardware and manageability.
One potential problem with the use of FC is the cost of fibre channel host bus adapters (HBA), the interface cards that plug into a server to allow it to connect to FC storage. The limited market for FC compared to IDE or SCSI (and the well-established vendor habit of gouging enterprise customers) means a dual-port 2GBps HBA usually sells in the $1500-$2000 range, but here again Apple is pushing to democratize FC by selling theirs for $500. The cards use copper connectors (Fibre channel is a misnomer - most FC is run over copper), but with the use of transceivers, it can be run over fiber optics over much longer distances than SCSI or SATA permit.
The Apple HBA cards are actually rebranded cards from LSI Logic, and while Apple would like you to use their XServe G5, we actually use them in Sun Sparc servers using the LSI Logic driver. The website www.alienraid.org has very useful resources for using the XServe RAID with non-Apple hardware. Sun's Solaris is not officially supported by Apple, but the XServe RAID is certified to run with Windows Server and Linux, Apple clearly understands the need for openness to compete in the enterprise market, a welcome change from past habits of theirs.
As I mentioned, the IDE technology used in the drives means they are less efficient at parallel random-access. This isn't much of an issue for video editing, which tends to sequential I/O, but for a database workload like ours, we need to take two precautions:
1. Use a modern journaling filesystem to optimize disk access with a bias toward sequential write patterns.
2. Ensure the unit has a lot of cache RAM so it doesn't have to do as many random disk accesses (flailing the disk arm around is a killer on performance). The XServe RAID is available in two versions, with 128MB or 512MB of RAM per controller - make sure you get the 512M version. You should also get the optional cache battery backup modules that allow this buffer to be used also for writes (without this module, the RAID controller has to wait for the writes to be flushed to disk, otherwise a power failure could cause data loss).
The disk drives themselves have a buffer, probably 8MB or 16MB. The Apple management software optionally allows you to use that buffer for writes. Be advised the cache battery backup only backs up the cache on the controller, not the one on the drives, and so this option should only be considered for non-critical data like a video rendering drive (Apple recommends this option only be used if the XServe RAID is connected to a UPS,).
The XServe RAID comes in a 3 rack unit package, and a rack-mount rail kit is supplied with (a refreshing change from vendors who try to nickel-and-dime you to death with extra charges for something like this). When fully populated with 14 drives, it is fairly heavy. The solid brushed aluminum chassis shows Apple's trademark design flair and build quality. The drives plug in the front, you press on the face plate to expose the spring-loaded recessed handle, and take the drive carrier out. There is a hex lock bolt to lock the drives in place to prevent accidental disconnection.
This unit has more lights than a Christmas tree. Every component has a status light that turns to red if it is faulty, and there are two columns of blue LEDs to indicate disk activity that would not be out of place on a stereo. In addition to the fault LEDs, there is a fairly strident audible alarm; fortunately, it can be turned off from the management software. Almost every component is hot-swappable, including the two redundant power supplies, blower fans, and battery-backed cache modules, the exception being the two RAID controller modules. The airflow works well at keeping the drive cool, and the noise levels are acceptable, but probably too high for deskside use (video editors will probably spring for a long-range FC cable and put the XServe RAID in a closet).
The two RAID controller modules each have their own Fast Ethernet port for management, and that leads me to the only weak point in this unit - everything else is fully redundant, but the RAID controllers themselves are not. Apple's marketing literature is evasive on this subject, verging on the deceptive by omission. This is not a RAID unit with redundant controllers accessing up to 14 drives, it is two non-redundant RAID units accessing up to 7 drives each, in a single chassis with redundant components. The two modules communicate among themselves, so both remain accessible even if the Ethernet connection on one is disabled. The disks are managed in separate pools, however, and you cannot pool them together in a single large volume, and you would have to use a volume manager on your server operating system to achieve that.
The IDE bus was not designed for sharing, which probably explains why Apple was not able to create a fully redundant design. This would require designing a passive backplane for IDE like those used for FC or SCSI (in theory, a passive backplane has no electronics, only connectors, and is more reliable, but in practice the FC backplanes on out NetApps failed more often than the RAID controllers). This lack of redundancy may exclude the XServe RAID from high-availability environments, but then again, there are limits to what you can expect for this price, and from our experience so far, this is not a deal-killer.
Apple provides RAID Admin, a Java-based management application. It does not have a Windows installer, however, and is a bit of a pain to run on non-Apple management machines for someone who isn't familiar with Java. You need to have a recent version of Java installed on your machine, unpack the supplied zip or tar.gz, and run "java -jar RAID_Admin.jar". It works fine on my Solaris 9/x86 workstation using Sun's standard JVM. On Windows, you can double-click on the jar.
Interestingly, the management application seems to use HTTP for management instead of the more usual SNMP, and Rendezvous/Zeroconf for discovery. The management interface uses a proprietary XML-RPC style interface over HTTP, you can't just point your browser to the XServe RAID's IP address and expect it to work the way some Cisco switches do. If you are planning on managing units across a firewall, as we do, this is good to keep in mind. There is no command-line management interface. SNMP support is minimal - only the system description is available, and there are no performance monitoring MIB or fault reporting traps available. The front panel LEDs show disk I/O in real time, it would be useful to have some this available over SNMP, as well as SMART events from the drive modules.
Fault management is performed by the management application, which can be set up to send alerts by email. This requires the management application to be always running, but is probably more reliable than SNMP traps. Older versions of the management app (specially 1.3) would tend to crash frequently, but 1.3.2 has proven more stable.
Each half of the XServe RAID can be split into up to 8 volumes, also known in FC/SCSI parlance as logical units (LUN). The volumes can be formatted as RAID 0 (striping, no data redundancy but high performance for applications like HD video editing), RAID 1 (mirroring), RAID 0 1, RAID 5 (data redundancy but less costly than mirroring) and RAID 3 (mostly useless, RAID 5 is preferable in almost all cases). Any unused drives are automatically used as hot spares.
There are constraints on the minimum amount of drives in a volume (at least 3 in a RAID 5 volume, for instance), and since in a production environment you will typically want to leave at least one hot spare drive, in practice you can't reasonably put more than two volumes per half. Queueing theory actually shows a single volume with more drives will have better performance than two smaller volumes, so the configuration we use is 6 drives in a RAID 5 volume, with 1 hot spare. Using 250GB hard drives, this yields two 1.1TB usable volumes per shelf.
Apple will allow you to divide a volume in "slices", up to 8, but they will all have the same size, you cannot choose the size of each slice individually. In a single-server situation, partitioning will provide the same effect (with more flexibility for partition sizes), but if you are sharing the XServe RAID between multiple servers through a SAN, this could be useful, and each slice would still benefit from the throughput of all available spindles.
The two fully-populated units we ordered were shipped with all drives and battery cache modules pre=installed, but one was formatted out of the box, the other wasn't. They put all 7 drives on each side in a single RAID 5 volume with no hot spares, however, so we had to redo it in any case. The initial format is very slow, taking nearly an entire day to format the volume. By default, the volume will be immediately available while volume construction is under way, but performance will be degraded until the volume is completed. This is something to allow for when planning for tight deployment schedules.
Finally, the support options are very reasonably priced. A 3-year AppleCare support contract will cost $1000 (compared with $12K/year on the NetApps...), but for mission-critical deployments, you are advised to buy the $2000 spares kit which includes one of every hot-swappable component to tide you over until you receive the RMA-ed part serviced under warranty. Make sure you get the right version, with a 250GB or 400GB spare drive. Disk drives in a RAID array are mechanical components that will wear out, and should really be treated like consumables. The support contract means you have peace of mind for 3 years for the price of two spare drives. After 3 years, the unit is going to be obsolete, anyway, and you will probably want to upgrade to the then current generation.
Recommended:
Yes
|
|
|
|
Epinions.com ID:
majid
|
|
Member: Fazal Majid
Location: San Francisco
Reviews written: 53
Trusted by: 5 members
About Me:
I'm the CTO of an Internet startup
|
|
|
- Add to My Yahoo!: 
- Add to Google Homepage: 
© 1999-2009 Shopping.com, Inc.