As new PC-oriented CPU and communications technologies and architectures crowd into the market for network server computing systems, it's easy to become enamored with the low prices for what seems to be commodity server systems. While competing performance claims abound, one thing is certain: underlying improvements in CPU, memory, network and communications hardware and software are driving down the cost of excellent computing technology, and will continue to do so. Today, as never before, the IS manager can configure higher performance, lower cost systems with hardware components based on well-accepted industry standards in both hardware and software. Well-accepted standards have opened the door to increased competition and comparable (i.e., definable) performance from a number of vendors.
And with increasing competition in both the high-end and low-end of the market, the choice of a high quality network building block such as network servers should be simple, right? Industry standard components, the proper benchmark figures and the best price should yield the right choice .... or does it?
Tricord Systems, the maker of the PowerFrame Enterprise Server and a pioneer in the development of the high performance super server marketplace has a different view. We believe that in choosing a network server, IS managers should widen their focus to include both price/performance and high system availability.
The reason is simple. Computer network users depend upon the smooth, continuous operation of the network in the same way businesses and homes depend upon the power company. Like the electricity that powers products from the mundane (refrigerators) to the indispensable (air traffic control systems), network servers provide computing power to drive applications ranging from simple word processing to enterprise-wide and mission-critical business applications. And like the electric power plants we depend upon, network servers are only noticed when they are down. And our dependence on our servers is increasing as more companies are migrating more and more applications to the less expensive and more powerful server platforms, such as the PowerFrame Enterprise Server, that feature open systems architectures, standard OSs and high availability features such as prefault notification, fault recovery and extensive management features.
As a result, for an ever increasing number of companies, when the network is down, the company is down. The impact ripples from end-users within the company, to the supporting infrastructure of the company, down to the company's customers, and eventually even through to its suppliers. It is fair to say that downtime (or more precisely, avoiding downtime) has become the single most important measure of success for most network computing systems. Thus, the selection of an enterprise server must be made within a strategy that addresses the issue of ensuring high availability. In fact, even though performance is always a critical requirement, for many, high availability has become even more important than meeting or exceeding the ever-changing performance benchmarks.
At the global level, Tricord defines high availability as "the ability to provide reliable, fault tolerant server performance with little or downtime." At a fundamental level, this requires innovative hardware and software design, featuring a redundant and load-balanced system architecture with built-in system management functions, and diagnostic tools. But, Tricord also defines high availability in terms not associated with system architecture at all. These include: high RAS (reliability, availability, and serviceability), ease of installation, local technical support and a corporate commitment to a Total Quality Management or TQM approach to overall product and process quality. The commitment from Tricord is to keep its PowerFrame servers up and running an average of 99.98 percent of the time (this equates to less than two hours of downtime per year). To achieve figures like these requires a high-availability strategy that includes a number of critical and related components.
High availability starts with the server. The ideal network server provides high performance, scalability, and fault tolerance in a complete system-level solution with fault prediction and management, hot-swapping of key components and hot-upgrades when performance enhancements are needed. In addition, the ideal server is backed by a high-quality approach to manufacturing, an easy-to-install startup procedure, local support, a comprehensive service program, and a company committed to providing ongoing solutions for MIS operations.
At the core of a high-performance enterprise server are the hardware design features that are most often touted in datasheets and product briefs, i.e., balanced, multiprocessor architecture, intelligent subsystems, redundancy, error-checking and correction features, data path parity, hot-swapping of components, and uninterruptable power supplies. These do indeed provide the basic fundamental hardware architecture that separates a high availability server from a simple multiprocessor PC system.
The PowerFrame Enterprise Server, for example, as shown in the accompanying figure, starts with a symmetric multiprocessing (SMP) CPU/memory architecture that is integrated into a Functional Multiprocessing (FMP) systems architecture. This divides the system into three major independent, intelligent subsystems - for CPU and memory, disk I/O, and network I/O. Intelligent subsystems use the 64-bit system bus (PowerBusŪ) with its 267 Mbyte/second bandwidth to communicate with each other.
The PowerFrame symmetric multiprocessing architecture allows multiple CPUs to share system memory, interrupts, and mass storage devices. With the major data processing and data moving and storing subsystems operating independently, overall system performance can be improved and balanced by adding just the right amount of resources to a given server function. This ensures a robust, scalable, yet balanced system that can be easily reconfigured as enterprise requirements change - a major ongoing task for network managers. By concentrating multiple processor and disk resources in a single system, the enterprise server simplifies the resource management task. By permitting hot-swapping and fast upgrading, high availability is maintained.
An important component of the PowerFrame is the Intelligent Management Subsystem or IMS. Its sophisticated system monitoring functions give the system administrator the information needed to streamline system operation. For example, one of the most intriguing questions about a given system concerns not just the raw performance specifications, but rather, how well a given system element will perform in a specific environment under a given load? And also, how well will the system perform over time, as application loading varies? The IMS reporting functions give the administrator a snapshot of system operation. These can be used to take base-line samples of system operation over time to highlight the operation of individual components. For example, a baseline measurement of system operation can be taken, a resource can be added or reconfigured, and then a second measurement taken to reveal the results of the change. This allows the administrator to pinpoint bottlenecks and to adjust server resources for optimum performance. It also provides the tools for forward capacity planning, giving the administrator the historical performance data from which to derive accurate understanding of system response to external loading.
It is the combination of symmetrical multiprocessing, functional multiprocessing and intelligent monitoring features that separate the PowerFrame Enterprise Server from PC-based servers.
The PowerFrame architecture is certified for operation with the single and multiprocessor operating systems including Microsoft Windows NT Server 3.x, Novell Netware 3.x and 4.x, and Netware 4.1 SMP. The fact that Tricord is a certified OEM with Novell and Microsoft, with personnel in both companies dedicated to supporting the Tricord solution, ensures a standard and open operating platform for all Tricord applications.
Using multiple Pentium processors, the PowerFrame can be configured with as many as eight high speed CPUs configured in CPU/Cache Subsystems (CCS). Tricord employs MP Accelerator Technology (MPAT), providing 2 Mbytes of second-level cache for each CPU, along with address pipelining, burst data transfers, bus snooping, etc., to maximize multiprocessing performance. This significantly improves Pentium processing performance. The PowerFrame utilizes the APIC (Advanced Programmable Interrupt Controller) architecture to provide dynamic symmetrical interrupt distribution across all processors. This keeps CPU loading balanced throughout the system.
The Intelligent Storage Subsystem (ISS) of the PowerFrame features dedicated processors for handling storage I/O tasks. It off-loads 90 percent of the I/O processing tasks from the CPU and operating system. A primary component of performance is the very high speed PowerBusŪ, a 64-bit wide bus with a high burst data rate. Multi-channel high speed SCSI disk controllers provide high speed data access, with no degradation even as additional drives are added to the system. For even greater performance, the PowerFrame ISS includes an optional 8 Mbyte fault-tolerant cache module that increases disk performance by 25 percent or more. Enabled on a logical device basis, it allows the server disk performance to be fine-tuned by dedicating the cache to selected write-intensive, transaction-based application storage. Even without employing dedicated cache, the independent I/O processor provides very high performance disk access to even the less access critical applications. Data access algorithms running in the independent I/O processor greatly speed up data access thanks to the ability to buffer and re-order data requests to optimize disk drive access and eliminate head thrashing. This sophisticated disk and data management takes place in the I/O processor, eliminating any overhead on the main applications processor.
For high performance network I/O, the PowerFrame includes either a 33 Mbyte/second EISA Bridge Subsystem (EBS) or a dual 133 Mbyte/second PCI Bridge Subsystem (PBS) that also supports a 33 Mbyte/second EISA bus. The PBS subsystem supports optimized operation with data buffering and high-speed data bursting techniques that allow maximum data transfer among all buses concurrently. All EISA or PCI bus slots are bus masters, allowing data transfers into and out of main memory with no CPU interaction.
With high speed, independent I/O subsystems and symmetric multiprocessor subsystems, the PowerFrame Enterprise Server provides a powerful, standardized platform for network server operating systems and applications. On performance alone, it would be worth a look as the choice for a network server building block. But, as previously pointed out, performance without high availability is no longer a viable option.
Embedded within the high performance architectures of the subsystems themselves are high availability design features that enhance system availability. In addition, there are system-level features that allow high availability to be maintained throughout the normal course of a system life-cycle.
An Intel study on server failure points identified the percentage of time that each component contributes to failure in a typical server configuration. (See figure 2.)
The PowerFrame Enterprise Server addresses these failure points with a number of fault tolerant redundancy features. These include RAID disk arrays, disk hot sparing, disk controller duplexing, redundant CPUs, redundant network interface cards, redundant and uninterruptable power supplies. The result is a set of embedded hardware features that protect the server system from any combination of internal hardware failures at every critical point in the system.
Disk drives are responsible for 50 percent of server downtime. RAID (Redundant Array of Inexpensive Disks) technology has become the technique of choice in protecting the server system from disk failure. RAID spreads data over an array of multiple disks so that overall disk operation is not interrupted in the event of a single drive failure. There are different levels of RAID operation (from 0 to 10) to provide for different levels of protection and expense. Often it is desirable to mix different levels of RAID within the system, in order to provide the correct level of protection at a justifiable expense.
For example, RAID 0 is data striping. This improves I/O performance and disk utilization by configuring multiple physical devices into a single, large logical device, balancing I/O requests evenly across all drives. While not an availability feature, combining the I/O performance of multiple drives increases performance as capacity grows. RAID 1 is disk mirroring and thus is an availability feature. Disk mirroring enhances performance of read operations and provides for fault tolerance. Data on one disk is duplicated on one or more other disks. If a drive fails, the system continues to operate on the surviving drive or drives. RAID 2 interleaves data across all disks with parity information contained on three extra disks. RAID 3 is similar to RAID 2 except that only the primary disks in the array can detect corrupted data. With parity information contained on a single disk, it is a more economical solution. RAID 4 and 5 are similar to RAID 0 in that they configure multiple drives into a single, logical device, but store parity information on a single drive (RAID 4) or among all the drives (RAID 5).
Heavy-use transaction processing environments produce a large number of transactions consisting of small block sizes. RAID 2, RAID 3 and RAID 4 are not conducive to this type of data transaction and have never been widely adopted.
An important feature of advanced RAID systems is the ability to use multiple levels of RAID at the same time. The PowerFrame architecture supports this ability and allows the network manager to configure different RAID levels for different applications.
For the highest performance and redundancy, RAID 10 uses a combination of data striping and mirroring on the same disk array that optimizes both disk capacity and performance. However, this maximum performance is more expensive because it requires more drives as compared to RAID 5. RAID 5 offers a comparable level of data integrity to RAID 10 and costs roughly 35 percent less to implement. But, because RAID 5 requires two additional disk reads and one additional disk write during disk write requests, performance drops significantly in a RAID 5 implementation while read performance remains excellent, as shown in the accompanying figure.
RAID 5 is often used in environments that require limited disk writing, such as database serving, reference libraries and multimedia serving.
The PowerFrame Enterprise Server provides PowerRAID as a standard feature. PowerRAID implements RAID levels 0, 1, 4, 5, and 10, as well as hot sparing, hot replacement and disk controller duplexing.
With disk hot sparing, network administrators can ensure that a failed disk drive does not bring down the system. Disk hot sparing gives the capability of assigning backup disks to a mirrored configuration for additional fault tolerance in a RAID 1, 4, 5 or 10 configuration. In the PowerFrame Enterprise Server, when one of the mirrored drives fails, a designated hot spare automatically replaces the failed drive during normal system operation, restoring the benefits of the mirrored configuration and protecting the data from a secondary failure.
Because the hot spare drive actuates without operator intervention, remote systems receive the same high level of disk fault tolerance as systems located nearby. Disk hot sparing eliminates the risk and inconvenience of system downtime caused by failed disk drives. When used with hot-swapping, disk hot sparing eliminates downtime for disk replacement and maintenance.
Used in complementary fashion with RAID and disk hot-swapping, disk hot sparing provides another important line of defense for protecting user data and ensuring seamless network operation. RAID is itself, of course, an important data defense mechanism. And hot-swapping also is a very good feature, but it leaves open the question of -- will the untested spare work in the system? Using inventoried spare drives to replace failed ones always introduces a new risk, since these components have not been tested in actual operation. The use of hot sparing eliminates this risk because the "spare" is already a fully operating part of the system with a history of performance (recorded by the Intelligent Management Subsystem). Using hot sparing, in conjunction with RAID and hot-swapping of components, allows the system to maintain operation with a minimum of risk to user data.
Disk controller duplexing provides an additional level of fault recovery in the disk subsystem as well as providing additional disk I/O performance. This is a special mirroring technique where mirrored drives are connected to separate disk controllers. (See figure 4.) This protects the system from a failure in the disk drive, the controller, or the data bus itself. Disk duplexing must be implemented in the operating system software. Both Microsoft Windows NT Server and Novell NetWare OS can support this capability.
With the Tricord multi-channel disk controllers, adding additional disk drives actually increases system I/O bandwidth. With additional channels, the Tricord system can distributed the drive load across the system, increasing the I/O potential of the server system.
One of the advantages of multiprocessor systems is the availability of additional CPUs. The PowerFrame Enterprise Server design supports the ability to isolate a failed CPU and continue operating with the failed CPU off-line. This automatic recovery mechanism causes a system reboot upon CPU failure detection. The failed board is placed in a reset mode and the system operates with one less CPU. This keeps the system running letting the system administrator replace the failed component with no downtime.
With today's IC technology, CPU failure is relatively rare. With the Tricord approach, additional CPU and cache subsystems resources, even when designated as hot standby components, are fully utilized by the system. They are fully utilized as additional application computing resources, until they are needed for redundant operation. This results in a system that is redundant, high performance, and cost-effective.
Likewise, redundant network interface cards (NIC) under the control of system software such as NSI's Balance and Redundancy operating in the Novell NetWare environment, can maintain network operation even with complete card or controller failure. If network administrators install multiple NICs within the same logical segment (using NSI's Balance), the network uses all NICs during normal operation. However, if a NIC fails, the NSI Redundancy software automatically redistributes the network load among the remaining NICs. This reduces the risk of network-related downtime. In addition, NSI's software constantly monitors a failed NIC; if its state changes from failed to functional, the load automatically is redistributed to include the recovered NIC.
The PowerFrame Enterprise Server uses modular, hot replaceable power supplies with integrated cooling fans that keep the server running and cooled if a power-supply failure occurs. As the second most vulnerable system component, according to the Intel study, power supplies have received special attention in the PowerFrame design. Multiple power supply modules provide redundant power for the system and each module is monitored by the server management software.
Hot replacement ensures continuous operation when internal failures occur. External options like Uninterruptable Power Systems (UPS) provide battery backup for continuous operation when external power failure or a general power outage occurs. Should the power loss continue for an extended time, system UPS software allows the server to perform a graceful, orderly shutdown if the battery system is in danger of being exhausted.
The PowerFrame Enterprise Server UPS also conditions the power source to eliminate the effects of power fluctuations, eliminating a common source of server failure.
Given the high heat levels of modern IC-based circuitry, thermal protection is more important than ever. The PowerFrame Server uses a mainframe-like cooling system which uses plenum directed airflow technology and incorporates thermal sensing and over-temperature protection to protect components and keep the system running at the appropriate temperature.
Naturally, like any large-scale system, an enterprise server should support industry standard tape backup systems for physical protection of sensitive data. With PCI and EISA bus support, and standard operating system support, the PowerFrame Enterprise Server supports a wide range of industry-standard tape backup subsystems. Network administrators have several options in configuring backup systems. One is to connect devices directly to the Intelligent Storage Subsystem or ISS SCSI bus. This multichannel bus provides for direct, high speed connection of Digital Linear Tape (DLT) drives. Alternatively, other backup options such as optical juke boxes can be connected via the PCI or EISA buses. Tricord users have employed a wide variety of backup systems including 20-40 Gbyte DLTs, carousels with up to fifty 8 mm DAT tapes, 4 mm tape stacks with auto changers and magneto-optic or WORM optical drives.
For the ultimate in redundant protection, PowerFrame mirrored server solutions can provide non-stop operation for a fraction of the cost of non-stop mainframe systems. While requiring supportive system software such as Novell's SFT III (System Fault Tolerance Level III), Vinca, and Octopus 1.3, these systems provide the highest level of server availability. SFT III, for example, allows two Novell servers to be identically configured and connected via a high-speed fiber optic link to provide total server redundancy. SFT III completely duplicates disk contents and memory images of the systems. Should a hardware error occur, the faulty server is shut down immediately and the mirrored server takes over servicing network clients.
In addition to redundancy at the component and even the complete system level, the PowerFrame provides high availability features embedded in the hardware design itself. These include ECC memory, data integrity (parity) checking on all data paths, a passive backplane design and a modular, design-for-serviceability for fast field replacement and technology updates. In addition, Tricord uses high MTBF components in the base design of its servers, as opposed to the low-cost implementation approaches used in PC-based servers.
The PowerFrame main memory subsystem or MMS employs an extra 8-bits of error correction coding (ECC) memory to provide the highest level of data security. This technique provides for single bit error detection and correction and double bit error detection. ECC hardware checks for data integrity before any memory information is sent from the memory subsystem to the system bus. During memory write cycles, 16 check bits are appended to each 128 bits of data, providing for correcting up to 4 memory errors per memory write. During memory refresh cycles, the data is checked for validity before transfer to the system bus.
In addition, the MMS continually scrubs memory to detect and correct single-bit errors. This improves system availability by eliminating most bit errors in memory before the data is ever needed by an application program. Every 70 minutes, the entire 4 Gbyte physical address range is cycled through, one bit at a time. All errors are corrected and logged by the IMS. The IMS keeps track of errors by a location and a SIMM by SIMM basis. After 5 reported errors, a system administration notification flag is set. This allows for early notification of potential problem hardware and planned replacement.
Because the PowerFrame ECC is implemented in hardware, there is no degradation of the memory subsystem performance during memory access. The CPU and other bus masters can access main memory at a very high, 267 Mbyte/second or greater read burst rate from contiguous memory locations thanks to the interleaved memory design and address pipelining. Single-bit errors are corrected transparently without introducing any wait states.
In addition to error checking and correcting techniques in the main memory, the PowerFrame Enterprise Server provides parity checking on all data, at every point in the data transfer. This protects the data from the moment it reaches the server, as it is written and read from disk, until it is transferred back out over the LAN. Parity checking throughout all data transfers is critical in protecting against loss or corruption of data moving through the server.
Many otherwise well designed high availability servers are dependent upon an active motherboard that acts as the Achilles heel when (not if) it fails. When it fails, all other subsystem availability features are rendered moot. The PowerFrame, on the other hand, features a highly modular design utilizing a passive backplane that contains no active electronics. This ensures flexible and upgradable configurations and minimizes system mean time to repair (MTTR).
In addition to basic hardware redundancy and fault tolerance, high availability servers such as the PowerFrame Enterprise Server include important system management features such as fault prediction, configuration management, and performance management functions. These give the system administrator the tools to manage the server resource in ways that will improve reliability and avoid downtime.
The Intelligent Management Subsystem (IMS) from Tricord provides powerful fault, performance and configuration management tools for the PowerFrame Server with highly integrated hardware and software features. The IMS architecture is shown in the following figure.
To achieve uptimes of 99.98 percent or better requires a corporate strategy of commitment to the highest levels of quality in all phases of design, integration, manufacture and support. Tricord currently meets this challenge with its PowerFrame Enterprise Servers, its corporate commitment to high quality, and its close relationship with its VAR-based sales/support teams. The end result puts Tricord in the forefront of the industry in both performance and reliability as well as customer satisfaction. In the future, technology will continue to advance and new CPU/memory/network architectures will emerge, but the Tricord commitment to high availability will remain the same. This will enable Tricord to consistently produce advanced systems that support IS and network managers in their quest to provide highly available computing resources to their end users.
END OF TECHNICAL ARTICLE