Special for Communications Systems Design
Dramatic forces at work in the enterprise network system are dictating major changes in the way that network equipment system designers are designing and developing their next generation systems. The massive increases in data traffic along with the deployment of new, much faster transmission technologies are forcing systems designers to rethink their architectures to encompass new ways of resolving new bottlenecks that are emerging in the network topology.
The underlying communications network is rapidly changing underneath the surface just as the growth of the Internet has exploded with the number of new users skyrocketing from 100 million to over 600 million. At the same time as the network is coping with these new users, there is greater deployment of new database and multimedia oriented applications that are sending even larger amounts of data through the communications pipelines.
One means of coping with this greatly expanded traffic load is to switch to higher bandwidth transmission technologies, primarily fiber optics. While this provides a tremendous increase in raw data transmission speeds, it is forcing a rethinking of how and where to handle network traffic bottlenecks. The network is made up of communication links (the copper or fiber data pipes) and the processing nodes (the switches and routers) that direct the traffic from node to node. In the past, using copper and twisted pair, the bottleneck was more in the link itself than the nodes. As long as communications nodes performed their switching and routing functions at Świre-speed" they added little delay to the network. Today however, as high bandwidth fiber becomes the backbone link technology of choice, the bottleneck in the network traffic system moves from the link to the communication node. This creates a new dynamic within the network, where the architecture of the node must be redesigned in order to maintain "light-speed" data throughput.
The primary element in delay at the network node is the increasing communications data processing demands in the node. The key to increasing bandwidth through the node is to repartition the internal network equipment systems by putting high performance distributed processing power closer to the communications link and offloading the host system processor. Prior generations of equipment put a high degree of intelligence in the system, but at the high level system control point. Low level data link controllers handle the front-end tasks of data encoding/decoding, packetizing, assembling frames, etc. Once these tasks strip out the data from the communications envelop, the system CPU and software handle the routing and system level connectivity. Now, however, the network nodes is facing data link congestion at the link level and processing overload at the host system level. The solution is putting greater intelligence in the front-end data handlers so that these tasks get performed simultaneously with the data routing functions. This approach calls for a new way of partitioning the data processing and communications tasks within the edge network switching and routing system distributed edge processing.
The evolution of the network backbone from Category 5 copper wiring to fiber optics is dramatically affecting the internal dynamics of the network equipment system. The transmission time of data traveling across the network is a combination of the propagation delay time between the nodes and the time spent in each node. As the data propagation between the nodes approaches the speed of light, the new bottleneck is the time spend in the node. This is the time that the switch or router spends receiving data, descrambling it, processing routing information, reformatting it and then sending it on its way. With copper wire as the transmission media, this data processing time is insignificant compared to propagation time through the communication link itself. Since copper wire transmission speeds top out at 100 Mbps or so, and fiber potentially provides bandwidth at speeds of upwards of 160 Gbps, the network bottleneck is now clearly in the network node. With fiber as the transmission media, almost all of the delay will be in the communication node. It is there that the most significant gains in network transmission will be gained.
The Internet explosion has nearly overloaded the network backbone. Millions of new users are being added rapidly and these new users are accessing new applications that are generating vast volumes of image, audio and multimedia data. At the same time as the data traffic load is increasing, the processing requirements in the node are also increasing. Techniques such as Time Division Multiplexing or TDM are being employed to allow hardware data links to be shared by multiple messages from multiple sources and destinations. These techniques place significant processing burdens on the network node to receive, descramble, assemble and reassemble data frames, and then to encrypt and send them on to their destinations.
The increasing use of TCP/IP or Internet Protocol is also increasing demands on the node for greater processing. TCP/IP requires many small, 40 byte or so, acknowledgement packets. As Internet data traffic increases, it is not just the large image and multimedia files that are being transferred, but also the many small data packets along with their acknowledgement receipts that are adding to the processing load in the network equipment system.
As the percentage of network traffic that is IP-based increases, the network is increasingly transmitting IP packets using ATM and Frame Relay over T1 lines. The requirement to process ATM cell-based packets is also contributing to the data processing overload at the switch and router level. ATM service is based on switching fixed length packets known as cells. It supports multiple services and can be switched at very high speeds. But ATM user traffic must be segmented into cells, transmitted and then reassembled back into ATM cells. On the other hand, frame relay service uses a variable length frame to transport user traffic. It is very bandwidth efficient and it can provide multiple, independent data links to multiple destinations. Since it does not always require full T1 bandwidth, T1 or E1 lines can be "fractionalized" to allow frame relay traffic and other types of services on the leftover bandwidth.
As a result, virtual channels are being increasingly used to share hardware links between data traffic using different protocols in order to take advantage of any leftover bandwidth that may be available. While this greatly increases bandwidth utilization, it again puts more processing burden on the network node.
Finally, as the diversity of the network traffic increases, new network management requirements also increase. Traffic such as voice over Internet, video, and audio mixed with traditional email, fax and data files is placing more demand for reliability of service and priority queuing features. Quality of Service or QoS is necessary to handle the priority and balancing of packet traffic. With voice and multimedia data traffic, functions such as filtering, routing and forwarding are increasing in complexity because data packets carrying voice data, for example, must be delivered and reassembled in correct order within a real-time frame of reference. Otherwise the voice data will be clipped or scrambled.
All of these requirements are now being handled in the network switch or router. With current usage patterns, the old design of such system is no longer adequate for handling the volume and complexity of the data traffic and therefore, new communications systems architectures are needed.
The traditional network system design provides high level system processing power at the heart of the system, surrounded by ASIC-based data link controllers operating at high speed, but with limited intelligence. The data pathway through the network switch or router begins with low level wire-speed processing at the lowest layer of the OSI (Open System Interconnection) model. ASIC-based link controllers handle the various protocols of the data link. Once packet data has been extracted from the raw incoming data stream, it is delivered to the host processing system for processing and forwarding. The host CPU manages large routing tables and performance management functions. Once the data has been processed and is ready to be forwarded, it is sent back to the lower level controllers, reformatted and transmitted out of the node.
This centralized approach is being overwhelmed by the amount of raw data to be handled, the new processing demands of mixed protocol traffic, and the new filtering demands of TDM and other fractional line sharing techniques. The breakdown is in the I/O channels to the host CPU connection. There is just too much data, coming in too fast for the host system to process, filter, encode and decode and route at "wire-speed" or near "light-speed."
One solution to this problem is to apply more processing power and intelligence at the front-end of the communication pathway. In much the same way as distributed processing impacted centralized mainframe computing, distributing the communications processing load and putting more processing power closer to the point of impact can greatly speed network system throughput.
For example, by placing more intelligence at the data link controller layer, the system can apply this new computing power to handling the data encoding/decoding function for a variety of mixed media, mixed protocol data links without involving the host processing system. This new system intelligence can support the high level processing required to untangle the various incoming data streams from the virtual channels, the mixed protocol packets and perform the raw data link processing before handing off the "clean" data to the host system processor. Such capabilities require massive processing power however.
New edge processing devices, such as the TS704 674 Channel HDLC Controller from T.sqware, for example, put four high-speed 32-bit RISC processors on a single chip device. This massive compute power in the data link channel allows system designers to repartition the network switching equipment architecture. Instead of working around the bottleneck between the I/O channels and the host system processor, they can now focus on putting more functionality closer to the data link.
With the ability to support large numbers of HDLC links with minimal involvement by the host system processor, the edge processor enables greater throughput through the network switch or router. For example, the edge processor, not the host processor, can perform layer 2 switching processing. In frame relay applications, instead of transmitting the entire IP packet upstream to the host processor (usually over a slow PCI bus), only the header need be transmitted. The data packet (the bulk of the data stream) can be buffered locally and sent on its way after the host processor restructures the header information. For instance, when the task is to modify the packet header and retransmit on another TDM line local to the line card, the host merely sends back the header and the edge processor sends out new (old) packet data. In mixed protocol applications, frame relay to ATM, for example, again only the header need be processed by the host, with the edge processor bursting the data to the specified destination.
To achieve the performance levels required by new switching equipment, devices such as the edge processor must provide hundreds of MIPS (millions of instructions per second) of compute power, as well as the ability to process communications tasks such as encoding/decoding and protocol handling. As shown in Figure 5, the first edge processor commercially available from T.sqware contains four 32-bit RISC processing engines operating in parallel within a system-on-a-chip that includes a full complement of computing resources and 16 high speed serial links. They provide over 400 MIPS of performance per device. This compute power provides the ability to handle the many processing tasks required in supporting fractional frame relay, ATM, IP over T1 and other network-to-network and internetworking functions at the data link level and above. Using edge processors, network equipment begins to resemble a distributed computer network with local edge processors being added as more data lines are supported. Like distributed networks, as more functionality is needed, more edge processors are added. The system can grow without overloading the central host processing system.
The implementation of edge processors into network equipment yields significant benefits in several areas. First of all and most importantly, the bandwidth of the network equipment is extended so that the bottleneck in the overall network is reduced. That alone makes rethinking the network switch/router worthwhile. But, there are other important benefits gained by putting more distributed communications processing power in the box. One is much greater flexibility. As new techniques and protocols are adopted in the marketplace, network switching equipment can be upgraded via software to adapt.
With the task of data link processing shifted to the software running on an edge processor, supporting new technologies, or mixed protocols or new TDM techniques becomes simply a matter of software coding versus creating new high speed and dedicated ASIC devices. Creating software is far faster and less expensive than trying to develop custom ASICs and integrate them into an existing communications processing hierarchy.
Making the support of new system features software dependent also results greatly reduced time to market. Once the edge processor element is embedded in the architecture, the development time of new systems is shifted from long cycles of development and testing to much faster cycles of software coding and debug. <
The massive increase in data traffic driven by network expansion, greater popularity of the Internet and new data intensive multimedia applications are overwhelming the ability of network equipment systems to provide switching and routing functions at adequate speeds. The greater utilization of fiber optics technologies and use of virtual channel techniques that use mixed protocols over fractional data links to increase bandwidth intensify this problem. These increased processing requirements in the switching and routing equipment can be met by the repartitioning of the system architecture to place high performance distributed communications processing elements, called edge processors into the switch or router. These high-speed programmable units can offload the host system processor and increase bandwidth dramatically by placing more intelligence, closer to the data link itself.