Content area
Utility computing - hailed as the biggest paradigm shift since the first disk drive spun up -- has stalled. One basic definition defines utility as an on-demand computing resource. On-demand computing allows companies to outsource significant portions of their datacenters, and even ratchet resource requirements up and down quickly and easily depending on need. The on-demand version of utility computing is the one closest to fruition. This year, Sun Microsystems has been the noisiest of the bunch, recently announcing that it wants to be the electric company of offsite computing cycles. The grand concept of utility computing is a solution for three key problems: 1. wasteful technology purchases, 2. unnecessarily laborious IT processes, and 3. rigid IT capabilities that by definition paralyze business processes. Grids provide a perfect entry into the utility-computing space because they follow the golden rule of offering more for less: namely, the power of a supercomputer for the price of a few workstations.
Full text
PARADIGM SHIFTS WERE EASIER BEFORE THE BUBBLE BURST. SERIOUS change costs serious money, and few IT organizations have gobs of green stuff to throw around anymore. So it's no surprise that utility computing - hailed as the biggest paradigm shift since the first disk drive spun up - has stalled.
It doesn't help that the marketing geniuses who came up with the concept still can't agree on what it means. There are three basic definitions.
Utility as an on-demand computing resource: Also called "adaptive computing," depending on which analyst or vendor you talk to, on-demand computing allows companies to outsource significant portions of their datacenters, and even ratchet resource requirements up and down quickly and easily depending on need. For those of us with gray whiskers in our beards, it's easiest to think of it as very smart, flexible hosting.
Utility as the organic datacenter: This is the pinnacle of utility computing and refers to a new architecture that employs a variety of technologies to enable datacenters to respond immediately to business needs, market changes, or customer requirements. Datacenters not only respond immediately, but nearly effortlessly, requiring significantly less IT staff than traditional datacenter designs.
Utility as grid computing, virtualization, or smart clusters: This is just one example of a spécifie technology designed to enable the above definitions. Other technologies that will play here include utility storage, private high-speed WAN connections, local CPU interconnect technologies (such as InfiniBand), blade servers, and more.
These three descriptions are different enough to seem unrelated, but in fact they're dependent on each other for survival. Should utility computing ever live up to its name - a resource you plug in to, as you would the electric power grid - then that resource needs to be distributed, self-managing, and virtualized. Whether that grand vision will ever be realized is an open question, but at leaast some of the enabling technologies are already here or on the horizon.
The On-Demand Adaptive Buzzword Enterprise
The on-demand version of utility computing is the one closest to fruition. Vendors such as Dell, EMC, HewlettPackard, IBM, and Sun have been selling it for some time. This year Sun Microsystems has been the noisiest of the bunch, recently announcing that it wants to be the electric company of offsite computing cycles.
"Sun has decided to take utility to a whole new level," says Aisling MacRunnels, Sun Microsystems' vice president of marketing for utility computing. "We're building the Sun Grid to be easy to use, scalable, and governed by metered pricing. We're also incorporating a multitenant model that allows us to provide a different scale of economy by pushing spare CPU cycles to other customers."
The Sun Grid is comprised of several regional computing centers (six throughout the United States, so far), each running an increasing number of computing clusters based on Sun's Nl Grid technology. Sun allowed us to visit its secaucus, N.J., Regional Center, which is supplying Grid resources to several Wall Street customers for complex financial modeling. Sun's centers boast rack after rack of 32-node compute clusters based on its SunFire V20z Opteron-based servers, interconnected using Infini-Band (via Topspin), and managed from master consoles located at central corporate sites rather than at the Regional Centers themselves.
Sun wants to cut through utility computing's confusion and attract customers. Hence the ultrasimple pricing scheme: "$1 per CPU per hour," MacRunnels says proudly, "with a four-hour minimum, which will probably drop over time as operations become more efficient."
Yet Sun also has a few chasms to cross, which is why the Sun Grid still isn't commercially available. "The goal once we're out there is to be able to give additional CPU resources to our customers immediately," MacRunnels says. "That's a big challenge for us. Right now we know we're not yet commercially viable, which is why we're only chasing specific application markets. We need to walk before we run."
Charles King, president and principal analyst at market research firm Fund-It, has a rather cynical take on Sun's offering. "What Sun is selling isn't really new; it's been offered by IBM and HP for several years. Sun has simply gotten more specific and done what they do very well, which is simplify something highly complex with a great marketing slogan."
Most analysts agree that IBM leads the field in offering utility-based services to clients of its On Demand and Global Services departments. "Other companies are wrapped up in the whole notion of access to compute power," states Dave Turek, vice president of deep computing at IBM. "But computing power comes in many forms, including not just grids and virtualization, but also more standard forms of hosting. It depends entirely on customer needs, and these change quickly."
According to Turek, IBM's On Demand service is all about providing solutions tailored to individual requirements. "Utility should be a base kind of service just like water or electricity. But where those services are rigid, On Demand's intrinsic value needs to be wrapped up in customer need, and that means exceptional flexibility."
HP agrees, having coined its service name as the Adaptive Enterprise, but touting the same organic message requiring IT infrastructure that responds to changing business requirements. "We've made an announcement on our grid strategy," says Russ Daniels, vice president and CTO of HP's Software and Adaptive Enterprise unit, "but that's really a specialized application. We feel utility computing refers to technology applied to business process." Today, HP has customers accessing its resources for increased computing power similar to the Sun Grid, but like IBM, it also places consulting, traditional hosting, and even several on-site products under its utility umbrella.
The Attractions of Utility
Most customers understand the benefits of flexible hosting. But what of the organic, virtualized, self-managing datacenter - assuming it can be achieved? Forrester sees the grand concept of utility computing as a solution for three key problems: wasteful technology purchases, unnecessarily laborious IT processes, and rigid IT capabilities that by definition paralyze business processes. Nail those three, and you can get a lot more out of its existing resources. The initial investment in provisioning and virtualization eventually justifies itself by reducing capital expenditures, slowing the growth of IT staff, and providing the business with new agility.
Ultimately, a company could run multiple workloads on fewer machines in fewer datacenters, and accomplish this through the use of multisystem architectures such as blade-based systems, clusters, or grids. That's only one example, of course. Combining that hardware with a reduced number of platform architectures means faster processing, faster reaction time, and less staff training. Such consolidation isn't a plug-and-play decision, however, but a gradual process that involves evaluating every technology purchase.
"This is really customer-dependent," says Ken Knotts, senior technologist at ClearCube, a blade workstation and grid computing vendor. ClearCube (infoworld.com/2527) is an excellent example of a utility-oriented product offering, because the company manufactures a blade-based workstation system. By pulling workstations back onto a central blade backplane, ClearCube's utility-style blade system is in a position to meet a variety of challenges that traditional workstations can't easily handle.
"Because we can reprovision a blade from scratch, drop a user's personal data and settings on it within 10 minutes or less, we're in a position to save customers loads of money on large IT support staffs," Knotts says. The company can also extend its functionality across the WAN. One customer uses the ClearCube system on a LAN during the day for U.S. developers and then opens those workstations at night to developers in India. "Not only is he saving money," Knotts says, "but he also doesn't have to worry about his code being stolen because none of the data is in India anyway."
The ClearCube blade system can also be converted into a grid computing system during off-hours using its fast reprovisioning capabilities coupled with a partnership with Data Synapse, whose GridServer Virtual Enterprise Edition amounts to a software layer that virtualizes application services and manages that process across distributed hardware systems. "To us, utility computing is about creating the interface between a computing device and what amounts to a floating datacenter," Knotts says. "In effect, you're clipping the cable between the user and the dedicated local datacenter. A big chunk of utility computing is about creating the technology that takes the place of that cable, and new technologies are bringing us very close to this goal."
Assembling the Grid In today's grid environments, a user application (1) makes calls to the grid management software (2), which then schedules resource allocation for the request (3). Manual adjustments can be made to this system using the master node console (4), which manages not only the grid management and scheduling systems but all provisioning required of the grid compute nodes (5). The underlying storage resource (6), may or may not be controlled by this system depending on vendor implementation.
Frank Gillett, an analyst at Forrester, emphasizes the business benefit. "Organic IT isn't just about IT being able to respond to business requirements," he says. "It's about doing that on the fly. And the technology you purchase has to manage that using standardization and automation to keep costs low." The utility computing services being offered by Sun, HP, IBM, and others are simply outsourced versions of this same concept.
Sun's MacRunnels doesn't think customers will view her product in traditional outsourcing terms at all in a few years. "We want it to be as easy as simply purchasing CPU cycles," she says. Sun claims it's talking to electronic trading exchange Archipelago about allowing Sun Grid customers to trade excess purchased CPU cycles to each other during down cycles.
Getting on the Grid
Grids provide a perfect entry into the utility-computing space because they follow the golden rule of offering more for less: namely, the power of a supercomputer for the price of a few workstations. They offer unheard of flexibility and they don't require you to rip out existing infrastructure. And these benefits extend to outsourcers as well as those running grids in-house.
Don Becker, CTO of Penguin Computing, a manufacturer of Linux-based grid solutions, offers a succinct definition of grid computing. "A grid cluster is a collection of independent machines connected together by a private network with a specific software layer on top," Becker explains. "This software layer has to make the entire cluster look like a single computing resource."
A master node controls a varying number of such processing nodes with the ultimate goal being that, to the operator of the master node, the entire ensemble looks like a single processing unit. The most common example of a grid in action is that of the suddenly stressed Web server.
"E-tailers, for example," Fund-It's King explains, "have 30 percent of their business happening between January and October and 70 percent occurring between October and December because of holiday sales." If the e-tailer is running a grid, the master node administrator can simply spawn off several more virtualizations of Apache in early October, and thus handle the additional traffic. Even better, he can do it all in a few minutes or even schedule it to happen automatically based on a performance policy.
Although the standards for hardware grid management are evolving rapidly, they're still missing a critical component. "One of the big challenges in running software in any grid environment amounts to reorganizing your software," says Brian Chee, a senior programmer on a 90-node utility cluster being built for the bioinformatics department at the University of Hawaii. "The problem needs to be divided up into chunks and assigned to each processing node, and the transfers of data and results needs to be organized synchronously or asynchronously. When you're linking two grids, the problem gets divided into two, sent to each grid, and is there again subdivided onto those grids. Results are reassembled the same way."
That means designing software for use on a grid is difficult and time-consuming because it requires a rewrite to comply with an MPI (message passing interface), the foundation of grid computing. "That's not something you can just bolt on," says David Aubrey, a software architectural consultant. "That's a ground-up rewrite and a very difficult one because of the communication issues."
By itself, this is reason enough for most enterprises to have ignored grid computing. During the past year or so, however, new toolkits - such as the one from the Globus Alliance (infoworld.com/2770) - have arrived to help this process.
"Toolkits like Globus," Chee says, "enable standard software to make calls to the grid using MPI." But although they take the headache out of grid data communications, porting applications to this model still isn't child's play.
"To work on a grid," Aubrey says, "the software still has to be multithreaded." This multithreaded architecture is outlined in the Posix 1003.1C standard posted by the IEEE. "Porting software to this model still means a good deal of work," Chee agrees, "but the MPI toolkits at least make it manageable. Before them, it may as well have been rocket science."
Aubrey points out that multithreaded application design has gotten easier thanks to the industry's concentration on Web services and SOAs. "SOA is by definition multithreaded," Aubrey says, "so managing the migration of these applications onto a utility paradigm is really becoming possible this year." The Globus Consortium, a group promoting the tools created by the Globus Alliance, announced its intent to extend the OASIS WS-RF (Web Services Resource Framework) to include specific features aimed at grid computing.
The Long Road Ahead
Grids are only one example of utility computing's technology challenges. Other areas that need work include storage, WAN issues, security, and compliance. "That's really unavoidable right now," HP's Daniels says, "since the ramifications of computing as a utility are so hugely complex."
Daniels goes on to cite HP's commitment to creating a utility-storage model. "After all," he says, "where's the data? Companies that need additional computing resources typically have large, even vast, quantities of data. That means for utility computing to be viable, you've got to have a working model for utility storage." HP has released several products and management initiatives aimed at providing a utility model for storage, but it has yet to tie any of them into a coherent utility-computing offering.
So how does IT plan for a migration to the utility model? "Start by understanding your application diversity," Penguin's Becker advises. "What runs on what? This is important, as you'll need a management solution that works for each platform." He also advises moving to a standard hardware platform, the Intel/AMD model being his favorite, for obvious reasons. "Finally, look to move to a single operating platform," he says. "Presently, Unix is the system of choice for all things utility, as you simply have more options under Unix than you do Windows."
Within this framework, begin evaluating all new technology purchases with utility goals in mind. "Don't just look at a single vendor's commitment to utility," King says. "Make sure that every vendor you work with from now on can support as much of your infrastructure as possible." Each technology player should be evaluated against a utility goal that reflects an organization's unique combination business needs.
"A real utility pioneer is Oracle," University of Hawaii's Chee says, "and its 10g database release, which is one of relatively few 'standard' enterprise applications available with utility features included in the form of specific support for grid computing." Oracle10g supports a feature called RAC (Real Application Clusters), which was first introduced in Oracle9i. RAC allows Oracle administrators to perform server virtualization and fast software provisioning in response to performance demands. It even performs these functions across pools of hardware resources, allowing new blades to be included or excluded as needed, with the whole pool behaving like a single computing resource.
Although software products such as Oracle 10g are still evolving, the hardware platforms are maturing rapidly. But even without specific software support, products such as Knotts' ClearCube have plenty of benefit to offer all by themselves, enabling IT managers to begin evaluating a move to a utility-based datacenter today.
"Sure, there are still important tools missing," Forrester's Gillett says. "But the cost benefits of this architecture are simply too compelling to ignore."
Copyright Infoworld Media Group May 23, 2005