How To Overcome The Data Center Challenges of High Performance Compute

How colocation helps customers focus on maximizing performance when leveraging high-performance computing environments or high density application.

As power densities increase, colocation will be a better option than on-premises. It’s no secret that compute density is increasing in data centers. A combination of powerful new processors and increased computing demand is requiring more and more power be pumped into the same space older equipment was housed. That power may be limited in some markets.

Server racks have been steadily increasing in intensity for some time, but it has accelerated in recent years. Racks that used to require 1-3kW of power are now consuming anywhere from 5-10kW and up. The latest generation of processors from leading chip manufacturers offers tremendous performance gains over prior chip generations, but the tradeoff is a considerable increase in power demands. New CPUs now draw upwards of 300 W; just a few years ago, only the very top server CPUs required more than 200 W of power. The latest GPUs are even more demanding, requiring up to 700 W of power.

Add to the equipment advances, the latest technologies also require exponentially more compute to process extremely large data sets. As AI becomes more widely accepted, used, and deployed, that means more processing power is needed. Processing and AI models can take weeks of running dozens or even hundreds of processors at full utilization. For typical server activities, such as file and print, database, and line of business apps, the CPU isn’t constantly taxed; it comes in bursts of use.

Finally, there are the challenges posed by updating an existing on-premise data center. The facility may have been designed and constructed a decade ago when power requirements were much lower. Now you need twice as much power as before, and you don’t have the capacity or the ability to handle it.

Some regions are hitting power caps, affecting businesses in areas where power is at a premium, such as dense urban locations or Tier I markets. So you may go to the power company and say you need 10 more megawatts of power for your data center, only to be waitlisted for several years.

All of these reasons are why businesses can benefit from outsourcing data center management and modernization to a colocation provider.

Data Center Providers Are Advancing Technology To Support High-Performance Compute Where In-House Data Center May Fall Short

The pressure and responsibility of managing the company data center falls to IT. Too often IT is focused on keeping the servers running and network data flowing. HVAC and electrical issues may not be their specialty, nor should they be. Colocation allows the IT department to focus on the IT and leave the facilities management up to their colocation provider.

For businesses that have run their own data centers for several decades, there are increasing challenges associated with operations beyond the pressure to modernize for high density computing. These can include adapting to a decentralized hybrid IT strategy, delivering more services and network performance at their edge, and increasing sustainability expectations that further tax the skills and operational efficiency of on-premise solutions.

IDC estimates an 18% to 20% savings over the course of a three-year data center lifespan when moving to a Colo scenario with the exact same hardware as on-prem. There are many variables in determining the savings, of course, such as the power density of the equipment, cooling required, and any custom or specialty hardware. But a 20% reduction in costs certainly has its appeal.

Always remember that just because your hardware is sitting in a third-party provider’s facility, off-site from your corporate headquarters, you still have total access and multi-layered security protection. That’s part of the contract. You are free to stop by to do maintenance on your hardware that can’t be done remotely. So your hardware will always be within reach.

Traditional Air-Cooling Technology May Not Be Sufficient to Meet Increasing Data Center Densities

One of the major considerations for colocation stems from the increase in rack densities. At some point, air cooling is simply not viable. There is too much heat and the fans can’t keep up with the heat buildup. The conventional wisdom is that air cooling ceases to be effective when you go over 30 kW per rack.

That’s where liquid-to-the-chip cooling comes in. Chilled water is pumped into a plate seated on top of the CPU to absorb heat and then the water is pumped a way to be recycled, re-cooled, and reapplied to the server. Recycled. For high-density scenarios, liquid cooling is more efficient. Water has 3,000 times the cooling efficiency of air and once you go beyond a certain threshold, like 50 kW and up, air cooling is no longer viable and liquid cooling becomes mandatory.

Liquid cooling has two challenges: first, you must design it into your environment. The piping and cooling of the water are not easily retrofitted into an existing data center. An entire rip and replace might be needed which would be prohibitively expensive.

To put it another way, you may want to have 50 kW racks in your data center for AI training. Still, you may not be able to do it because retrofitting a data center for liquid cooling is neither trivial or something to be done piecemeal. It usually requires an entire infrastructure retrofit, rebuilding the data center with liquid cooling in mind.

Second, the skills required to deploy a liquid-cooled system are not very common. Liquid cooling is about plumbing, not server management. That means expensive consulting services on top of the retrofit to modernize and manage the new facility and systems.

Liquid-to-the-chip and innovative immersion cooling trends may tax enterprises and businesses that run data centers in-house. However, purpose-built data centers can support the next-generation cloud, enterprise, and IT infrastructure required to support High-Performance Compute (HPC) and higher-density deployments. Higher-performing data center facilities are vital in maximizing the economics around more power, the best use of space, and increasingly dense footprints.

Colocation Brings Several More Benefits

There are other obvious benefits to colocation, which include:

• Offloading the management of the facility: This benefit is obvious. You no longer have to worry about maintaining and managing your data center facilities and can focus on just managing your computing environment.

• Simplified scaling: When a company looks to expand its on-premise infrastructure, it will likely need new space for the equipment and upgraded electrical and cooling facilities. This can take months. With colocation, space can be scaled rapidly as infrastructure is already built and in place to accommodate customer growth or transitions.
• Increased bandwidth: Big colocation providers often have very fat network pipes to accommodate many customers. That can mean having much more bandwidth than you had in your data center.
• Easy connection with cloud providers: Some colocation providers have direct, private connections with the major cloud service providers and help to connect customers to the CSP for a hybrid cloud scenario. So rather than striking a deal with AWS et. al., on your own, the colo provider offers direct cloud on-ramps to the CSP at more affordable rates for transferring data.
• Better security: Because of their valuable contents, both in hardware, software, and data, colocation has multiple layers of the most stringent security and continuous monitoring of the facilities
• System stay put: If your company is growing and you need to move to new offices, congratulations. That will be easy, but at least you won’t have to worry about packing up and moving your data center since all your equipment lies safely in colocation.
• Assistance with Corporate Compliance: If you are in a heavily regulated industry with complex corporate compliance standards, you know the challenge of meeting that compliance with your IT systems. Colocation providers take much of that worry away because they handle the security and data protection compliance. You’ll find that many data centers have these compliance measures baked into daily operations.
• Environmentally-Friendly: There is increasing pressure on companies to increase their sustainability efforts. Data centers are under increased scrutiny because of their notorious power consumption. Data center providers keep green policies top of mind and often use it as a selling point. Sustainability means efficiency, so it benefits the data center provider as much as the customer.

Data centers, in general, and HPC, in particular, require two separate disciplines, computing and infrastructure management. The computing part is where you make your money. The infrastructure part costs you money and demands skills separate from computing.

Element Critical can spare you the burden of deploying and managing your High Performance Compute (HPC) systems and let you focus on running your computation. We create customizable space and connect businesses with our partner ecosystem to help businesses achieve HPC environments. Book a tour of our facilities and talk to a solutions architect about the time and expense savings possible by letting us host your HPC systems. Download our HPC guide here.