Hosted, Outsourced, and Cloud Data Centers
Considerations for Overall SLA's for Facility Owners and Hosting Providers
Hosted data centers, both outsourced/managed and collocation varieties, provide a unique benefit for some customers through capital savings, employee savings and in some cases an extension of in-house expertise. Traditionally, these facilities were thought of as more SME (Small to Medium Enterprise) customers. However, many Global 500 companies have primary, secondary or ancillary data centers in outsourced locations. Likewise, collocation data centers are becoming increasingly popular for application hosting such as web hosting and SaaS (Software as a Service), Infrastructure as a Service (IaaS), Platform as a Service (PaaS) in Cloud computing. These models allow multiple customers to share redundant telecommunications services and facilities while their equipment is colocated in a space provided by their service provider. In-house bandwidth may be freed up at a company's primary site for other corporate applications.
Considerations for the Hosted/Cloud Facilities Owner
The challenges for a hosted or cloud facility owner are similar to the user considerations mentioned above, but for different reasons. While most facilities are built with the expectation of full occupancy, the reconfiguration of occupancy due to attrition and customer changes can present the owner with unique challenges. The dynamic nature of a tenant-based data center exacerbates problems such as cable abatement (removal of abandoned cable), increasing power demand and cooling issues.
Data centers that have been in operation for several years have seen power bills increase and cooling needs change - all under fixed contract pricing with their end-user, tenant customers. The dynamic nature of the raised floor area from one tenant to the next compounds issues. Some collocation owners signed fixed long-term contracts and find themselves trying to recoup revenue shortfalls from one cage by adjusting new tenant contracts. Renegotiating contracts carries some risk and may lead to termination of a long-term contract.
Contracts that are based on power per square foot plus a per square foot lease fee are the least effective if the power number is based on average wattage and the contract does not have inflationary clauses to cover rising electricity costs. Power usage metering can be written into contracts, however in some areas this requires special permission from either the power company or governing regulatory committees as it may be deemed as reselling power. As environmental considerations gain momentum, additional focus is being placed on data centers that use alternative energy sources such as wind and solar.
There are however, additional sources of revenue for owners that have traditionally been overlooked. These include packets passed, credits for power saving measures within tenant cages, lease of physical cabinets and cabling (both of which can be reused from one tenant to the next) and monitoring of physical cabling changes for compliance and/or security along with traditional network monitoring.
For new spaces, a collocation owner can greatly mitigate issues over time with proper space planning. By having at least one area of preconfigured cages (cabinets and preinstalled cabling), the dynamic nature in that area and the resulting problems are diminished. This allows a center to better control airflow. Cabling can be leased as part of the area along with the cabinets, switch ports, etc. This allows the cabinets to be move-in ready for quicker occupancy. This rapidly deployed tenancy area will provide increased revenue as the space does not need to be reconfigured for each new tenant. This area can also be used by more transient short term tenants that need space while their new data center or redundant site is built.
If factory terminated and tested trunking cable assemblies aren't used, it is important to use quality cabling so that the cable plant does not impact Service Level Agreements (SLAs). Both TIA 942 and ISO 24764 recommend a minimum of category 6A/Class EA cabling. The minimum grade of fiber is OM3 for multimode. Singlemode is also acceptable for longer distances and may be used for shorter distances, although the singlemode electronics will be higher priced.
Owners must insist on quality installation companies if they allow tenants to manage their own cabling work. An owner may want to maintain a list of approved or certified installers. One bad installer in one cage can compromise other users throughout the facility. Approved installers provide the owner with an additional control over pathways and spaces. Further, owners want to insist on high performing standards-based and fully tested structured cabling systems within the backbone networks and cages. Higher performing systems can provide a technical and marketing advantage over other owners that use minimum compliant or outdated cabling infrastructures.
While collocation owners historically stop their services at the backbone, distributed switching via a centralized cabling plant and patching area can provide significant power savings through lower switch counts, enhanced pathway control and decreased risk of downtime during reconfigurations. All the while, the additional network distribution services provide increased revenue for the collocation owner. Managed and leased cabling ports can be an additional revenue stream.
Understanding that some tenants will have specific requirements, a combination of preconfigured and nonpreconfigured cages may be required. For more dynamic non-preconfigured areas, trunking assemblies, which are factory terminated and tested, allow the owner to offer various cabling performance options, such as category 6 or 6A UTP, 6A shielded or category 7A fully shielded, to best suit the end-user's needs. The owner can lease these high performing cabling channels and, on the greener side, the cabling can be reused from one tenant to the next, eliminating on site waste and promoting recycling.
Whether pre-cabled or cabled upon move in, owner leased or customer installed, category 6A or higher copper and OM3/OM4 fiber or better should be used. Higher performing cabling conforms to the minimum recommended standards, allows for higher speed applications while providing backwards compatibility to lower speed technologies. Category 6A/Class EA, 7/Class F and 7A/Class FA allow short reach (lower power mode) for 10GBASE-T communications under 30m for an additional power savings to the owner. Category 7/7A and class F/FA also provides the most noise immunity and meets strict government TEMPEST/EMSEC emissions tests, meaning they are suitable for use in highly classified networks alongside fiber. Installing the highest performing cabling up front will result in longer cabling lifecycles thus reducing the total cost of ownership and maximizing return on investment.
For non-configured areas, the backbone can be distributed into zones. The zone distribution area can be connected to pods or modular units within a space. This modular approach allows customers to move equipment into their areas one pod at a time. Preterminated copper and fiber trunking cables are easily configured to known lengths allowing the location owner to have stock on hand for rapid deployment of customer areas. These trunks can be reused and leased from tenant to tenant increasing revenue and enabling near instant occupation.
Facility owners are typically under some type of SLA requirements. SLA's can be for performance, uptime, and services. There are some network errors that are caused by poorly performing or underperforming cabling plants. Selecting high performing quality cabling solutions is only partial protection. The quality of the installation company is key for pathways, spaces, performance and error free operation. Cabling has historically been an afterthought or deemed to be the tenant's decision. By taking control of the cabling in hosted spaces, the building owner removes the cabling issues that can cause SLA violations, pathway problems, and ensure proper recycling of obsolete cabling.
While network monitoring can pinpoint ports that cause bit errors and retransmission, determining if the cause is cabling related can be difficult. Noise is harder to troubleshoot as it is intermittent. Testing the cable requires that a circuit is down for the period of testing, but may be necessary when SLAs are in dispute. While intermittent retransmissions are relatively benign in normal data retrieval, poorly performing cabling can make this intermittent issue more constant. This can slow down transmissions, or in the case of voice and video, can become audible and visible. In short, cabling is roughly 3-5% of the overall network spend, but that 3-5% can keep the remaining 95-97% from functioning properly and efficiently.
Modularized Deployment for the Collocation/Hosted Facilities Owner
Hosted and collocation facilities lend themselves well to modular POD-type scalable build outs. It is rare that these centers are built with full occupancy on day one unless there is a sizeable anchor tenant/tenants. Spatial planning for tenant considerations can sometimes be problematic due to varied size, power and rack space required by customers. These facilities are generally an open floor plan to start. Configuring spaces in a cookie cutter manner allows the owner to divide space in parcels while addressing hot/cold aisle requirements, cabling, and most importantly scalability and versatility within the floor plan space. In a typical scenario, the space is allocated based on cage layouts. The rows can be further subdivided for smaller tenants, or cage walls can be removed for larger tenants. If racks are populated with cabling, then the cabling can be leased to tenants.
Cloud facilities are generally highly occupied day one. A modularized design approach in these environments allows rows of cabinets to be deployed in a cookie cutter fashion. A structured cabling system that is pre-configured within cabinets, or ready for connection to banks of cabinets allows the owner to have a highly agile design that accommodates a wide variety of equipment changes without the need to run additional cabling channels in the future. There are two ways to deploy a modularized cloud or collocation data center. The first entails pre-cabling cabinets and rows to a centralized patching area. The second involves pre-cabling to zones within the data center. Once the zones are cabled, the addition of rows of cabinets within the zone becomes a matter of moving in the new populated cabinets, and connecting them via patch cords to the zone cabling distribution area. One common complaint with high density centers, such as clouds, is that equipment is often moved in with little to no notice. By pre-cabling the data center to a centralized patching area or to zones, the reactionary and often expensive last minute rush is eliminated.
If a centralized patching area is used, equipment changes become a patch cord or fiber jumper change, allowing rapid deployment. In a central patching (any to all) configuration, copper and/or fiber patch panels are provided in the central patching area that corresponds to patch panels in each cabinet. Connections to switching, servers, SAN, etc., are achieved via patch cords rather than having to run new channels as new cabinets are deployed.
The Need for Space Planning
One historical problem in open non-configured spaces has been the varied customer configuration requirements and the need to fit as many customers into the floor space as possible. As with any data center, growth without planning can cause serious issues in a collocation/shared space. One cage's equipment running perpendicular to another cage can cause undesirable hot air to be introduced into cold aisle of adjacent spaces. Haphazard and inconsistent cabling practices can block air flow. Improper use of perforated tiles can cause loss of static pressure at the far sides of the space. In short, in a hosted space that is not properly planned, problems can arise quickly. For space planning, an owner typically defines zones within the open space. Due to deeper equipment, a minimum of 3 feet (800 mm) should be allowed in all aisles, or slider cage doors should be installed that will provide full access. If that is not possible, deeper equipment should be housed in the cabinets in front of the sliding doors so that cage walls don't block access. A facility owned and operated cage can provide facility wide networking, monitoring and connectivity services to other cages via preconfigured, pre-cabled, cabinets allowing servers to be moved in and plugged in on demand. The cabinets and networking services become part of the tenant lease.
To allow for a variety of customer size requirements, a set of caged areas can be provided with 2-4 preconfigured cabinets for smaller tenants. By preplanning the spaces, cages do not need to move, pathways and spaces are predefined and airflow can be optimized in hot/cold aisles. In reality, there may be tenants that move into one of these areas that do not need to fill the cabinets provided. Some facilities allow for subleasing within cages. This allows underutilized cabinets to be occupied by another tenant as long as access to the area is supervised and cabinets have segmented security access via different combinations and/or key locks. Even in a tenant designed space it is common for a cabinet or partial cabinet to go unused. The benefit over time in pre-configured areas is that the floor will remain unchanged from one tenant to the next.
Another area with 8-10 cabinets is preconfigured for medium size tenants. And another section/area is left blank for those tenants that require their own configuration. The layout of that area should be completed by the building owner to assure that hot aisle/cold aisle planning is consistent throughout the floor area.
Figure 1 – Sample space plan
In the sample space plan above, we see caged areas of various sizes. Cage walls are static, cabling is centralized, and air flow is optimized. By providing varied numbers of cabinets within each cage, the floor plan can accommodate a variety of tenants. Tenants can occupy one or more cages depending on needs. For smaller tenants, individual cabinets or smaller spaces can be leased providing room for growth. The static cage configuration provides a significant cost savings over time. Centralized patching may be provided for the entire floor or in each zone with connections to core services. This keeps cable lengths shorter, less expensive, and easier to manage.
The above plan takes advantage of Siemon's VersaPOD cabinet line. The VersaPOD is available with a variety of integrated ZERO U vertical patch panels (VPP) for support of copper and fiber patching. The VPP's supply up to 12U of patching and cable management in the front and/or rear vertical space between two bayed cabinets without consuming critical horizontal mounting space. By utilizing the vertical space adjacent to the vertical mounting rails, the VPP's provides ideal patching proximity to active equipment, minimizing patch cord runs and slack congestion. Zero-U vertical patching areas can also be used to mount PDU's to service the equipment mounted in the adjacent 45 U of horizontal mounting space. This increases versatility and eliminates cabling obstructions and swing arms within equipment areas which can block air flow from the equipment. The ZERO-U patching and cable management channels further free up horizontal rack mount space and provides better managed and controlled pathways.
Figure 2: Swing arm cable managers vs.
VersaPOD Zero-U vertical cable management
The highly perforated (71%) doors allow greater airflow into equipment whether it be from an underfloor system or if cooling is supplemented by an in row cooling unit. To increase heat egress, optional fans can be installed in the top of the cabinets.
Cabinets in all areas should be outfitted with blanking panels that can be removed/moved as equipment is installed. An overall cooling plan must include intra-cage support. Blanking panels can have a significant impact on cooling expenses. Likewise, brush guards where cabling penetrations pass through floor tiles can help to maintain static pressure under the raised floor.
IIM (Intelligent Infrastructure Management)
By using a central patching area or zone patching areas, Intelligent Infrastructure Management can be deployed in a very cost effective manner. It is understood that the equipment that moves in and out of cabinets will vary over time regardless if there is one continuous tenant or several changing tenants.
The connections in the central patching area are monitored dynamically and in real time by analyzers that monitor continuity via a 9th pin on the patch cords and fiber jumpers. Because the software can see the equipment at the end of each channel via SNMP, it really doesn't matter what that the equipment is or if it changes.
Using Cross Connections in a Central patching area eliminates the need for sensor strips that attach to active equipment in each cabinet. Without a cross connect, sensor strips must be replaced as equipment changes either due to failure, upgrade, replacement or new deployment. As new equipment is introduced into the market, there may be a void in time between equipment deployment and the corresponding sensor strip being available.
Figure 3: IIM in cross-connect configuration
With IIM, moves, adds and changes are logged for date and time (necessary for most compliance requirements), and can be accompanied by photographs of the person making the change if the central patching area/zone is outfitted with either cameras or video equipment. For companies that have requirements for HIPAA, Sox, CFR-11, and other data protection laws, this audit trail maintains networking documentation.
For the facility owner, this software will also allow visibility into switch ports that are patched but not passing traffic. This enables better asset/port utilization reducing the need to add equipment and the resulting additional power consumption. Because the cabling channel is added to overall troubleshooting, it becomes much easier to identify and locate equipment for repair. The faster reaction times for troubleshooting can increase SLA performance while providing necessary audit trails. A premium may also be charged for Intelligent Infrastructure monitoring.
As more companies look towards collocation/hosted and managed services for their primary and backup data center, there are considerations both short and long term when selecting a provider. For the provider, there are lessons to be learned from prior poor practices that can lead to increased profitability and revenues. As this market grows continually more competitive, customers will demand long-term planning to assure that their equipment and service is at least as good, if not better, than what they could provide in-house. Higher performing cabling and design will differentiate hosting providers, monitoring capabilities provide a benefit to both the owner and the tenant. The lines between the various types of providers and services are becoming blurred, giving end-users increasing numbers of options for services and space.
For detailed configuration assistance, aids for planning moves, and further discussion of these topics, contact Siemon Data Center Services or a local sales representative or Ask Siemon.
- Podcast Audio - Overview of the TIA 942 Data Center Standard
- Siemon's Carrie Higbie presents an overview of the TIA 942 Data Center standard released in April 2005. The standard is designed for anyone building out a data center infrastructure. (22:39 mins - 17 Aug 2005)
Rev. A 4/10