Play among clouds (II) - network environment problems

If we regard the computing, storage, network and personnel required by game operation as resources, network is definitely the core resource for game operation. Because computing or storage resources are of low costs and are available in many means, but network resources, especially high-quality network resources, are quite valuable. In China in particular, the policy restrictions and various network operators make high-quality network resources hard to find. For traditional internet applications, such as web page visits, a delay or a temporary interruption is not a big deal - you can just reload the page. However, things are different for online games, especially real-time online games. A player may be shot in the head because of tens of milliseconds of delay, and top-notch gear may be looted by others because of one second of interruption. The network quality, especially the network speed, not only determines the survival or death of a player in the game, but also seals the fate of a game. This is because the network has a great impact on the user experience.  

I remember several years ago, many online games required you to select the server before login: China Netcom server, Telecom server or dual-line server. This phenomenon was caused by restrictions of history policies . The interconnectivity between different operators was not only highly costly, but also slow in speed, and cross-operator networks were extremely unstable. To secure better user experience and save costs, many games divide servers by operator network. This can definitely ensure the network speed, but a problem arises: we usually play games together with friends who may have network connections provided by various operators. Just imagine when you and your teammates are teaming up to do a raid, and the boss is going to die soon, you get stuck, leading to the total party kill (TPK). What else can you say except the frustration?

So it is not exaggerating to say the network may make or break a game. Many games do not require a powerful CPU, a big memory or a high I/O, but they require a good network. For all online games, whether real-time or turn-based, the network quality is the top element. Given the significant importance of the network for games, let’s talk about the difference of networks in the cloud age from that in the past. In the past, the management of gaming hardware and network falls broadly into two forms: moneyed companies could build their own server rooms and lease network lines. But such cases were few because such companies should acquire the qualification, and are financially powerful. This solution is not realistic for a majority of companies, except for some tycoons. The other common solution is IDC hosting which is adopted by most companies.

Network speed first. Most of today’s IDCs adopt two lines (Telecom and Unicom) and the network speed is also related with the IDC scale. Large IDCs may enjoy a good network speed because of sufficient bandwidth resources, while small and medium-sized IDCs usually offer a good performance in one network, and a barely-satisfactory performance in the other due to cost and scale reasons. However, dual-line connection alone is not enough, and there is also the education network among other networks domestically. Now many IDCs claim they adopt a BGP network, but truly reliable BGP networks are few and very expensive. Here can we briefly introduce BGP. BGP is short for Border Gateway Protocol, and is a route selection protocol used to connect independent systems on the internet. BGP is primarily used for the connection between internet AS (autonomous system). The main function of BGP is to control the route propagation and select the best routes. China Netcom, China Telecom, China Railcom and some large private-owned IDC operators all have AS numbers. Most of major network operators in China achieve multi-line connection through the BGP and their AS numbers. To achieve multi-line interconnection using this solution, the IDC should get the AS number from CNNIC (China Internet Network Information Center) or APNIC (Asia-Pacific Network Information Center), and then broadcast this IP segment through BGP to the networks of other operators. After a network gets connected through the BGP, all the backbone routing devices of the network operator will determine the optimal route to the IP segment of the IDC to ensure high-speed access of users of different network operators. From this piece of introduction, we can see the high threshold required to implement the BGP network. Even if you have implemented the BGP connection, you need to invest a huge amount to ensure the supplied bandwidth and stability. But because of the special application and extensive user base of online games, BGP network is the top choice when cost is not one of the considerations.

Cost-wise, IDC hosting charges are dominated by the network fees. The single-line network is much cheaper than the double-line connection. If you require a high bandwidth and speed, the cost will be prohibitively expensive, and the case will be worse for a BGP network. Not a few IDC hosting charges are collected by bandwidth, and other charges can even be included in the bandwidth. It is thus not surprising that most of your annual revenue has been spent on bandwidth.  

Management. IDCs vary from each other in network or server room stability level because of the gap in their strengths and management levels. A well-managed IDC will have fewer failures in a year, while the opposite may suffer network interruptions or issues every two or three days, and even frequent occurrences of backup power supply faults in case of power-down. A more offbeat case is that a company found it could not connect to the server hosted in an IDC. They finally located the cause being the cleaner carelessly kicking out the power cord of the cabinet. This is definitely not the responsibility of the cleaner, but because of the absence of management and operation regulations of the data center. It is a pity that too many domestic data centers do remain in the server room level, namely they just offer the room for placing the machines, with no standard management, not even to mention information security. It is a frequent scene that the whole IDC network gets paralyzed because a server in the same server room is under attack.  

Last, flexibility. The bandwidth services of IDC server rooms are usually offered at a fixed price and discount as agreed in advance, in the unit of half a year or year. It is not that convenient to increase or decrease the bandwidth specification at will. But in the mobile internet age today, the user volume of online games often surges all of a sudden. If the bandwidth cannot be adjusted flexibly, it is hard to guarantee a sound user experience. Of course you can pre-configure your bandwidth according to the maximum visit traffic, but the rise in cost will be remarkable.

In the cloud era, will these network problems faced by traditional IDC hosting be different? If a cloud service provider tells you they offer double-line network connection, you can just pass them over. Because in the cloud era, BGP network is the standard configuration. Cloud means users may come from across the country or even the world. Without a sound network condition, cloud services will just be empty talk. Another advantage of BGP network is its ease of use. You don’t need to configure multiple IP addresses for the same server to connect to operator networks that are seldom used. Instead, one IP address is enough to ensure the connectivity of all networks. Of course, the requirement on the cloud service providers will be quite high for them to meet these conditions. First of all, the cloud service providers must be large enough in scale, because only a large scale can ensure cloud service providers to get high-quality network resources to ensure bandwidth quality. Second, they should possess technical strength, and a very strong technical and O&M team to manage clusters with tens of thousands of physical servers. In addition, the data centers must also be available in major cities nationwide, and it will be better if they have overseas nodes, such as Hong Kong and Southeast Asia. Now a lot of domestic game companies have set foot in Asian markets, so a good cloud service provider must also have a certain number of overseas data nodes. With regard to the cost, it is just because of the scaled effect of cloud service providers that it is possible for them to coordinate various operators in the country, to get high-quality network resources at a lower cost, further cutting down the cost. The bandwidth prices offered by many cloud service providers are quite attractive.

Elasticity is the most important of course. You don’t have to predict the traffic and bandwidth after the game is launched on line. In fact, it is hard for you to predict it as well. Given the hard prediction, we can just drop it. On the cloud, bandwidth is just one of the resources subject to convenient allocation. Thanks to the flexibility of bandwidth adjustment, you can increase or decrease the bandwidth whenever you need. For example, you can increase the bandwidth temporarily during the heavy gaming period, or during 7 to 9 o’clock every day for doing raids, and decrease the bandwidth in other time periods. This not only guarantees the network quality in the peak periods, making users feel the fast network connection at any time. More importantly, it achieves on-demand usage and pay-as-you-go to make every cent of expenditure count.

Speaking of management, I believe all cloud service providers will announce and incorporate their server room management specifications and requirements as well as SLA in the contract. Management, however, differs with companies, because it doesn’t only require the procedures and specifications, but also technical strength, and especially the experience in operating large IDCs. Only with these skills can a powerful and stable service be established.