In Chapter 1, you implemented a very simple way of server hosting. In this chapter, you go further and implement a more advanced solution with the help of PlayFab, as well as in Azure with Kubernetes and Agones. You will explore and learn what you need to create online multiplayer games.
First, you will review different server hosting alternatives. Then, using PlayFab’s multiplayer server hosting feature and the provided SDK, you will enable online multiplayer mode for your game.
In the second part of this chapter, you move on and build a server hosting environment in the cloud. Agones, on top of Kubernetes, will manage and allocate the servers for each online game session. You will learn how to build your own backend infrastructure.
Server Hosting Alternatives
Dedicated game server hosting means basically that you want your server to run somewhere in the cloud and your clients to have access to it, even if the clients are located all over the world. Cloud-hosted game servers allows players to play together, without being in the same location or on the same network.
Server hosting. The server must run on a machine somewhere on the Internet and be reachable by all clients.
Server management. Some logic must manage the lifecycle of game servers. For example, you want to allocate servers for specific players or shut down servers if the players finish the game. If you consider it in broader terms, scaling up or down the servers can also belong here. This implies that a single virtual machine will not be sufficient.
Three columns and three rows represent a relation between the game server, server management, and server hosting versus the application servers named IAAS, PAAS, and GBAAS.
Dedicated game server hosting in the cloud
The most basic and traditional way is when you have a (virtual) machine with public Internet access or pay for a hosting provider to host your server. You can also buy virtual machines from a cloud provider, then you decide on the Infrastructure as Service (IaaS) model. Here, you only have a machine with no capability to manage your game servers or even scale on demand. You need to write your custom logic to solve these problems, which can take considerable effort. You can see an example of this solution in Chapter 1.
The most convenient way is to let a Game Backend-as-a-Service (GBaaS) provider do everything. You use their API to access their services and you only need to worry about making your game. In this chapter, we implement this approach with the help of PlayFab.
In between, you find the Platform as a Service (PaaS) model. Azure Kuberentes Service (AKS) is a fully managed PaaS offering from Microsoft. Still, you have to configure it, which can be a daunting task. But it implements a lot of useful functionalities, for example, it scales automatically when demand grows. On top of this, you need the game server management functionality. You can use an existing production-proof solution, such as Agones. In this chapter, you implement this approach in Azure with the help of Kubernetes.
Now that you have understanding the various server hosting options, let’s start by exploring the GBaaS way.
PlayFab
PlayFab provides a convenient way for server hosting. You only need to configure PlayFab and use the PlayFab Game Server SDK (GSDK) to access the servers. PlayFab arranges all the management tasks. It scales the servers according to your need.
Configuring PlayFab
By default, PlayFab disables the multiplayer server feature. After you log in to PlayFab, you should enable it from the Multiplayer menu. If you have not yet added your credit card information, choose Add Credit Card.
PlayFab will not charge you until you reach the limit. Currently, it is 750 free core hours and 10GB of egress (outgoing) traffic. It is good enough to experiment with PlayFab.
Be aware that this means core hours. If you choose, for example, a two core virtual machine, which is the minimum, you only have 375 core hours. You can run out of the free tier limit easily and then PlayFab will charge you.
On this page, you can see the current pricing of PlayFab services: https://playfab.com/pricing/
After you submit your credit card information, you can enable the multiplayer server feature of PlayFab.
A game server is basically a process that runs on an operating system, which runs on a machine. Nowadays, we use the more flexible virtual machines (VMs) instead of real hardware. So you need a provider, which hosts the virtual machine and the server process on it. But what happens if thousands of players suddenly want to use your game?
You then need more virtual machines to serve the increasing demands, and an automatism which spins up additional VMs. You cannot know how many VMs you need in advance. Instead of saying you need a specific amount of VMs, you define how and when PlayFab should create VMs.
Virtual machine size. PlayFab currently provides Dasv4 (2-16 cores) VMs. This type comes from Azure’s virtual machines instances. The D-type is for running general-purpose workloads.
Servers per machine/server type. If you choose the Containers server type, PlayFab will generate as many containers on each virtual machine as many servers per machine you choose. Each container will have a different port number, but they will all map to the game port you define for Mirror in Unity (the default value is 7777).
Assets. These are files you upload to the server in compressed (ZIP) format. This should contain your server process, which will automatically run on the VM (as defined under the Start command).
Network. The port and protocol for incoming network traffic. If you choose the default Mirror settings, the port is 7777 and the protocol is UDP (KCP counts as UDP).
- Region. Currently the options are EastUS or NorthEurope. You define two important parameters here. These are the only parameters that you can also change later for your build:
StandingBy server. This is the number of servers in StandingBy state waiting to become active. These are running servers flagged as StandingBy because currently no player is using it. Starting up a server can take time, so PlayFab starts up these servers before players actually allocate them. You can also set the Dynamic Standby option later, which will provision additional StandingBy servers dynamically with the growing demand.
Max servers. The maximum number of servers in the chosen region. Obviously, the number of StandingBy servers must be smaller than or equal to the configured max servers.
Except for the number of StandingBy and Max servers, you cannot change any parameters of your build later.
To each build, you will get a BuildID. Figure 3-2 illustrates the key components of the multiplayer server hosting implementation in PlayFab. You can have multiple builds, and within each build, you have multiple regions. Each region can have multiple servers (virtual machines), and each server has containers where the server process runs.
Classification of multiplayer represents build ID, preferred location, host ID, session ID, and clients.
Multiplayer server hosting implementation
When first creating a new build, it will turn into the Initialized state. You need to increase the number of StandingBy servers, so that the build goes into the Deployed state. First, it goes through the Deploying state, checking if the server you created in Unity can turn into the StandingBy state. This happens only if you implemented the ReadyForPlayers method and call it periodically.
Configuring Unity
First, you need to download the PlayFab Game Server SDK (GSDK) for the Unity game engine. The simplest way to download the Unity package is from the following URL. Then import all the assets into your project:
https://github.com/PlayFab/gsdk/raw/main/UnityGsdk/MpsGsdk.unitypackage
In order to use it, you need to enable the ENABLE_PLAYFABSERVER_API directive in the Unity Editor. Go to File ➤ Build Settings…and select Dedicated Server and Switch Platform. Now go to Player Settings… ➤ Player ➤ Dedicated Server Settings ➤ Other Settings ➤ Scripting Define Symbols. Then add the directive to it and click Apply.
Without this directive, you cannot compile, because of the #if ENABLE_PLAYFABSERVER_API directive in the GSDK code.
Create two game objects, called PlayFabServer and PlayFabClient, and add them to your scene as a child of the PlayFab game object. Also create two scripts (such as PlayFabServer.cs and PlayFabClient.cs) which will contain the code. You should add these scripts to the respective game objects, so that Unity can run them.
Inherit the server and the client scripts from Mirror’s NetworkManager. Make sure that you have only one NetworkManager at one time on the scene.
You can put the client and server in separate projects if you want. This can be useful, especially if it is a bigger project. For this tutorial, we keep them in one project.
Implementing the Server
The server will then listen to a port for incoming remote client connections.
Here, you merely start a server and not a host. In case of a host, you would also start a client on the same machine.
Transport. By default, Mirror currently uses KCP transport, which claims to be the fastest and best suited protocol for game-related traffic. You can choose from many, but basically use TCP if reliability is important, and UDP if speed has precedence over losing data.
Network address. For servers, it is always the localhost.
Max. connections. The maximum number of allowed concurrent connections.
Disconnect inactive connections. When you want to remove players whose clients are not responding to the given Disconnect Inactive Timeout time period.
Although the Transport and the NetworkManager cannot live without each other, they have a separate component by design. Mirror wanted to separate the way of transport, so that you can change it any time and use the most suitable one.
Check your PlayFabServer in the Inspector. If your Transport field is None, drag-and-drop the Kcp Transport component into it.
You can change the transport parameters in Unity Inspector. For now, the most important parameter is the port number. It is the port your game server is listening on. By default, Mirror uses 7777. Be sure to avoid port conflicts; in most cases, the default value just works fine. Other parameters help optimize the performance of data transfer between the clients and the server.
Shutting Down the Server
Building the Server
Make sure you disable the NetworkManager and the PlayFabClient game objects. Drag-and-drop the PlayFabServer.cs script to the PlayFabServer game object. Add the player prefab (Slime) to it just as in case of the NetworkManager. Then, go to File ➤ Build Settings… and select Dedicated Server.
In Unity Hub, the Linux Dedicated Server Build Support must be installed.
Select a folder and generate your game server.
Testing Your Server
LocalMultiplayerAgent. Previously called MockVMAgent, which emulates the PlayFab environment on your local machine. With the VmAgent, the game server can go through the states you will also have in your real PlayFab deployment (https://github.com/PlayFab/MpsAgent).
Thundernetes. A newer initiative to debug your server locally, where you deploy your server to a customized Kubernetes cluster. The PlayFab team implemented custom resources for Kubernetes to manage the state transitions of your server (https://github.com/PlayFab/thundernetes).
PlayFab Connect. You can connect to your virtual machine directly through your PlayFab Game Manager. Here you use a remote desktop or SSL and go to your virtual machine directly. You can inspect each server and log with the help of simple command-line tools.
Unfortunately, PlayFab does not give you too much information when there is an issue with your server. For example, when it is hanging at deployment. It is a good idea to test it locally first. You will see how to use Thundernetes for local testing.
Thundernetes
You can create a Docker image, which includes an operating system and the game server files. Then, you’ll create running containers from this image when a new session starts. Now you just need something that automates this process. Kubernetes is a production-proof container orchestration system, which is exactly what you need.
Thundernetes runs on top of Kubernetes. This is an extra layer that implements the behavior you need for running game servers.
Docker Engine for containerizing:
Chocolately (on Windows) as package manager:
Kubernetes CLI to run commands against the cluster:
choco install kubernetes-cli
Minikube as a local Kubernetes implementation:
choco install minikube
Install Thundernetes on the cluster.
You can find more information about the installation steps of Thundernetes at https://playfab.github.io/thundernetes/quickstart/installing-thundernetes.html.
The logs should show a StandingBy state. Let’s allocate a server to see if the transition to the active state happens, and if the client receives a server IP of the allocated server.
If the server works on the local Thundernetes cluster, it should also work on PlayFab. You can now move on to deploy it to PlayFab.
Creating a Build in PlayFab
Now you have everything to create a build in PlayFab. Choose Multiplayer ➤ Builds ➤ New Build. Upload the earlier created docker image to the provided registry.
The user interface of My game studio represents servers including new build details of name, server details, container image, and assets.
Creating a new build in PlayFab
It will take a few minutes before PlayFab provisions your server and accepts client requests.
Troubleshooting Problems
If your server code has some issue, PlayFab cannot deploy the server, and it will go into the Unhealthy state. Here, you can only try to debug locally, as you cannot go to the server directly with a remote desktop.
Otherwise, if the deployment was successful, but the server still has some issue or you want to investigate the logs, you can use PlayFab Connect.
Choose PlayFab ➤ Multiplayer ➤ <your build name> ➤ Servers ➤ Connect, and RDP onto the machine. You should start a command prompt by searching for cmd on the server. Then right-click it and choose Run as Administrator.
Creating the Client in Unity
BuildId. This is the ID you get after creating the build in PlayFab’s Game Manager. You can find the ID under your build’s name (choose Multiplayer ➤ Server ➤ Builds).
PreferredRegions. This must fit to the region configured for your build (currently either EastUS or NorthEurope). You can have multiple regions configured for one build. With this parameter, you can route the player to the closest region to reduce latency.
SessionId. This can be an existing ID, which means the player joins an active game session. Or this can be a new ID, which means the player starts a new game where other players can join.
If the request was successful, the OnRequestMultiplayerServer method provides RequestMultiplayerServerResponse, which contains the IP address and port number of the server assigned to this session. You pass this information to the Mirror client, so that it can connect to the listening server.
Extending the ControlPanel
Testing
You can start your client from a Unity Editor instance and see if it joins the server automatically. Disable PlayFabServer and enable the PlayFabClient game object. Drag-and-drop your player prefab (Slime) to the Player Prefab field of PlayFabClient. When you start your game, don’t forget to authenticate the player first.
If you want to join to the same session, open the cloned project in another Unity Editor. Copy the session ID from PlayFabSettings in the original project so a second player can join to the same session. With matchmaking, you can simplify this step. The matchmaker will decide if you can join to an existing session or start a new one.
You can observe how the server changes from StandingBy to Active in PlayFab’s Game Manager, and also back to StandingBy when all the players leave.
Now that you have learned how to build and use multiplayer game servers in the backend in PlayFab, the next section shows you how to build the same functionally with Azure.
Azure
It is time to build your server in Azure and compare it to the GBaaS solution. You need a basic understanding of containerization, Docker, Kubernetes, and a general understanding of cloud computing and Azure, as well as Terraform. The earlier chapters help with in this, explaining the main concepts and commands to move forward.
The Problems
Basically, you want your Mirror server to run on a machine, listening on a port, to which you can connect with your clients. The clients only need an IP and a port number for the connection. In theory, you could simply put the Mirror server to a virtual machine in Azure and connect to it to the clients.
Every time a new player starts a new game, a new session will start. For additional game sessions, you need additional servers. Thus, you need an automatism to create new servers generated when the players start new game sessions. On the other hand, you need an automatism to deallocate the game server when the game finishes and the players leave the session. You need a mechanism to handle the transition of game server states, i.e. some kind of logic that manages your servers.
Your player base will increase (hopefully) with time. At the beginning, you might have only ten players, but that can go up to millions of players. The backend should automatically increase the number of servers, or even decrease when fewer player want to use it. This scaling up and down of the infrastructure should happen automatically.
If you go with pure Azure services, you would quickly encounter these problems. You can decide to implement game server management and scaling on your own. That’s a lot of work, and I would not suggest it for individuals or small development teams. Here is where Agones, on top of Kubernetes, can be a valid option.
Kubernetes and Agones
Agones is a free, open-source library to host, run, and scale dedicated game servers on top of Kubernetes. Kubernetes is the de facto standard for container orchestration used by many companies around the world. The problem with Kubernetes is that Google originally invented it to manage web-based workloads.
A web session is usually stateless, so a client’s request can go once to one server, the next time to another, and it will not disturb the user. In case of games, however, your client needs to communicate with the same server. Also, if the server goes down, the game session is over.
For a stateless web application, this would not be a problem. So Kubernetes could solve the scaling problem, but you need something specific for the game. That is where Agones helps. Agones will manage the state of your game servers throughout the complete lifecycle, from their creation to shutting them down.
With that, you can conclude that the combination of Kubernetes and Agones provides a powerful foundation to host your game servers. If you got to know Thundernetes earlier, you will notice the similarity between them.
Be aware that building a Kubernetes cluster with Agones is relatively easy, but more challenging than building it with a GBaaS solution, such as PlayFab. This chapter goes through each step, explaining what happens and why. For this solution, the developer should understand not only the game code but also the infrastructure code.
Building Agones
Nothing is easier than installing Agones on a Kubernetes (AKS) cluster. Download the following Terraform script:
The Terraform script will not run through completely at first. You can start debugging to determine why.
You don’t know what exactly you have deployed with the Terraform script. That can cost a lot of money. Using resources in the cloud costs money, so you have to be careful as to which services you use.
You want to understand the infrastructure (code) as much as you want to understand the application code. Otherwise, you cannot fix, extend, or operate it.
You should only deploy the minimum required infrastructure resources to minimize costs.
The solution is to build and learn your infrastructure step-by-step.
Three Steps to Build the Infrastructure
I suggest using Visual Studio Code with the related plugins and terminal window for developing the Terraform scripts and executing CLI commands.
To have some structure, you can put each resource in its own file (such as aks.tf, acr.tf, and so on). When you execute terraform apply, Terraform will go through all the files in the current folder.
Later, if you want to extend the infrastructure with additional resources, you can also distribute the Terraform code in multiple folders and modularize it.
- 1.
Provisioning the Azure infrastructure resources (such as the AKS cluster, public IP, or the container registry).
- 2.
Deploying Agones and the game server image on top of the infrastructure you created in the earlier step.
- 3.
Creating Agones resources (such as fleets) for running game servers, automatically scaled, and providing the server IP and port of ready servers for clients.
Block diagram of Azure agons represent local machine, Azure with user ID location and network security rule, and Agones .dev file.
Implementing Agones on Azure
To summarize what you need to build: first, you will provision a couple of resources with the help of Terraform in Azure. This starts with an AKS cluster, which comes with a lot of additional resources such as load balancer, network security group, and public IP. After the Kubernetes cluster is running, you will deploy and configure Agones on top of it.
You will also need to create a container registry to store the server image. From this registry, Kubernetes deploys the game servers as needed. Agones will manage the game servers by organizing them in fleets and allocate according to the needs of your clients.
The next sections go through these steps in detail.
Provisioning Azure Resources
You take the Terraform code from the previous chapters and extend it with the resources built in this chapter.
Create a Resource Group
Build Azure Kubernetes Service
You create the core component, the Azure Kubernetes (AKS) cluster, next. Note that even though the Terraform code looks short, Azure will generate several additional resources as well in the subscription.
Here you refer to the location you set in the previous step on the resource group and to the resource group (agones-resources) itself, so you should add the AKS cluster to that group.
A node on Kubernetes means a virtual machine, a virtual machine is a scale set in Azure. At the default_node_pool you define the type of the virtual machine you want to use under the AKS cluster. The Standard_B2 is one of the cheapest (if not the cheapest) virtual machine, so for experimenting and building the infrastructure, it is the most cost effective. You can also choose to have only one node (one VM) for the same reason. In production, you can increase this based on the number of players.
Network security group: This is a kind of firewall to control which inbound and outbound traffic Azure allows. It is associated with the AKS subnet. You can extend this with your own network security rule.
Route table: This determines where to route the traffic; what the next hop IP address is.
Virtual machine scale set: This includes the VM instances; the nodes under the Kubernetes cluster.
Load balancer: This is only for outbound traffic. The VMs under AKS sometimes have to reach the Internet, for example, to download updates.
Public IP: This is attached to the Load Balancer for outbound traffic.
Virtual network: All the components are in a virtual network, which you can further divide into subnets. This determines the internal address space and IPs for the servers.
Verifying the Cluster
If Terraform finishes without any problem, you have the Kubernetes cluster in Azure. You can check this in the Azure Portal or through the command line.
Add a Network Security Rule
Agones will generate the port numbers for the servers between 7000 and 8000. With the AKS cluster, you have already received a network security group. However, you need to add a new rule to allow inbound UDP traffic for the ports between 7000 and 8000.
Be aware that this may not give you the network security group name upon the first Terraform apply. This is an API issue. In this case you need to run Terraform later again or copy the network security group’s name directly into the Terraform code. Fortunately, it is generated from the DNS prefix of the nodes. If it is agones, then the name of the network security group will always be aks-agentpool-55978144-nsg.
Build Azure Container Registry
Accessing ACR from AKS
With this method, you have allowed AKS to pull images from ACR.
Install Agones Using Helm
Creating and Deploying the Server
After you have deployed the Agones infrastructure, you have to develop the server code in Unity that will receive the client requests. After building the code, create an image from it and push it to the container registry in Azure.
Implementing the Server
You have to add Agones Unity SDK to Unity first. Go to Unity ➤ Window ➤ Package Manager ➤ + ➤ Add Package from GIT URL…, and copy the following URL to the text box:
https://github.com/googleforgames/agones.git?path=/sdks/unity#v1.22.0
In Unity Editor, create two new game objects, AzureServer and AzureClient, and add them as child objects of the Azure game object. You should also add the two core Mirror components, NetworkManager and KcpTransport, to them. Drag-and-drop the player prefab to the Player prefab field.
Without these calls, the game server will be stuck in Unhealthy state. You have to add the AgonesSDK component to the AzureServer game object.
Now leave only the AzureServer game object enabled and create a server build. Go to Unity Editor, File ➤ Build Settings…, select Dedicated Server as Platform, and set the Target Platform to Linux. Click Build and select a new folder for the server files.
Create Your Game Server Image
Push the Image to ACR
Now AKS will be able to pull the image from ACR as soon as Agones requests it.
Agones Configuration
An illustration of player 1 and player 2 with game clients. a block show node with azure services.
Access of game servers in Agones
In this file, you describe the parameters of a GameServer, such as its name, the container’s port, the image used, the requested CPU, and the container memory. It is important to give it enough resources or the pod will simply crash and never run.
You can test if your game client can connect to the address and port. To do so, you have to temporarily disable the Azure game object and enable NetworkManager. Use a NetworkManagerHUD component to provide the server IP and add the port number to the KcpTransport in the inspector.
Note that the IP is the public IP address of the node, and the port is generated and assigned to the game server by Agones.
Create a Fleet
Having one game server is not enough, because then you can only start one game session. You need a fleet of servers, and Agones helps with this. Fleet is a custom resource in Kubernetes implemented by Agones, which generates multiple servers according to your definitions.
It is similar to the GameServer resource. The only difference is the replicas parameter, which will determine the number of game servers generated automatically. If you remove a game server, the fleet will generate a new one. Practically, if players exit a session, the game server goes into shutdown state, and it will be removed from Kubernetes.
Fleet Autoscaler
The fleet autoscaler allows you to create and remove game servers dynamically according to the current demand. Also, you don’t want to reserve CPU and memory in advance.
The fleetName must match the name of the earlier defined fleet. Using min and maxReplicas, you can set the limits of scaling. The bufferSize shows the number of game servers that are always ready. Note that the replicas setting of the fleet resource will be overwritten by the fleet autoscaler. In this example, by default there will be only two game servers ready. If one becomes allocated, a new one will be added as ready. That way, there are always two servers in the ready state.
Game Server Allocation
Extending the Control Panel
Copy any running server’s IP and port into the text fields of the control panel, click Start Azure Client, and verify that the client can connect to the server and the character appears on the screen.
The next chapter extends this with an automatic matchmaker, so that players will be automatically connected to ready servers without having to type in the server IP and port.
Summary
In this chapter, you implemented the PlayFab and Azure dedicated servers, which allow online multiplayer gaming from any distance between players. You learned that you need much more effort to implement it in Azure. On the other hand, you have endless possibilities to customize the solution to your needs. But you may think that PlayFab costs much more. Let’s do a quick comparison.
Prices of PlayFab and Azure Virtual Machines
Instance | vCPU | RAM | Storage | PlayFab | Azure | Difference |
|---|---|---|---|---|---|---|
D2asv4 | 2 | 8 GiB | 16 GiB | $0.208/hour | $0.199/hour | $0.009/hour |
D4asv4 | 4 | 16 GiB | 32 GiB | $0.4159/hour | $0.398/hour | $0.0178/hour |
D8asv4 | 8 | 32 GiB | 64 GiB | $0.8318/hour | $0.796/hour | $0.0358/hour |
D16asv4 | 16 | 64 GiB | 128 GiB | $1.6636/hour | $1.592/hour | $0.0716/hour |
Prices of PlayFab and Reserved Virtual Machines in Azure
Instance | PlayFab | Azure 1Y | Azure 3Y | Difference |
|---|---|---|---|---|
D2asv4 | $0.208/hour | $0.1549/hour | $0.1323/hour | $0.0757/hour |
D4asv4 | $0.4159/hour | $0.3098/hour | $0.2645/hour | $0.1514/hour |
D8asv4 | $0.8318/hour | $0.6198/hour | $0.5290/hour | $0.3028/hour |
D16asv4 | $1.6636/hour | $1.2394/hour | $1.0579/hour | $0.6057/hour |
If you decide to host your server purely on Azure VMs, you will pay around the same price. But if you are ready to reserve VMs for up to three years, you can save 30-40 percent, compared to PlayFab hosting. Be aware that on Azure you will have additional costs (such as load balancers, firewalls, and monitoring), while in PlayFab you only have to pay for the VMs.
Review Questions
- 1.
Why do you need dedicated game servers hosted in the cloud?
- 2.
What are the possible ways to host game servers? How do they compare?
- 3.
What is a build in PlayFab?
- 4.
What are the options to troubleshoot issues with the PlayFab server?
- 5.
What are the problems with a single virtual machine when implementing server hosting for multiplayer games?
- 6.
Which features of Kubernetes and Agones support multiplayer games?
- 7.
What is the role of containerization when using Kubernetes?
- 8.
How do you access game servers in Agones?
- 9.
What is the purpose of fleets in Agones?
- 10.
Compare the realization of multiplayer game servers regarding cost and complexity.