If you take the time to browse through the folders of nearly any game that's installed on your hard drive, you will see lots of files with interesting extensions: mpq, pk3, dat, upk, and so on. What you most likely will not find are images, models, sounds, or scripts. But then you start the game and everything appears on the screen and runs perfectly fine—why is that?
As you may have already guessed, it has something to do with these strange files that are carrying even stranger name suffixes. These files are containers, hiding away the game resources from the end user. Basically, they are similar to ZIP or RAR archives. Most of them are using a proprietary file format that sometimes even resembles a small file system, just as it might be implemented in an operating system.
Besides the obfuscation of resources, such file containers provide at least two more very important advantages over distributing assets openly. The first one concerns the installation process: The setup program only has to copy a few large container files compared to transmitting hundreds or even thousands of small to medium sized files. This can speed up the process of copying files by a substantial amount.
The design of the game engine consuming these files is the other reason. Containers allow developers to store game assets in an optimized form and order that makes them easier and faster to load and process at runtime, possibly decreasing loading time and increasing the game's performance.
Panda3D provides the right tools and built-in features for creating and loading container files. In the terminology of this engine, they are called multifiles and the following steps will show you how to work with them.
This recipe extends upon the code created in Loading models and actors found in Chapter 2, Creating and Building Scenes. Please review said recipe before going on.
Additionally, you have to copy the files panda-walk.egg.pz, panda.egg.pz, and teapot.egg.pz from C:Panda3D-1.7.0models to the models subdirectory of your project directory tree.
Let's create a sample program:
- Additional to the existing
importstatements at the top ofApplication.py, make sure to have the following lines:from panda3d.core import * import glob import os import os.path
- Add the following lines of code to the constructor of the
Applicationclass directly after the call toShowBase.__init__(self):mf = Multifile() mf.openWrite(Filename("../models/models.mf")) for f in glob.iglob("../models/*.egg.pz"): filename = os.path.split(f)[1] mf.addSubfile(filename, Filename(f), 9) mf.repack() mf.close() fs = VirtualFileSystem.getGlobalPtr() fs.mount(Filename("../models/models.mf"), ".", VirtualFileSystem.MFReadOnly)
The preceding code actually combines two steps that are normally not found within the same program.
The first step, which we would normally put inside a build script, is to build the multifile—to add files to the container. We are using a Multifile object to add so-called subfiles to the multifile. The addSubfile() method is the key point here. The first parameter is the name the added file will be accessible as inside the multifile. The second parameter is the full path to the file that is to be added. The third parameter sets the compression level.
The multifile libraries and tools of Panda3D are able to compress and decompress files contained in a multifile on the fly, as they are added at build time and loaded at runtime. This can help to keep the size of a game's distribution package small, and also provides a simple form of obfuscation. When working with multifiles, use 0 for no compression at all and 9 for the highest level of minimization. Choosing a higher compression level will keep package sizes small, but in exchange we must accept that creating these packages will take more time to finish.
Before we are done creating the multifile and close it, we need to repack()our container file. A multifile contains an index that stores the names and offsets of the subfiles. Repacking reorders this index, and makes sure all metadata and subfile content is in the proper place. We must not forget this step to ensure that the engine will be able to load data from the multifile.
The second part of working with multifiles happens in the last two lines shown in the previous code. Here we mount the contents of the multifile into the root folder of Panda3D's virtual file system, which makes it possible to transparently access all of the files found within the container.
There are a few more things you can do with multifiles.
During development, you will repack your assets over and over again. The following method call will only add a subfile if it is different from the one already present in the multifile. This may help you to decrease the time needed to regenerate asset containers.
mf = Multifile() mf.updateSubfile(internalName, Filename(fullPath), compressionLevel)
Subfiles can also be extracted from a multifile like this:
mf = Multifile() index = mf.findSubfile(internalName) mf.extractSubfile(index, Filename(fullPath))
To protect your data, you can encrypt subfiles on a per-file basis. After calling the two following methods, all subsequent reads or writes will decrypt and encrypt subfiles respectively. Similar to using compression, encoding, and decoding your data using an encryption algorithm requires additional CPU resources. Also note that Panda3D uses the same key for encrypting as well as decrypting data. This means that the password required for accessing multifile data needs to be stored somewhere in your code, exposing it to possible reverse-engineering attacks. This kind of encryption should be seen as a time-consuming obstacle to be put in the way of attackers. It makes accessing the data harder, but not impossible.
To protect your data, you can encrypt subfiles on a per-file basis. After calling the two following methods, all subsequent reads or writes will decrypt and encrypt subfiles respectively. Similar to using compression, encoding, and decoding your data using an encryption algorithm requires additional CPU resources. Also note that Panda3D uses the same key for encrypting as well as decrypting data. This means that the password required for accessing multifile data needs to be stored somewhere in your code, exposing it to possible reverse-engineering attacks. This kind of encryption should be seen as a time-consuming obstacle to be put in the way of attackers. It makes accessing the data harder, but not impossible.
mf = Multifile()
mf.setEncryptionFlag(True)
mf.setEncryptionPassword("arewesafenow?")
The Panda3D SDK also includes a command line tool for working with multifiles. The following line shows a possible way to invoke the program from the command prompt:
multify -c z -9 -e -p pass -f models.mf panda.egg.pz panda-walk.egg.pz teapot.egg.pz
This will create (-c) a new multifile called models.mf (-f), containing the subfiles panda.egg.pz, panda-walk.egg.pz and teapot.egg.pz. The subfiles will be compressed using the highest compression level (-z -9) and encrypted with the given password (-e -p pass). The multify tool also has several other options and flags which can be viewed using the command multify -h.
Note the .egg.pz file extension of the model files. These files are normal .egg files that were compressed using pzip. You can unpack the raw .egg data using the punzip tool.
Time is a precious resource, even in people's spare time. If a TV show isn't compelling after seeing 3 seconds of it, we zap on to the next channel. If we don't like what we hear, we skip to another song when listening to music.
The same principles apply to video games. But not only do games have to be compelling and fun to keep players engaged—in the world of PC gaming we also need to provide an uncomplicated experience for installing and launching a game. If players needed to do lots of configuration work to get the game running, then that would drive off most of them from actually playing it rather quickly.
To prevent this from happening and to make launching our games as easy as possible, Panda3D gives us two things: The Panda3D Runtime and the Panda3D applet file that packs an entire game into one file. All players need to do is install the runtime and double-click the applet file.
This recipe will show you how to obtain the Panda3D Runtime and how to build a game package for easy redistribution of your games.
This recipe requires some runtime components to be present to work. Open your browser and download the Panda3D Runtime from the Panda3D website. The runtime download can be found at www.panda3d.org/download.php?runtime. Then go to runtime.panda3d.org, download the file packp3d.p3d and copy it to C:Panda3Din.
You can use one of the samples found in this book or one of your personal projects as a starting base for this recipe. As long as it is built after the project setup described in Setting up the game structure found in Chapter 1 you are set and ready to go.
You can create and launch a redistributable game package like this:
- Open
main.pyand change its contents to this:from Application import Application gameApp = Application() gameApp.run()
- Create a new batch file in the top-level directory of your project. Name it
deploy.bat. - Edit
deploy.batand insert the commands found below:@echo off mkdir deploy xcopy /E /Y src* deploy xcopy /E /Y models* deploy xcopy /E /Y shaders* deploy xcopy /E /Y sounds* deploy packp3d -o pandagame.p3d -d deploy -r models rmdir /S /Q deploy
- Invoke
deploy.batfrom a command prompt or by double-clicking it. This step might take a little time, as some additional components need to be downloaded. - Double-click the newly created
pandagame.p3dfile to launch your game. Before it starts, you will most likely see the following screen while additional data and assets are downloaded:
To fulfill the requirements of the p3d game package format and the Panda3D Runtime, we first need to make a minimal change to main.py. Then we can go on to package our project. The packp3d expects all files, assets as well as scripts, to be present in one folder, which is why we copy all of our files to the temporary deploy directory.
Essentially, a p3d file is a special version of a multifile with a slightly different file header. While multifiles are mainly used for packaging game assets, p3d files contain all files needed to run a Panda3D based game. This includes assets as well as scripts and executables. Multifiles are just containers, but p3d files can be launched using the Panda3D runtime. Therefore they need to store some extra info to allow the engine to properly load and run p3d packages. For example, the version of Panda3D used to create the package, the application main file name as well as the names of referenced external packages are all included in a p3d file.
When you double-click the p3d file, the Panda3D runtime starts up and mounts the file. The engine then looks for a file called main.py and starts running the Python code found in it. If our main file has a different name, we can pass it to packp3d using the -m parameter.
To close off the discussion, we take a look at what might be the most important parameter of packp3d, which is -r. The initial Panda3D Runtime installation is very lightweight and does not contain all components of the engine. Instead, it relies on game packages to reference the various engine components. If a referenced package is not found in the local cache, or a newer version is available, it is downloaded from runtime.panda3d.org, which is the default package host.
This means that we have to use the -r parameter to add a reference for every package we use. Without these references Panda3D assumes that we are only using the packages included in the minimal runtime installer. So building a project using the default models means appending -r models to the command. If we use sound, we need to add -r audio. Using PhysX means an extra -r physx and so on.
In the previous recipe, Creating a redistributable game package, you learned about building one monolithic container file that stores all of a game's code and assets. Of course, this already made things simpler, as you only need to provide this one file and the information that the Panda3D Runtime needs to be installed to run.
In this recipe, you will take things one step further. Instead of just putting everything into one file, you will split your code and assets into two separate packages. You will be able to drop the requirement of having to copy all data into one directory and additionally, you will learn how to build these packages in a way that allows you to host them on the web using an HTTP server. This will allow you to easily distribute games and the data they require. Additionally, this will allow you to provide distinct packages containing game-specific data and others that are filled with common libraries and resources. This way, code and assets shared across multiple releases need to be downloaded only once, which helps in keeping client side loading times smaller as well as keeping server system loads and bandwidth costs low.
After finishing this recipe you will be able to control the packaging process in a more detailed way. This will allow you to further optimize the user experience and help you to save file hosting bandwidth and traffic because they will only be downloading what they need.
Additional to the prerequisites of the recipe Creating a redistributable game package, you need to download the file ppackage.p3d from runtime.panda3d.org and copy it into the C:Panda3D-1.7.0in directory.
Panda3D expects packages referenced by the main p3d file to be hosted on a web server. Therefore you need to either set up a local web server or get some hosted webspace. There are many free HTTP servers like HFS, Cherokee, or IIS Express, for example, that all allow you to set up a simple web server very quickly.
Follow the steps below:
- Create a new file called
deploy.pdefin the top-level directory of your project. - Open
deploy.pdefand enter the following code. Make sure to replace the URL with the one of your server:from panda3d.core import * packager.setHost("http://localhost:8000/") class myresources(package): file(Filename("models/*")) file(Filename("shaders/*")) file(Filename("sounds/*")) class pandagame(p3d): require("panda3d", "models", "myresources") config(display_name="My Game") dir("src") mainModule("main") - Open a command prompt window, navigate to your project's directory and enter this command:
ppackage -i deploy deploy.pdef
- The last command will create a new directory called
deploy.Upload its entire contents to your server. - Double-click
pandagame.py.The Panda3D runtime should start, download all necessary files, and then run your project. The following screen denotes the runtime not being able to reach the server:
Instead of pointing a tool to a directory containing all of our data, we write a package definition file containing detailed information about the contents and name of each package. The syntax of a .pdef file is based on Python and even allows a subset of the Python language constructs, like importing modules or if conditionals. In a .pdef file, there are two classes you derive from to create either multifile packages or p3d applet containers. These two classes are package and p3d.
Within each of these classes, we use file() and dir() to add arbitrary files to a container file. What's nice about these two is that the packaging tool is able to detect model files in plain-text .egg format and automatically converts them into the more space and loading time efficient .bam format, for example.
In our sample we also set a configuration variable and define the main module of our p3d file using the self-explaining config() and mainModule() functions. Additionally, we must not forget to add a call to require() inside our .p3d definition. This references an external package—just like the running packp3d with the -r parameter flag.
Before we are through with this recipe, there are just a couple of things you ought to know when working with packages.
In this recipe we only created a game package and one containing asset. One thing that's important to know additionally, though, is how to package Python modules. This enables us to create libraries of common classes and functions the user has to download only once, even though it might be used by multiple games.
There are two functions for working with modules: module() and excludeModule(). The module() function will add a Python module and all of its dependencies to a package. If we want to explicitly exclude a module from being included as a dependency, we add a call to excludeModule(). This is used in the scripts building the Panda3D core runtime packages to exclude the PhysX and ODE libraries to decrease the package size for example.
We created a new package using the ppackage tool in this recipe. But as it happens, games change slightly over time as bugs in assets and the codes are fixed later after the initial release.
When hosting packages on a server, Panda3D makes sure it always uses the latest version of all referenced applet and package files. But wouldn't it be a big waste of resources and time to download the whole package all over again because one little thing has changed? Definitely! And this is exactly what happens if we run ppackage for every patched release we make.
To overcome this problem and deliver updates more conveniently, Panda3D includes the ppatcher tool. Simply run the following command line after the initial release and after you are ready to distribute a new update (assuming our ppackage output is in a subdirectory called deploy):
ppatcher -i deploy
This creates all the necessary files for patching data on the client side to the newest version, only needing to transmit the data that has changed since the last version.
While providing a downloadable easy-to-run package of your game is a good step towards making users happy, we can take this to the next level. Why require players to ever leave our website? Why even bother them with downloading data to their hard disks where players, need to find the file again before they are able to double-click to launch?
Minor nuisances, we might say. But the truth is that people might have already lost the will to play a game after hitting the first few bumps in the road.
In this recipe, we take a look at how Panda3D allows us to embed games within websites. This allows us to make many things easier for potential players. Using the web plugin, we can create a website that just prompts players to press a button to launch the game without leaving the site, or even the browser. If the plugin isn't installed on a player's system, it's no problem either—we just forward her or him to the appropriate download page.
Online gaming websites have had great success with this operating model, so why shouldn't we, too?
This recipe assumes you have read and followed the recipe Creating a redistributable game package. Also be prepared to work with web technologies like JavaScript, HTML, and DOM.
The files DetectPanda3D.js and RunPanda3D.js are not part of the Panda3D SDK distribution and are only located in the source code package. Be sure to download the source code from www.panda3d.org to have the required files present on your system.
Adding to that, you will need a digital X.509 certificate in PEM format to sign your game package. You can (and should) retrieve a trusted certificate from many commercial certificate authorities for redistribution of your games over the web.
For testing purposes, you can create a self-signed certificate using OpenSSL. You can download OpenSSL from gnuwin32.sourceforge.net/packages/openssl.htm. Select the download link next to where it says Complete package, except sources as shown in the following screenshot. Save the file and install OpenSSL.
To generate a new test certificate, open a new command prompt and first create a new private key using the following command:
openssl genrsa 2048 > cacert.pem
Then, issue the following command to obtain a new certificate:
openssl req -config "C:Program Files (x86)GnuWin32shareopenssl.cnf" -key cacert.pem -new -nodes -x509 -days 1095 >> cacert.pem
The OpenSSL program will ask a few questions about your location and company name. You don't need to answer them for internal use and can accept the default values by just pressing the Enter key. After finishing these steps, the certificate will be stored in a file called cacert.pem in the current working directory.
Embedding a game based on Panda3D in a website is done like this:
- Copy your certificate file to the top-level directory of the project structure and rename it to
cacert.pem. - Create a new directory called
webinside the top-level directory of your project. - Copy the files
DetectPanda3D.jsandRunPanda3D.jsfrom thedirectsrcdirectscriptssubdirectory of the source code package to the web directory of your project. - Inside the
webdirectory, create a new file calledweb.htm. Open it in an editor and enter the following HTML code:<html> <script src="DetectPanda3D.js" language="javascript"></script> <script src="RunPanda3D.js" language="javascript"></script> <body> <script language="javascript"> detectPanda3D('noplugin.htm', false) P3D_RunContent('id', 'pandaPlugin', 'src', 'pandagame.p3d', 'width', '800', 'height', '600') </script> </body> </html> - Add another HTML file to the web directory. Name it
noplugin.htmand enter the following markup:<html> <body> <p>Panda3D Web Plugin not found!</p> <p>Download it <a href="http://www.panda3d.org/download.php?runtime">here</a> </body> </html>
- Edit the
deploy.batfile in your top-level project directory so it looks like this:@echo off mkdir deploy xcopy /E /Y src* deploy xcopy /E /Y models* deploy xcopy /E /Y shaders* deploy xcopy /E /Y sounds* deploy multify -S cacert.pem -uf pandagame.p3d xcopy /Y pandagame.p3d webpandagame.p3d rmdir /S /Q deploy
- Open the
web.htmfile found inside the web subdirectory in a browser. After the Panda3Dpluginhas downloaded some additional data, you will see something similar to this:
- Press the play button to start the game running. If you are using a self-signed test certificate, the first time the program is started you will see the following message:
- Click View Certificate to proceed. Next you will see this window:
- Your own test certificate can be considered as trustworthy, so you can press Run to allow your application to start.
There are three important points to look at in this recipe. First, the detectPanda3D() function iterates over the list of installed browser plugins, trying to find an entry for the Panda3D runtime. If it can't be found, the function immediately redirects the browser to noplugin.htm, preventing the execution of any further JavaScript statement.
If detectPanda3D() was able to find the Panda3D Runtime, JavaScript execution goes on to P3D_RunContent(). This function takes care of placing the necessary<object> tags that make our application appear in the browser window, interpreting its parameters as key-value pairs used for configuring the plugin.
These scripts have one great advantage—detecting plugins and placing the correct objects in the document object model tree work slightly different across browsers. Luckily, the Panda3D developers have already handled this for us, taking a lot of tedious work off of our shoulders.
Note
The web plugin of Panda3D accepts only signed packages and refuses to run any unsigned piece of code. This happens with good reason, as players are executing code locally, making it possible to run media-rich applications, but also opening the door for possible attacks. Therefore we need to guarantee users that the packages are coming from us and haven't been changed while they were on the way through the tubes of the Internet.
Additionally, we should use a certificate issued by one of the big trust providers when distributing our games. This prevents the certificate warning from popping up, as these certificates can be checked against their respective root certificates, guaranteeing our trustworthiness to the user.
The Panda3D web plugin allows you to deeply integrate your games with the website they are embedded in. Using JavaScript in the website, code from outside of the plugin can access values and call functions implemented in Python. This also works in the opposite direction, as code running within the plugin may make calls to JavaScript or access the document tree defined by the website's markup.
This opens many possibilities for interesting mashups between games and web technology. Automatically setting a player's name within your game based on login data, showing toast notifications when achievements are unlocked or posting status updates and screenshots to social networking sites using their JavaScript APIs. These are just a few of the many interesting things you could do using this feature of the Panda3D web plugin. But always keep in mind the security issues involved, especially with accessing personal information or login information!
This recipe carries on where the previous recipe Embedding a game into a website left off. Make sure you have a working version of the code produced in that recipe before going on.
Web technologies like HTML, JavaScript, and DOM will be used in this recipe. A basic level of knowledge in this area is assumed.
Let's add some interaction between the website and the Panda3D plugin:
- Open the file
webweb.htmin an editor and replace its contents with the code below:<html> <script src="DetectPanda3D.js" language="javascript"></script> <script src="RunPanda3D.js" language="javascript"></script> <script language="javascript"> function enableButtons() { var buttons = document.getElementsByTagName('input') for (var i = 0; i < buttons.length; i++) buttons[i].disabled = false } </script> <body> <script language="javascript"> detectPanda3D('noplugin.htm', false) P3D_RunContent('id', 'pandaPlugin', 'src', 'pandagame.p3d', 'width', '800', 'height', '600', 'onWindowOpen', 'enableButtons()') </script> <p> <input type="button" disabled="true" value="Wireframe" onclick="pandaPlugin.main.base.toggleWireframe()"> <input type="button" disabled="true" value="Get Value" onclick="alert(pandaPlugin.main.gameApp.foo)"> <input type="button" disabled="true" value="Load Smiley" onclick="pandaPlugin.main.loader.loadModel('smiley').reparentTo(pandaPlugin.main.render)"> <input type="button" disabled="true" value="Lights Out!" onclick="pandaPlugin.main.gameApp.lightsOut()"> </p> </body> </html> - Edit
srcApplication.pyand add the following lines to the constructor ofApplication:self.foo = 42 base.appRunner.main.base = base base.appRunner.main.render = render base.appRunner.main.loader = loader base.appRunner.main.gameApp = self
- Also add this member method to
Application:def lightsOut(self): base.appRunner.evalScript('document.body.style.backgroundColor="#000"') - Open
deploy.batin your editor and change the line that runs the packp3d tool to this:packp3d -c script_origin="**" -o pandagame.p3d -d deploy -r models
- Run
deploy.bat. - Open
web.htmin your browser. The following buttons the Panda3D plugin will be enabled after the application has started successfully:
Before we go on to discuss the communication between website and plugin, we need to first think a second about security. By default, JavaScript code is not allowed to call into the plugin, and the Python code running within it prevents arbitrary JavaScript from messing with our games. To explicitly enable this channel, we need to set the script_origin flag to the URL of the server that is going to call runtime functions from JavaScript code. In this sample, we set it to allow any host, but when making our games available to a broader public, we need to lock this down to a specific URL. For example, we could pass scripts.example.com to allow only scripts from this URL to access the runtime. We can also use wildcards for specifying these URLS: The rule *.example.com matches a.example.com, b.example.com but not example.com, while the rule **.example.com would match all of the aforementioned URLs.
The key component in enabling communication between the website, JavaScript, the Panda3D plugin and Python is base.appRunner. Using the evalScript() method of the AppRunner class, we can run arbitrary JavaScript code within the context of our website. This can really be anything allowed in JavaScript: A call to a function, a DOM tree manipulation or even a bigger string containing loops, calls, conditionals, and variables.
The other calling direction, which is from JavaScript into the plugin, also involves the AppRunner class. Here, any attribute attached to base.appRunner.main becomes visible to JavaScript code.
Lastly, the plugin provides the possibility to assign functions to a set of callbacks. This allows us to react to various events occurring during the runtime of the Panda3D plugin. In our case, we use the onWindowOpen event callback that is called right before the game actually starts running and produces the first frame.
There are several other events we can react to, like onPluginLoad after the plugin is loaded and initialized, onDownloadComplete after all dependencies are downloaded or onReady when the plugin is ready to launch our application. There are several more of these. You should see the official documentation of Panda3D on this topic, found at www.panda3d.org/manual/index.php/Plugin_notify_callbacks, to get a comprehensive list.