Proksim’s Net-Z 2.0

by Crosbie Fitch  (originally for Game Developer, June 2001, but not published)

 

Net-Z represents an exciting step into the next generation of networking middleware.

Before I go into Net-Z though, perhaps I should first define ‘previous generation’. I don’t mean complete game engines that have already had networking built into them, rather the networking layer APIs, e.g. WinSock, DirectPlay, RTime, etc. These provide an ‘easier to use’ networking API, i.e. ‘sockets’ but easier to use. Today such middleware has become a little more refined and polished, and ever more facilities have been added to help the developer monitor and compensate for the behaviour of the underlying network. Further goodies have also been added over the years to try and sweeten the bitter pill that is the task of incorporating multiplayer capabilities into a game. Typical sweeteners are multiplayer services such as lobby server, chat, score ladders, teams management, account and subscriptions management, digital rights management, the kitchen sink, etc. When it comes down to it though, a pig is still a pig no matter how you tart it up.

Net-Z moves into the next generation by abandoning the low level notion that it’s just messages that are being sent between computers, and adopting as fundamental, the concept that objects are duplicated across computers. Net-Z takes care of the messages used to synchronise these duplicates, leaving you to worry about which objects these should be and how they need to be synchronised in more abstract terms.

In other words Proksim has noted like few others in the games industry that there is a higher level objective to address in enabling multiplayer games, and this distributed systems approach is to obtain a consistent or synchronised game state (or simulation model). Naturally, there are many aspects of the synchronisation process which require knowledge specific to each application, and sensibly Net-Z has many features to address this.

So with Net-Z you’re straight in at the level of thinking about how you’re going to architect your distributed system, whether it’s simple client/server, totally egalitarian peer to peer, or somewhere within the spectrum of those bounds. Thus Net-Z allows you to control how object state is arbitrated, whether tied to a specific machine or spread across all or a subset of machines. Given that there is some burden to hosting an arbitrated object, or ‘duplication master’ in Net-Z terminology, the policy by which this burden is shared is where Net-Z’s feature of load balancing comes into play. You can either use a built-in policy (‘equal object numbers’ or ‘equal weights of weighted objects’) or you can hook in your own if, say, some objects need a dynamic weighting.

Scalability

Where Net-Z really demonstrates its next generation credentials is in the aspect of scalability, i.e. being able to cope with ever greater numbers of players and an ever increasing size of game world – without necessitating the use of expensive game servers and administrative nightmares.

In its earlier releases, Net-Z duplicated the complete game state across all participating computers. This meant that although Net-Z based games were able to break through the bandwidth barrier of the client/server architecture, they were still ultimately bound by bandwidth limitations, i.e. although spreading arbitration across multiple computers greatly reduces the need for a dedicated high bandwidth game server, the entire game state still needs to be synchronised (and 56Kbps isn’t much). However, now Net-Z supports a feature called Duplication Spaces. This is an extremely useful and flexible concept that can be used to decompose a game world into relatively independent elements – termed ‘cells’ in Net-Z’s literature. This is better than imposing an ad hoc requirement as to how you segment your game world, e.g. into a cubic grid (regular or octree), portal connected rooms, or server tied territories. Whilst these ad hoc zoning mechanisms might still allow location transparency (zones could be mobile between computers for load balancing), they’d still exhibit a hardwired nature, i.e. a solution oriented for a specific class of application. A duplication space allows you (rather than the middleware vendor) to decide your own zoning policy because it is an associatively computed mechanism. It could be based on spatial co-ordinates; it could be based on which room an object is associated with; it could even be based on a dynamic spatial partitioning scheme. You can have different schemes for different purposes, e.g. duplication spaces for high-scores or other sensitive player stats, duplication spaces for static scenery, and duplication spaces for dynamic objects. Perhaps visibility based cells might be useful?

What duplication spaces do is allow a much larger world to be simulated by breaking it down across clear or fuzzy boundaries so that players’ computers only need to synchronise state concerning the parts of the game world that are relevant to them. Cells do not have to be exclusive either, you can even have a multi-resolution duplication space with small cells intended for distributing across small computers, medium, and large super-cells for more capable computers.

In Use

Figure 1 – How Net-Z libraries integrate into the game

Given the principle that every player sees the same set of objects (for each cell they subscribe to), your job is to decide what objects in your game need to be synchronised, and how.

Net-Z has a pre-processing phase which processes your description of the objects and data needing to be duplicated. These are defined in DDF (Duplicated object class Definition File) and declared in DDL (data definition language). Don’t worry, it’s not as bad as IDL. Alongside objects, Net-Z can distribute data such as primitive values (int, float, etc.), strings, vectors, lists, and queues.

This has the effect of creating class definitions with properties and methods that a C++ programmer such as you might expect, but with a ‘hidden implementation’ base class that provides the distribution mechanism.

Your job after this is to keep making refinements, using Net-Z features as necessary (dead reckoning, say), in order to fine tune the bandwidth consumption and keep the impact of latency to a minimum.

 

Figure 2 – The Atlantis Demo of Net-Z with on screen bandwidth utilisation graphs

Features

Load Balancing

Net-Z automatically distributes duplication space cells and object arbitration (duplicate master vs duplica). It is possible to have complete control over this process. You could even make Net-Z behave in the classic client/server configuration, which might be sensible for some classes of duplicated object.

When computers drop out (unexpectedly or otherwise), Net-Z allows you to control how the duplicated objects should be redistributed – if at all. There is still the idea of one computer being the game master, even if it doesn’t take on all the workload. It is to this computer that responsibility reverts to by default if any players drop out, or can be obtained from if players join.

Control

Simple ‘reliable/unreliable’ flags determine whether an attempt should be made to ensure updates occur, or whether a fire & forget policy should be sufficient.

The programmer is given some facility to interfere in the distribution process and take a more active role in fine tuning it.

Dead Reckoning

Linear and Quadratic prediction algorithms are used within a dead reckoning scheme to reduce updates whilst they remain within a certain tolerance (optionally distance-based). This can reduce bandwidth where data values exhibit linear or quadratic trends. Values only need to be synchronised when they diverge too greatly from those predicted. Unfortunately, this has the tendency of smoothing behaviour, but you can get involved to identify the cases where discontinuities are actually required.

Encryption

Packet encryption is available to encrypt all information sent between players’ computers. Naturally, tampered or resent packets are detected and rejected. However, there is still a responsibility for the game to protect itself against attack whilst running. Obviously you’ll be using a secure server for financial or other critical transactions.

Compression

Packet level compression is available in conjunction with encryption to squeeze just that little extra out of the available bandwidth, though it has to be appreciated that this will then compete with any hardware level compression. Three compression algorithms are currently available (LZRW, LZSS, Splay).

Limitations

Compile Time

Duplicated Objects must be defined at compile time. Now this may not seem much of a limitation, but it means it could be difficult to produce your own debugging/editing tools to try things out at run-time. It also means that content duplication behaviour is pretty much tied to the game code, i.e. difficult to provide tools for players to define their own objects. Sure, you can probably get round it by defining objects with many different permutations of duplication attributes, but do be aware that some things are defined at compile time. However, this is probably just me being picky, and isn’t likely to be a problem for most games.

Maximum duplicated object classes=2¹⁰. Maximum object instances=2²².

Capacity

Net-Z by making all duplicated objects accessible via in-memory object instantiation has a scalability limit in terms of the numbers of objects that one can fit into memory at any one time. However, because Net-Z was designed with exhaustive object synchronisation in mind, this is so limited by the peer bandwidth, that it’s likely for this to limit the number of duplicated objects before memory does (perhaps why Net-Z doesn’t support caching). However, if you’re hoping to produce the next Ultima Online, but in a peer-to-peer incarnation, you’re either going to have to whack up the virtual memory setting, or be very careful how you size your cells and define the load balancing policy. This isn’t an impossible task, but maybe we can look forward to a lazy scheme in a future version, e.g. ‘low priority cells to cache’.

Players

Without the use of duplication spaces Net-Z would be limited to total players of the order of around 10-100, with variation due to degree of game state coherence and players’ bandwidth (<56Kbps). Even with duplication spaces, you still have to be careful how you allow players to congregate as this will affect the fidelity of their mutual interaction. But hey, everyone will have DSL in a couple of years, and 2Mbps real bandwidth with 10ms pings will be commonplace – won’t it?

Diagnostics

A reasonably extensive set of performance counters are provided via the API to assist programs that need to monitor the performance of Net-Z and the underlying network. The API also provides the ability for the simulation of networking conditions typical to the Internet, and allows various parameters to be controlled by the programmer.

What I’d have hoped against hope for though was a nice suite of graphical tools to configure the simulation of a virtual network and an arbitrary number of players. You might need to borrow the company’s network of Win2K PCs in order to host umpteen spoof players per machine, but it would help stress testing, even scalability to some extent.

Bugs

Net-Z v1.6 documentation lists a number of known bugs typical to a teething phase. These are predominantly those that occur in some expected failure modes, i.e. as long as players’ computers are well behaved, Net-Z is well behaved. The bugs (from ‘mild’ to ‘crash’) tend to arise in some obscure situations where more than one failure mode happens at the same time. With Net-Z v2.0 (not available at the time of review) presumably there are no known bugs.

The Competition

Net-Z may be the most advanced and stable implementation of a distributed or duplicated objects based middleware solution to date, but as this is still a bleeding edge field you should reassure yourself by checking out other networking solutions. See Table 1 for a few pointers to a selection of networking solutions (of which support for distributed objects varies).

 

Table 1 – Pointers to a selection of networking solutions (usefulness and readiness varies)

 

Who should use it?

If you’re comfortable with the distributed objects paradigm, particularly in terms of object location transparency, then Net-Z is ideal. If you still prefer to think in terms of passing messages between machines, or in terms of collecting player input and publishing client updates, then you can still use Net-Z, but you may not be fully exploiting its next generation features.

If you’re converting a single-player game, or are interested in going beyond the ceiling of players provided by your current engine, then get the evaluation version of Net-Z now – it might be your best hope of avoiding ‘multiplayer’ being the feature that makes you miss your release date.

Let’s not repeat the bad old days of the transition from the ease of home-brew 2.5D sprite engines to the not so easy home-brew software 3D rendering engines. There will be those who reckon they can knock up a good distributed objects engine over a couple of weekends, try and possibly succeed, and those who wisely use Net-Z in the interim, and perhaps still have a go at the weekend – if they’re that keen. As Dirty Harry says: “A man’s got to know his limitations”. Use Net-Z first. You never know, you might use it happily ever after, never stopping to think of doing a home-brew engine.

If you’re happy with your current engine (Quake, Unreal, etc.) and aren’t finding its scalability limits cramped, then stick with it. Net-Z is unlikely to bring greater efficiencies for this specific application class.

If you’re using a military or academic solution such as DIS you might find Net-Z more in tune with the Game developer ethos.

Conclusion

Net-Z represents a programmer friendly integration of a distributed objects system into a game – not too isolated, not too spaghetti-like, just enough hooks and involvement to reassure you. There’s still a big benefit if you understand what’s going on down below, but it’s nice to leave the mucky plumbing to someone else. Of course, if you want to get your hands dirty, Net-Z is flexible enough to allow you precise control and the ability to explore a wide range of options.

If I had to convert a game, or was comfortable with a single player game engine and had to use it to produce a multiplayer game before Christmas (the one after next), then I’d use Net-Z. If I had to write a game from scratch then it’s probably a matter of economics as to whether to go for a full engine with networking built in, or whether to hook up a cheap engine to NetZ. Of course, if I had two years going on five, then I’d write everything from scratch, but then I’m a bit mad and don’t know my limitations...

Summary

NB Net-Z v1.6 software used, but features of Net-Z v2.0 described.

Contact Details

NB The following info is now out of date. Please see www.quazal.com for latest contact info.

General inquiries: info@proksim.com

Sales information: sales@proksim.com

Technical questions: support@proksim.com

 

Proksim Software Inc.
433 Place Jacques-Cartier
Suite 200
Montréal, QC
H2Y 3B1

Tel: (514) 940-9090

www.proksim.com

Supported platforms

Windows 95 / 98 / ME / NT 4.0 SP3 / 2000

WindowsCE / PocketPC (Compaq iPAQ, HP Jornada)

Linux (Intel Architecture (i386))

Sony PlayStation 2

Further Development

Proksim is committed to making its technologies cross-platform, i.e. PC, console, wireless, etc.

It is also exploring arrangements with network providers able to provide enhanced network performance, e.g. quality of service guarantees, packet prioritisation, dedicated backbones, etc.

Further tools for simulating network and player behaviour, scalability and stress testing, are also in the pipeline.

Pros

1.      Next generation middleware, in a more reliable, ‘version 2’ release

2.      Supports a more scalable, higher level networking model

3.      Still provides facilities for fine control and low level access

Cons

1.      Not yet a battle hardened product (Zero-G Marines is one of the first games)

2.      Scalability limitations still present, and as yet unmeasured

3.      Proksim is not owned by Microsoft, Sony, Vivendi, et al (yet)

Product Rating

Three and a half stars (promising, but needs battle hardening)