|
|
QSDK 1.1 Documentation |
|
|
This technical note describes how streaming in Q functions, how it can benefit game design, and how to use it to best effect in your authoring and code.
Traditional game engines operate by loading into memory all game data for a "level". This level may consist of several "rooms" worth of data, but at the end of the level a loading screen must be displayed and the next level loaded into memory.
Q provides a seamless, continuous, environment suitable for producing immersive 3D worlds through its streaming technology. Worlds are divided into a set of contiguous "Zones" of content. While a given Zone is being simulated and displayed neighboring content is being streamed into memory ready for when the participant moves.
Loading data from CD/DVD brings other challenges as seek times are high and data transfer rates low compared with hard disk. Q provides mastering technology that enables creating a disk image with data organized into large contiguous chunks suitable for streaming in quickly and efficiently.
Q content is divided into "Zones". Each Zone represents an atom of simulation and would typically contain all authored scene data for a single "room" or "area". This is achieved at author-time by organizing the 3D Studio MAX scene file into individual hierarchies labeled by Zone name. Other content, e.g. a character, is also exported in a Zone, but this is usually a temporary "holding" Zone. In practice in the game the character hierarchy is re-parented into the appropriate Zone containing the room geometry.
Zones are connected together by "Portals". A Portal-pair is a pair of associated matching polygons authored into the scene file, one half of the Portal lives in the scene graph of one Zone, the matching half lives in the scene graph of the neighboring Zone. As a "character" or other geometry moves through a Portal in the simulation the engine automatically re-attaches them to the new Zone. A Portal is normally authored to be in line with a visible feature, e.g. a door or opening, but this does not have to be the case.
Application code indicates which Zones' data should be held in memory and simulated by creating objects called "Scopes". A Scope is placed on a Group (the scene graph under a Zone is constructed from a hierarchy of Groups & associated objects) with a given radius, normally 1. The radius is specified in terms of the number of zones, not including the one containing the Scope object. So if the Scope was placed in Zone "E", a radius of 2 would place all zones in scope, as no zone is further than 2 zones away from the Scope object. The figures below show the function of a Scope within an example set of Zones "A" to "J":
The circles represent the Zones and the lines represent Portals connecting them. The following figure shows a Scope created with radius 1 on a Group in Zone "A", e.g. the Zone root or a Camera within that Zone:
The red and dark-orange Zones ("A" and "B") are "in-scope". The light-yellow Zones ("C", "D" and "E") are in the "fringe".
The Zone that a Scope is placed on, and all of the neighboring Zones within radius portals of it are referred to as being in-scope. You would normally use a Scope of radius 1 - the default - so only the nearest-neighbor Zones and the Scope Zone are in-scope. This means that they are in memory and being simulated.
If the in-scope Zones have not been requested earlier (preloaded) or streamed in by moving through the level then the in-scope Zones will be demanded from the CD or disk, i.e. loaded in the foreground; this would cause a stall. Zones being simulated can have animations playing on them if appropriate, can be rendered, can be intersected/collided with, and any Entities in those Zones are embodied and ticked.
While the in-scope Zones are being rendered and simulated the fringe Zones are loaded in the background so that if you move to an neighboring Zone, "B" in this example, the fringe Zones can come into scope and have all their data available in memory for simulation and rendering avoiding stalls. Fringe Zones are not rendered.
When the Scope moves, i.e. the Group it is attached to moves, from Zone "A" to Zone "B" the picture changes to:
this will normally be because the Camera or character with the Scope on it has walked through a Portal. The in-scope Zones are now "A", "B", "C", "D" and "E" and the fringe is "F", "G", "H" and "I". An important fact to note here is that while the Scope was in Zone "A" Q was loading the fringe of Zones "C", "D" and "E" (see previous picture). If the Scope moves to Zone "B" before the fringe loaded completely then as soon as the Scope enters Zone "B" the in-scope Zones will be demanded and rendering/animation will stall until the scope content has loaded.
The authoring tool QStudio contains a Mastering Tool that indicates, for a given Scope Zone and Scope radius, which neighboring Zones are "in-scope" and which are in the fringe. It will also calculate the disk and memory sizes of all the Zones and show you the loading time assuming a certain bandwidth. By playing through your game you can then measure the time taken to cross a Zone and ensure, by tweaking the walk speed of the character for example, that the fringe will always be loaded in time.
If the Scope now moves from Zone "B" to Zone "C":
we can see that Zones "F", "G", "H" and "I" that were previously in the fringe will no longer be in the fringe. They are therefore flushed from memory. The objects for Zones "E", "D" and "A" are retained. If you were to move back to Zone "B", the data for Zones "F", "G", "H" and "I" must be reloaded.
At any point in the game you must be able to fit in memory all of the data for the Scope Zone, the in-scope Zones and the fringe Zones. So for the example shown next, where the Scope has moved to Zone "E":
all of the Zones shown must be loadable into memory. If this is not possible the Q Engine will swap data in and out constantly causing a performance penalty. The Mastering Tool in QStudio can be of great use here in calculating the total memory usage for a given Scope Zone.
The in-scope Zones are all being simulated, i.e. any Entities in those Zones have been embodied and are being ticked if required. In that tick they are executing AI code and other game logic. This will obviously use processor time. Various statistics are available through the Q API that show the time spent in the application tick and can be used by application developers to optimize performance. For details refer to Q::Application::getApplicationCounters. The first, most obvious, step in performance analysis however is to simply graph the connectivity of the Zones as we have done above and count the number of Zones in scope at any given time, and the number of Entities within those Zones.
For geometry to be rendered or be intersectable/collideable it must be both:
the render will not stall to load data however. It will only render whatever has been loaded and made available. This impacts how a level is designed.
If, for example, you were to construct a level out of a set of 5 rooms (e.g. Zones A, B, E, I and J in the figure above), each room in its own Zone, all in a line with the doors between them lined up and portals on all the doors then you might think that you would be able to see the content of all of the rooms. If you are using a Scope of radius 1 this would not be true.
When the Camera is in Zone A, only Zones A and B will be in scope, therefore loaded and simulating and rendering. So you would not be able to see into Zone E. You would just see black (or the fog color) where the portal to Zone E is located.
There are several solutions to this with varying suitability depending on the platform/memory available, the style of game-play you want and the visual appearance you want:
Note also that the simulation/load characteristics will also impact how sounds are heard. If a monster two Zones away is not being simulated it will not be playing any sounds.
To improve the performance of loading data from CD or disk there are a couple of optimizations that Q provides: clumping and mastering.
Clumping works by pre-calculating all of the Objects required for a given Zone - i.e. the scene graph hierarchy of Groups, Instances and Geoms together with any marked up assets. It also calculates the differences in objects between all neighboring pairs of Zones.
If, for example, we only consider the Zones A, B and C from the above picture and suppose they have the following objects in them:
Zone A Zone B Zone C a a a b d b c e g d f h
The clumps calculated will have the following object-sets:
Zone A Zone B Zone C Zone A->B Zone B->A Zone B->C Zone C->B a a a e b b d b d b f c g e c e g h f d f h
there are no transition Clumps for A->C or C->A in this example as there is no Portal connecting Zones A and C.
When the Scope is first created in Zone A, Q will load the "Zone A" clump of objects a, b, c, d. As there is a Portal connecting to Zone B, Q will also load objects e and f using the Zone A->B clump. The fringe, Zone C, will be background loaded using the Zone B->C clump (objects b, g and h).
Load performance can be further increased by arranging for all of the objects required for a Clump to be placed contiguously on the disk or CD so that they can be read in a sweep without having to re-seek for each object. The mastering tools, QDub and QDS are used to do this after all data has been been exported from source art packages to Q files, extra objects imported from QML files and Entity definitions compiled in. Clumping and mastering are normally performed at the same time right at the end of the data build process.
Not all objects go into the Clumps. Q divides objects into "essential" objects, i.e. objects that are strictly required to render or interact with a scene, and "luxury" objects, i.e. objects that increase the visual quality or detail but are not strictly required.
Luxury objects include:
Luxury objects are always loaded after Clumps.
Entities are created at runtime in game-code to control e.g. flickering lights, moving monsters, characters, pickups, etc.. Since they are not authored into the scene we need an alternative mechanism to indicate how their objects are to be loaded and streamed.
To achieve this we use the Game Schema and Placement files, authored in QStudio. The Game Schema defines distinct types of Entity, e.g. BigMonster, PlayerCharacter, AmmoPickup, and also specifies the "preload assets" required. The preload assets are all of the other persistent objects in the Q database that the Entity will ask for and use/instantiate at runtime. There are two kinds of preload assets: the lifetime assets, which remain resident throughout the lifetime of the entity, and the load assets, which can be discarded after the entity was embodied. For example, a PlayerCharacter will use a Cluster (probably containing its Skin, Meshes and collide-box), a QAM file Clip containing its animation machine definition. Those will be load assets because they will be used for embodying the entity but will not be touched afterwards. The PlayerCharacter may also use some Sound objects for footsteps, breathing. These are lifetime entity because they will be used throughout the lifetime of the character and therefore needs a longer residency. The PlayerCharacter will probably "use" other Entities as well, e.g. bullets, flares, pickups. These dependencies can all be expressed in the Game Schema.
There may also be dependencies specific to a single instance of an Entity in the scene. Say, for example, you have 5 bridges but one of those bridges makes a different sound when you walk on it. You can express the dependence on the Sound asset only for a single bridge using the EntityInstance definition in QStudio, which edits the Placement files.
All EntityInstances are placed in a given position using QStudio - this may be a "start" position for moving objects. Before Clumping and Mastering you can compile the Game Schema and Placement files into the Q files. When this is done, the Clumping process will add the Entity preload assets into the Clumps so that they are controlled by the streaming mechanism discussed above.
If an Entity moves, e.g. a monster, its assets must not be tied to its start Zone. If they were, when that Zone went out of Scope the Entity may have to redemand all of its assets back and cause stalls. The Game Schema allows you therefore to mark Entity types as moving. The engine will then automatically lock the assets into memory for the lifetime of the Entity, regardless of where it is.
There may be some objects that you can't express as dependencies of Entities as they don't have any particular spatial position within the game (e.g. they are for the User Interface), or because you need them for start-up.
For this reason Q allows the game-code to manually request loading of Objects using the Object::adviseResidency() API call. This call takes the following form:
Utils::Result Q::Object::adviseResidency(Q::Object::residencyAdvice,
Q::Object::residencyQualification,
Q::ResidencyListener* = 0);
The residencyAdvice parameter states what you want to do with the object: acquire it - i.e. bring it into memory; discard it - i.e. remove it from memory; or hold it - i.e. acquire the object then hold it in memory.
Note that this call also operates on all of the dependents of the object that you call it on. So, for example, if you do:
Q::Zone z = Q::Zone::find("zone_start");
z.adviseResidency(Q::Object::acquire, Q::Object::soon);
then Q will start to load the Zone as well as the entire scene graph under it.
The residencyQualification parameter states when you want it done. The API offers the following possibilities: now, soon or later.
Q has 6 priority classes of which 3 are offered to the API:
Priority Meaning now serviced immediately without queuing soon highest priority queued, non-stalling, request normal used for individual objects that have not been clumped. There shouldn't be any of these in an ideal, clumped set of Q files. clump clump loading of fringe Zones later can be used in a game for readying assets for a couple of Zones away. Also useful for putting a hold on things that have already been loaded as it doesn't overload the background loading system. low luxury asset loading
"now" should be used sparingly as it causes a foreground load/discard to take place which will cause a stall if the data is not already available. "soon" should be used for most items in conjunction with a Q::ResidencyListener object that provides notification of when the load has taken place.
Bootstrapping a game and teleporting within locations in a game are related problems. Bootstrapping is efficient startup loading, i.e. loading the 3D world ready to play while a UI or animation is playing. Teleporting is the creation of a non-overlapping Scope without stalling.
If you aren't too worried about a load stall because you're not actively animating something you can use the following process:
Q::Zone z = Q::Zone::find("zone_start");
Q::Group g = z.root();
Q::Scope s = Q::Scope::create(g);
void MyApplicationListener::onTick(double t)
{
...
if (state == loading) {
unsigned int countdown;
Q::Application::app().preloadScope(&countdown);
if (countdown == 0) {
// set state to finishedLoading... then carry on with using the camera.
state = loaded;
}
}
...
}If you definitely don't want any stalls at all because you're animating or displaying other geometry, or walking through the world but you want to relocate the camera/player to a different area then you can use the following procedure. At present this requires knowledge of the surrounding Zones, but this will be improved for a future API release:
Q::Zone z = Q::Zone::find("zone_jump"); Q::Zone zn1 = Q::Zone::find("zone_jump_neighbor_1"); Q::Zone zn2 = Q::Zone::find("zone_jump_neighbor_2");
MyListener listener(z, 3); z.adviseResidency(Q::Object::acquire, Q::Object::soon, &listener); zn1.adviseResidency(Q::Object::acquire, Q::Object::soon, &listener); zn2.adviseResidency(Q::Object::acquire, Q::Object::soon, &listener); // now keep rendering etc. while it loads. ...
// MyListener listens for when the individual Zones have been loaded struct MyListener : public Q::ResidencyListener { MyListener(const Q::Object& z, int target) : jumpZone_(z), target_(target), count_(0) {} ~MyListener() {} void onLoaded(Q::Object&) { // this method is called each time a Zone has completed loading. ++count_; if (count_ == target_) { // we've loaded it all - create the scope to start the fringe loading. Q::Scope s = Q::Scope::create(jumpZone_.root()); // tell the game it can now teleport and hand the scope to it as we've now finished our // work, so the game can happily destroy this listener at its leisure Game::instance()->getOnWithIt(s); } } private: Q::Zone jumpZone_; int count_; // how many things we've loaded so far int target_; // the number of things we're loading };
Note that "teleporting" in a game doesn't need to take place in this way - it is only offered here as an alternative. Another viable technique is to use a small, "hidden", portal - e.g. up in the air or behind a wall - that connects the Zones that you are teleporting from/to. In this way the normal Q streaming functionality can be used to ensure that the content for the Zone you're teleporting to is loaded/simulated in time.
|
|||
|
|
Qube Software Limited © 2000-2004
|
|