Mobile 3D
Graphics with OpenGL ES and M3G
by Kari Pulli, Tomi
Aarnio, Ville Miettinen, Kimmo Roimela, Jani Vaarala
Link
to Elsevier / Morgan Kaufmann page
Buy the book from Amazon.com
Send feedback to the authors
Appears in the Morgan Kaufmann Series in Computer Graphics
ISBN: 978-0-12-373727-4
ISBN10: 0-12-373727-3
Hardback, 452 pages
Publication Date: November 2007
Price: $59.95
Book Description
Graphics and game developers must learn to program for mobility. This
book will teach you how.
"This book - written by some of the key technical experts...provides a
comprehensive but practical and easily understood introduction for any
software engineer seeking to delight the consumer with rich 3D
interactive experiences on their phone. Like the OpenGL ES and M3G
standards it covers, this book is destined to become an enduring
standard for many years to come."
- Lincoln Wallen, CTO,
Electronic Arts, Mobile
"This book is an escalator, which takes the field to new levels. This
is especially true because the text ensures that the topic is easily
accessible to everyone with some background in computer science...The
foundations of this book are clear, and the authors are extremely
knowledgeable about the subject. "
- Tomas Akenine-Möller,
bestselling author and Professor of Computer Science at Lund University
"This book is an excellent introduction to M3G. The authors are all
experienced M3G users and developers, and they do a great job of
conveying that experience, as well as plenty of practical advice that
has been proven in the field."
- Sean Ellis, Consultant
Graphics Engineer, ARM Ltd
The exploding popularity of mobile computing is undeniable. From cell
phones to portable gaming systems, the global demand for
multifunctional mobile devices is driving amazing hardware and software
developments. 3D graphics are becoming an integral part of these
ubiquitous devices, and as a result, Mobile 3D Graphics is arguably the
most rapidly advancing area of the computer graphics discipline.
Mobile 3D Graphics is about writing real-time 3D graphics applications
for mobile devices. The programming interfaces explained and
demonstrated in this must-have reference enable dynamic 3D media on
cell phones, GPS systems, portable gaming consoles and media players.
The text begins by providing thorough coverage of background
essentials, then presents detailed hands-on examples, including
extensive working code in both of the dominant mobile APIs, OpenGL ES
and M3G.
C/C++ and Java Developers, graphic artists, students, and enthusiasts
would do well to have a programmable mobile phone on hand to try out
the techniques described in this book.
The authors, industry experts who helped to develop the OpenGL ES and
M3G standards, distill their years of accumulated knowledge within
these pages, offering their insights into everything from sound mobile
design principles and constraints, to efficient rendering, mixing 2D
and 3D, lighting, texture mapping, skinning and morphing.
Along the way, readers will benefit from the hundreds of included tips,
tricks and caveats.
- Written by key industry experts who helped develop the
standards of the field
- Hands-on code examples are presented throughout the book,
and are also offered on the companion website
- Provides examples in the two most popular programing
interfaces, OpenGL ES and M3G
OpenGL ES example set
This archive contains a set of OpenGL ES example programs,
a simple example framework, and ports of it for various platforms (desktop Windows, S60 2nd ed, S360 3rd ed, and UIQ3).
Also, S60 bindings for Python can be downloaded.
The source code is free to use in any application, and is provided
as-is without any warranties. If you wish to distribute the source
code, we ask that you refer back to http://www.graphicsformasses.com
for the original, latest version.
Java example code
Working versions of the Java example code from the book is available (both
source code and binaries).
The source code is free to use in any application, and is provided
as-is without any warranties. If you wish to distribute the source
code, we ask that you refer back to http://www.graphicsformasses.com
for the original, latest version.
The main "com.graphicsformasses" package contains the executable example
MIDlets. Each MIDlet corresponds to a chapter in the book. The source
code for the actual examples is contained in the "ch12"-"ch16"
sub-packages.
Additionally, there are two common packages. The "framework" package
contains our demo framework, which lets you quickly try out M3G
features by just overloading a couple of methods in the provided
classes. The "tools" package contains some helper classes and methods
used in the examples. For more details, refer to documentation in the
source code for those packages.
The Earth images in the "res" folder are derived from images owned by
NASA and used subject to NASA Terms Of Use. The original images are
located in the NASA Visible Earth repository.
Errata
Several errors have unfortunately crept into the book, but
will be fixed in the next revision.
SECTION 13.4.2 Iterating and cloning
The getUniqueObjects method does not work properly. Corrected code
example follows (also better documented, and renamed to be in line
with M3G naming conventions).
/**
* Finds all Object3D instances that are reachable from the given
* root object. The returned objects are guaranteed to be unique,
* but they are not in any particular order.
*
* This method is not very efficient: it takes O(N^2) time and
* O(N) memory, where N is the number of objects traversed. On
* the other hand, finding objects in the scene graph is typically
* a one-off process.
*
* @param found an Object3D array to store the found objects;
* all elements must be initialized to null
* @param root the Object3D to start searching from
* @param type the Class whose instances to find
*
* @return the number of matching Object3Ds found
*
* @throws IndexOutOfBoundsException if the found array
* is not large enough to accommodate the matching Object3Ds
*/
public static int findUniqueObjects(Object3D[] found,
Object3D root,
Class type)
{
int i, numUnique = 0;
// Retrieve the scene graph objects that are directly referenced
// by 'root' and process them recursively. Note that we create
// a new 'references' array at each step of the recursion; this
// is not recommended as a general practice, but in this case we
// are favoring brevity and clarity over efficiency.
//
Object3D[] references = new Object3D[root.getReferences(null)];
root.getReferences(references);
for (i=0; i < references.length; i++) {
numUnique += findUniqueObjects(found, references[i], type);
}
// Check whether 'root' is an instance of 'type', and if so,
// insert it at the end of the 'found' array, provided that it
// has not been inserted before. If there are no empty slots
// at the end of the array, an IndexOutOfBoundsException will
// be thrown.
//
if (type.isAssignableFrom(root.getClass())) {
for (i=0; found[i] != root && found[i] != null; i++);
if (found[i] == null) {
found[i] = root;
numUnique++;
}
}
return numUnique;
}
SECTION 14.4.1 Background
Example: scrolling background
In the code example, the cropX parameter is erroneously
restricted to the [0, 239] range. Instead, cropX should
be increased without limit. Also, it would be more
intuitive to load the Image2D from a PNG file than to
construct it from a byte[]. Corrected code example
follows:
// initialization
myBg = new Background();
myBg.setColor(0x00CCFFCC);
myBg.setImage((Image2D)Loader.load("/bg.png")[0]);
myBg.setImageMode(Background.REPEAT, Background.BORDER);
// per frame stuff: scroll the background horizontally.
// the screen is 240 pixels wide
myBg.setCrop(myBg.getCropX()+1, 18, 240, 320);
g3d.clear(myBg);
Relevant links
Below are links to some of the most important sites related to mobile 3D graphics.
Mobile 3D graphics
OpenGL ES
M3G
About the authors
Kari Pulli contributed to both OpenGL ES and M3G from the very beginning, and was
among the most active technical contributors to each API. Kari, originally Principal
Scientist and later Research Fellow, headed Nokia's graphics research, standardization,
and technology strategy and implementation, and was Nokia's contact person for both
standards.
Tomi Aarnio, Senior Research Engineer, mostly concentrated on the M3G standard. He
was the specification editor of all versions of M3G, and headed the implementation
project of both its Reference Implementation and the engine that is shipping on Nokia
phones.
Ville Miettinen was active and influential on the definition of the first versions of both of
these graphics standards. At the time he acted as the CTO of Hybrid Graphics, nowadays
he is a freelance consultant.
Kimmo Roimela, Senior Research Engineer at Nokia, also concentrated on the M3G standardization
and implementation. He was the main architect of the M3G's animation
model and an associate editor of the M3G specification. He was also the lead programmer
of the Nokia M3G implementation.
Jani Vaarala, Graphics Architect at Nokia, was very active in the definition of OpenGL ES
standard. He also headed the team that implemented and integrated Nokia's first OpenGL
ES and EGL solution.
Table of contents
Preface xii
About the Authors xv
CHAPTER 1. INTRODUCTION 1
1.1 About this Book 2
1.1.1 Typographic Conventions 3
1.2 Graphics on Handheld Devices 3
1.2.1 Device Categories 4
1.2.2 Display Technology 5
1.2.3 Processing Power 6
1.2.4 Graphics Hardware 8
1.2.5 Execution Environments 9
1.3 Mobile Graphics Standards 12
1.3.1 Fighting the Fragmentation 12
1.3.2 Design Principles 14
1.3.3 OpenGL ES 18
1.3.4 M3G 19
1.3.5 Related Standards 21
PART I ANATOMY OF A GRAPHICS ENGINE
CHAPTER 2. LINEAR ALGEBRA FOR 3D GRAPHICS 27
2.1 Coordinate Systems 27
2.1.1 Vectors and Points 29
2.1.2 Vector Products 29
2.1.3 Homogeneous Coordinates 31
2.2 Matrices 31
2.2.1 Matrix Products 32
2.2.2 Identity and Inverse 33
2.2.3 Compound Transformations 33
2.2.4 Transforming Normal Vectors 34
2.3 Affine Transformations 35
2.3.1 Types of Affine Transformations 35
2.3.2 Transformation Around a Pivot 39
2.3.3 Example: Hierarchical Modeling 39
2.4 Eye Coordinate System 42
2.5 Projections 44
2.5.1 Near and Far Planes and the Depth Buffer 45
2.5.2 A General View Frustum 47
2.5.3 Parallel Projection 50
2.6 Viewport and 2D Coordinate Systems 51
CHAPTER 3. LOW-LEVEL RENDERING 55
3.1 Rendering Primitives 57
3.1.1 Geometric Primitives 57
3.1.2 Raster Primitives 60
3.2 Lighting 61
3.2.1 Color 61
3.2.2 Normal Vectors 63
3.2.3 Reflection Models and Materials 64
3.2.4 Lights 68
3.2.5 Full Lighting Equation 70
3.3 Culling and Clipping 70
3.3.1 Back-Face Culling 71
3.3.2 Clipping and View-Frustum Culling 71
3.4 Rasterization 73
3.4.1 Texture Mapping 74
3.4.2 Interpolating Gradients 82
3.4.3 Texture-Based Lighting 83
3.4.4 Fog 88
3.4.5 Antialiasing 90
3.5 Per-Fragment Operations 92
3.5.1 Fragment Tests 92
3.5.2 Blending 95
3.5.3 Dithering, Logical Operations, and Masking 99
3.6 Life Cycle of a Frame 100
3.6.1 Single versus Double Buffering 101
3.6.2 Complete Graphics System 101
3.6.3 Synchronization Points 102
CHAPTER 4. ANIMATION 105
4.1 Keyframe Animation 105
4.1.1 Interpolation 106
4.1.2 Quaternions 111
4.2 Deforming Meshes 113
4.2.1 Morphing 113
4.2.2 Skinning 114
4.2.3 Other Dynamic Deformations 116
CHAPTER 5. SCENE MANAGEMENT 117
5.1 Triangle Meshes 118
5.2 Scene Graphs 120
5.2.1 Application Area 120
5.2.2 Spatial Data Structure 121
5.2.3 Content Creation 123
5.2.4 Extensibility 125
5.2.5 Class Hierarchy 125
5.3 Retained Mode Rendering 128
5.3.1 Setting Up the Camera and Lights 129
5.3.2 Resolving Rendering State 130
5.3.3 Finding Potentially Visible Objects 130
5.3.4 Sorting and Rendering 132
CHAPTER 6. PERFORMANCE AND SCALABILITY 133
6.1 Scalability 134
6.1.1 Special effects 135
6.1.2 Tuning Down the Details 136
6.2 Performance Optimization 136
6.2.1 Pixel Pipeline 137
6.2.2 Vertex Pipeline 139
6.2.3 Application Code 140
6.2.4 Profiling OpenGL ES Based Applications 141
6.2.5 Checklists 142
6.3 Changing and Querying the State 145
6.3.1 Optimizing State Changes 146
6.4 Model Data 146
6.4.1 Vertex Data 147
6.4.2 Triangle Data 148
6.5 Transformation Pipeline 148
6.5.1 Object Hierarchies 148
6.5.2 Rendering Order 149
6.5.3 Culling 150
6.6 Lighting 151
6.6.1 Precomputed Illumination 151
6.7 Textures 152
6.7.1 Texture Storage 152
PART II OPENGL ES AND EGL
CHAPTER 7. INTRODUCING OPENGL ES 157
7.1 Khronos Group and OpenGL ES 157
7.2 Design Principles 158
7.3 Resources 159
7.3.1 Documentation 160
7.3.2 Technical Support 160
7.3.3 Implementations 160
7.4 API Overview 161
7.4.1 Profiles and Versions 161
7.4.2 OpenGL ES 1.0 in a Nutshell 161
7.4.3 New Features in OpenGL ES 1.1 164
7.4.4 Extension Mechanism 165
7.4.5 OpenGL ES Extension Pack 166
7.4.6 Utility APIs 166
7.4.7 Conventions 167
7.5 Hello, OpenGL ES! 170
CHAPTER 8. OPENGL ES TRANSFORMATION AND LIGHTING 173
8.1 Drawing Primitives 173
8.1.1 Primitive Types 174
8.1.2 Specifying Vertex Data 177
8.1.3 Drawing the Primitives 179
8.1.4 Vertex Buffer Objects 180
8.2 Vertex Transformation Pipeline 183
8.2.1 Matrices 183
8.2.2 Transforming Normals 185
8.2.3 Texture Coordinate Transformation 186
8.2.4 Matrix Stacks 188
8.2.5 Viewport Transformation 188
8.2.6 User Clip Planes 189
8.3 Colors and Lighting 189
8.3.1 Specifying Colors and Materials 189
8.3.2 Lights 190
8.3.3 Two-Sided lighting 192
8.3.4 Shading 193
8.3.5 Lighting Example 193
CHAPTER 9. OPENGL ES RASTERIZATION AND FRAGMENT PROCESSING 195
9.1 Back-Face Culling 195
9.2 Texture Mapping 196
9.2.1 Texture Objects 196
9.2.2 Specifying Texture Data 197
9.2.3 Texture Filtering 202
9.2.4 TextureWrap Modes 205
9.2.5 Basic Texture Functions 205
9.2.6 Multi-Texturing 206
9.2.7 Texture Combiners 207
9.2.8 Point Sprite Texturing 209
9.2.9 Implementation Differences 209
9.3 Fog 210
9.4 Antialiasing 211
9.4.1 Edge Antialiasing 211
9.4.2 Multisampling 212
9.4.3 Other Antialiasing Approaches 213
9.5 Pixel Tests 214
9.5.1 Scissoring 214
9.5.2 Alpha Test 214
9.5.3 Stencil Test 215
9.5.4 Depth Testing 218
9.6 Applying Fragments to the Color Buffer 218
9.6.1 Blending 219
9.6.2 Dithering 220
9.6.3 Logic Ops 220
9.6.4 Masking Frame Buffer Channels 220
CHAPTER 10. MISCELLANEOUS OPENGL ES FEATURES 223
10.1 Frame Buffer Operations 223
10.1.1 Clearing the Buffers 223
10.1.2 Reading Back the Color Buffer 224
10.1.3 Flushing the Command Stream 225
10.2 State Queries 225
10.2.1 Static State 226
10.2.2 Dynamic State Queries 227
10.3 Hints 233
10.4 Extensions 234
10.4.1 Querying Extensions 234
10.4.2 Query Matrix 234
10.4.3 Matrix Palette 235
10.4.4 Draw Texture 238
10.4.5 Using Extensions 238
CHAPTER 11. EGL 241
11.1 API Overview 242
11.2 Configuration 244
11.3 Surfaces 248
11.4 Contexts 252
11.5 Extensions 253
11.6 Rendering into Textures 254
11.7 Writing High-Performance EGL Code 255
11.8 Mixing OpenGL ES and 2D Rendering 257
11.8.1 Method 1:Window Surface is in Control 257
11.8.2 Method 2: Pbuffer Surfaces and Bitmaps 258
11.8.3 Method 3: Pixmap Surfaces 258
11.9 Optimizing Power Usage 259
11.9.1 Power Management Implementations 259
11.9.2 Optimizing the Active Mode 261
11.9.3 Optimizing the Idle Mode 262
11.9.4 Measuring Power Usage 262
11.10 Example on EGL Configuration Selection 264
PART III M3G
CHAPTER 12. INTRODUCING M3G 269
12.1 Overview 270
12.1.1 Mobile Java 270
12.1.2 Features and Structure 272
12.1.3 Hello,World 276
12.2 Design Principles and Conventions 277
12.2.1 High Abstraction Level 278
12.2.2 No Events or Callbacks 279
12.2.3 Robust Arithmetic 280
12.2.4 Consistent Methods 281
12.2.5 Parameter Passing 282
12.2.6 Numeric Values 283
12.2.7 Enumerations 284
12.2.8 Error Handling 284
12.3 M3G 1.1 285
12.3.1 Pure 3D Rendering 285
12.3.2 Rotation Interpolation 285
12.3.3 PNG and JPEG Loading 286
12.3.4 New Getters 287
12.3.5 Other Changes 288
CHAPTER 13. BASIC M3G CONCEPTS 289
13.1 Graphics3D 290
13.1.1 Render Targets 290
13.1.2 Viewport 293
13.1.3 Rendering 294
13.1.4 Static Properties 296
13.2 Image2D 297
13.3 Matrices and Transformations 300
13.3.1 Transform 300
13.3.2 Transformable 303
13.4 Object3D 306
13.4.1 Animating 306
13.4.2 Iterating and Cloning 306
13.4.3 Tags and Annotations 308
13.5 Importing Content 311
13.5.1 Loader 311
13.5.2 The file format 313
CHAPTER 14. LOW-LEVEL MODELING IN M3G 319
14.1 Building meshes 319
14.1.1 VertexArray 319
14.1.2 VertexBuffer 320
14.1.3 IndexBuffer and Rendering Primitives 323
14.1.4 Example 325
14.2 Adding Color and Light: Appearance 326
14.2.1 PolygonMode 327
14.2.2 Material 328
14.2.3 Texture2D 329
14.2.4 Fog 332
14.2.5 CompositingMode 333
14.3 Lights and Camera 337
14.3.1 Camera 337
14.3.2 Light 339
14.4 2D Primitives 343
14.4.1 Background 343
14.4.2 Sprite3D 346
CHAPTER 15. THE M3G SCENE GRAPH 349
15.1 Scene Graph Basics: Node, Group, and World 350
15.2 Mesh Objects 351
15.3 Transforming Objects 354
15.3.1 Camera, Light, and Viewing Transformations 355
15.3.2 Node Alignment 356
15.4 Layering and Multi-Pass Effects 360
15.5 Picking 362
15.6 Optimizing Performance 364
15.6.1 Visibility Optimization 364
15.6.2 Scope Masks 365
CHAPTER 16. ANIMATION IN M3G 367
16.1 Keyframe Animation: KeyframeSequence 367
16.2 Animation Targets: AnimationTrack 372
16.3 Timing and Speed: AnimationController 374
16.4 Animation Execution 377
16.5 Advanced Animation 378
16.5.1 Deformable Meshes 378
16.5.2 Animation Blending 385
16.5.3 Creating Discontinuities 387
16.5.4 Dynamic Animation 388
PART IV APPENDIX
A FIXED-POINT MATHEMATICS 393
A.1 Fixed-Point Methods in C 395
A.1.1 Basic Operations 395
A.1.2 Shared Exponents 397
A.1.3 Trigonometric Operations 399
A.2 Fixed-Point Methods in Assembly Language 400
A.3 Fixed-Point Methods in Java 405
B JAVA PERFORMANCE TUNING 407
B.1 Virtual Machines 408
B.2 Bytecode Optimization 409
B.3 Garbage Collection 410
B.4 Memory Accesses 411
B.5 Method Calls 413
C GLOSSARY 415
Bibliography 419
Index 425