Lesson 3

In this lesson, we will start to add interactive features to our visualization. Panning around, and zooming in and out of a picture is one of the simplest navigation tasks available. In a computer graphics application, this is typically done by managing a "world to viewport transformation". That is, we create a coordinate system that we consider to be "the world", and then map a region of the world to the extent of the screen.

We add interactivity by map actions in screen space (like panning a map, or zooming into a particular pixel) to changes in this world-to-viewport transformation. This combination of event handling (mouse clicks, drags, moves, etc) and coordinate transformations is managed by Lux by an interactor. To enable simple zooming and panning interaction to a Lux visualization, one adds an interactor object to the initialization routine:

Lux.init({
    interactor: Lux.UI.center_zoom_interactor({zoom: 0.3}),
// ...

The interactor object does a few things. First, it replaces the default Lux scene with one that performs a correct window-to-canvas transformation. This makes it easy to support canvases of different proportions; compare this (bad) to this (better). More importantly, it knows how to change the transformation in case the user drags the mouse around. Try panning the map, or zooming in (by scrolling) at a specific position. This transformation obviously influences how the dots are drawn. Because a Lux scene includes the transformation infrastructure, the dots object is automatically handled by virtue of being inside the interactor scene.

The resulting code is nicely structured. The specification of dots is only concerned with its own coordinate system, and the specification of the window-to-viewport transformation is only concerned with how to translate the mouse events to the right transformation parameters.

Extra material

The following is a deeper discussion of issues that arise in graphics applications and drive some of the decisions behind Lux. The discussion will help you understand the philosophy behind Lux, but is skippable if you are only interested in using Lux.

Handling interactions requires inverse transformations

Even though the interactor transformed the position attribute of the dots object from world coordinates to canvas coordinates, computing the right viewport transformation from user interaction is a transformation in the opposite direction! This happens because mouse events (and user interactions in general) is given in screen coordinates.

If we're performing all transformations manually, not only do we have to make sure that they're done for every object in the program, we also have to separately implement (and keep track of) the correct inverses, in case we need to use it later. In Lux, you'll still need to implement both the function and its inverse, but you can tell Lux about this relationship, and Lux will be able to use it. We will come back to this point when adding interactive selection of regions.

Transformations happen on the GPU, but event handling happens on the CPU

Perhaps the main reason for the existence of Lux is bridging the gap that exists between programming in a graphics API like WebGL and the rest of a computer graphics program. This is a serious problem that causes all sorts of code duplication in a computer graphics application.

As illustration, consider the following. How would inverting the window-to-viewport transformation look like in an API that used WebGL for forward-transforming a point (for performance reasons), but the CPU for back-transforming it (because we need to convert the mouse position appropriately)? The forward transformation would be implemented in GLSL (so that we used the GPU), but the backward transformation would be implemented in Javascript. So not only we would have two separate functions, but they would be implemented in two separate languages.

Lux, on the other hand, knows about the semantics of Shade expressions, and can convert them to GLSL if necessary, but can also evaluate them directly in Javascript. As a result, if you write the transformations using Shade expressions, they can be executed wherever it's more convenient. This means we can, for example, write test suites that check if transform functions are actually inverses of each other and, more generally, use Shade expressions.

Aside: the OpenGL <3.0 matrix stack

(If you've never used the venerable OpenGL matrix stack, feel free to ignore this aside.) Another interesting thing about the Lux scene graph transformation is that it is a really small and straightforward extension of the OpenGL matrix stack. Pushing and popping matrices on the matrix stack is classically equivalent to adding affine transform scene graph nodes in APIs like OpenSceneGraph.

In Lux, on the other hand, we have the ability to transform the vertex and fragment program specifications. The matrix stack can be directly implemented in terms of transformations that only involve matrix multiplication. In addition, nonlinear transformations like geographical projections are implemented in the same way, and transformations can be used to change other aspects of the visualization completely independently.

This seems to be a powerful way to structure graphics programs. For example Recording the depth of a fragment (used for shadow mapping, for example) is done simply by adding a transformation that replaces the color with an encoding of the z position; picking replaces the color with an object id; implementing a color-blindness simulator is a matter of creating a transformation that projects the fragment colors into the right subspace, and adding the objects into the new scene.

Lux is a big hammer, but graphics programming is a tough nail

Lux includes a compiler of Shade expressions to GLSL, and an interpreter that matches much of the GLSL semantics. On first inspection it seems like implementing such a large amount of programming-language technology is massive overkill. However, not having a programming-language level integration between Javascript and WebGL brings much worse consequences for programming on the resulting API. I currently know of no other way to build library-level abstractions that reuse shader code, and that avoid the code duplication that ends up littering the vast majority of graphics program. That's mostly the reason I wrote Lux: I think the source code of computer graphics programs is much messier than it needs to be. While I don't think Lux is anywhere near perfect, it seems to be a step in the right direction.

Back to the index.