This is an initial version of the VisAD DataRenderer Tutorial. This is a very complex topic. The tutorial starts with an overview and general theory of operation of DataRenderers, then considers the specific design of some example DataRenderers. Please send any suggestions for how it can be improved to hibbard@facstaff.wisc.edu.

In the VisAD system, visad.DataRenderer is an abstract class of objects that transform Data objects into depictions in Display objects, and in some cases transform user gestures back into changes in Data objects. Whenever a visad.Data object is linked to a visad.Display object, via a visad.DataReference object, an object of some concrete subsclass of visad.DataRenderer is part of the linkage. The current hierarchy of DataRenderer subclasses distributed with VisAD is:

visad.DataRenderer
	visad.java2d.RendererJ2D
		visad.java2d.DefaultRendererJ2D
			visad.bom.BarbRendererJ2D
		visad.java2d.DirectManipulationRendererJ2D
			visad.bom.BarbManipulationRendererJ2D
	visad.java3d.RendererJ3D
		visad.java3d.DefaultRendererJ3D
			visad.java3d.AnimationRendererJ3D
			visad.bom.BarbRendererJ3D
			visad.bom.SwellRendererJ3D
			visad.bom.ImageRendererJ3D
			visad.bom.TextureFillRendererJ3D
			visad.cluster.ClientRendererJ3D
			visad.cluster.NodeRendererJ3D
		visad.java3d.DirectManipulationRendererJ3D
			visad.bom.BarbManipulationRendererJ3D
				visad.bom.SwellManipulationRendererJ3D
			visad.bom.CurveManipulationRendererJ3D
			visad.bom.PickManipulationRendererJ3D
			visad.bom.PointManipulationRendererJ3D
			visad.bom.RubberBandBoxRendererJ3D
			visad.bom.RubberBandLineRendererJ3D

The major division of this class hierarchy is between the two graphics APIs currently used to implement VisAD Displays: Java2D and Java3D. The classes under visad.java3d.RendererJ3D generate Data object depictions as Java3D scene graphs. The classes under visad.java2d.RendererJ2D generate Data object depictions as VisAD internal scene graphs, using subclasses of visad.VisADSceneGraphObject, which are rendered using Java2D.

When applications link a DataReference to a DisplayImpl by invoking the DisplayImpl's addReference() or addReferences() method, they can optionally pass an object of class DataRenderer. If they do not pass such an object then a default is constructed by the DisplayImpl. This default is a visad.java2d.DefaultRendererJ2D object for a DisplayImplJ2D and a visad.java3d.DefaultRendererJ3D object for a DisplayImplJ3D. These default DataRenderers implement logic for generating depictions of virtually any VisAD Data object according to virtually any set of ScalarMaps. This generality is necessary for the default DataRenderers in order for applications to be able to visualize virtually any data in any way, without needing to define their own non-default DataRenderers. However, non-default DataRenderers can be selective about which Data objects and which sets of ScalarMaps they accept.

Non-default DataRenderers exist for the following reasons:

We will give examples of DataRenderer subclasses that exist for each of the first three reasons. The visad.bom.ImageRendererJ3D class is designed to generate depictions of rectangular images and image sequences more efficiently than the defaults. The Data object linked via a visad.bom.ImageRendererJ3D object must have a MathType that conforms to one of the patterns:

	((x, y) -> z)
	(t -> ((x, y) -> z))
	((x, y) -> (r, g, b))
	(t -> ((x, y) -> (r, g, b)))

and the Display object must have ScalarMaps (actually a subset of these as needed for the RealTypes in the MathType):

	t -> Animation
	x -> a spatial axis
	y -> a different spatial axis
	z -> RGB
	r -> Red
	g -> Green
	b -> Blue

Further, the domain Set of the Field with MathType ((x, y) -> …) must be a GriddedSet.

The visad.bom.BarbRendererJ3D class is designed to render identically to the default except that flows are rendered by wind barbs rather than arrows. Thus its linked Data object and Display object can have the same broad range of MathTypes and sets of ScalarMaps allowed by the default DataRenderers.

The visad.java3d.DirectManipulationRendererJ3D class is designed to interpret user gestures as data changes for a variety of simple MathTypes and sets of ScalarMaps. The Data object linked via a visad.java3d.DirectManipulationRendererJ3D object must have a MathType that conforms to one of the patterns:

	x
	(..., x, ...)
	(..., x, ..., y, ...)
	(..., x, ..., y, ..., z, ...)
	(x -> (..., y, ...)
	(x -> (..., y, ..., z, ...)

The exact criteria on ScalarMaps of the Display object are complex. In the case of a RealType 'x' or a RealTupleType (..., x, ...), there must be a ScalarMap of x to a spatial axis, with other ScalarMaps of x allowed. In the case of a RealTupleType (..., x, ..., y, ...) or (..., x, ..., y, ..., z, ...) there must be a ScalarMap of at least one of the component RealTypes to a spatial axis. In the case of a FunctionType (x -> (..., y, ...) or (x -> (..., y, ..., z, ...) there must be a ScalarMap of x to a spatial axis and a ScalarMap of at least one of y or z to a spatial axis. These ScalarMap criteria are designed so that there is a way to interpret spatial gestures as unambiguous modifications of at least some values in the Data object.

There is no DataRenderer subclass currently part of VisAD that exists for the fourth reason: to combine multiple Data objects in a single depiction. However, a user did create such a DataRenderer almost immediately after the system's initial release in early 1998 (a heroic effort). The purpose was to texture map one image Data object onto a surface defined by another Data object, where the two Data objects had different spatial sampling resolution. Technically this DataRenderer should have been accompanied by new instances of DisplayRealType to be used in ScalarMaps defining what RealType values would be used to compute pixel coordinates in the texture image Data object. However, the DataRenderer used the existing DisplayRealTypes Red and Green for that purpose (somewhat of a hack solution, but effective).

If you are defining a new subclass of DataRenderer, it should be for one of the five reasons listed in this section. You will need to think about what restrictions your DataRenderer will place on the MathType of its Data object and the set of ScalarMaps in its Display object. You may also need new instances of DisplayRealType to define various parameters of novel rendering techniques.

If you are contemplating writing your own subclass of DataRenderer, a key question is whether there is some other way to accomplish your goals. The alternative to a custom DataRenderer is often a network of Data and Cell (i.e., computational) components, possibly including existing non-default DataRenderers. For example, applications can alter the appearance of Data depictions by replacing the simple network:

	Data -> DisplayImpl

with:

	Data -> CellImpl -> Data -> DisplayImpl

The idea is that the CellImpl computes a new Data object (or perhaps several Data objects) whose depiction generated by existing DataRenderers will have the desired appearance for the original Data object. For example, the derived Data object or objects may include new RealType vaues mapped to Shape, that can be used to "draw" virtually any depiction.

Complex manipulation of Data objects can sometimes be accomplished by linking auxilliary Data objects to the DisplayImpl via existing direct manipulation DataRenderers, with CellImpl components that compute new values for the original Data object based on the user's manipulation of the auxilliary Data objects. For example, RealTuple data objects, draggable via DirectManipulationRendererJ3D, can be placed at vertices of the depiction of a complex Data object, with a CellImpl that moves the corresponding "vertex" of complex Data object.

The visad.bom.FrontDrawer class is a good example. It enables users to draw weather fronts. It includes a Set object linked to the DisplayImpl via a CurveManipulationRendererJ3D. When the user finishes drawing the Set, a CellImpl is executed that smooths the curve represented by the Set, and uses it to derive a complex FieldImpl whose depiction is a repeating frontal shape along the smoothed curve.

It is generally true that most goals can be met with clever networks of existing VisAD components and DataRenderers, allowing programmers to avoid creating new DataRenderer subclasses.

Your new subclass of DataRenderer will be a subclass of visad.java2d.RendererJ2D or visad.java3d.RendererJ3D, unless you are implementing VisAD displays for a new graphics API or doing something equally radical. In fact, your new DataRenderer will probably be a subclass of visad.java2d.DefaultRendererJ2D or visad.java3d.DefaultRendererJ3D if it does not interpret user gestures as Data object changes, and a subclass of visad.java2d.DirectManipulationRendererJ2D or visad.java3d.DirectManipulationRendererJ3D if it does. All of these classes have constructors with no arguments, so your new DataRenderer subclass does not need an explicit constructor unless it needs special arguments from the constructor. For example, the visad.bom.CurveManipulationRendererJ3D is a subclass of visad.java3d.DirectManipulationRendererJ3D that allows users to define UnionsSets of Gridded2Dets with manifold dimension = 1 (typically used to define map outlines) by free hand drawing. Its constructors define arguments for defining conditions on the shift and control keys for enabling user drawing, and a boolean argument to restrict the UnionSet to a single Gridded2Dset.

The real work of generating depictions of Data objects is done by subclasses of visad.ShadowType. Every Data object has a MathType, which is really a tree structure of various subclasses of MathType. For example, the shorthand MathType notation:

	(hour -> ((line, element) -> brightness))

actually represents the tree structure:

        FunctionType (image_sequence_type)
            /                      \
    function domain           function range
    RealType (hour)         FunctionType (image_type)
                               /               \
                      function domain      function range
                      RealTupleType        RealType (brightness)
                      /           \
           RealType (line)     RealType (element)

Recursive algorithms that traverse this tree structure are used to generate depictions of Data objects with this MathType. These recursive algorithms need to be able to store temporary information in the nodes of the tree structure. However, since a Data object may be linked to many Display objects, each with their own DataRenderer, using the MathType objects for temporary storage would lead to conflicts. Furthermore, the recursive algorithms may vary between different DataRenderers. Hence another class hierarchy is needed for building tree structures that "shadow" the MathType tree structure. This is the class hierarchy under visad.ShadowType. A tree structure of ShadowTypes is created for each link between a Data object and a Display object, and different subclasses of ShadowType can be used to define different algorithms for generating Data depictions. The ShadowType class hierarchy includes one set of classes that are independent of graphics API, a set for each graphics API (Java2D and Java3D), and others as needed for non-default DataRenderers. The hierarchy independent of graphics API is:

	ShadowType
		ShadowScalarType
			ShadowRealType
			ShadowTextType
		ShadowTupleType
			ShadowRealTupleType
		ShadowFunctionOrSetType
			ShadowFunctionType
			ShadowSetType

Note the neat correspondence of this hierarchy to the MathType hierarchy, except for the addition of ShadowFunctionOrSetType. This exists because the visualization algorithms for Set and Function objects are essentially identical (Sets are treated as the domain Sets of Fields without any range values), and common code for ShadowFunctionType and ShadowSetType can go in ShadowFunctionOrSetType.

The classes visad.java2d.ShadowTypeJ2D and visad.java3d.ShadowTypeJ3D are subclasses of visad.ShadowType, and these have subclass hierarchies:

	ShadowTypeJ2D
		ShadowScalarTypeJ2D
			ShadowRealTypeJ2D
			ShadowTextTypeJ2D
		ShadowTupleTypeJ2D
			ShadowRealTupleTypeJ2D
		ShadowFunctionOrSetTypeJ2D
			ShadowFunctionTypeJ2D
			ShadowSetTypeJ2D

	ShadowTypeJ3D
		ShadowScalarTypeJ3D
			ShadowRealTypeJ3D
			ShadowTextTypeJ3D
		ShadowTupleTypeJ3D
			ShadowRealTupleTypeJ3D
		ShadowFunctionOrSetTypeJ3D
			ShadowFunctionTypeJ3D
			ShadowSetTypeJ3D

Because Java does not allow multiple inheritance, objects of these classes adapt objects of the corresponding graphics-API-independent classes in order to have access to their methods. For example, the ShadowTypeJ3D class includes the variable:

  ShadowType adaptedShadowType;

and the ShadowRealTupleTypeJ3D class includes the method:

  public ShadowRealTupleType getReference() {
    return ((ShadowRealTupleType) adaptedShadowType).getReference();
  }

ShadowRealTupleTypeJ3D includes similar implementations for every other method it needs to "inherit" from ShadowRealTupleType, and other graphics-API-dependent classes include similar sets of method implementations invoked via adaptedShadowType.

Understanding logic in the ShadowType classes can be a bit tricky, because it moves between methods in the graphics-API-independent classes and methods in the graphics-API-dependent classes. Much of the logic of generating depictions is done in the graphics-API-independent classes which construct VisAD's internal scene graphs (subclasses of visad.VisADSceneGraphObject). These are either converted to Java3D scene graphs by subclasses of ShadowTypeJ3D, or left as is by subclasses of ShadowTypeJ2D (for later rendering using Java2D). Throughout the rest of this tutorial, we will use the notation ShadowTypeJ*D to indicate any graphics-API-dependent analog of ShadowType, and similarly ShadowFunctionTypeJ*D and so on for graphics-API-dependent analogs of subclasses of ShdowType.

A subclass of DataRenderer defines the subclasses of ShadowType it will use to generate Data depiction by implementing a set of factory methods. Here are the implementations of these methods in visad.java3d.RendererJ3D:

  public ShadowType makeShadowFunctionType(
         FunctionType type, DataDisplayLink link, ShadowType parent)
         throws VisADException, RemoteException {
    return new ShadowFunctionTypeJ3D(type, link, parent);
  }

  public ShadowType makeShadowRealTupleType(
         RealTupleType type, DataDisplayLink link, ShadowType parent)
         throws VisADException, RemoteException {
    return new ShadowRealTupleTypeJ3D(type, link, parent);
  }

  public ShadowType makeShadowRealType(
         RealType type, DataDisplayLink link, ShadowType parent)
         throws VisADException, RemoteException {
    return new ShadowRealTypeJ3D(type, link, parent);
  }

  public ShadowType makeShadowSetType(
         SetType type, DataDisplayLink link, ShadowType parent)
         throws VisADException, RemoteException {
    return new ShadowSetTypeJ3D(type, link, parent);
  }

  public ShadowType makeShadowTextType(
         TextType type, DataDisplayLink link, ShadowType parent)
         throws VisADException, RemoteException {
    return new ShadowTextTypeJ3D(type, link, parent);
  }

  public ShadowType makeShadowTupleType(
         TupleType type, DataDisplayLink link, ShadowType parent)
         throws VisADException, RemoteException {
    return new ShadowTupleTypeJ3D(type, link, parent);
  }

In each of these method signatures, the 'type' argument is the corresponding object from the tree structure of MathTypes, the 'link' argument is the DataDisplayLink object that defines the link between a Data object and a Display object, and the 'parent' argument is the parent ShadowType in the tree structure (or null if this ShadowType is the root of the tree).

It is possible that a non-default DataRenderer would consist solely of implementations of some of these factory methods, defining alternate subclasses of ShadowType for generating Data depictions.

Instances of the visad.DisplayRealType class define various types of values used by algorithms for generating Data depictions. These include display spatial axes (e.g., XAxis, YAxis, ZAxis), color components (e.g., Red, Green, Blue), Animation, IsoContour, flow components (e.g., Flow1X, Flow1Y, Flow1Z), etc. Applications do not define subclasses of DisplayRealType. Instead they define new instances of DisplayRealType.

New DisplayRealType instances may imply new rendering algorithms and hence require new subclasses of DataRenderer and ShadowType. However, the default DataRenderers can detect and interpret new instances of DisplayRealType for new spatial, color or flow coordinates, as long as they are components of a DisplayTupleType with a CoordinateSystem whose reference is Display.DisplaySpatialCartesianTuple = (XAxis, Yaxis, Zaxis), Display.DisplayRGBTuple = (Red, Green, Blue), Display.DisplayFlow1Tuple = (Flow1X, Flow1Y, Flow1Z), or Display.DisplayFlow2Tuple = (Flow2X, Flow2Y, Flow2Z). This enables applications to define new spatial, color and flow coordinates without defining new DataRenderers.

A Display is either a local DisplayImpl or a RemoteDisplayImpl, which adapts a local DisplayImpl. Methods of RemoteDisplayImpl simply invoke the corresponding methods of the adpated DisplayImpl, so we only need to understand DisplayImpl. Its doAction() method is invoked when one of its linked Data or DataReference objects changes value, or when some other event such as a Control change occurs, that may require the scene graph for any linked Data object to be recomputed. The doAction() method invokes the prepareAction() method of each linked DataRenderer, which determines if recomputation of the scene graph is required for this DataRenderer, and computes the ranges of RealType values in the linked Data object for autoscaling, if requested by the DisplayImpl (this will happen if this is the first attempt to display any linked data, if the application calls a method of DisplayImpl requesting autoscaling, or if a previous autoscaling request failed to establish a value range for some RealType because of null or missing data).

No current subclass of DataRenderer overrides the implementation of prepareAction() in DataRenderer. This invokes the DataDisplayLink.prepareData() method, which computes default values for DisplayRealTypes, analyzes the ScalarMaps linked to the DisplayImpl via calls to the ShadowType.checkIndices() method, and calls the DataRenderer.checkDirect() method to determine whether this DataRenderer supports direct manipulation for the linked Data object and set of ScalarMaps. The checkIndices() recursively calls itself down the tree structure of ShadowTypes to determine which ScalarMaps are relevant to each subtree of the MathType tree structure, including especially which are relevant to each RealType and TextType (these are the leaves of the MathType tree). The checkIndices() method determines whether the combination of MathType and ScalarMaps are feasible for rendering (e.g., a ScalarMap to Animation is illegal for a RealType occurring in a Function range) and generates an appropriate Exception if not. The checkIndices() method also precomputes lots of information useful for generating Data depictions and saves it in the ShadowTypes.

You probably do not need to override the prepareAction() method in your new DataRenderer. If you need new instances of DisplayRealType that are not new spatial, color or flow coordinates, then you probably do need to override the checkIndices() methods of your new ShadowTypes. The default implementations of checkIndices() are complex in order to deal with arbitrary MathTypes and sets of ScalarMaps. However, most custom DataRenderers deal with restricted MathTypes and ScalarMaps and hence can have much simpler implementations of checkIndices() (and other methods that you may need to override).

After DisplayImpl.doAction() invokes the prepareAction() method for each linked DataRenderer, it uses the RealType range data to autoscale the ScalarMaps if autoscaling is requested, then invokes the doAction() method for each linked DataRenderer. This method has implementations in visad.java2d.RendererJ2D and visad.java3d.RendererJ3D, which invoke the DataRenderer.doTransform() method if the prepareAction() method determined that the scene graph for the linked Data needs to be recomputed. These RendererJ2D and RendererJ3D implementations of doAction() manage the attachment and de-attachment of the scene graph depicting their Data objects to and from the overall scene graph for the DisplayImpl.

You probably do not need to override the doAction() method in your new DataRenderer. None of the DataRenderer subclasses distributed with VisAD override the implementations in visad.java2d.RendererJ2D and visad.java3d.RendererJ3D.

A number of non-default DataRenderers override the implementation of the doTransform() method. This method returns a scene graph depicting the linked Data and has different signatures for different graphics APIs. Hence doTransform() is not declared as a method of the abstract DataRenderer class, but is rather a method name reused in similar ways by DataRenderer's subclasses for different graphics APIs. The implementations of doTransform() in visad.java2d.DefaultRendererJ2D and visad.java3d.DefaultRendererJ3D are quite similar. They both construct a scene graph group node to serve as the parent for the scene graph depicting the linked Data object (a javax.media.j3d.BranchNode for DefaultRendererJ3D and a visad.VisADGroup for DefaultRendererJ2D). They get the root ShadowTypeJ*D for the linked Data object, which will be the root of a ShadowType tree containing results computed by DataRenderer.prepareAction(). They get the linked Data object and catch any RemoteException indicating failure to access a remote Data object (if the Data object is null, they simply return a null value for the parent group of the scene graph which will trigger a "data is null" message in the display). The real work of DataRenderer.doTransform() is done by the call to doTransform() method of the root ShadowTypeJ*D (note that doTransform() is not a method of ShadowType, but is a method of its subclasses). This doTransform() call is bracketed by calls to the preProcess() and postProcess() methods of ShadowType. These were designed into the system as a way to accumulate information during the ShadowType.doTransform() call that is only assembled into a scene graph after it is all accumulated, but this feature has never been used. Your DataRenderer can probably ignore the preProcess() and postProcess() methods. The doTransform() method in both DefaultRendererJ2D and DefaultRendererJ3D calls the DataDisplayLink.clearData() method. The reference to Data is cached in the DataDisplayLink during a DisplayImpl.doAction() cycle, in order to maintain consistency in case Data changes during the process, and in order to avoid multiple retrievals of remote Data. The clearData() method clears this cache. The DataRenderer.doTransform() method also initializes some arrays passed to the doTransform() method of the root ShadowType - more about these later.

The visad.java2d.DirectManipulationRendererJ2D and visad.java3d.DirectManipulationRendererJ3D classes define their own implementations of doTransform() in order to add a test for whether direct manipulation is supported for the linked Data and Displays (this test is primarily on the Data's MathType and the Display's ScalarMaps).

This is a key section because the tree structure under a root ShadowType provides the basis for the way that a scene graph is constructed to depict a Data object. In fact, there are four related tree structures involved:

As noted, the doTransform() recursive calls do not always descend all the way to the leaf nodes in the ShadowType tree structure. Rather, the analysis in the recursive ShadowType.checkIndices() calls determines that certain nodes in the ShadowType tree structure are designated as "terminal" nodes, meaning that doTransform() is not called recursively to the children of these nodes. A ShadowFunctionTypeJ*D node is terminal if it is "flat" (i.e., the range of its FunctionType is a RealType, a RealTupleTyple, or a TupleType of RealTypes and RealTupleTypes). A ShadowSetTypeJ*D is terminal. A "flat" ShadowTupleTypeJ*D (including any ShadowRealTupleTypeJ*D), ShadowRealTypeJ*D or ShadowTextTypeJ*D is terminal if it is not the child or descendant of a terminal ShadowType.

The ShadowType tree structure plays one more key role in the way scene graphs are constructed: DisplayRealType values are passed down the tree structure and used to determine the locations, colors and other graphical attributes of scene graph nodes. Any Real values in a non-terminal Tuple (i.e., a Tuple corresponding to a non-terminal ShadowTupleTypeJ*D) are converted to DisplayRealType values via any applicable ScalarMaps and passed down to any non-Real components of the Tuple. Similarly, Real values from the domain of a non-terminal Field are converted to DisplayRealType values via any applicable ScalarMaps and passed down to the corresponding range Data objects. These values are passed down in the 'value_array' argument of the doTransform() method. At terminal nodes, these "passed down" DisplayRealType values are combined with DisplayRealType values computed from the corresponding Data object to create scene graph nodes.

The key method of the ShadowType subclasses is doTransform(). It has one signature in the graphics-API-dependent subclasses and a different signature in the graphics-API-independent subclasses. Specifically, the signature in the graphics-API-independent subclasses is:

  public boolean doTransform(Object group, Data data, float[] value_array,
                             float[] default_values, DataRenderer renderer,
                             ShadowType shadow_api)
         throws VisADException, RemoteException

and the signature in the graphics-API-dependent subclasses is:

  public boolean doTransform(Object group, Data data, float[] value_array,
                             float[] default_values, DataRenderer renderer)
         throws VisADException, RemoteException

The reason for these different signatures is that the recursive calls to doTransform() are made on the tree of graphics-API-dependent ShadowTypes, but these generally delegate their work by calling doTransform() for their adapted graphics-API-independent ShadowType and that call has an extra 'shadow_api' argument where the graphics-API-dependent ShadowType can pass a 'this' reference to itself. The doTransform() method of the graphics-API-independent ShadowType can use this to invoke methods that require graphics-API-dependent logic (for example, adding geometry and appearance informaiton to a scene graph group).

The arguments to doTransform() are:

Different ShadowType subclasses have different implementations of doTransform(), but they all work in two basic stages: 1) converting data values into DisplayRealType values via ScalarMaps, and 2) for non-terminal ShadowTypes passing the DisplayRealType values to recursive calls to doTransform(), and for terminal ShadowTypes using the DisplayRealType values to construct scene graph nodes. The doTransform() method of a graphics-API-dependent ShadowType typically just invokes the doTransform() method of its adapted graphics-API-independent ShadowType. This starts by checking for null data and other error conditions, then getting a Vector of ScalarMaps and associated housekeeping information. It constructs an array 'float[][] display_values' for accumulating DisplayRealType values converted via ScalarMaps from data values. Then it fills the 'display_values' array with any DisplayRealType values in the 'float[] value_array' argument passed down the tree, as determined from the inherited_values array computed by the checkIndices() method during the prepareAction() phase. The way that 'display_values' is filled with data values varies for different MathTypes. A Real object only has one value, but its RealType may occur in multiple ScalarMaps and so it may fill multiple entries in 'display_values'. A RealTuple object or terminal Tuple object has multiple Real values, each used to fill entries in 'display_values' according to relevant ScalarMaps. A non-terminal Tuple object similarly accumulates entries into its 'display_values' array, then passes this as the 'value_array' argument in recursive doTransform() calls for each Tuple component that is not a Real or a RealTuple. Field and Set objects will have multiple values for the same RealType, one for each sample of the Field or Set. Thus 'float[][] display_values' is doubly indexed to permit an array of multiple values for some of its entries. Note that RealTuple, Set and Field objects may have CoordinateSystems with reference RealTuples whose Real values may occur in ScalarMaps: these must also be converted to DisplayRealType values and put into the 'display_values' array. Finally, Text values are handled specially. They are not passed as arguments down the recursive doTransform() calls, but are stored in variables of one ShadowType and then retrieved by its child nodes in the ShadowType tree. This works because it only makes sense to have a single Text value at any terminal node in the ShadowType tree.

Once all data values have been converted to DisplayRealType values in the 'display_values' array, they are either passed to recursive calls to doTransform() or in terminal ShadowTypes they are used to construct scene graph nodes. In a non-terminal ShadowType the scene graphs returned by the recursive doTransform() calls are all made children of a scene graph group. In a non-terminal Field ShadowType whose domain is a single RealType mapped to Animation or SelectValue, the scene graph group is a Switch, which is linked into the AnimationControl or ValueControl which selects a scene graph child based on Animation or SelectValue behavior.

In a terminal ShadowType, a sequence of calls are made to methods that assemble various kinds of graphical information from appropriate DisplayRealType values in the 'display_values' array. These methods are implemented in ShadowType and are:

If there are any DisplayRealType values for Shape, Text, Flow or IsoContour these are handled specially. Each of these results in data being depicted by some specialized "shape" other than a 0-D, 1-D, 2-D or 3-D "graph" of the data. There are methods in ShadowType named makeFlow(), makeText() and makeContour() which make these various specialized shapes. These methods can be over-ridden in extensions of ShadowType to change the appearance of data depictions.

If there are no DisplayRealType values for Shape, Text, Flow or IsoContour then data are depicted directly via a 0-D, 1-D, 2-D or 3-D "graph". The makePointGeometry() method in ShadowType depicts data as isolated points, eliminating NaN values (i.e., missing values). This is used for data that have manifold dimension = 0, and as a "punt" for data with manifold dimension = 3 but where volume rendering is not done because the topology in (XAxis, YAxis, ZAxis) coordinates is not a LinearSet. Otherwise data are depicted by a 1-D, 2-D or 3-D graph. The only 3-D graph option is volume rendering, which is done in visad.ShadowOrFunctionSetType.doTransform() via 3-D textures (actually a stack of 2-D textures because 3-D texture mapping is not implemented on Windows NT) when the boolean isTexture3D = true. 2-D graphs may be implemented by shaded triangles or by 2-D texture mapping in visad.ShadowOrFunctionSetType.doTransform() when either of the booleans isTextureMap or curvedTexture = true. Note isTextureMap is true only if the topology in (XAxis, YAxis, ZAxis) coordinates is a LinearSet. If curvedTexture = true then the data texture is laid on a sub-sampled surface (for efficiency) and hence rendering is not an exactly accurate depiction. The degree of sub-sampling is controlled by the curvedSize variable in GraphicsModeControl.

For 2-D or 3-D linear textures, missing data (including data not selected in SelectRange) is depicted as either black or transparent, depending in the missingTransparent flag in GraphicsModeControl. For curvedTexture and for non-texture 1-D and 2-D graphs, missing data is handled by removing missing points from the geometry via the VisADGeometryArray.removeMissing() method (with different implementations for different sub-classes). Note this is preceded by a call to Set.cram_missing(), which sets NaNs in the spatial Set (normally NaNs are illegal as Set coordinates) to be detected later by removeMissing(). Map projection discontinuities are removed from 1-D and 2-D geometries via calls to VisADGeometryArray.adjustLongitude() and VisADGeometryArray.adjustSeam(). adJustLongitude() detects and removes lines and triangles crossing a longitude seam (often at the 180 degree date line, but not always). adjustSeam() detects and removes lines and triangles crossing any map projection seam (it is not always accurate in detecting seams, since it uses a heuristic method based on derivatives of DisplayTupleType CoordinateSystem transforms).

Direct manipulation DataRenderers translate user mouse or wand gestures (generally with the right mouse button held down) as changes to Data values. The visad.DataRenderer class defines a context for doing this, as a set of methods that direct manipulation DataRenderers need to implement (these methods have non-abstract implementation in DataRenderer, which must be over-ridden for a direct manipulation DataRenderer to function correctly). Their signatures are:

  // determine if the MathType and ScalarMaps are valid for direct manipulation
  public void checkDirect()
         throws VisADException, RemoteException

  // return reason why direct manipulation is invalid
  public String getWhyNotDirect()

  // save array of spatial locations for manipulation "grab points"
  public synchronized void setSpatialValues(float[][] spatial_values)

  // return minimum distance from mouse ray to a "grab point"
  public synchronized float checkClose(double[] origin, double[] direction)

  // interpret mouse ray as a manipulation of data
  public synchronized void drag_direct(VisADRay ray, boolean first,
                                       int mouseModifiers)

Other methods that direct manipulation DataRenderers may implement (but are not required to) include:

  // may be called by drag_direct() for temporary scene graph change
  public void addPoint(float[] x)
         throws VisADException

  // called when mouse button is released, ending manipulation
  public synchronized void release_direct()

  // may be called by applications to stop manipulation
  public void stop_direct()

  // return the index of the "grab point" closest to the mouse ray
  public int getCloseIndex()

Note that some direct manipulation DataRenderers include implementations of the doTransform() method (with signature appropriate for their graphics API). For example, visad.bom.PointManipulationRendererJ3D, visad.bom.RubberBandBoxRendererJ3D and

visad.bom.RubberBandLineRendererJ3D all include implementations that return empty BranchGroups, since none of them actually creates Data depictions.

The checkDirect() method is called by DataDisplayLink.prepareData() and decides whether this DataRenderer supports the Data's MathType and the Display's ScalarMaps. Rather than returning a boolean, it records its decision by a call to:

  public void setIsDirectManipulation(boolean b)

If checkDirect() decides that it doesn't support the MathType and ScalarMaps, it records a reason in a String to be returned by a call to getWhyNotDirect().

The setSpatialValues() method is called by doTransform() (or by the methods it invokes) to record the "grab points" of the Data depiction in 3-D graphics coordinates. Note that its spatial_values argument array is organized float[3][number_of_points]. If the Data object is a Real or RealTuple, then number_of_points will be 1, but if the Data object is a Field or Set then the depiction will be a curve and there will be many grab points along that curve. Note that for visad.bom.BarbManipulationRendererJ2D and visad.bom.BarbManipulationRendererJ3D the grab point location is head of the wind barb, whereas the (latitude, longitude) location of the wind determines the location of the barb's tail. However, for most direct manipulation DataRenderers the grab point locations coincide with Data spatial locations.

The MouseBehavior invokes the checkClose() and drag_direct() methods when the user holds down the right mouse button (the choice of mouse button can of course be changed by custom MouseBehavior subclasses, and note a wand is substituted for the mouse by visad.java3d.WandBehaviorJ3D). Mouse locations define rays in 3-D space (for 2-D graphics the ray is simply into the screen, i.e., parallel to the Z axis). The ray is passed to checkClose() as an origin and direct, but passed to drag_direct() as a VisADRay (these are equivalent).

The checkClose() method returns the minimum distance from the ray to the grab points passed to the DataRenderer via setSpatialValues(). When the right mouse button is first pressed, the MouseBehavior compares the distances it gets from each direct manipulation DataRenderer linked to the Display. All subsequent mouse motion events with the right button pressed generate calls to the drag_direct() method of the DataRenderer whose checkClose() returned the least distance.

The checkClose() method computes the perpendicular distance from the ray to each grab point. For the closest grab point it determines the closest point on the ray and stores a 3-D vector offset (in variables named offsetx, offsety and offsetz) from the closest point to the grab point. This offset vector is used in drag_direct() to avoid having the data values "snap" to the cursor, if the application has called the DataRenderer method:

  public void setPickCrawlToCursor(boolean b)

with b = true. In this case, the Data value gradually "crawls" toward the mouse location.

Some DataRenderers, such as visad.bom.CurveManipulationRendererJ3D, allow the user to draw new Data depictions even where no depiction exists. In such circumstances their checkClose() implementations sometimes return 0.0f as a way to assert their claim to the manipulation. In order to avoid such DataRenderers monopolizing all manipulations, their constructors have arguments where application can specify conditions on SHIFT and CTRL key states under which they are active. The checkClose() methods of such DataRenderers can call the DataRenderer method:

  public int getLastMouseModifiers()

to get the SHIFT and CTRL key states when the right mouse button was pressed.

When a DataRenderer may have multiple grab points, they may implement the getCloseIndex() method to allow application to retrieve the index of the closest grab point as determined by checkClose(). For example, getCloseIndex() is implemented by visad.bom.PickManipulationRendererJ3D to enable applications to discover which point along a curve (and hence which Field or Set sample) was picked by the user.

The drag_direct() method does the real work of a direct manipulation DataRenderer. It determines a 3-D graphical location from the cursor ray (this involves picking a point along the cursor ray, which is a bit subtle - more about this below), converts this back through any applicable display spatial CoordinateSystem, then back through applicable ScalarMaps, to get up to 3 visad.Real values. These are used to update Real sub-objects of the Data object being manipulated. The Real values are also used to generate Strings passed to the DisplayRenderer.setCursorStringVector() method (to be displayed as a cursor location in the upper left corner of the Display window unless the application has disabled the cursor location display).

The default implementation in DataRenderer.drag_direct() illustrates the functions required of any implementation of this method. First, it checks to make sure that critical information is available (non-null). Then it checks whether the applications has called stop_direct(). Then it extracts the origin and direction of its VisADRay argument and, if pickCrawlToCursor has been set, adds a decreasing fraction of the pick offset to the origin. If it is the first call to drag_direct() after the right mouse click, it gets the grabbed spatialValues location in point_x, point_y and point_z.

Next comes the subtle problem of determining unique new RealType values, which requires a point in 3-D (or 2-D), whereas a mouse location defines a ray consisting of an infinite numbers of points. In the default implementation in DataRenderer.drag_direct(), this ambiguity is resolved in one of two ways. If only one or two ScalarMaps of RealTypes are relevant for the MathType of the linked Data, then these determine a one- or two-dimensional sub-manifold of display space (a line or a plane). In this case the ambiguity is resolved by finding the intersection of the cursor ray with the plane or finding its closest point to the line. Note that the default implementation of DataRenderer.drag_direct() requires that spatial ScalarMaps are to the Cartesian spatial DisplayRealTypes (i.e., XAxis,YAxis and ZAxis) rather than through display CoordinateSystems, just so these one- and two-dimensional sub-manifolds are lines and planes rather than curved. If three ScalarMaps of RealTypes are relevant, then the ambiguity is resolved by intersecting the ray with the plane perpendicular with the ray and containing (point_x, point_y, point_z).

Some non-default implementations of drag_direct() resolve this ambiguity in other ways. For example, visad.bom.CurveManipulationRendererJ3D.drag_direct() allows ScalarMaps to be to non-Cartesian spatial DisplayRealTypes, and resolves the ambiguity by using Newton's method to find the intersection of the cursor ray with curved two-dimensional sub-manifolds in display space. This enables users to draw curves on the surfaces of spheres, for example.

Once a drag_direct() implementation has determined unique new RealType values, it must use them to appropriately modify Data objects. The default implementation in DataRenderer.drag_direct() provides a nice example of doing this in cases when the linked Data is a Real, a RealTuple and a FlatField.