Winter 2008
Solving the McIDAS equation
This is a McIDAS-V 3D display of GOES East, 1 km visible channel (base image), NCEP model moveable vertical cross-section of wind speed (colored contours) and 925 mb wind vectors (white arrows) from near 18 UTC on 7 January 2008 near the time of the Northern Illinois/ Southern Wisconsin EF-3 tornado.
The future of data analysis and visualization looms on the horizon as an SSEC development team puts the finishing touches on the beta release of the next step in the evolution of the Man-computer Interactive Data Access System (McIDAS). Appropriately named McIDAS-V, this iteration of the software will constitute the fifth generation of SSEC’s 35-year legacy. The development team has spent the last year bringing the McIDAS concept to a modern, more versatile computing environment.
In preparation for the impending influx of multispectral and hyperspectral satellite data, McIDAS needed to evolve. Building on the solid, ingenious origins of the system in the early 1970s, the data analysis and visualization capabilities of McIDAS-V integrate powerful and innovative methods that will engage many atmospheric scientists from both research and operations. With new features and functionalities, McIDAS-V will have all the capabilities to support a smooth transition to the next generation of environmental satellites.
We like to think of the development of McIDAS-V using the following “equation:”
McIDAS-V >> VisAD + IDV + HYDRA + McIDAS-X Bridge + New Development
Defining the terms in the equation provides a clear picture of how the integration of these elements creates a powerful system capable of handling complex data analysis and visualization tasks.
McIDAS-V >> VisAD + IDV + HYDRA + McIDAS-X Bridge + New Development
Figure 2: Consecutive overpasses from MODIS (base image, color enhanced 11 um channel) and CALIPSO (cross section of lidar cloud information).
Figure 1: IDV rendering of color-enhanced 850mb temperature field with 3-D wind field greater than 50 kts.
The foundation of McIDAS-V is the Visualization for Algorithm Development (VisAD) open source, Java™ library. VisAD serves as a building block for creating interactive and collaborative data analysis and visualization tools. Developed by SSEC’s Bill Hibbard in the 1990s, VisAD is used by a wide variety of scientists. VisAD combines several important attributes, including:
Figure 3: HYDRA analysis of Mt. Etna eruption of October 2002. The user has selected two EOS/AIRS channels (top left) and by differencing them brings out the SO2 signature of the volcanic eruption (lower left). An image from the IR window channel is shown in the lower right. The top right image shows the AIRS spectra from the SO2 plume and a nearby clear sky area.
- The use of Java for platform independence
- A mathematical data model that describes virtually any numerical data in a systematic way
- A general display model that supports 2-D and 3-D displays, and also supports direct manipulation of multiple data views
- Metadata that is integrated into each data object (e.g., units, error estimates)
- Adapters for multiple data formats (NetCDF, FITS, HDF-EOS, McIDAS-AREA, MD and Grid files, Vis5D, etc.) and access to remote data servers through HTTP, FTP, DODS/OPeNDAP, and ADDE protocols
McIDAS-V >> VisAD + IDV + HYDRA + McIDAS-X Bridge + New Development
The Integrated Data Viewer (IDV) adds interactive data selection and visualization capabilities to McIDAS-V. Developed at Unidata, the IDV is also an open source, Java-based software framework built on top of the VisAD library to provide a versatile data analysis and visualization toolkit for geoscience data. The IDV brings together the ability to display and work with multiple observations within a unified interface, including satellite observations, surface and upper air (radiosonde) observations, gridded data, and radar and profiler data. Together, VisAD and the IDV provide 3-D views of the atmosphere and allow users to interactively slice and probe the data. Users can create cross-sections, profiles, animations and value read-outs of multi-dimensional data sets. The combination will allow McIDAS-V users to easily integrate data from various sources and to interact with flexible displays. The IDV software is freely available from Unidata under the terms of the GNU Lesser General Public License. Figures 1 and 2 provide examples of IDV displays.
McIDAS-V >> VisAD + IDV + HYDRA + McIDAS-X Bridge + New Development
Figure 4: The user has selected a MODIS visible channel image (top left) and an IR window channel image (lower left) and displayed the visible reflectance vs. IR emitted radiance in the scatter plot (right). The user can select a group of pixels and color enhance those pixels on the imagery, identifying specific features from the data. The cold 11um radiances and low reflectance, indicative of cirrus clouds, are colored turquoise, pixels with 11um radiances near freezing and high visible reflectances, indicative of snow cover, are colored red, and pixels with very warm 11um radiances and medium visible reflectance, indicative of hot desert sand, are colored purple.
SSEC scientists have developed a powerful tool to interrogate multispectral and hyperspectral satellite data. The Hyperspectral Data Research Application (HYDRA), based on VisAD, allows scientists to delve deeply into these data. Using HYDRA, scientists can display radiance spectra, multi-band imagery, scatter plots and transects. Created to facilitate the manipulation of observations from the aircraft-based Scanning HIS and EOS/AIRS, HYDRA has also been used to work with EOS/MODIS, MSG/SEVIRI, and METOP/IASI data. With the forthcoming NPP/NPOESS and GOES-R programs in the U.S., the HYDRA data interrogation capability is a key feature being added to the McIDAS-V suite.
Figures 3 and 4 show examples of HYDRA applications to detect the SO2 plume in the October 2002 Mount Etna volcanic eruption and to differentiate cloud types and surface features in western Europe and the Mediterranean Sea. These examples demonstrate how HYDRA capabilities can assist scientists in probing hyperspectral and multispectral data and in developing useful products from current and forthcoming advanced remote sensing instruments. Integrating HYDRA into McIDAS-V will connect this powerful tool with the many features of IDV.
McIDAS-V >> VisAD + IDV + HYDRA + McIDAS-X Bridge + New Development
SSEC developers have ensured that the McIDAS-X user community will have
a smooth transition to McIDAS-V. While the original McIDAS software was developed in the
1970s, McIDAS-X, McIDAS-V’s
predecessor, debuted in the early 1990s (see timeline on page 14) to operate on computers
running the UNIX operating system (the X in McIDAS-X represents the X UNIX). McIDAS-X contains
over one million lines of code, written primarily in the Fortran and C programming languages,
and has a tremendous amount of functionality. Many McIDAS-X users have developed their own software
to enhance McIDAS-X for their specific needs. In developing McIDAS-V, it was immediately evident
that to rewrite the software to reproduce the functionality in Java would be a huge undertaking … there
would need to be another solution.
Figure 5. A McIDAS-X session is running on the right side of the PC or Mac screen, and a McIDAS-V session is running on the left side. Using the Bridge, the GOES image in its native resolution was brought into –V; the projection was changed to account for the GOES E-W oversampling. Next, the Milwaukee/Sullivan Level 2 radar was overlaid in -V. This figure demonstrates the ability to bring an –X image and data into –V and the powerful capabilities of –V to add 3-D information.
That solution is a two-way communication between McIDAS-V and a session running McIDAS-X. The “Bridge” consists of two parts: a McIDAS-V data chooser that communicates with a McIDAS-X session and the McIDAS-X “Listener” which allows outside clients to communicate with the McIDAS-X session via any available, user-specified network port. Together, these two components provide a “Bridge” between McIDAS-X and McIDAS-V. Commands initiated from within McIDAS-V are run in McIDAS-X and the results are visualized in the McIDAS-V environment. This allows sites to continue to use McIDAS-X (including locally developed code) while transitioning to McIDAS-V.
Figures 5 and 6 show an example of the Bridge being used to display McIDAS-X data in McIDAS-V.
An example of how McIDAS-V will provide greatly improved analysis and visualization capabilities is demonstrated in Figure 6, which displays an aviation weather Icing Potential product. A 2-D display can only slice through a single layer of the atmosphere (Fig 6A), while the 3-D display from McIDAS-V (Fig 6B) allows the scientist to see both the horizontal and vertical depth of the super-cooled water potential. This user can also zoom into and around the display, using probes to get precise readouts or cross sections to plan routes.
McIDAS-V >> VisAD + IDV + HYDRA + McIDAS-X Bridge + New Development
Figures 6A & 6B: The Total Icing Potential Diagnostic, a product developed by the NOAA Aviation Weather Center, displays the probability that icing will be found in a given area of the atmosphere. Since McIDAS-X displays only 2-D data, Figure 6A (top right; generated with McIDAS-X) displays a slice of the atmosphere where the potential for icing exists. Figure 6B (left) is from McIDAS-V, and displays the full 3-D depiction where the potential for icing exists. The 3-D display provides a much greater understanding of the depth of this field, as well as its horizontal distribution.
With the completion of the first four elements of McIDAS-V, the development team will turn to new development. The McIDAS-V environment will offer great opportunities to scientists and other users of McIDAS-V to not only create new ways to analyze combinations of data, but also invent new ways to visualize the data. One focus for the future is producing unique combinations of data from disparate sources. The rich environment that the VisAD data and visualization models bring to this equation provides an excellent platform for these explorations.
Developing for the Future
McIDAS-V is a software ensemble containing many valuable features and builds upon the enhanced functionality stemming from more than ten years of VisAD and several years of IDV development. McIDAS-V will also integrate the multispectral and hyperspectral satellite data capabilities from HYDRA and allow a two-way dialog with a McIDAS-X session to make available those data and products. Along with these powerful attributes are improved data and server management, and a friendly and user-configurable interface. Finally, since the software is freely available and open source, we will add our own improvements and those of our collaborators to continue to evolve the capabilities of the McIDAS-V software.
An alpha version of McIDAS-V has been demonstrated and released to McIDAS Users’ Group (MUG) members at their annual meeting in October 2007 (see sidebar). Work continues on completing the integration of HYDRA into the McIDAS-V core, adding features to the Bridge and working with Unidata to enhance the IDV. We anticipate a beta version of McIDAS-V will be available for general release in spring 2008.
