Space Science and Engineering Center University of Wisconsin-Madison

SSEC/CIMSS scientist Kathy Strabala teaching students at a Direct Broadcast seminar in Honolulu, Hawaii, August 2013. Photo credit: Liam Gumley, SSEC/CIMSS.

As part of its core mission, the Space Science and Engineering Center (SSEC) has long promoted environmental awareness, shared data, and the latest in satellite remote sensing techniques through seminars held around the world…for graduate students and for young researchers already in the field.

As SSEC scientists began actively developing the content and framework for the Cooperative Institute for Meteorological Satellite Studies (CIMSS)/SSEC Remote Sensing and Direct Broadcast seminars, one problem persisted: access to free, specialized visualization software that workshop attendees could use without licensing restrictions.

Out of that need more than a decade ago came the original Hyperspectral Data Viewer for Development of Research Applications (HYDRA) software, a unique visualization tool that allowed users to interrogate and analyze multispectral and hyperspectral data. HYDRA became an integral part of those seminar series developed by SSEC scientists Paul Menzel, Paolo Antonelli, and Allen Huang. But its impact did not end there.

In the last ten years, development of HYDRA has continued with a new purpose – to accommodate the latest hyperspectral meteorological satellite instruments – leading to the release of HYDRA2, as noted in a paper recently published in the Bulletin of the American Meteorological Society.

Example of hyperspectral data interogation using HYDRA2 taken from a classroom laboratory: Analysis of low level water vapor gradients combining CrIS SDR and derived CIMSS dual-regression SFOV retrievals. Users can interactively manipulate various probes to compare full spectra and resulting retrieval. Note how the spatial variation of water vapor (red line in Transect display) tracts the online-offline brightness temperature difference (blue line) on the water vapor absorption line centered on 815.375 cm-1. Image credit: Tom Rink, SSEC/CIMSS.

“The most important thing is that [HYDRA2 is] more flexible in terms of accommodating new polar sensors. The original HYDRA was specific to working with MODIS [Moderate Resolution Imaging Spectroradiometer] and AIRS [Atmospheric Infrared Sounder]. In the process of the new development, I wanted to make sure it had more flexibility,” says Tom Rink, a software developer at CIMSS who worked on both HYDRA and HYDRA2. In fact, “the main motivation [for HYDRA2] was Suomi NPP [National Polar-orbiting Partnership] and JPSS [Joint Polar Satellite System] 1 and 2.”

Expanding the number of satellite sensors covered and making it easier to incorporate additional polar sensors in the future was a significant upgrade in the new version, says Rink. Rink worked as part of a team to create and establish the HYDRA software, and later became the primary developer for the HYDRA2 effort.

The HYDRA2 design expanded to accommodate data from Suomi NPP instruments (Visible Infrared Imaging Radiometer Suite, VIIRS; Cross-track Infrared Sounder, CrIS; Advanced Technology Microwave Sounder, ATMS) and the future JPSS satellites (scheduled to launch in 2017), Europe’s Metop instruments (Infrared Atmospheric Sounding Interferometer, IASI; Advanced Very High Resolution Radiometer, AVHRR; Microwave Humidity Sounder, MHS), and the Chinese FY-3 satellites (Medium Resolution Spectral Imager, MERSI).

While Rink was developing the code for HYDRA2, he had the benefit of a small “steering committee” to guide him, not to mention a history of visualization software success at the Center. In particular, CIMSS scientists Kathy Strabala and Liam Gumley sought to expand on the success of the International MODIS/AIRS Processing Package (IMAPP) software for MODIS data through a newer software package known as the Community Satellite Processing Package (CSPP). While it is possible to use HYDRA2 independently of CSPP, the goal was to integrate HYDRA2 into CSPP, complementing the software package’s capabilities.

“[HYDRA2] been supported by CSPP and IMAPP, and released under their suite of software packages… The intent is for it to work with the generated data products, particularly Level 1B, which contain geolocated spectral measurements calibrated to physical units, and selected Level 2 derived products,” says Rink. The capability to integrate the spectral information with the derived atmospheric parameters can assist students, researchers or weather forecasters in understanding the strengths and weaknesses of the Level2 products.

And CSPP users responded very positively to the addition of HYDRA2 and its unique strengths. When HYDRA2 was initially released as part of CSPP, the number of new registrants rose significantly, with 30 new registrants from around the world, including users in Australia, Peru, Argentina, China, Indonesia, Vietnam, and the United Arab Emirates.

Example of multispectral analysis comparing SuomiNPP VIIRS SDRs with derived CLAVRx products taken from a CSPP workshop in Korea. The statistical relationship between bands and/or band combinations can be examined for a given spatial scene selected as a subset from the full dataset (green box in main window). Users can interactively select clustered points in the scatter plot, say between bands with certain know absorption characteristics, and see their spatial
relationship in the scene. The derived product can be co-located in the same or different window
to provide insight into the algorithm’s process. Image credit: Tom Rink, SSEC/CIMSS.

Getting to that response was the result of not just a lot of hard work, but a lot of planning and discussions. The steering committee—consisting of Strabala, Gumley, and Menzel—provided the vision and requirements for HYDRA2. The driving philosophy, says Rink, was “to do fewer things, but do those things well.” Specifically, the committee urged him to focus on streamlining the process of interactive multispectral interrogation and analysis.

Throughout the development process, Rink regularly sought feedback from the committee. And while their suggestions helped to shape HYDRA2, they were not the only updates that Rink made. Software development has changed dramatically in the last 10 years since HYDRA was first released. Many of the tools and technologies that paved the way for improvements and greater efficiencies within HYDRA2 were not yet available.

“The first version… we were still using text editors to write our code,” says Rink.

For HYDRA2, Rink took advantage of new development tools, including NetBeans (a software development platform known as an Integrated Development Environment, or IDE), profiling tools, such as JProfiler, to improve memory and performance issues, and Install4j to generate cross-platform point-and-click installers. Rink also leveraged work on other SSEC visualization projects, noting that HYDRA2 is “still a VisAD-based application” – using VisAD libraries, or components, that strengthen the visualization and interactivity. In addition, HYDRA2 uses the Java-NetCDF (Network Common Data Form) library developed by Unidata to provide cross-platform access to different data formats from a single Application Programming Interface (API).

The overall software design uses division of labor and what Rink described as a “simple hierarchy.” HYDRA2’s capabilities are compartmentalized into different classes or modules, making edits and updates far easier to manage.

“You can bring in a new dataset, for instance AVHRR from AAPP. I just have to make some modifications which are isolated to the part of the library that deals with adapting the data, while the workflow from the user’s perspective is not affected,” says Rink.

Making the software easy for users to understand and work with was equally important. Describing the primary HYDRA2 user as “anyone who has a DB [direct broadcast] antenna can use this application with the data their site produces,” Rink aimed to make it as intuitive as possible from the beginning. So the software is now easier to install on a variety of platforms. Users do not need to be experts on data formats—they select the data they wish to interrogate and how and HYDRA2 handles the details behind the scenes.

SSEC/CIMSS scientist Liam Gumley teaching students at a Direct Broadcast seminar in Miami, Florida, February 2015. Image credit: Jordan Gerth, SSEC/CIMSS.

Workshop users, as well as users who have downloaded the software on their own, are important in the development of HYDRA2 as they provide useful feedback, whether it be compliments, complaints about bugs, or suggestions for more significant changes. Questions and suggestions have come from around the globe, including Japan, Philippines, Russia, EUMETSAT, as well as places closer to home such as NASA and within our own center. For example, SSEC scientist Bryan Baum has been using HYDRA2 to explore Himawari AHI channels related to fire and smoke detection; he recently suggested a new interactive tool to quickly mask scenes according to threshold or range—a tool that that is now under development. Rink continues to update and improve the software as needed or requested—a process that never really ends.

At the same time, HYDRA2 remains an integral part of SSEC/CIMSS Direct Broadcast and Remote Sensing seminars. SSEC scientists continue to share its specialized capabilities with new users around the globe. That support and user feedback serve as an important measure of the success of HYDRA2—proof that what you have designed and built serves a valuable purpose.

by Leanne Avila