McIDAS-X User Responses:
Pros and Cons of the IDV

Original Request:

Hello all,

As you know we have the next generation of McIDAS under development and we need your input so that McIDAS-V will meet your needs.

If you are an IDV user,  please tell us what you feel are the pros and the cons of the IDV and what you'd change if you could.

Also please let us know what parts of the IDV you currently use.

Thanks in advance for your comments.
Dee

I am a user of both IDV and McIDAS and have a few comments.

IDV PROS

I really like that I can load a much broader spectrum of data into IDV.  NetCDF datasets are very important to my work, and if they are structured according to IDV standards, I can easily visualize these data in IDV.  In addition, I can pull datasets from places all over the country without having to do a bunch of DATALOC's and deal with ADDE dataset numbers.

I like that all data are properly navigated to the same view so one can combine various data sources together.  This would require quite a number of IMGREMAP's in McIDAS.

I do not use the GUI in McIDAS because I really don't understand its functionality.  The IDV GUI is very user friendly in my opinion, and it is easy to click between datasets and overlay fields.  McIDAS would require 10 commands to do some of the things I can do in a few clicks.

I like the color bar editor in IDV much better than the EU in McIDAS.  I have much more control in IDV.  In addition, the color bar displays in the data units, not in brightness counts.  IDV color bars should have some sort of a "gamma" parameter, where I can focus the bulk of the color variability into one portion of the data range.  This is tedious in IDV, but probably shouldn't be.


IDV CONS

It seems as though you need to know the "magic formula" to get NetCDF into IDV.  There are annoying little things that I have to keep asking Tom Whittaker about that seem to prevent me from properly viewing my data the first time around.   This is still better than current McIDAS, where NetCDF is essentially unsupported.

I would change the huge memory requirements of IDV.  I've had IDV freeze up numerous times on a powerful machine because I was trying to look at a Visible satellite image from the Rockies to the eastern U.S.  I do not want to degrade the resolution of my imagery in order to display them, as my work focuses on studying detailed mesoscale features

I would incorporate some of the scatter plot/transect features available from hydra in IDV.  I find myself frequently going back and forth between IDV and hydra to do my analysis

HDF data should be supported in IDV.  This is an area where IDV is lacking in a major way.

IDV should support more graphical output types such as tiff and postscript.  One should be able to control the dpi of the resulting graphic as well.  I would also add adjustable font sizes and font types for labels and color bars.  The graphics from IDV are not currently publication quality...but they should be if I take the time and detail to set things up the way I want them.

Kris Bedka - SSEC

I am a novice user of idv.  I've used it mainly for 3-d wind plots for some presentations.  I also use it as a quick way to plot up netcdf and grib files.  Its much easier to use for those things than setting up McIDAS to read them.

On the con side, it is a resource hog.  The ability to easily plot different MD file types would be nice.

Steve Wanzong - SSEC

Use IDV extensively to display and analyze (2D and 3D) Doppler weather radar data both Level II and Level III data. Also use to display icing grib files although we are currently having trouble doing this.

Greg Salottolo - NTSB

I haven't used IDV in a while (years), but here goes....

UNITS -- When in mcidas, I just do a "D" key and I get the raw, radiance, TBB and brit values. This is very nice. To do this in IDV (when I looked at it a while ago), I would have to point to the dataset 4 times and request it in 4 units. And then it came out in 4 different windows (I think). This was a big step back. I hope the latest version of IDV is better.

A PIXEL -- In mcidas I can zoom way in a see the size of a given pixel. Say, to measure it's size. But, in IDV, everything was "mapped" and/or "shaped" to be a surface. Yes, this looks nicer, but I couldn't see a 'real' pixel. Again, maybe this has changed, or maybe there's a "turn off interpolation" button that I didn't know about.

Tim Schmit - SSEC

Pros:

The IDV is free and easy to install and configure on my Windows and Linux machines (and on my Mac, if I had one ;-) .  It does not require any "root" or "admin privileges" and comes self-contained.

It continues to improve in capability and performance, with guidance from the community it serves.   Substantial changes between version 2.0 and 2.1 improved performance and usability.  2.2 (now in beta) offers even more improvements and expanded capabilities.

The IDV has no "commands", and thus no "command line" per se; instead, it is driven through the drag-and-drop / point-and-click.  Scripting uses Python for both formula evaluation and "shell scripting" (thinking "mcenv" kind of things here -- often used to create movies or single images for the web).    This means I don't have to constantly look up "help" on command line options.  Data choosers are presented as screens tailored to the unique characteristics of the data type (grids, images, point, aircraft, etc).

It also means that when I want to create a tailored display for making images for the web, that I don't run a bunch of "commands" to create the elements of the display -- I do this with the IDV tools, and it as a "bundle" and then simply invoke the bundle in a shell-script from cron (or whatever).  It's kind of like the difference between creating a text document with "vi" compared to "word" (no judgment intended -- hey, I still use 'vi').

3D displays -- aircraft tracks, iso-surfaces, vertical cross sections.
New displays we haven't even dreamed of yet...

Probes -- vertical profile (like from model data), time-height display, etc.

Doing computations not provided by the library does not require installing compilers and writing Fortran; instead, the built-in Jython library and editor can be used.  For example, recently we needed to compute and display the summation of hourly precip data from a model run -- basically needed to add up the values at grid points for the N hours of the run.  The "code" we put into the IDV Jython user library was this simple:

def addEmUp(grid):
 n = getNumberOfTimes(grid)
 a = 0
 for k in range(n):
   a = grid[k] + a
 return a

The 4-panel (or 2-panel) display mode is great -- I can have 4 independent displays that can be linked for zooming or roaming, and each can have a different color enhancement, etc.

The display window GUI is quite configurable to add/remove tools and bundle references.  "Bundles" are the equivalent of using Mcbasi or mcenv to create a "favorite" display that you do often.  In the IDV, you do this once (complete with data, labels, etc), save it as a "bundle" and put a button in the GUI toolbar so you can just "one-click" it.  [One "edits" a bundle by reloading it, making changes and saving it again...trust me, one should never try to "read" a bundle file!]

(My favorite is a "local radar" display that combines the latest 4 radar images of base reflectivity and velocity, the last hour of GOES images, a county map, a street map for Dane County, and finally a probe into the velocity field so I can roam around and get numerical readouts of the values.  One-button.  GREAT!)

And the "bundle" specified that each field should automatically update every 5 minutes -- so I don't have to worry about manually refreshing.
The "preferences" in the IDV certainly help in the configuration of the "dashboard", too -- since I never use profiler data, I can simply eliminate that Chooser from the GUI, for example.  But GUI design is a big challenge -- perhaps that's why so many books have been written on the subject.

[By the way, Version 2.2 will also allow for saving remote data with the bundle, or copying it to a local disk (think"imgcopy").  Great for making quick case studies...  (It also adds annotation on the screen using HTML-embedded strings for those who want it...)]

Quick, automated remapping.  If I don't like the projection I can change it with a couple of clicks.  there is even an interactive way to define my own new basemap.  Of course, lat-lon lines and dashed maps are available, too.

Data "fusion" -- not only into a common projection (automatically), but numerical combinations without much fuss and bother (and programming).  I can easily put up a LEO and GEO in the same projection.  I can get Google Earth files and include them, or data from WCS servers or shape files.  All together...

Tailoring the mouse button actions.  I use the "roller" on my mouse for zooming in and out.  I use the right-button drag to roam (the default is to rotate the 3D display, but I roam more than I rotate ;-)
Speaking of zooming:  I can load up 1km data on the national scale and then zoom into it to reveal the higher res data.  Of course, this one is easy to abuse, since 1km data gobbles up a lot of memory....

Finally, the "plug-in" architecture will allow tailoring the functionality to particular environments.  This is an excellent approach to take to help with the first "Con" I list, below.

Cons:

There are essentially two GUIs in the IDV -- the main display and the 'dashboard'.  The IDV is a "reference application", meaning it is there to test all the capabilities built into the library.  As such, there is a lot there, and it can be daunting.  To that end, a "simple IDV" plug-in was created that removes many of the 'dashboard' options.
It helps...but as data needs expand, you can bet that there will be more "Choosers" and probably more "Controls".

Memory usage is a "con" for me, for right now.  The biggest culprit is images -- partly due to my tendency to put up full-sized, full-resolution images (point, above), and partly due to how images are stored in memory.  Fortunately, work is being done on the latter -- I'm not sure I can change that easily.  The last two updates to the IDV have improved performance in grids and point data manipulation and display so that I don't see that as an issue anymore....

I'd like to mention one more item -- more of a "challenge", though, than a pro or con:

The Unidata program has branched out beyond Meteorology at the encouragement of NSF, their primary funding agency.  To that end, they are embracing other fields (hydrology, oceanography, etc) and thus need to insure that their flagship tools can work with all these data. The plethora of data formats and lack of adherence to conventions by data providers.  It's bad enough that even in meteorology we cannot agree on a format or convention for our data files; add to that the other disciplines that Unidata needs to integrate, and you see a big, on-going challenge.

Also please let us know what parts of the IDV you currently use.

Mostly the displays and analysis I mentioned above for "weather briefing", plus the ability to generate images and movies in the background, off a scheduler.

Tom Whittaker - SSEC

Is it fair to add as a temporary "con" that IDV on a Mac is a 6-bit display for the satellite imagery? (But that is going to be fixed, right?)

Matthew Lazzara - SSEC

Greetings, MUG -- My comments have been raised before, but at least for the McIDAS-X work done within the Retrievals/Clouds group within CIMSS,  I think they bear repeating:

We need to be able to replicate what we now do in "standard" McIDAS-X within  McIDAS-V, so that we will be able to migrate our work as painlessly as possible when McIDAS-V supplants McIDAS-X.

1.)   We need to be able to run McIDAS commands directly from the UNIX prompt.

You are assuming that there are such things as "McIDAS commands".  I recently showed Kathy Strabula an example, using the IDV, of how one of her scripts might appear in Mc-V.  In the "shell", her entire set of "Mcidas commands", which loaded up and remapped 8 images into a common projection and then added a bunch of labeling to the display, and then saved the display as a JPG image, was replaced by an IDV "bundle" and 3 lines of Python code in the "shell script".

It's a different "model"....in Kathy's case, all the "commands" that were used to create the display were replaced with a "bundle" -- using "drag and drop" to do things instead of a series of Mc-X commands.

Also, keep in mind that the current direction will allow you to "be in" the Mc-V environment and run your old "Mc-X" commands, from a "command line" built into the Mc-V and communicating with a "mcenv" session....somewhere (on your machine or elsewhere).  We believe this will allow you to continue to use your Mc-X commands and batch files during the transition.

2.)   We use  mcenv  very heavily, almost exclusively within UNIX  ksh  scripts. This functionality *must*  be maintained in  McIDAS-V.

Same comment....

3.)    This is probably ignorance on my part, but I have heard comments raised that indicate, for example, that something as basic as the D command is not as easy to run in McIDAS-V as it is in the current McIDAS-X.   There seem to be more steps required to get data at the cursor position in all available calibration units.   This ties in to my #4 concern:

The issue is that the current "probe" is for the data that is displayed -- and you read and display "data", not just 8-bit colors. In Mc-X, when you do a "D" key, several messages are exchanged with the ADDE server to fetch the "values" calibrated by the server in a variety of ways.  It's just different -- and there's no reason why we cannot build this capability to return to the server to get these other calibrated values, but for now the "probe" only gets you the values from the data as displayed (in whatever units you specified). But, again, the "model" is different -- in Mc-V you have "data" in your session, in Mc-X you do not...

4.)    Again, probably ignorance on my part, but I am a little concerned about the tendency of McIDAS-V to "remap" or "cast" all input data for a given display into a common projection.   I realize that this is necessary if one wishes to view cross-sections of multiple types of data, but it would be nice to somehow still be able to deal with the "raw" data in its native projection/units.   (Actually, even when the current McIDAS-X plots data on a satellite image, for example, the locations of the data being plotted are being converted to the satellite projection prior to being plotted.)  I guess what I'm really saying is that I still want to be able to deal with raw data in my old, clunky Fortran applications, recompile/link my code, and easily see the results within the McIDAS-V environment.

You have the wrong impression.  You can specify in the preferences that you want all displays to be remapped to a specific projection, but you can also say "use the data's projection"....so if you have a GOES or POES image, you can use that projection.  And of course, once you've displayed a bunch of different types of data ("fusion") you can choose to change the projection to common ones, ones you've created, or the "native" projection of any of the data sets.

Jim Nelson - SSEC (with comments from Tom Whittaker - SSEC)

At one stage I was regularly downloading IDV and evaluating it. Most of the comments below were based on a much earlier release of the IDV, but a quick look at the latest version indicates that they are still relevant.

Things I _really_ like about it
* User's Guide is well formatted
* IDV Web pages, with "stored datasets" for training, sharing interesting cases etc
* saving Bundles
* Preferences viewer/editor
* Interactive Formula Creation
* standard animation control used throughout
* derived parameters are automatically calculated eg dew point, mixing ratio, wind speed, wind vectors/ barbs, 1000-500 thickness
* Projection Manager, which allows interactive definitions of new maps to be used in the applications
* Plan View Control has an "animate level" option
* Colour Table Editor looks quite sophisticated & includes ability to add "breakpoints"
* "Drag and Drop" capability by dragging legend into another app
* Vertical XSection Control looks cool
* Vertical Profile Control to view parameter through height at a particular point (includes animation!)
* Data Probe to get paramter values at a particular x,y point (and altitude for 3D)
* Time Height Display
* Omni control launches a VisAD Spreadsheet display of your data
* vector control includes options for: streamline display, skip interval, vector size & colour
* skewt display of NWP temperature/dew point for any nominated location (includes animation)
 ... coming soon is the full linked displays of hodograph & wind staff & thermodynamic parameters
* time series view of any 2D dataset, for a nominatedex,y point

* reading Tom W's email, I like the sound of the "local radar" display with 1 click but haven't tried it out
* mouse roller for zoom ... nice... see further comments below

Things I don't like about it
* two lots of pull down menus (eg View/Projections vs File/Edit/Displays etc)
* the 3 d controls... our operational users don't use 3d at all... we'd want to disable that
* the default pan/zoom toolbar on the left hand side (see below)
* search help facility throws an exception (in version 2.1 downloaded 23/3/07)
* too may keystrokes/mouse clicks to load initial data sets
(but this could probably be circumvented by creating a bundle & loading it by default, or creating a web page of the required data sets)
* no quick way of switching between maps
* preferred map setting does not continue throughout a session (seems to go back to global setting)
* no quick way (single click/keystroke) of switching between levels and/or parameters
(_but_, there is "animate level" in the Plan View Control; also  it could be done, again, by bundle and using the 1/2/3 keys... or function key control to do the switching)

Things I'd change:
* mouse roller zoom : I'd zoom to where the mouse is currently located... that way you get "pan" for free... you successively zoom & unzoom using the roller (not in the centre of the display) and the display pans at the same time
* while I understand that the mouse behaviour can be customised, I'd prefer the default pan tool to behave differently eg:
 - a single "pan" tool in the toolbar (which I'd have across the top, not along the left hand side)
 - select the "pan" tool
 - once selected, you then pan in the display area by left mouse dragging
 - this behaviour continues until another tool is selected
 (other than the "roller", all our apps can be controlled by left mouse only, in combination with selection of an appropriate "tool". This makes training a bit easier for one thing.)
* the "Layer Manager" is pretty good but could be more succinct:
 - I wouldn't put colour legends on the layer manager by default (user can click on it to find that out)
 - and I'd consider grouping layers together (eg all "model" data) using a "tree" type structure
 - similary all "obs" and "satellite" data could be grouped


As I said, not routinely using it now, but had been evaluating IDV.

James Kelly - Australian Bureau of Meteorology