McIDAS Programmer's Manual
Version 2015

[Search Manual] [Table of Contents] [Go to Previous] [Go to Next]


Grid data

McIDAS-X grids are typically composed of atmospheric and oceanographic data, which are produced by numerical models or derived from observational data using an objective analysis scheme. Grids may contain any data that can be represented in a two-dimensional matrix.

Because grid data and image data can both be represented in two-dimensional matrices, it's important to know how they're different. The attributes that distinguish them are listed in the table below.

Attribute Grid data Image data

data volume

low

high

data value resolution

high

low

geographic resolution between data points

low

high

representation to users

graphical contours

gray shading


This section describes:

Basic concepts

Grid data has five unique attributes: valid time, level, parameter, origin and navigation. Each is described below.

Valid time

Grid data is often used to store output from numerical models that simulate the atmosphere and ocean. Because these models predict what the atmosphere or ocean will look like at some time in the future, grids must contain two different time attributes when representing model forecasts:

For example, if a model run on 17 January at 00 UTC generates forecast fields valid 36 hours later, the primary time attribute is 17 January at 00 UTC and the secondary time attribute is 18 January at 12 UTC. If the grid does not represent a model forecast, no secondary time attribute is needed.

Level

Grids store information that represents data at some vertical location in the atmosphere. The level can be the constant height above the surface, such as 100 meters or 32000 feet; the constant pressure surface, such as 500 millibars; or some other meteorological surface, such as the level of the tropopause or isentropic surface.

Parameter

The grid parameter is the data type the grid represents. Parameters can be any field that can be stored as a number, such as temperature, wind speed or dew point. Grids must also contain the stored units used in the grid.

Origin

A grid's origin refers to the process that created the grid. For example, if a grid is created by a forecast model, the origin is the name of the model, such as ECMWF or GFS. If the grid is created from another objective analysis process, that name can appear as the origin.

Navigation

Navigation is the process of determining the latitude and longitude location of data points on a grid. This information is needed when colocating data points from a grid with another data type. For example, you can use this information to contour the gridded field on a satellite image displayed on the McIDAS-X Image Window.

McIDAS-X currently recognizes these grid projection formats:

The diagrams below show how a three-dimensional Earth is represented on a two-dimensional surface for the pseudo-Mercator, polar stereographic and Lambert conformal projection formats, and the orientation of the data points on these projections. A sixth grid projection format is also available, but since it has no navigation, the data points can't be converted to planetary coordinates.

 

For more information about grid projections and McIDAS-X navigation, see the section titled McIDAS navigation later in this chapter.

 

Figure 5-3. Pseudo-Mercator projection.


Figue 5-4. Polar stereographic projection.


Figure 5-5. Lambert conformal secant cone (left) and tangent cone (right) projections

What is a grid object?

Gridded data is two-dimensional data representing an atmospheric or oceanic parameter along an evenly spaced matrix. For the matrix to be useful, ancillary information about the grid must also be known. This ancillary information, along with the gridded data, is collectively called a grid object. Grid objects in McIDAS-X contain two blocks of information.

The API functions and procedures for reading and writing grid objects are described below.

Reading grid objects

Most applications for reading grid objects will do one of the following:

The table below lists alphabetically the McIDAS-X library functions for performing these tasks.

Function Description

mcgdrd

reads the grid header from a grid object

mcgget

opens a connection to read the data block of a grid object

mcgdir

opens a connection to read the grid header of a grid object

mcgfdrd

retrieves the grid file header

mcgridc

reads the grid object and returns the grid in row-major ( C ) format

mcgridf

reads the grid object and returns the grid in column-major (Fortran) format

m0gsort

gets the parameters from the command line and adds them to the selection array for future mcgget call

These functions are described below along with sample code, following an explanation of the selection conditions for requesting grid objects.

Defining selection conditions

Applications use selection clauses to restrict the information sent from the server to the client. For example, selection clauses can restrict the search based on grid time attributes, the parameter type and level, or the process that generated the grid. Below is a list of the valid grid selection clauses as sent to the server. Additional information for each follows.

Selection clause Description

DAY day1 .. dayn

list of primary grid days

DRANGE bday eday inc

range of primary grid days with a day increment

FDAY day

forecast day

FHOUR hour1 .. hourn

list of forecast hours

FRANGE bvt evt inc

range of forecast hours with an hour increment

FTIME time

forecast time

GRIB geographic parameter model level

specific grib numbers to find

GRID bgrid egrid

specific range of grids in a grid file

LEV lev1 .. levn

list of data levels

NUM numgrid

number of grids to find

OUT option

header output format to return

PARM p1 .. pn

list of parameters

POS offset

relative position offset in a dataset

PRO projection

grid projection type

SRC src1 .. srcn

list of sources that generated the grid

TIME time1 .. timen

list of primary grid times

TRANGE btime etime inc

range of primary grid times with a time increment

DAY -- Use this clause to identify a list of primary days that the grids represent. For model forecast grids, these values will be the day the model was initialized. Otherwise, DAY is the Julian day the data represents.

DRANGE -- Use this clause to identify a range of primary days that the grid represents. Enter the beginning and ending day numbers and the increment between days in the range. The increment default is one day.

FDAY -- Use this clause to identify the secondary day attribute for requested forecast grids. For example, if you want only the forecast fields valid for day 1997017, specify FDAY=1997017.

FHOUR -- Use this clause to identify a list of secondary forecast hours that the grids represent. For example if you want only the 12-, 24- and 48-hour forecasts from a model run, specify FHOUR=12 24 48.

FRANGE -- Use this clause to identify a range of forecast hours that the grids represent. Enter the beginning and ending forecast hours and the increment between hours in the range. The increment default is one hour.

FTIME -- Use this clause to identify the secondary time attribute for the grids requested. Use this field with the FDAY clause to isolate grids that are valid at a particular time. The format for the arguments is hhmmss. For example, to request all grids valid at 12 UTC on day 96017, specify FDAY=96017 FTIME=120000.

GRIB -- Use this clause to select grids with the GRIB values specified for the geographic region, the parameter, the model, or the level. This is a useful way to find grids when other sort conditions provide insufficient information to differentiate between grids.

GRID -- Use this clause to access grids based on their position in specific grid files. Since this is an artifact of previous McIDAS-X API functions, avoid using this clause if a practical alternative exists.

LEV -- Use this clause to identify a list (not a range) of levels in the atmosphere or ocean that this data represents. This field is typically filled with height in millibars or words such as SFC, MSL or TRO. To retrieve data for several levels, enumerate them individually.

NUM -- Use this clause to specify the maximum number of grids returned from the server. The default is one grid. To receive all grids matching the selection conditions, use NUM=ALL.

OUT -- Use this clause to receive only the grid header. To get the entire grid header, specify ALL (default). To get only the grid file header, use FILE.

PARM -- Use this clause to specify a list of parameter types to retrieve from the server. To retrieve temperature and height fields, enter PARM=T Z.

POS -- Use this clause to specify a grid file in a dataset. This is a relative position based on the dataset description. For example, to request grid file 5010 from a dataset that contains grid files 5001 to 5100, specify POS 10.

PRO -- Use this clause to specify a projection type for a grid. The valid entries are LAMB, CONF and MERC.

SRC -- Use this clause to specify a list of grid source names to retrieve from the server. This is usually the name of the model or process that generated the grid, such as ETA, NGM or MDX.

TIME -- Use this clause to identify a list of primary times that the grids represent. For model forecast grids, these values will be the time of day that the model was initialized. The format for the arguments is hhmmss.

TRANGE -- Use this clause to identify a range of primary times that the grid represents. Enter the beginning and ending times and the increment between times in the range. The increment default is one hour. The format for the arguments is hhmmss.

You can use the m0gsort function with any application-level program to retrieve command line keyword parameters and translate them into equivalent selection clauses. The table below lists the keywords for accessing grid objects and their equivalent selection clauses. You can also set a flag in m0gsort to disable a request to contain multiple grid selection matches.

Command line keyword Equivalent selection clause m0gsort restrictions

DAY=d1 .. dn

DAY

cannot use with DRANGE

DRANGE=bday eday inc

DRANGE

cannot use with DAY

FDAY=day

FDAY

cannot use with FHOUR or FRANGE

FHOUR=h1 .. hn

FHOUR

cannot use with FDAY, FRANGE or FTIME

FRANGE=bhr ehr inc

FRANGE

cannot use with FDAY, FHOUR or FTIME

FTIME=time

FTIME

cannot use with FHOUR or FRANGE

GPRO=g1 ..gn

PRO

validate with projection: MERC, PS, LAMB, EQUI

GRIB=geo param model level

GRIB

 

GRID=bgrid egrid

GRID

use LAST to get the last grid in the dataset and position; when specified, all other selection conditions are ignored

LEV=l1 .. ln

LEV

 

PARAM=p1 .. pn

PARM

 

SRC=s1 .. sn

SRC

 

TIME=t1 .. tn

TIME

cannot use with TRANGE

TRANGE=btim etim inc

TRANGE

cannot use with TIME

Opening a connection to read the grid header

In ADDE, a client may request only grid headers from a server. This allows the client to sample information on a server without transferring large amounts of grid data that may not be necessary for an application. For example, the McIDAS-X GRDLIST command reads only grid headers. The ADDE interface to the grid directory is through mcgdir, which opens a connection based on a set of selection clauses for a given dataset name.

Reading the grid header

Once mcgdir opens the connection, the application makes repeated calls to mcgfdrd and mcgdrd until all grid and grid file headers are retrieved. The mcgfdrd function is called first to retrieve the grid file header, then mcgdrd is called until it can't find any more grids in the grid file. Then mcgfdrd is called again and the loop continues, as shown below.

    character*32 selects(5)
    character*32 dataset
    integer      grid_header(64)
    integer      file_header(64)
    integer      nselects
    integer      error_flag
    integer      status

c---    assign the dataset name

    dataset    = 'RTGRIDS/ALL'

c---    assign the selection conditions to retrieve six grids from
c---    the dataset RTGRIDS/ALL that are from the ETA, NGM or
c---    MRF model with a primary day of either 96017 or 96019

    selects(1) = 'DAY=96017 96019'
    selects(2) = 'SRC=ETA NGM MRF'
    selects(3) = 'NUM=6'
    nselects   = 3

c---    set an error flag to print a message if an error occurs

    error_flag = 1

c---    open the connection to the server

    status     = mcgdir(dataset, nselect, selects, error_flag) 
    if (status .lt. 0)then
       return
    endif

c---    every time statement 100 is reached, try to read a new grid
c---    file header

100 continue

    hdread = mcgfdrd(file_header)

c---    if you have successfully read the grid file header

    if (hdread .eq. 0)then

c---       every time statement 200 is reached, try to read a new grid header
200 continue

       gread = mcgdrd(grid_header) 

c---       if you have successfully read the grid header

       if (gread .eq. 0)then
          {process grid header here}

c---          see if there are any more grid headers to read

          goto 200

c---       if you have read the last grid from this file, go see if there 
c---       are any more grid files to read from

       elseif (gread .eq. 1)then

          goto 100

c---       if there was a problem reading the grid header

       elseif (gread .lt. 0)then

          return
       endif

    elseif (hdread .lt. 0)then
       call sdest(`Unable to read grid file header',0)
    endif

c---       if you make it to here, hread has returned the value 1,
c---       which means the server has finished sending data


The grid header contains a list of ancillary information about the grid. The entries in the grid header are described in the table below.

Header Word Description

1

total size; rows * columns (not to exceed the value of MAXGRIDPT in gridparm.inc)

2

number of rows

3

number of columns

4

Julian date of the data, ccyyddd

5

time of the data, hhmmss

6

forecast time for the grid, if applicable

7

name of the gridded variable, four character ASCII

8

scale of the gridded variable, specified as a power of 10

9

units of the gridded variable, four character ASCII

10

value of the vertical level
1013 = 'MSL'
999 = ' '
0 = 'TRO'
1001 = 'SFC'
(Otherwise, it is displayed as entered.)

11

scale of the vertical level

12

unit of the vertical level

13

gridded variable type:
1 = time difference
2 = time average
4 = level difference
8 = level average
(or any sum of 1, 2, 4 and 8)

14

used if the grid parameter is a time difference or time average, hhmmss

15

used if the grid parameter is a level difference or level average; values are the same as Word 9

16 - 32

reserved

33

grid origin; identifies the type of program that generated the grid data

34

grid projection type:
1 = pseudo-Mercator
2 = polar stereographic or Lambert conformal
3 = equidistant
4 = pseudo-Mercator (more general)
5 = no navigation
6 = tangent cone

35 - 40

varies, depending on the grid type; see the GRIDnnnn data file in Chapter 6 for more information

41 - 52

reserved; filled only if the grid was created by the McIDAS-XCD GRIB decoder

49

geographic grib number

50

parameter grib number

51

model grib number

52

level grib number

53 - 64

grid description

Opening a connection to read the grid object

In ADDE, a client can request an entire grid object, including the grid header and data block from the server. For example, the McIDAS-X GRDDISP and GRDCOPY commands request entire grid objects.

The mcgget function opens a connection to read the data block of a grid object. It passes an application's selection conditions for requesting grid objects from the client to the server. The return status from mcgget indicates if the application's request can be fulfilled.

The mcgget function also allows the application to separately specify the units and format of the data returned. These are not part of the selection conditions because they are required. Units may be any unit identifier valid for the data type being retrieved. The format parameter can be either I4 for integer or R4 for real number.

Reading the grid data

If a grid in the dataset satisfies the client request, a connection is established between the server and the client and the transaction proceeds. The requested grid objects are read with either the mcgridf or mcgridc function; mcgridf reads the column-major (Fortran) format, while mcgridc reads the row-major (C) format.

Unlike image objects, which require multiple calls to mcalin to get an entire object, mcgridf and mcgridc return an entire grid object with each call. Below is a sample code fragment demonstrating a mcgget/mcgridf calling pair. Note that the mcgget call must occur before mcgridf.

    
        include     'gridparm.inc'
    character*24    dataset
    integer     grid(maxgridpt)
    integer     header(64)
    character*24    selects(8)

c---    set up a request to get the 24-hour 500 mb temperature
c---    field forecast grids from the ETA and NGM model runs at 
c---    12 UTC on day 96017

    dataset    = 'RTGRIDS/ALL'
    selects(1) = 'DAY=96017'
    selects(2) = 'TIME=12'
    selects(3) = 'SRC=ETA NGM'
    selects(4) = 'PAR=T'
    selects(5) = 'LEV=500'
    selects(6) = 'FHOUR=24'
    selects(7) = 'NUM=2'
    nselects   = 7

c---    send the request to the server to return the data as
c---    scaled integers in Celsius

    status = mcgget(dataset, nselect, selects, 'C   ', 'I4',
     &               maxpts*4, 1, numgrids, totbyts) 

c---    if there was an error finding the data requested

    if (status .ne. 0)then
       return
    endif

c---    if you have made it to here, numgrids contains the number
c---    of grid objects the server wants to return to you, so call
c---    mcgridf to retrieve the grid objects

    do 100 I = 1, numgrids

      status = mcgridf (grid, header) 
      if (status .lt. 0)then
         call sdest('error retrieving grid',0)
         goto 100
      endif

c---      do some processing..

100 continue

Writing grid objects to a dataset

Writing a grid object to a grid dataset has two restrictions:

Use the API functions below to write grid objects to a dataset.

Function Description

mcgoutc

writes a grid object stored in row-major format to a file

mcgoutf

writes a grid object stored in column-major format to a file

mcgput

opens a connection to write a grid object

Opening a connection to write a grid object

The request to open a connection for writing a grid object is performed by the function mcgput, which requires the following:

When writing a grid object, the application may create and initialize a file in the destination dataset using the selection conditions below.

Selection clause Description

DEL=YES or DEL=NO

deletes the destination dataset file before recreating it to write the grid object (default=NO)

GRID=num

grid number in a dataset location to write the grid object to; the previous grid stored in this location is overwritten

LABEL=

label to attach to the dataset when the file is created

MAX=num

maximum number of grid objects that can be stored in the newly created file

NUM=

number of grid objects to write to the dataset

POS=pos

position number in the dataset to write the grid object to

Writing the data block

Once the connection is open, the server expects to transfer the number of bytes specified by the entries in the grid header. Transferring too few or too many bytes will result in a error. The mcgoutf and mcgoutc functions send the grid objects to the server. mcgoutf assumes the data block is stored in column-major order; mcgoutc assumes the data block is stored in row-major order. You must call mcgput before calling mcgoutf or mcgoutc, as shown in the sample code fragment below.

    integer grid1(MAXGRIDPT), grid2(MAXGRIDPT)
    integer grid_h1(64) grid_h2(64)
    character*48 selects(10)
    character*48 dataset

c---    write to the dataset LOCAL/GRIDS

    dataset = 'LOCAL/GRIDS'

c---    set up the selection conditions. We will write 2 grid object
c---    to dataset position 3 on the server.

    selects(1) = 'POS=3'
    selects(2) = 'NUM=2'
    selects(3) = 'MAX=100'
    nselects   = 3

c---    initialize the grid headers. Not all fields are shown
c---    in this example. The first grid object will contain 300
c---    mb height fields, the second will contain 250 mb height
c---    fields

c---    assign the size of the first grid object

    grid_h1(1) = 500
    grid_h1(2) = 20
    grid_h1(3) = 25

c---    assign 300 mb level

    grid_h1(10) = 300
    :
    :

c---    copy the first grid header to the second

    call movw(64, grid_h1, grid_h2)

c---    change the level of the second grid header to 250 mb

    grid_h2(10) = 250

c---    assign the total number of bytes that will be transmitted.
c---    this number will be the total number of data points in 
c---    each of the data blocks, plus the size of the grid header
c---    of each of the objects. We multiply the total by 4 because
c---    the storage format of the grid object is whole words and
c---    there are 4 bytes per word.

    ngrids=2
    total_bytes = 8+(grid_h1(1)+grid_h2(1)+(ngrids*64)+ngrids)*4

c---    send the request to write data to the server

    ok = mcgput(dataset, nselects, selects, 1, total_bytes) 
        if (ok .lt. 0)then
       return
    endif

c---    write the first grid object

    ok = mcgoutf(grid1, grid_h1) 
    if (ok .lt. 0)then
       return
    endif

c---    write the second grid object

       ok = mcgoutf(grid2, grid_h2) 
    if (ok .lt. 0)then
       return
    endif

Calculating the number of bytes to transfer

In the sample code above, you will notice that the last parameter in the mcgput function is the total number of bytes to transfer from the client to the server. You must provide this number using the equation below.

total bytes = 8 + 4 * ((number of grids * 64) + (number of data points) + (number of grids))

For example, to transfer two grids to a server where the first grid has 200 data points and the second grid has 300 data points, the total number of bytes to transfer is 2528, as shown below.

8 + 4 * ((2 * 64) + (200 + 300) + 2) = 2528


[Search Manual] [Table of Contents] [Go to Previous] [Go to Next]