McIDAS Programmer's Manual
Version 2015

[Search Manual] [Table of Contents] [Go to Previous] [Go to Next]


Point data

McIDAS-X point data is typically composed of atmospheric and oceanographic data occurring at irregularly spaced locations on the Earth or vertically within the atmosphere or ocean. This type of data storage is most often used with station observations such as synoptic, RAOB or ship reports.

This section describes:

Basic concepts

Point data has two unique attributes:

Each of these attributes is described below along with the limitations imposed on point data.

Point structure

Conceptually, you can think of point data as a spreadsheet with each cell containing a predefined number of data values. Each cell contains data for a specific location at a given instant in time. For example, one cell may contain all the mandatory level RAOB data from Green Bay, Wisconsin, at 12 UTC on 17 January 1996.

Mixed data types

Unlike grids, point objects may contain a combination of data types and units for different elements within a cell. For example, in a point cell of surface hourly information, you can store the following:

Limitations

When accessing point data, the MD (Meteorological Data) file structure has four limitations, which are explained below. These limitations pertain only to the MD file structure and are not limitations of the ADDE point object subsystem. Point servers for other formats, netCDF for example, use configuration files to map the parameter names in the file to McIDAS-compatible names.

Cells

Each cell is limited to 400 elements. For most observational data, this isn't problematic. The exception occurs with observations containing data at several levels of the atmosphere. For example, you can't store all the information for an upper air observation reporting values for eight different parameters at 50 levels of the atmosphere in one cell. Although the cell can accommodate the 400 values (8 x 50), it won't have enough space for the time and geographic location of the observation, which are also provided.

Element names and units

Element names and units are limited to four characters, which can be restricting when designating parameter units, especially derived parameters.

Character string elements

Character string elements are limited to four characters, which can be limiting for any type of alphanumeric parameter, such as station ID or country. You can bypass this restriction by using several parameters strung together to represent strings. McIDAS-XCD uses this method when representing five-character IDs associated with ship reports.

Numeric values

Numeric values can be stored only as scaled integers.

What is a point object?

McIDAS-X point data typically represents data occurring at irregularly spaced locations on the Earth. For this data to be useful, ancillary information about the data must also be known. This ancillary information, combined with the actual point data values, is collectively called the point object.

Each point object in McIDAS-X contains five blocks of information.

The API functions and the procedures for reading point objects are described below.

Reading point objects

Most applications for reading point objects will request one of the following:

The McIDAS-X command PTLIST is an example of an application that reads point objects. The table below lists alphabetically the McIDAS-X library functions for acquiring point data.

Function Description

m0ptget

opens a connection to read a point object

m0ptparm

extracts the parameter information from the command line

m0ptrd

reads the point data block from the server

m0psort

gets the selection parameters from the command line and adds them to a selection array used by m0ptget

These functions are described below along with an explanation of the selection conditions for requesting point objects.

Defining selection conditions

Applications use selection clauses to restrict the information sent from the server to the client. You can tell the server to return only fields that fall within certain thresholds. These selection limitations may include a list of stations, a time range, or a level in the atmosphere. Below is a list of the valid point selection clauses. Additional information for each follows.

Selection clause Description

MAX

maximum number of point objects to find (default=1)

POS

position number in a dataset to retrieve data from (default=ALL if day is specified; default=0 if day is not specified)

SELECT s1 .. sn

list of selection conditions

MAX -- Use this clause to specify the maximum number of point objects returned from the server. To receive all point data matching the selection conditions, use MAX=ALL.

POS -- Use this clause to identify a specific point file in a dataset. This is a relative position based on the dataset description. For example, to request point file 5010 from a dataset containing point files 5001 to 5100, specify POS=10. To read from all the files in a dataset, specify POS=ALL.

SELECT -- Use this clause to identify the selection conditions for limiting the objects returned to the client. The syntax of this clause varies, depending on the request. To include multiple conditions, enclose each clause in single quotes. For example, to limit the list of stations in a surface hourly observation to include only Madison and Milwaukee, you can use: 'ID KMSN, KMKE'. To limit a selection to include only those stations with a temperature between 0° and 32° Fahrenheit, include: 'T 0 TO 32 F'.

Since piecing the selection conditions together can be a difficult task, the m0psort function will build the appropriate SELECT string for you. Use m0psort with any application-level program to retrieve command line keyword parameters and translate them into equivalent selection clauses. For example, if you enter a point command with the following arguments:

SELECT='T[F] 40 50; ST WI, MI; TIME 12 13'

m0psort will return the following information, which can then be passed to the server via m0ptget.

'T 40 TO 50 F'
'ST WI,MI'
'TIME 12 TO 13'

Opening a connection to read the point object

Once the selection conditions are made, the m0ptget function opens a connection to read point data from the server. The calling sequence for m0ptget allows the client to access the data in one of two modes.

In the first mode, the server returns all the parameters for a given data type matching the selection conditions, such as all the decoded information from a METAR report for a given station. If you use the PTLIST command without specifying the PARAM keyword in this mode, the client may not know the valid parameters and units for the data type until a successful return from m0ptget.

In the second mode, the client knows the list of parameters to request and the units they can be returned in, such as temperature in Celsius and wind speed in knots.

The API for m0ptget contains the field asknparm for input and output. If this field is zero, the first mode of data acquisition is assumed and the client retrieves all data associated with this data type. If a list of parameters is specified from the client, the second mode of data acquisition occurs and asknparm will contain the number of parameters to return.

When accessing point data in the second mode, you must supply m0ptget with a list of parameters and units to retrieve. If you adhere to the McIDAS-X command line syntax PARAM=parameter[units], you can use the m0ptparm function to build the parameter and unit list.

Upon successful return from m0ptget, all elements of the point object are returned by the server, except the actual data block. If the client requests all parameters, the parameter and unit block are returned. The data form is also returned, indicating the type of data each parameter is stored in. For character strings, the form is C#; for integer values, it is I#; and for floating point values, it is F#, where # is the number of bytes for this parameter. The scaling factor is also returned for floating point numbers.

If the request can be fulfilled by the server, the parameter, unit, scale and form blocks are filled accordingly, regardless of which mode the client uses for m0ptget. The example in the next section demonstrates the second mode of data retrieval.

Reading the point data

If the point request for a given dataset can be fulfilled, a connection is established between the server and the client and the transaction proceeds. The m0ptrd function reads the point data. It is called continuously until all the data is read. Because each call to m0ptrd may yield character strings, integers and floating point numbers, the McIDAS-X library contains a group of functions for extracting these mixed data type values. You can call the m0ptbufinit function after each m0ptget call to initialize the application environment to more easily extract data from the buffer.

The example below demonstrates point data acquisition, using the m0ptbufc, m0ptbuff and m0ptbufi functions to extract characters, floating point, and integer values from the buffer filled by m0ptrd.

    subroutine main0
    
    implicit none
    include 'ptparm.inc'
    integer      MAXBYTE    ! max bytes the buffer may contain
    parameter    (MAXBYTE = MAXNUMPARM * 4)

    character*24 dataset    ! adde dataset name
    character*80 select(2)  ! list of selection conditions
    integer      nselect    ! number of selection conditions
    integer      nparams    ! number of parameters to return
    character*1  quote  ! single quote
    character*4  params(MAXNUMPARM) ! list of parameters
    character*4  units(MAXNUMPARM)  ! list of units
    character*4  form(MAXNUMPARM)   ! list of return forms
    integer      scales(MAXNUMPARM) ! list of scales
    integer      buffer(MAXNUMPARM) ! data buffer
    integer      ok ! function return value
    double precision  temperature   ! temperature extracted from the
                    ! buffer
    double precision  cloudheight   ! cloud height extracted from the
                    ! buffer
    character*4       id    ! station id extracted from the
                    ! buffer
    character*12      ct    ! temperature
    character*12      czcl  ! cloud height
    character*12      chms  ! ob time
    
    integer           obtime    ! observation time extracted
                    ! from the buffer
    
c---    external library routines

    integer      m0ptbufinit
    integer      m0ptbufc
    integer      m0ptbufi
    integer      m0ptbuff
    integer      m0ptget
    integer      m0ptrd

    data         units/MAXNUMPARM * ' '/

    quote = char (39)

c---    assign the dataset

    dataset = 'RTPTSRC/SFCHOURLY'
c---    set up the selection conditions to retrieve the station id,
c---    the temperature in Celsius, the observation time and the 
c---    first non-ceiling cloud height for five SFCHOURLY stations 
c---    in Wisconsin with a temperature between 60 and 70 Fahrenheit 
c---    for day 1996274

    params(1) = 'ID'
    params(2) = 'T'
    units(2)  = 'C'
    params(3) = 'HMS'
    params(4) = 'ZCL1'
    nparams   = 4

    select(1) = 'MAX=5'

c---    the assignment below is usually more easily performed by the
c---    function m0psort. it is demonstrated in this example in a fully
c---    expanded manner to show the most basic access method.

    select(2) = 'SELECT='
        &              //quote//'ST WI'//quote//' '
        &              //quote//'T 60 TO 70 F'//quote//' '
        &              //quote//'DAY 1999274'//quote

c---    select(2) will contain the following. note that the single
c---    quote characters are an important element of the string:
c---    SELECT='ST WI' 'T 60 TO 70 F' 'DAY 1999274'

    nselect = 2

c---    make the request to the server

    ok = m0ptget (dataset, nselect, select, nparams, params, 
        &                units, form, scales, MAXBYTE, 1) 

    if (ok .lt. 0)then
       goto 999
    endif

c---    upon a successful return from m0ptget, the arrays will contain
c---    the following:

c---    location    params  units   form    scales
c---    1       ID  CHAR    C4  0
c---    2       T   C   F4  2
c---    3       HMS HMS I4  0
c---    4       ZCL1    FT  F4  -2

c---    initialize the system so m0ptbufc/f/i can be used to 
c---    extract the buffer values

    ok = m0ptbufinit (nparams, form, scales) 
    if (ok .lt. 0)then
       goto 999
    endif

c---    continuously call m0ptrd until it indicates there are no more
c---    point data blocks to return.

    call sdest('ID    T[C]  Time   ZCL1[FT]',0)
100 continue
       ok = m0ptrd (buffer) 
       if (ok .lt. 0)then
          goto 999
       endif

c---       if data was found, process the data

       if (ok .eq. 0)then

          ct    = ' '
          chms  = ' '
          czcl  = ' '

c---        extract the station id

        ok    = m0ptbufc (1, buffer, id) 

c---        extract the temperature

        ok    = m0ptbuff (2, buffer, temperature) 
        if (ok .eq. 0)then
            write(ct,FMT='(f9.1)')temperature
        endif

c---        extract the observation time

        ok    = m0ptbufi (3, buffer, obtime) 
        if (ok .eq. 0)then
            write(chms,FMT='(i6)')obtime
        endif

c---        extract the cloud height

        ok    = m0ptbuff (4, buffer, cloudheight) 
        if (ok .eq. 0)then
            write(czcl,FMT='(f7.1)')cloudheight
        endif

        write(cline, 
     &         FMT='(a4,2x,a5,1x,a6,1x,a6)')
     &         id,ct(6:),chms,czcl
          call sdest(cline,0)
          goto 100
       endif

999 continue
    call edest('done',0)
    return
    end

Reading the point-data file header

Occasionally, an application may need to access the file header associated with point data without accessing the data itself. The McIDAS-X ADDE command PTLIST with the FORM=FILE option is an example of such an application.

The m0pthdr function opens a connection to read point data file headers from the server based on the selection conditions shown in the table below.

Selection clause Description

BPOS

beginning file in the dataset or ALL

EPOS

ending file in the dataset

Upon a successful return from m0pthdr, the m0ptrdhdr function is repeatedly called until all the data is returned. Below is a sample code fragment demonstrating the use of m0pthdr and m0ptrdhdr.

    include 'ptparm.inc'
    character*24 dataset
    integer      header(HEADSIZE)
    character*24 selects(2)
    integer      nselects 

c---    request file header positions 1 through 5 from the dataset
c---    RTPTSRC/SFCHOURLY

    dataset = 'RTPTSRC/SFCHOURLY'
    selects(1) = 'BPOS=1'
    selects(2) = 'EPOS=5'
    nselects   = 2

    ok = m0pthdr(dataset, nselects, selects) 
    if (ok .lt. 0)then
       return
    endif

100 continue
       ok = m0ptrdhdr(header) 

c---       if there was an error

       if (ok .lt. 0)then
          return

c---       if a header was successfully returned

       elseif (ok .eq. 0)then
          (process header)
          goto 100
       endif

The contents of a header returned from m0ptrdhdr are described in the table below.

Word Description

0

schema name

1

schema version number

2

schema registration date, ccyyddd

3

default number of rows

4

default number of columns

5

total number of keys in the record

6

number of keys in the row header

7

number of keys in the column header

8

number of keys in the data portion

9

1-based position of the column header

10

1-based position of the data portion

11

number of repeat groups

12

size of the repeat group

13

starting position of the repeat group

14

missing data code

15

integer ID of the file

16 - 23

text ID of the file

24

creator's project number

25

creation date, ccyyddd

26

creator's ID

27

zero-based offset to the row header

28

zero-based offset to the column header

29

zero-based offset to the data portion

30

first unused word in the file

31

start of the user record

32

start of the key names

33

start of the scale factors

34

start of the units

35 - 38

reserved

39

beginning Julian day of the data, ccyyddd

40

beginning time of the data, hhmmss

41

ending Julian day of the data

42

ending time of the data, hhmmss

43 - 60

reserved

61 - 62

file name

63

MD file number

64 - 463

user record, MD coordinates (0,0); not described by the schema; use for storing arbitrary information

464 - 863

names of the file keys

864 - 1264

scale factors for the keys

1264 - 1663

units of the keys

1664 - 4095

reserved

Words 39-42 are filled in only during the production of real-time files. When real-time file data is copied to other MD files, words 39-42 in the destination files are set to null.


[Search Manual] [Table of Contents] [Go to Previous] [Go to Next]