McIDAS Programmer's Manual
Version 2015

[Search Manual] [Table of Contents] [Go to Previous] [Go to Next]


McIDAS calibration

Calibration is the process of converting data values sensed by an instrument to useful, physical quantities such as temperature, radiance and albedo. The McIDAS-X calibration subsystem is designed to:

Thus, you can define your own calibration modules that will allow McIDAS-X applications to view the data that you prescribe.

This section describes:

Basic concepts

McIDAS-X has a defined Application Program Interface, including subroutine names, calling sequence and functionality, that all calibration modules must adhere to. This section provides a brief history of how the calibration API was developed and an overview of the applications interface.

Historical perspective

In the mid-1980s, the McIDAS-X area-format image file was redesigned to accommodate the ever increasing number of remotely sensed data types. The redesign allowed programmers to add new datasets from a variety of platforms to McIDAS-X without changing the existing software or file format. The new area file format accommodated multibanded, multibyte data along with a variety of ancillary data. The redesign process also provided a general method of storing, navigating and calibrating these data. It defined an API that all navigation and calibration modules would adhere to, and a mechanism for accessing the appropriate module at application runtime. The previous section in this chapter described the navigation system; this section describes the calibration system.

 

For information about the McIDAS-X area format image file, see Chapter 6, Format of the Data Files.

Calibration API

The calibration API is defined at several levels. For ADDE applications, the call to mcaget has a parameter to request data to be returned in a specific physical quantity. The calibration process occurs on the data server, and although calibration information is returned, it is usually not needed by the ADDE client application.

The next level down in the API is used by non-ADDE applications and by the ADDE data servers. A call to the araopt function, which sets the area options, will use a calibration module if the physical quantity specified in its UNIT option is different than that stored in the image file (word 53 of the area directory). Subsequent calls to redara, which reads the data, will use the calibration module if needed. This provides the application with calibrated data in a rather transparent and data-independent way.

Below araopt in the API is the subroutine kbprep, which provides the interface to the dynamic calibration modules. Most applications don't call the lower level API functions directly. However, you must understand this lowest API definition to incorporate new calibration modules.

The function names used to access the calibration are listed below:

These are not the names of the functions within the calibration module itself. A mapping is done in kbprep that allows applications to use multiple instances of different calibrations. kbprep builds the name of the module to load, based on the slot number and data type.

The numeric (1, 2 or 3) is the slot number, which allows simultaneous use of up to three different calibration modules. The data type, which is stored in word 52 of the area directory, is needed to construct the name of the module to call. For example, the data type for GOES-8 is GVAR. Thus, the name constructed, using slot number 1, is KB1GVAR, which uses the functions KB1INIGVAR, KB1CALGVAR, and KB1OPTGVAR.

Designing your calibration module

All calibration modules have the same framework. They must conform to the McIDAS-X convention for functionality, names of functions, and types of arguments. This standardization allows applications to make use of these modules in a generic, yet powerful way. It is usually not necessary for applications to have any private knowledge of the data it's working with; the calibration interface provides a means to acquire certain aspects that are common to all.

Naming your calibration module

When naming your calibration module, use this form: kbxtype.dlm, where type is the same four characters in word 52 of the area directory. For example, type could be GVAR, TIRO or AAA.

Conforming to module requirements

Each calibration module has identical function names and interface, and performs similar operations. The actual algorithm for calibrating the data is hidden from the application, regardless of the calibration type. All calibration modules contain these three functions:

Function Description

kbxini

initializes and verifies the requested calibration

kbxcal

calibrates the data

kbxopt

provides additional operations, which are usually queries from the application

The structure of these functions is designed so that any required ancillary data is passed in as arguments or handled by kbxopt. An exception is the access to the calibration block. Current McIDAS-X calibration modules requiring the calibration block read the block from disk from within kbxcal. A few modules have the capability to receive the calibration block through kbxopt, kbxgvar for example.

All the functions should return either the requested data or an error status. They should never terminate, but rather rely on the exception handling of the application.

Storing parameters

Most McIDAS-X calibration modules retrieve necessary parameters from one of these locations:

Some calibration algorithms, such as VISR (GOES 1-byte data) don't require additional parameters, or the amount is fixed and relatively small. The VISR calibration was designed for the original GOES satellites, which transmitted its data as 1-byte values. For the visible (even sensor source numbers), the RAW value is the same as the BRIT, and no conversion is necessary. For the IR (odd sensor source numbers), an option for TEMP (temperature) is available through a lookup table coded in the module.

The calibration of these images are handled in the code without an external data structure.

The calibration parameters stored in the area directory, calibration block, line prefix and disk files are described below.

Directory block

The MSAT (Meteosat) calibration parameters are stored in the area directory of an area format image file. Although the entire image only has three constant parameters, they change twice per day. SSEC does not recommend that you store parameters in the area directory, since it lacks available words. A calibration block is the preferred location.

Calibration block

The calibration block contains the calibration parameters for:

This block consists of 128 words and is the most common location for storing calibration parameters. The values in this block are used for the entire image file. Sections in the calibration block are defined for AAA and GVAR.

Normally, integers or scaled integers are stored so that moving the data to different platforms is not a problem. However, potential problems exist when storing character data. For example, when accessing files that are not native to the platform, byte flipping may be needed for integers but not character data. The problem arises when deciding how to flip the bytes, since a schema for storing calibration parameters is not defined. Currently, 4-byte words are tested to determine if all the bytes are printable characters. This does not always work for large or scaled integers. Be aware of this situation when developing new calibration modules, if problems seem to be platform-dependent.

 

For more information about the calibration blocks for AAA and GVAR, see the section titled Image-specific characteristics in the area file description in Chapter 6, Format of the Data Files.

Line prefix

The prefix part of the data line contains the calibration parameters for:

This is the preferred location for image data, where calibration parameters can change throughout the image. For AAA data, two different channels may alternate through the image; TIRO calibration has a different set of parameters every five lines. Although you can define the documentation and calibration sections for specific data, use only the calibration section to store calibration parameters. The documentation section is not guaranteed to move with some copy commands.

 

For more information about the line prefix, see the section titled Data block in the area file description in Chapter 6, Format of the Data Files .

McIDAS-X disk files

Disk files, which are described earlier in this chapter, are used to store calibration parameters for:

The use of these files is decreasing as the definition of the calibration block is now less restricted. Previously, there was a 128-word limit in the calibration block; thus, storing large lookup tables in a disk file was preferable to including them as DATA statements in the code.

For VAS calibration, the line-to-line variability in the calibration coefficients required that the lookup tables be pregenerated for better performance. The file VASTBLS, which accounts for all possible lookup tables, is over six megabytes.

GMS data is received from the satellite as 1-byte values. For the IR, each value corresponds to a temperature; for the visible, an albedo. Because this table is fixed for GMS-3, and the calibration block was restricted to 128 words, a disk file was chosen to store the calibration information. For GMS-5, the tables are provided in the satellite data stream and are stored in a now-expanded calibration block. The disk file is not needed in this case, and will not be used for any future GMS satellites.

Generating lookup tables

Most calibration modules in McIDAS-X generate lookup tables to convert the stored data to some output physical quantity. This is usually preferred over performing the computation for every input data value. For example, a standard McIDAS-X image frame is about 300,000 pixels. Since most input data is eight or 10 bits per value, performing 256 or 1024 computations to create a lookup table is much more efficient than doing 300,000. These are many byte and word manipulation routing in McIDAS that can help to simplify the code in calibration modules. These can be found in the Conversion Utilities section of Chapter 4.

The table below describes some of the data-specific functions you will use in your calibration modules. These functions pass values through a lookup table for performance and memory considerations.

Function Description

maaatb

AAA-specific mpixtb

mavhtb

AVHRR-specific mpixtb

mvastb

VAS-specific mpixtb

mgvatb

GVAR-specific mpixtb

For example, AAA data is sent by the satellite as 10-bit data, but is stored on disk as 15 bits. Rather than generating a lookup table of 32,768 values (for 15 bits), a table of 1024 values is made for the 10-bit data. maaatb does the bit shifting to take the raw 15-bit data to 10-bit, and then passes it into the lookup table.

 

For more information on conversion utilities, see the section titled Conversion utilities in Chapter 4, McIDAS-X Utilities.

Writing your calibration module

The sample calibration program named KBXSIN.DLM, which follows, illustrates the structure of a calibration module. It accepts input data, from 0 to 255, and returns values modified by a sine curve. To use this module with existing 1-byte data, run the following McIDAS-X command:

Type:  IMGCHA dataset STYPE = SIN CTYPE = RAW

After compiling KBXSIN.DLM and any appropriate applications, McIDAS-X applications can be run against the data.

The three required functions below are present in the sample code. An additional subroutine, maktab, generates the lookup table.

kbxini takes its input, usually from araopt, and copies it to a local buffer, as shown in line 53. It then verifies that the calibration requested is valid; see lines 59 and 60.

kbxcal is usually not called directly from the application, but rather when required by redara. It takes as input:

The calibrated data is returned through the same buffer. kbxcal checks if the lookup table was generated (line 118). If not, a call is made to maktab ( line 119). mpixtb completes the calibration (line 125) by taking the data, passing it through the lookup table, and expanding or packing the bytes.

kbxopt contains additional operations for querying information about the calibration. The KEYS option, shown in lines 182-187, passes a frame directory block to the calibration module; the number and list of physical quantities are returned.

This option was written for the McIDAS-X IMGPROBE command, which lists the stored data values converted to appropriate quantities. The ADDE image directory servers actually use this option to pass the information block to the client application. Because the information returned by KEYS was incomplete, the option INFO (lines 191-202) was added to provide scale factors and units. The input for INFO is:

Most calibration modules contain code to handle stretch tables generated by the SU command. By calling kbxopt with BRKP as the option and the name of the stretch table, the calibration module computes a modified brightness value based on the table.

To identify the sections of code where this is done, find the CALTYP variable, which is held in COMMON/BRKPNT. In ADDE, this function is done in the client application instead of the calibration module. This code will be removed from the calibration modules in the future.

Sample program

The sample calibration module, KBXSIN.DLM is provided below.

       
   1:       INTEGER FUNCTION KBXINI(CIN,COUT,IOPT)
   2: 
   3: *$ Name:
   4: *$     kbxini - Initialize for sine modified calibration
   5: *$
   6: *$ Interface:
   7: *$     integer function
   8: *$     kbxini( character*4 cin, character*4 cout, integer iopt(*))
   9: *$
  10: *$ Input:
  11: *$     cin  - input physical quantity ('TEMP', 'BRIT', 'RAW', etc.)
  12: *$     cout - output physical quantity
  13: *$     iopt - 
  14: *$             iopt(1)   precision of stored data (1, 2 or 4 bytes)
  15: *$             iopt(2)   spacing of output data (1, 2 or 4 bytes)
  16: *$             iopt(3-5) filled by araopt but should not be used
  17: *$
  18: *$ Input and Output:
  19: *$     none
  20: *$
  21: *$ Output:
  22: *$     none
  23: *$
  24: *$ Return values:
  25: *$      0    - success
  26: *$     -1    - unit conversion not possible
  27: *$
  28: *$ Remarks:
  29: *$      This calibration module will only accept values from 0 to
  30: *$      255 and will return values modified by a sine curve. There
  31: *$      is no check for input data out of range.
  32: *$
  33: *$ Categories:
  34: *$       calibration
  35: 
  36:       CHARACTER*4 CIN
  37:       CHARACTER*4 COUT
  38:       INTEGER IOPT(*)
  39: 
  40:       INCLUDE 'areaparm.inc'
  41:  
  42:       INTEGER JTYPE
  43:       INTEGER ISOU
  44:       INTEGER IDES
  45:       INTEGER JOPT(NUMAREAOPTIONS)
  46: C
  47: C--- Store information needed in other functions
  48: C
  49:       COMMON/M0SIN/JTYPE,ISOU,IDES,JOPT
  50: C
  51: C--- Copy what araopt sent in
  52: C
  53:       CALL MOVW(NUMAREAOPTIONS,IOPT,JOPT)
  54: 
  55:       JTYPE=0
  56:       ISOU=IOPT(1)         ! length in bytes of input data
  57:       IDES=IOPT(2)         ! length in bytes to output data
  58: 
  59:       IF(CIN.EQ.'RAW'.AND.COUT.EQ.'SIN')  JTYPE=1
  60:       IF(CIN.EQ.'RAW'.AND.COUT.EQ.'BRIT') JTYPE=2
  61: C
  62: C--- If not one of the 2 cases above is true, error
  63: C
  64:       IF(JTYPE.EQ.0) GO TO 900
  65:  
  66:       KBXINI=0
  67:       RETURN
  68: 
  69: 900 CONTINUE
  70:       KBXINI = -1
  71:       RETURN
  72:       END
  73: 
  74:       INTEGER FUNCTION KBXCAL(PREFIX,IDIR,NVAL,IBAND,IBUF)
  75: *$ Name:
  76: *$     kbxcal - Calibrate data
  77: *$
  78: *$ Interface:
  79: *$     integer function
  80: *$     kbxcal( integer prefix(*), integer idir(*), integer nval, 
  81: *$                 integer iband, integer ibuf(*))
  82: *$
  83: *$ Input:
  84: *$     prefix - prefix part of image line            (not needed)
  85: *$     idir   - area directory                       (not needed)
  86: *$     nval   - number of values to calibrate
  87: *$     iband  - band number                          (not needed)
  88: *$
  89: *$ Input and Output:
  90: *$     ibuf - buffer containing data 
  91: *$
  92: *$ Output:
  93: *$     none
  94: *$
  95: *$ Return values:
  96: *$      0     - success
  97: *$     -1     - error                                (not needed)
  98: *$
  99: *$ Categories:
 100: *$       calibration
 101: 
 102:       INTEGER PREFIX(*)
 103: 
 104:       INCLUDE 'areaparm.inc'
 105:  
 106:       INTEGER JTYPE
 107:       INTEGER ISOU
 108:       INTEGER IDES
 109:       INTEGER JOPT(NUMAREAOPTIONS)
 110: 
 111:       INTEGER ITAB(256)
 112: 
 113:       COMMON/M0SIN/JTYPE,ISOU,IDES,JOPT
 114:       DATA IFLAG/0/
 115: C
 116: C--- If the calibration type changes, remake the lookup table
 117: C
 118:       IF( JTYPE .NE. IFLAG) THEN
 119:          CALL MAKTAB(JTYPE, ITAB)
 120:          IFLAG = JTYPE
 121:       ENDIF
 122: C
 123: C--- Pass the data IBUF through the lookup table ITAB
 124: C
 125:       CALL MPIXTB(NVAL,ISOU,IDES,IBUF,ITAB)
 126: 
 127:       KBXCAL=0
 128:       RETURN
 129:       END
 130: 
 131: 
 132:       INTEGER FUNCTION KBXOPT(CFUNC,IIN,IOUT)
 133: *$ Name:
 134: *$     kbxopt - Additional operations
 135: *$
 136: *$ Interface:
 137: *$     integer function
 138: *$     kbxopt( character*4 cfunc, integer iin(*), integer iout(*))
 139: *$
 140: *$ Input:
 141: *$     cfunc - function ('INFO', 'KEYS')
 142: *$     iin   - for cfunc 'KEYS', iin contains frame directory block
 143: *$             for cfunc 'INFO'
 144: *$                  iin(1) - band number
 145: *$                  iin(2) - sensor source number
 146: *$                  iin(3) - calibration type ('GVAR', for example)
 147: *$
 148: *$ Input and Output:
 149: *$     none
 150: *$
 151: *$ Output:
 152: *$     iout - for cfunc 'KEYS'
 153: *$              iout(1)   - number of physical quantities,'TEMP' etc.
 154: *$              iout(2-n) - list of physical quantities
 155: *$     iout - for cfunc 'INFO'
 156: *$              iout(1)   - number of physical quantities,'TEMP' etc.
 157: *$              iout(2-n) - list of physical quantities, units,
 158: *$                          and scale factors
 159: *$
 160: *$ Return values:
 161: *$      0   - success
 162: *$      1   - invalid function
 163: *$
 164: *$ Categories:
 165: *$       calibration
 166: 
 167:       CHARACTER*4 CFUNC
 168: 
 169:       INTEGER IIN(*)
 170:       INTEGER IOUT(*)
 171: 
 172:       INCLUDE 'areaparm.inc'
 173:  
 174:       INTEGER JTYPE
 175:       INTEGER ISOU
 176:       INTEGER IDES
 177:       INTEGER JOPT(NUMAREAOPTIONS)
 178:       COMMON/M0SIN/JTYPE,ISOU,IDES,JOPT
 179: C
 180: C--- KEYS option
 181: C
 182:       IF( CFUNC .EQ. 'KEYS') THEN
 183:          IOUT(1) = 3              ! Number of types
 184:          IOUT(2) = LIT('RAW ')    ! Physical quantities
 185:          IOUT(3) = LIT('SIN ')
 186:          IOUT(4) = LIT('BRIT')
 187:       ENDIF
 188: C
 189: C--- INFO option
 190: C     
 191:       IF( CFUNC .EQ. 'INFO') THEN
 192:          IOUT(1) = 3              ! Number of types
 193:          IOUT(2) = LIT('RAW ')    ! Physical quantities
 194:          IOUT(3) = LIT('SIN ')
 195:          IOUT(4) = LIT('BRIT')
 196:          IOUT(5) = LIT('    ')    ! Units
 197:          IOUT(6) = LIT('none')
 198:          IOUT(7) = LIT('    ')
 199:          IOUT(8) = 1              ! Scale factors
 200:          IOUT(9) = 1000
 201:          IOUT(10)= 1
 202:       ENDIF
 203: 
 204:       RETURN
 205:       END
 206: 
 207: 
 208:       SUBROUTINE MAKTAB(JTYPE, ITAB)
 209: *$ Name:
 210: *$      maktab - Make lookup table for sine modified calibration
 211: *$
 212: *$ Interface:
 213: *$      subroutine
 214: *$      maktab( integer jtype, integer itab(*))
 215: *$
 216: *$ Input:
 217: *$      jtype  - calibration type
 218: *$               1   - sine
 219: *$               2   - grayscale
 220: *$
 221: *$ Input and Output:
 222: *$      none
 223: *$
 224: *$ Output:
 225: *$      itab   - lookup table of 256 values
 226: *$
 227: *$ Remarks:
 228: *$      This routine makes a lookup table by computing the sine
 229: *$      for all possible values from 0 to 255 (the range of the 
 230: *$      input data). Rather than computing the sine directly on
 231: *$      the values 0 to 255, it is initially scaled to 0 to 10
 232: *$      which is approximately 3 sine waves (3 * PI = 10)
 233: *$
 234: *$ Categories:
 235: *$      calibration
 236: 
 237:       INTEGER ITAB(*)
 238: 
 239:       REAL SINVAL
 240:       REAL X
 241: C
 242: C---
 243: 
 244:       DO 100 I = 1, 256
 245: 
 246:          X = I - 1              ! X goes from 0 to 255
 247:          X = 10. * (X / 255.)   ! Normalize and scale to 3 sine waves
 248:          SINVAL = SIN(X)
 249: C
 250: C--- Output sine value
 251: C
 252:          IF (JTYPE .EQ. 1) THEN
 253:             ITAB(I) = NINT(SINVAL * 1000.)       ! Scale sine by 1000
 254: C
 255: C--- Output grayscale value
 256: C
 257:          ELSE IF (JTYPE .EQ. 2) THEN
 258:             ITAB(I) = NINT( 127. + 128 * SINVAL) ! Scale to 0 to 255
 259:          ENDIF
 260: 
 261: 100   CONTINUE
 262: 
 263: 
 264:       RETURN
 265:       END


[Search Manual] [Table of Contents] [Go to Previous] [Go to Next]