This chapter describes the formats of the data files developed for applications running under McIDAS-X. The data files are presented alphabetically with the following information:
This chapter describes these file formats:
Area (image) files, where nnnn is a user-defined number.
In McIDAS-X, images are stored in binary files called areas. Each area file is a collection of information that defines the image and its associated ancillary data.
Area files are usually named AREAnnnn, where nnnn is a four-digit number between 0000 and 9999. This number is called the area file number. For example, AREA0013 is the name of the file containing area 13.
Areas do not have to follow this standard naming convention. The file masking option of DSSERVE may be used to access a data file of any name through the ADDE.Area files consist of these six blocks:
Some blocks also contain satellite-specific information, which is located in the Image-specific characteristics section. For area file Application Program Interfaces (APIs), refer to the API functions list.
|
For more information about reading, writing and deleting image data, see the section titled Image data in Chapter 5, Accessing Data . |
The first 64 words of an area file contain the directory block for the image. The directory lists ancillary information about the image, such as the number of lines and data points per line, the satellite ID and the number of spectral bands. The data in the directory is stored as 32-bit (4-byte) twos- complement binary integers or as ASCII characters.
Each of the directory's 64 words is described below. Since some of the words are satellite specific, see the section titled Image-specific characteristics that follows. All byte offsets and pointers are zero-based. Note that all data shown as yyyddd are the year and day-of-year. The yyy values are the actual year modulo 1900.
|
Word |
Description |
|---|---|
|
actual image start time, hhmmss; in milliseconds for POES data |
|
|
For more information about image data, see the section titled Image data in Chapter 5, Accessing Data. For more information about coordinate systems, see the section titled Coordinate systems in Chapter 2, Learning the Basics . |
The navigation (NAV) block contains the information for determining the location of data points in physical space. Word 35 of the directory block contains the byte offset to the start of the navigation block. If an image isn't navigated, word 35 is zero. The NAV block format varies with each satellite; see the section titled Image-specific characteristics that follows.
The calibration (CAL) block contains information for converting image data from its stored (internal) units to more meaningful units such as radiance or albedo. The presence of this block depends on the implementation of the satellite-specific calibration. Word 63 of the directory block contains the byte offset to the start of the calibration block. If there is no CAL block, word 63 is zero. The calibration block format varies with each satellite; see the section titled Image-specific characteristics that follows.
The supplemental (or auxiliary) block contains additional information that is specific to a data type. For example, information specific to radar data is stored in this block. Also, the latitude/longitude grid for the LALO navigation is stored in this block. Word 60 of the directory block contains the byte offset to the start of the supplemental block. Word 61 contains the total number of entries in the supplemental block. If there is no supplemental block, words 60 and 61 are zero.
The data block contains the actual image data values. Any data point in an image or image sector can be located with image and file coordinates.
An area file may be produced from an image by sampling or averaging the data. In the case of multiband images, the file may include only a portion of the measured spectral bands, so that each element contains fewer data values than are contained in the original image. To map an area back to the original image, these two formulas are used:
Image Line = UpperLeftLine + (File Line * LineResolution)
Image Element = UpperLeftEle + (File Element * ElementResolution)
UpperLeftLine is the line coordinate of the first image line and UpperLeftEle is the element coordinate of the first image element. File Line and File Element are zero-based.
When LineResolution and ElementResolution are both 1, the image resolution is 1, or full resolution. If the image resolution is 4, every fourth line and element of an image originally at resolution 1 are included in the image. Each sensor has its own scan resolution, so an image resolution of one will mean different geographic resolutions from one satellite to another.
Each line is divided into two parts: the line prefix and the actual data values as shown below. The line prefix contains information about the image and the particular line.
line prefix 1 line data 1 line prefix 2 line data 2 etc.
|_____________|___________|_____________|___________|___...
0 byte numbers increase >>
Although the size and content of the line prefix depend on the image source defined in word 52 of the directory block, each line in an image has the same prefix length. Word 15 of the directory block contains the length of the line prefix, in bytes. If no line prefix exists, word 15 is zero.
The line prefix may contain any region shown in the diagram below and described in the following table; the regions' lengths are multiples of four bytes.
validity code documentation calibration band list
|______________|_______________|_____________|___________|
0 byte numbers increase >>
Word 34 of the directory block contains the byte offset to the start of the data block. Each line in an image is the same length and a multiple of four bytes. To calculate the length of a line prefix, the line data, or the entire data block, use the formulas below.
line prefix length = doc + cal + band + 4 (if valcode is present)
line data length = nbands*nele*nbytes
line length = line prefix length + line data length
data block length = nlines * line length
The parameters used in these formulas are defined in the directory block and provided in the table below.
|
Parameter |
Directory block word |
Definition |
|---|---|---|
|
length of the prefix validity code; if nonzero, the length is four bytes; otherwise it is zero |
||
An area file may contain a comment (AUDIT) block containing a variety of textual information such as a list of commands run on the image object to date. Each comment record is 80 ASCII characters. Word 64 of the directory block contains the number of comment records, or cards.
TIP (TIROS-N Information Processor) data is extracted from the AVHRR, GAC and LAC data described in the previous section. TIP data also contains information received from the MSU (Microwave Sounding Unit), the HIRS (High Resolution Infrared Sounder), and the SSU (Stratospheric Sounding Unit). These three sources measure incoming radiation in the infrared and microwave spectrum.
The navigation block for TIP data is filled with zeros; no calibration information is needed.
TIP data is transmitted as 10 bits and stored as 16 bits. The 10-bit data is formatted as follows, with x representing a data bit and the rest being zero-filled after shifting: | 0 | x | x | x | x | x | x | x | x | x | x | 0 | 0 | 0 | 0 | 0 |
The line prefix for the data block contains the information below.
|
Region |
Description |
|---|---|
|
optional, but recommended to flag missing data, which requires zeros as placeholder data or a validity code that does not match the value in the area directory |
|
The following section describes the navigation and calibration for the visible and infrared channels of the Geostationary Meteorological Satellites, GMS-4 and GMS-5. GMS-4 has one channel of visible and one channel of infrared data. GMS-5 has two channels for visible and two channels for infrared data; only one channel of each is used. The remaining two channels are reserved for backup.
|
For a complete description of the line documentation fields, see NOAA Technical Memorandum NESS 107 , 1988. The band information for TIP data is provided in this manual in Appendix B, Image Information. |
|
|
The tables referenced in this section are from the document Revision of GMS Stretched-VISSR Data Format , Japan Meteorological Agency, October 1993. |
The following three tables list the words used in the GMS navigation block. These are 1-byte words. The first table lists all the words in the GMS navigation block and the following two tables list the words used in the attitude prediction data sub-blocks and the orbit prediction data sub-blocks.
The Type column in the tables below shows scaled integers in the format R*M.N. The R indicates real numbers, M is the number of bytes and N is the exponent.
|
The GMS data is also formatted using the navigation block format of GOES-7 data. The section GOES-7 AAA navigation block describes the GOES-7 navigation block. |
The table below describes an attitude data prediction sub-block. The GMS navigation block contains ten attitude data prediction sub-blocks that occupy words 257-617. Each block occupies 36 words.
The table describes the orbit prediction data sub-block. The GMS navigation block contains eight orbit prediction sub-blocks that occupy words 618-2352. Each block occupies 182 words.
The GMS calibration block contains both directory and data conversion tables located in sub-blocks that follow the directory. The 128-word directory block, shown below, indicates the locations of the six sub-blocks. The starting byte offset for each sub-block varies with the data; therefore, it is shown as a variable in the directory below.
|
Word |
Value |
Description |
|
Words 1 to 256 of GMS 4-or -5 calibration block; see the table below |
||
The table below describes the calibration data sub-blocks, not all of which may be filled. The calibration data, which follows the directory, has a length of 6400 bytes.
|
See table A-7 in the Revision of GMS stretched-VISSR data format document for a complete description of the data block. |
|
Byte |
Descriptions |
|---|---|
|
sub-block 2; visible level-to-albedo conversion tables; four 64-level tables for VIS1 through VIS4 detectors |
|
The prefixes for each scan line of visible data must contain a code indicating the detector used. The first four bytes of the documentation section should contain the following information.
|
Code |
Description |
|---|---|
The Defense Meteorological Satellite Program (DMSP) satellites are polar orbiting satellites. DMSP has two sensors: the Operational Linescan System (OLS) and the Special Sensor for Microwave Imagery (SSM/I) data. The following sections describe the navigation and calibration blocks for each sensor.
The table below lists the contents of the DMSP navigation block. The block contains 128 words and is used for all DMSP signal types.
|
Word |
Value |
Description |
|---|---|---|
|
right ascension of ascending node from the ASCII record below; deg*1.e6 |
||
|
argument of perigee from the ASCII record below; per,deg*1e7 |
||
The DMSP navigation block includes a 0-based, type 8 ASCII record, in words 120-127, that contains the scaled integer values used by the navigation block. The table below describes the ASCII record.
|
Character numbers |
Description |
|---|---|
The OLS and microwave sensors use different calibration. The OLS calibration uses only one pair of of gain and offset parameters to transform infrared pixel values to radiance. The offset is set to 190.0 and the gain is set to .4706. The calibration module kbxols.dlm contains the offset pair. The visible pixels are not calibrated.
For the microwave sensors, each data line is calibrated and the information is stored in the documentation section. A postprocessor, DMSPCAL, calculates the gains and offsets from the information in the documentation section. The table below describes the calibration documentation for each line of data.
When DMSPCAL is run, a gain and offset for each channel is placed in the calibration section of each scan line prefix. The table below describes the byte location of each gain and offset pair.
|
Bytes |
Description |
|---|---|
|
Word |
Description |
|---|---|
|
2 |
image line of the North Pole |
|
3 |
image element of the North Pole |
|
4 |
standard latitude 1, DDDMMSS |
|
5 |
standard latitude 2, DDDMMSS |
|
6 |
spacing at standard latitude, M |
|
7 |
normal longitude, DDDMMSS |
|
8 |
radius of the planet, M |
|
9 |
eccentricity of the planet, x1000000 |
|
10 |
coordinate type, &ge 0 planetodetic, < 0 planetocentric |
|
11 |
longitude convention, &ge 0 west positive, < 0 west negative |
|
12-120 |
reserved |
|
121-128 |
memo entry; up to 32 characters of comments |
|
Word |
Description |
|---|---|
|
2 |
image line of the equator |
|
3 |
image element of the equator |
|
4 |
standard latitude |
|
5 |
spacing at standard latitude, meters |
|
6 |
normal longitude, DDDMMSS |
|
7 |
radius of the planet, M |
|
8 |
eccentricity of the planet, x1000000 |
|
9 |
coordinate type, &ge 0 planetodetic, < 0 planetocentric |
|
10 |
longitude convention, &ge 0 west positive, < 0 west negative |
|
11-120 |
reserved |
|
121-128 |
memo entry; up to 32 characters of comments |
|
Word |
Description |
|---|---|
|
1 |
PS |
|
2 |
image line of the North Pole |
|
3 |
image element of the North Pole |
|
4 |
standard latitude, DDDMMSS |
|
5 |
spacing at standard latitude, M |
|
6 |
normal longitude, DDDMMSS |
|
7 |
radius of the planet, M |
|
8 |
eccentricity of the planet, x1000000 |
|
9 |
coordinate type, &ge 0 planetodetic, < 0 planetocentric |
|
10 |
longitude convention, &ge 0 west positive, < 0 west negative |
|
11-120 |
reserved |
|
121-128 |
memo entry; up to 32 characters of comments |
|
Word |
Description |
|---|---|
|
2 |
row (image coordinates) of the radar site |
|
3 |
column (image coordinates) of the radar site |
|
4 |
latitude of the radar site, DDDMMSS |
|
5 |
longitude of the radar site, DDDMMSS |
|
6 |
pixel resolution, meters |
|
7 |
rotation of north from vertical, degrees x1000 |
|
8 |
if present, same as 6, but only for longitude direction |
|
Word |
Description |
|---|---|
|
2 |
a particular image row number |
|
3 |
latitude corresponding to word 2, degrees x10000 |
|
4 |
a particular image column number |
|
5 |
longitude corresponding to word 4, degrees x10000 |
|
6 |
latitude degrees/image line, degrees x10000 |
|
7 |
longitude degrees/image line, degrees x10000 |
|
8 |
radius of the planet, meters |
|
9 |
eccentricity of the planet, x1000000 |
|
10 |
coordinate type, &ge 0 planetodetic, < 0 planetocentric |
|
11 |
longitude convention, &ge 0 west positive, < 0 west negative |
|
Function |
Description |
|---|---|
|
opens a connection to read the directory block from an image object |
|
|
opens a connection to read the data block from an image object |
|
