8. Utility Scripts

The following are scripts that can be used to aid in the creation of customized Geo2Grid products. All utility scripts are stored in the bin directory:

$GEO2GRID_HOME/bin/<script>.sh ...

For simplicity, the sections below will specify the script directly, but note the scripts exist in the bin directory above.

8.1. Defining Your Own Grids (Grid Configuration Helper)

This script is meant to help those unfamiliar with PROJ.4 and projections in general. By providing a few grid parameters this script will provide a grid configuration line that can be added to a user’s custom grid configuration. Based on a center longitude and latitude, the script will choose an appropriate projection.

usage: p2g_grid_helper.sh [-h] [-p PROJ_STR] [--legacy-format] grid_name center_longitude center_latitude pixel_size_x pixel_size_y grid_width grid_height

8.8.1. Positional Arguments

grid_name

Unique grid name

center_longitude

Decimal longitude value for center of grid (-180 to 180)

center_latitude

Decimal latitude value for center of grid (-90 to 90)

pixel_size_x

Size of each pixel in the X direction in grid units, meters for default projections.

pixel_size_y

Size of each pixel in the Y direction in grid units, meters for default projections.

grid_width

Grid width in number of pixels

grid_height

Grid height in number of pixels

8.2.2. options

-p

PROJ.4 projection string to override the default

--legacy-format

Produce a legacy ‘.conf’ format grid definition.

Example:

p2g_grid_helper.sh my_grid_name -150.1 56.3 250 -250 1000 1000

Will result in:

my_grid_name:
  projection:
    proj: lcc
    lat_1: 56.3
    lat_0: 56.3
    lon_0: -150.1
    datum: WGS84
    units: m
    no_defs: null
    type: crs
  shape:
    height: 1000
    width: 1000
  center:
    x: -150.1
    y: 56.3
    units: degrees
  resolution:
    dx: 250.0
    dy: 250.0

The above example creates a YAML formatted block of text for the grid named ‘my_grid_name’. It is defined to have a pixel resolution of 250m, have 1000 rows and 1000 columns, and be centered at -150.1 degrees longitude and 56.3 degrees latitude. The projection is a lambert conic conformal projection which was chosen based on the center longitude and latitude.

Once this text has been output, it can be added to a text file ending in .yaml and referenced in the geo2grid.sh command line. For instance, if I save the output text to a file named /home/user/my_grids.yaml, I can create a GeoTIFF from satellite data by executing a command like this:

geo2grid.sh -r abi_l1b -w geotiff --grid-configs /home/user/my_grids.yaml -g my_grid_name -f <path_to_files>

8.2. Add Overlays (Borders, Coastlines, Grids Lines, Rivers)

Add overlays to a GeoTIFF file and save as a PNG file.

usage: add_coastlines.sh [-h] [--add-coastlines] [--coastlines-resolution {c,l,i,h,f}] [--coastlines-level {1,2,3,4,5,6}] [--coastlines-outline [COASTLINES_OUTLINE ...]] [--coastlines-fill [COASTLINES_FILL ...]]
                         [--coastlines-width COASTLINES_WIDTH] [--add-rivers] [--rivers-resolution {c,l,i,h,f}] [--rivers-level {0,1,2,3,4,5,6,7,8,9,10}] [--rivers-outline [RIVERS_OUTLINE ...]] [--rivers-width RIVERS_WIDTH]
                         [--add-grid] [--grid-no-text] [--grid-text-size GRID_TEXT_SIZE] [--grid-font GRID_FONT] [--grid-fill [GRID_FILL ...]] [--grid-outline [GRID_OUTLINE ...]] [--grid-minor-outline [GRID_MINOR_OUTLINE ...]]
                         [--grid-D GRID_D GRID_D] [--grid-d GRID_D GRID_D] [--grid-lon-placement {tl,lr,lc,cc}] [--grid-lat-placement {tl,lr,lc,cc}] [--grid-width GRID_WIDTH] [--add-borders] [--borders-resolution {c,l,i,h,f}]
                         [--borders-level {1,2,3}] [--borders-outline [BORDERS_OUTLINE ...]] [--borders-width BORDERS_WIDTH] [--add-colorbar] [--colorbar-width COLORBAR_WIDTH] [--colorbar-height COLORBAR_HEIGHT] [--colorbar-extend]
                         [--colorbar-tick-marks COLORBAR_TICK_MARKS] [--colorbar-minor-tick-marks COLORBAR_MINOR_TICK_MARKS] [--colorbar-text-size COLORBAR_TEXT_SIZE] [--colorbar-text-color [COLORBAR_TEXT_COLOR ...]]
                         [--colorbar-font COLORBAR_FONT] [--colorbar-align {left,top,right,bottom}] [--colorbar-vertical] [--colorbar-min COLORBAR_MIN] [--colorbar-max COLORBAR_MAX] [--colorbar-units COLORBAR_UNITS]
                         [--colorbar-title COLORBAR_TITLE] [--shapes-dir SHAPES_DIR] [--cache-dir CACHE_DIR] [--cache-regenerate] [-o OUTPUT_FILENAME [OUTPUT_FILENAME ...]] [-v]
                         input_tiff [input_tiff ...]

8.8.1. Positional Arguments

input_tiff

Input geotiff(s) to process

8.2.2. options

--shapes-dir

Specify alternative directory for coastline shape files (default: GSHSS_DATA_ROOT)

--cache-dir

Specify directory where cached coastline output can be stored and accessed in later executions. The cache will never be cleared by this script. Caching depends on the grid of the image and the decorations added to the image.

--cache-regenerate

Force regeneration of any cached overlays. Requires ‘–cache-dir’.

-o, --output

Specify the output filename (default replace ‘.tif’ with ‘.png’)

-v, --verbose

each occurrence increases verbosity 1 level through ERROR-WARNING-INFO-DEBUG (default INFO)

Default: 0

8.2.3. coastlines

--add-coastlines

Add coastlines

--coastlines-resolution

Possible choices: c, l, i, h, f

Resolution of coastlines to add (crude, low, intermediate, high, full)

Default: “i”

--coastlines-level

Possible choices: 1, 2, 3, 4, 5, 6

Level of detail from the selected resolution dataset

Default: 4

--coastlines-outline

Color of coastline lines (color name or 3 RGB integers)

Default: [‘yellow’]

--coastlines-fill

Color of land

--coastlines-width

Width of coastline lines

Default: 1.0

8.2.4. rivers

--add-rivers

Add rivers grid

--rivers-resolution

Possible choices: c, l, i, h, f

Resolution of rivers to add (crude, low, intermediate, high, full)

Default: “c”

--rivers-level

Possible choices: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

Level of detail for river lines

Default: 5

--rivers-outline

Color of river lines (color name or 3 RGB integers)

Default: [‘blue’]

--rivers-width

Width of rivers lines

Default: 1.0

8.2.5. grid

--add-grid

Add lat/lon grid

--grid-no-text

Add labels to lat/lon grid

--grid-text-size

Lat/lon grid text font size

Default: 32

--grid-font

Path to TTF font (package provided or custom path)

Default: “Vera.ttf”

--grid-fill

Color of grid text (color name or 3 RGB integers)

Default: [‘cyan’]

--grid-outline

Color of grid lines (color name or 3 RGB integers)

Default: [‘cyan’]

--grid-minor-outline

Color of tick lines (color name or 3 RGB integers)

Default: [‘cyan’]

--grid-D

Degrees between grid lines (lon, lat)

Default: (10.0, 10.0)

--grid-d

Degrees between tick lines (lon, lat)

Default: (2.0, 2.0)

--grid-lon-placement

Possible choices: tl, lr, lc, cc

Longitude label placement

Default: “tb”

--grid-lat-placement

Possible choices: tl, lr, lc, cc

Latitude label placement

Default: “lr”

--grid-width

Width of grid lines

Default: 1.0

8.2.6. borders

--add-borders

Add country and/or region borders

--borders-resolution

Possible choices: c, l, i, h, f

Resolution of borders to add (crude, low, intermediate, high, full)

Default: “i”

--borders-level

Possible choices: 1, 2, 3

Level of detail for border lines

Default: 2

--borders-outline

Color of border lines (color name or 3 RGB integers)

Default: [‘white’]

--borders-width

Width of border lines

Default: 1.0

8.2.7. colorbar

--add-colorbar

Add colorbar on top of image

--colorbar-width

Number of pixels wide

--colorbar-height

Number of pixels high

--colorbar-extend

Extend colorbar to full width/height of the image

--colorbar-tick-marks

Major tick and tick label interval in data units

Default: 5.0

--colorbar-minor-tick-marks

Minor tick interval in data units

Default: 1.0

--colorbar-text-size

Tick label font size

Default: 32

--colorbar-text-color

Color of tick text (color name or 3 RGB integers)

Default: [‘black’]

--colorbar-font

Path to TTF font (package provided or custom path)

Default: “Vera.ttf”

--colorbar-align

Possible choices: left, top, right, bottom

Which side of the image to place the colorbar

Default: “bottom”

--colorbar-vertical

DEPRECATED

--colorbar-min

Minimum data value of the colorbar. Defaults to ‘min_in’ of input metadata or minimum value of the data otherwise.

--colorbar-max

Maximum data value of the colorbar. Defaults to ‘max_in’ of input metadata or maximum value of the data otherwise.

--colorbar-units

Units marker to include in the colorbar text

--colorbar-title

Title shown with the colorbar

Examples:

add_coastlines.sh GOES-18_ABI_RadF_true_color_night_microphysics_20221115_123020_GOES-West.tif --add-coastlines --add-rivers --rivers-resolution=h --add-grid -o abi_true_color_coastlines.png
add_coastlines.sh --add-coastlines --add-borders --borders-resolution=h --borders-outline='red' --add-grid GOES-17_ABI_RadF_natural_color_20181211_183038_GOES-West.tif -o abi_natural_color_coastlines.png

8.3. Add Colormap

Add a GeoTIFF colortable to an existing single-band GeoTIFF.

usage: add_colormap.sh [-h] ct_file geotiffs [geotiffs ...]

8.8.1. Positional Arguments

ct_file

Color table file to apply (CSV of (int, R, G, B, A)

geotiffs

Geotiff files to apply the color table to

Colormap files are comma-separated ‘integer,R,G,B,A’ text files.

A basic greyscale example for an 8-bit GeoTIFF would be:

0,0,0,0,255
1,1,1,1,255
...
254,254,254,254,255
255,255,255,255,255

Where the … represents the lines in between, meaning every input GeoTIFF value has a corresponding RGBA value specified. The first value is the input GeoTIFF value, followed by R (red), G (green), B (blue), and A (alpha).

This script will also linearly interpolate between two values. So the above colormap file could also be written in just two lines:

0,0,0,0,255
255,255,255,255,255

Often times you may want to have the 0 value as a transparent ‘fill’ value and continue the colormap after that. This can be done by doing the following:

# 0 is a fill value
0,0,0,0,0
# 1 starts at bright red
1,255,0,0,255
# and we end with black at the end
255,0,0,0,255

Note

Not all image viewers will obey the transparent (alpha) settings

Blank lines are allowed as well as spaces between line elements.

Note this script is no longer needed in modern versions of Geo2Grid if the original geotiff (no color) is not needed. The colormap can be specified directly in the enhancement YAML file for a product. For example, for the AMSR-2 L1B product “btemp_36.5h” we could add the following to a etc/enhancements/amsr2.yaml (or generic.yaml):

yaml

amsr2_btemp_365h:
  name: btemp_36.5h
  sensor: amsr2
  operations:
    - name: add_colormap
      method: !!python/name:polar2grid.enhancements.palettize
      kwargs:
        palettes:
          - filename: $POLAR2GRID_HOME/colormaps/amsr2_36h.cmap
            min_value: 180
            max_value: 280

When saved using the ‘geotiff’ writer this will be converted to an RGB/RGBA image. Optionally you can provide the --keep-palette flag to your geo2grid.sh call which will add the colormap as a geotiff color table.

8.4. GeoTIFF to KMZ Conversion

The gtiff2kmz.sh script converts a single GeoTIFF file into a Google Earth compatible Keyhole Markup language Zipped (KMZ) file. It is a wrapper around the GDAL tool gdal2tiles.py. The script can be executed with:

gtiff2kmz.sh input.tif [output.kmz]

Where output.kmz is an optional parameter specifying the name of the output KMZ file. If it isn’t specified it defaults to the input filename with the extension changed to .kmz.

Example:

gtiff2kmz.sh  GOES-18_ABI_RadF_natural_color_20221115_183020_GOES-West.tif

8.5. Overlay GeoTIFF Images

The overlay.sh script can be used to overlay one GeoTIFF image (ex. Gridded Geostationary Lightning Mapper (GLM)) on top of another image (ex. GOES infrared brightness temperature Image). This script uses GDAL’s gdal_merge.py utility underneath, but converts everything to RGBA format first for better consistency in output images.

usage: overlay.sh background.tif foreground.tif out.tif

Example: The following example shows how you would overlay the GOES ABI AIT Level-2 Cloud top Tempetaure Product on top of a GOES ABI Band 14 brithtness temperature image.

 overlay.sh GOES-17_ABI_RadF_C14_20221123_183031_GOES-West.tif GOES-17_ABI_TEMP_20221123_183031_GOES-West.tif abi17_fd_overlay.tif

overlay GOES-18_ABI_RadF_true_color_20221110_190020_GOES-West.tif  GOES-18_GLM_flash_extent_density_20221110_190000_GOES-West.tif overlay_true_color_flash_extent_density.tif

8.6. Convert GeoTIFFs to MP4 Video

The gtiff2mp4.sh script converts a series of GeoTIFF files in to a single MP4 video file. This script uses default video creation settings to support most video players. If an image is too large for the video creation they will be automatically scaled to a smaller size.

gtiff2mp4.sh out.mp4 in1.tif in2.tif ...

This will create a MP4 video file called out.mp4 with 24 images (frames) per second.

Example:

gtiff2mp4.sh my_natural_color_animation.mp4  *natural_color*.tif

8.7. Remap GOES GeoTIFFs

The projection of the GOES-East and GOES-West satellites uses special parameters that are not always supported by older visualization tools. While new versions of GDAL and PROJ.4 libraries can often fix these issues, this is not always an option. Geo2Grid provides the reproject_goes.sh script to remap GOES GeoTIFFs to a nearly identical projection that is more compatible with older visualization tools. The script can be called by executing:

reproject_goes.sh in1.tif in2.tif in3.tif

The script will take the original name and add a -y to the end. So in the above example the results would be in1-y.tif, in2-y.tif, and in3-y.tif. The y refers to the sweep angle axis projection parameter that differs between the input geotiff (x) and the output geotiff (y).

8.8. Convert legacy grids.conf to grids.yaml format

Convert legacy grids.conf format to Pyresample YAML format.

usage: 
To write to a file:
    convert_grids_conf_to_yaml.sh input_file.conf > output_file.yaml

8.8.1. Positional Arguments

grids_filename

Input grids.conf-style file to convert to YAML.

Example:

convert_grids_conf_to_yaml.sh old_file.conf > new_file.yaml