Introduction ============ Overview -------- .. include:: overview.rst Software Design --------------- .. graphviz:: digraph glue_flow { rankdir = LR; node [shape = rectangle, fillcolor="#C3DCE7:white", gradientangle="90.0", style="filled"]; "Reader" -> "Remapper"; "Remapper" -> "Writer"; "Remapper" -> "Compositors" [style=dashed]; "Compositors" -> "Writer" [style=dashed]; } |project| has a modular design operating on the idea of satellite “products” or "datasets"; data observed by a satellite instrument. These products can be any type of raster data, such as temperatures, reflectances, radiances, or any other value that may be recorded by or calculated from an instrument. There are 4 main steps or components involved when working with these products in |project|: reading, writing, compositing, and remapping. |project| makes it possible to access and configure these steps with a simple bash script called |script_literal| and other helper scripts. More information on accessing |project|'s features and running its scripts can be found in the :doc:`getting_started` section or the examples following each reader section. Note that although an example may be written for a specific reader the same operations can be applied to all readers unless mentioned otherwise. For more low-level information on the design and responsibility of each component see the :doc:`design_overview` Appendix. In |project| a majority of the functionality is provided by the open source SatPy library created by the Pytroll group. More information on SatPy and the capabilities it provides to python users can be found in the `SatPy documentation `_. What's New? ----------- .. ifconfig:: not is_geo2grid Polar2Grid Version 3.0 is now available. This is a major update that includes changes to basic Polar2Grid execution. These changes bring Polar2Grid in conformity with the execution strategy of Geo2Grid, and takes advantage of the Xarray and Dask python libraries. Please see the example executions listed at the end of every reader description in this document, as well as the updated examples in the :doc:`examples/index` section. Finally, the Appendix includes a longer list of changes and direct comparisons of Polar2Grid V2.3 to V3.0 executions. See :doc:`version3_implementation`. .. include:: NEWS.rst :start-line: 6 :end-line: 14 For more details on what's new in this version and past versions see the `Polar2Grid Release Notes `_ in the github repository. .. ifconfig:: is_geo2grid Included in this release: .. include:: NEWS_GEO2GRID.rst :start-line: 7 :end-line: 15 For more details on what's new in this version and past versions see the `Geo2Grid Release Notes `_ in the github repository. .. raw:: latex \newpage System Requirements ------------------- .. ifconfig:: is_geo2grid For minimal processing requirements (i.e. not realtime) the following system specifications are required: * Intel or AMD CPU with 64-bit instruction support (2+ cores - 2.4GHz) * 16 GB RAM (minimum) * CentOS 7.9 64-bit Linux (or other compatible 64-bit Linux distribution) * 10 GB disk space (minimum) For a more demanding processing load, like realtime generation of all GOES-16 ABI channels, true color, and natural color RGB images at full resolution, a system should have at least: * Intel or AMD CPU with 64-bit instruction support (20+ cores - 2.4GHz) * 64 GB RAM (minimum) * CentOS 7.9 64-bit Linux (or other compatible 64-bit Linux distribution) * 1 TB disk space (minimum for ~1 week of images, does not include long-term storage) Local storage (i.e. not network file systems) are preferred to limit any effect that network congestion may have. If additional satellites are included in the processing requirements then the above system requirements will need to be adjusted accordingly. Execution Times --------------- The following table provides execution time averages for creating all default GeoTIFF images at full spatial resolution for the given instrument and sector. Eight computer threads were used. The times are provided for the higher end system defined above. Execution times decrease when fewer bands and smaller regions are processed. **Table of Execution Times for Creating GeoTIFF default Images** (All bands plus true and natural color images) +------------------+---------------------+-----------------+-----------------------------+ |**Instrument** |**Full Disk Sector** |**CONUS Sector** | **1000x1000 pixel subset** | +==================+=====================+=================+=============================+ | GOES ABI | 2m50s | 29s | 14s | +------------------+---------------------+-----------------+-----------------------------+ | AHI HSD | 5m40s | N/A | 23s | +------------------+---------------------+-----------------+-----------------------------+ | AHI HimawariCast | 23s | N/A | 11s | +------------------+---------------------+-----------------+-----------------------------+ | GEO-KOMPSAT AMI | 2m48s | N/A | 12s | +------------------+---------------------+-----------------+-----------------------------+ | FY4 AGRI | 5m45s | N/A | 29s | +------------------+---------------------+-----------------+-----------------------------+ .. ifconfig:: not is_geo2grid System requirements for the |project| software are as follows: * Intel or AMD CPU with 64-bit instruction support (2+ cores - 2.4GHz) * 16 GB RAM (minimum) * CentOS 7.9 64-bit Linux; the software has been tested successfully on Rocky Linux 8.5 * 5 GB disk space (minimum) Improved Execution Times ------------------------- Updates in Polar2grid Version 3.0 result in improved image creation times. The table below presents a comparison of the unix `real` time required to create VIIRS and MODIS imager GeoTIFF files for the given segments of data in the default WGS84 projection. In these examples, the default 4 computer threads were used in the Version 3.0 executions. Execution times decrease when fewer bands and smaller data segments are processed. **Table of Execution Times for Creating GeoTIFF Default Projection Images** +------------------+-----------------+-----------------+------------------------+-----------------------+ |**Instrument** |**Polar2Grid** |**Polar2grid** |**Polar2Grid2 V2.3 All**|**Polar2Grid V3.0 All**| |**Input** |**V2.3 True and**|**V3.0 True and**|**Bands plus True** |**Bands plus True** | | |**False Color** |**False Color** |**and False Color** |**and False Color** | +==================+=================+=================+========================+=======================+ |**VIIRS SDR** | | | | | |10 - 86 second | 4m52s | 2m46s | 12m54s | 4m32s | |granules | | | | | +------------------+-----------------+-----------------+------------------------+-----------------------+ |**MODIS Level 1B**| | | | | |3 - 5 minute | 4m11s | 3m55s | 9m08s | 4m51s | |granules | | | | | +------------------+-----------------+-----------------+------------------------+-----------------------+ License and Disclaimer ---------------------- Original scripts and automation included as part of this package are distributed under the :download:`GNU GENERAL PUBLIC LICENSE agreement version 3 <../../LICENSE.txt>`. Software included as part of this software package are copyrighted and licensed by their respective organizations, and distributed consistent with their licensing terms. The University of Wisconsin-Madison Space Science and Engineering Center (SSEC) makes no warranty of any kind with regard to the |cspp_abbr| software or any accompanying documentation, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. SSEC does not indemnify any infringement of copyright, patent, or trademark through the use or modification of this software. There is no expressed or implied warranty made to anyone as to the suitability of this software for any purpose. All risk of use is assumed by the user. Users agree not to hold SSEC, the University of Wisconsin-Madison, or any of its employees or assigns liable for any consequences resulting from the use of the |cspp_abbr| software.