{"id":1085,"date":"2017-05-30T01:34:50","date_gmt":"2017-05-30T06:34:50","guid":{"rendered":"http:\/\/www.ssec.wisc.edu\/aeri\/?page_id=1085"},"modified":"2018-07-17T21:33:52","modified_gmt":"2018-07-17T21:33:52","slug":"tpss-operation-guide","status":"publish","type":"page","link":"https:\/\/www.ssec.wisc.edu\/aeri\/kb\/tpss\/tpss-operation-guide\/","title":{"rendered":"TPSS Operation Guide"},"content":{"rendered":"<h3>Overview<\/h3>\n<p>The Thermodynamic Profiling Software System is a software package that processes raw AERI data and optional meteorological surface observations in order to produce thermodynamic profiles and uncertainties using an optimal estimation method.\u00a0 The principal components of TPSS are<\/p>\n<ul>\n<li>rsync data transfer\n<ul>\n<li>transfer the raw AERI data (<tt>.RNC<\/tt> and <tt>.SUM<\/tt> files) from an AERI or other source into the\u00a0<strong><tt>\\raw<\/tt><\/strong> directory<\/li>\n<li>(optional) transfer the MET and ceilometer data into the\u00a0<strong><tt>\\met<\/tt><\/strong> and\u00a0<strong><tt>\\ceil<\/tt><\/strong> directories<\/li>\n<li>specified by <tt>\\config\\rsync_update.sh<\/tt>, see <a href=\"https:\/\/www.ssec.wisc.edu\/aeri\/kb\/tpss\/tpss-customization-options\/\">TPSS Customization Options<\/a>.<\/li>\n<\/ul>\n<\/li>\n<li>Ingest of AERI data and (optional) MET and ceilometer data\n<ul>\n<li>Converts raw AERI data into NetCDF files and stores them in the\u00a0<strong><tt>\\aerich1<\/tt><\/strong> and\u00a0<strong><tt>\\aerich2<\/tt><\/strong> directories<\/li>\n<\/ul>\n<\/li>\n<li>Quality control of data<\/li>\n<li>Noise filtering of AERI data\n<ul>\n<li>Note:\u00a0 4-7 days of ingested data must already exist in the\u00a0<strong><tt>\\aerich1<\/tt><\/strong> and\u00a0<strong><tt>\\aerich2<\/tt><\/strong> directories in order for the noise filter to get generated<\/li>\n<\/ul>\n<\/li>\n<li>Application of optimal estimation algorithm<\/li>\n<li>Output of results and quicklooks<\/li>\n<\/ul>\n<h3>&nbsp;<\/h3>\n<p>&nbsp;<\/p>\n<h3>Setup and Customization<\/h3>\n<p>Prior to running TPSS, the retrieval configuration file (<tt>vip.txt<\/tt>), climatology and data transfer need to be configured.\u00a0 See <a href=\"http:\/\/www.ssec.wisc.edu\/aeri\/kb\/tpss\/tpss-customization-options\/\">TPSS Customization Options<\/a>.<\/p>\n<h3>&nbsp;<\/h3>\n<p>&nbsp;<\/p>\n<h3>TPSS File Structure<\/h3>\n<p>The files related to TPSS are in the directory:\u00a0 <tt>C:\\Users\\AERI\\docker<\/tt><\/p>\n<ul>\n<li>Configuration files (see <a href=\"https:\/\/www.ssec.wisc.edu\/aeri\/kb\/tpss\/tpss-customization-options\/\">TPSS Customization Options<\/a>)\n<ul>\n<li><strong><tt>\\config<\/tt><\/strong><\/li>\n<\/ul>\n<\/li>\n<li>Input files\n<ul>\n<li><strong><tt>\\raw<\/tt><\/strong>: raw AERI data, directories of the form <tt>AEYYMMDD<\/tt> e.g. <tt>AE180301<\/tt><\/li>\n<li><strong><tt>\\ceil<\/tt><\/strong>: (optional) ceilometer data for cloud base height determination<tt><br \/>\n<\/tt><\/li>\n<li><strong><tt>\\met<\/tt><\/strong>: (optional) ground-based met data (temperature, pressure, relative humidity)<tt><br \/>\n<\/tt><\/li>\n<\/ul>\n<\/li>\n<li>Executable scripts\n<ul>\n<li><strong><tt>\\run<\/tt><\/strong>: tpss.bat &amp; tpss_stop.bat<\/li>\n<\/ul>\n<\/li>\n<li>Intermediary files\n<ul>\n<li><strong><tt>\\aerich1<\/tt><\/strong>: ingested AERI ch1 data<\/li>\n<li><strong><tt>\\aerich2<\/tt><\/strong>: ingested AERI ch2 data<\/li>\n<li><strong><tt>\\aerinfch1<\/tt><\/strong>: noise filtered AERI ch1 data<\/li>\n<li><strong><tt>\\aerinfch2<\/tt><\/strong>: noise filtered AERI ch2 data<\/li>\n<li><strong><tt>\\aerisum<\/tt><\/strong>: ingested AERI summary data<\/li>\n<\/ul>\n<\/li>\n<li>Diagnostic files\n<ul>\n<li><strong><tt>\\logs<\/tt><\/strong>: log files<\/li>\n<li><strong><tt>\\runflags<\/tt><\/strong>: flags indicating running processes<\/li>\n<\/ul>\n<\/li>\n<li>Output files\n<ul>\n<li><strong><tt>\\aerioe<\/tt><\/strong>: output files and quicklooks<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h3>&nbsp;<\/h3>\n<p>&nbsp;<\/p>\n<h3>Initialization<\/h3>\n<p>A recent noise filter file must exist in order for TPSS to run.\u00a0 The noise filter is generated automatically, but in general 4-7 days of recent ingested data must exist to generate the noise filter.\u00a0 Run TPSS in reprocessing mode on 4-7 days of recent data:\u00a0 this will run ingest, but not the retrieval algorithm.\u00a0 After 4-7 days of data have been ingested, TPSS can run normally in either reprocessing or real-time mode.<\/p>\n<p>e.g.<\/p>\n<p style=\"font-family: monospace; background: #000000; color: #c6c3c6; border: 2px solid #C6C3C6; padding: 10px;\">cd c:\\Users\\AERI\\docker\\run<br \/>\n.\\tpss.bat 20180301<br \/>\n.\\tpss.bat 20180302<br \/>\n.\\tpss.bat 20180303<br \/>\n.\\tpss.bat 20180304<\/p>\n<p>TPSS will run for <tt>20180304<\/tt>, after enough data have been ingested from the previous days to generate the noise filter.<\/p>\n<h3>&nbsp;<\/h3>\n<p>&nbsp;<\/p>\n<h3>Running TPSS in Reprocessing Mode<\/h3>\n<p>In reprocessing mode, TPSS is run on a given day <tt>YYYYMMDD<\/tt> specified by the user.\u00a0 The desired temporal resolution and other parameters can be set in <tt>vip.txt<\/tt>.\u00a0 To run TPSS in reprocessing mode:<\/p>\n<ul>\n<li>Run <em>Docker for Windows<\/em>.\u00a0 Note:\u00a0 this is configured to start automatically at startup.\u00a0 Ensure that <em>Docker<\/em> is running.<\/li>\n<li>Open <em>Command Prompt<\/em> and type<\/li>\n<\/ul>\n<p style=\"font-family: monospace; background: #000000; color: #c6c3c6; border: 2px solid #C6C3C6; padding: 10px;\">cd c:\\Users\\AERI\\docker\\run<br \/>\n.\\tpss.bat YYYYMMDD<\/p>\n<p>Where <tt>YYYYMMDD<\/tt> is the specified date to reprocess (e.g. <tt>20180301<\/tt>).<\/p>\n<ul>\n<li>Progress on the processing will be displayed in the\u00a0<em>Command Prompt<\/em> window.<\/li>\n<li>By default, TPSS will always append to existing output data for the specified day.\u00a0 If brand new output is desired, then ensure that previous NetCDF files for the specified day have been deleted.<\/li>\n<li>TPSS will stop and shut down automatically when processing is complete.<\/li>\n<li>Quicklooks will be generated at the end after the processing is complete.<\/li>\n<li>If processing needs to be stopped before the process has completed, type <tt>&lt;ctrl&gt;+C<\/tt>, <tt>&lt;ctrl&gt;+D<\/tt> in the<em> Command Prompt<\/em> with the running process.\u00a0 This may need to be repeated until the process is stopped.<\/li>\n<\/ul>\n<h3>&nbsp;<\/h3>\n<p>&nbsp;<\/p>\n<h3>Running TPSS in Real-time Mode<\/h3>\n<p>In real-time mode, TPSS is run continuously in order to output results and quicklooks of the latest available AERI data.\u00a0 It is recommended to select an appropriate temporal resolution (<tt>tres<\/tt> in <tt>vip.txt<\/tt>) to match the processing speed of the hardware being used.\u00a0 To run TPSS in real-time mode:<\/p>\n<ul>\n<li>Run <em>Docker for Windows<\/em>.\u00a0 Note:\u00a0 This is configured to start automatically at startup.\u00a0 Ensure that <em>Docker<\/em> is running.<\/li>\n<li>Click on the <tt>tpss<\/tt> icon on the desktop<\/li>\n<\/ul>\n<p>Or<\/p>\n<ul>\n<li>Open <em>Command Prompt<\/em> and type<\/li>\n<\/ul>\n<p style=\"font-family: monospace; background: #000000; color: #c6c3c6; border: 2px solid #C6C3C6; padding: 10px;\">cd c:\\Users\\AERI\\docker\\run<br \/>\n.\\tpss.bat<\/p>\n<ul>\n<li>Progress on the processing can be inspected in the <tt>log<\/tt> files in\u00a0<strong><tt>\\logs<\/tt><\/strong> (not the <em>Command Prompt<\/em> window).<\/li>\n<li>By default, TPSS will always append to existing output data for the current UTC day.<\/li>\n<\/ul>\n<h3>&nbsp;<\/h3>\n<p>&nbsp;<\/p>\n<h3>Stopping TPSS in Real-time Mode<\/h3>\n<ul>\n<li>Click on the <tt>tpss_stop<\/tt>\u00a0icon on the desktop<\/li>\n<\/ul>\n<p>Or<\/p>\n<ul>\n<li>Open <em>Command Prompt<\/em> and type<\/li>\n<\/ul>\n<p style=\"font-family: monospace; background: #000000; color: #c6c3c6; border: 2px solid #C6C3C6; padding: 10px;\">cd c:\\Users\\AERI\\docker\\run<br \/>\n.\\tpss_stop.bat<\/p>\n<h3>&nbsp;<\/h3>\n<p>&nbsp;<\/p>\n<h3>TPSS Output<\/h3>\n<p>When TPSS is run in either real-time or reprocessing mode, output files and quicklooks will be incrementally generated in the output directory:\u00a0 <tt>C:\\Users\\AERI\\docker\\aerioe<\/tt><\/p>\n<ul>\n<li>Output NetCDF files <tt>*.cdf<\/tt> are self-describing files that contain the full results of the retrieval. e.g.\n<ul>\n<li><strong><tt>height<\/tt><\/strong>: height above ground level in km<\/li>\n<li><strong><tt>temperature<\/tt><\/strong>: retrieved temperature in Celsius<\/li>\n<li><strong><tt>waterVapor<\/tt><\/strong>: retrieved water vapor mixing ratio in g\/kg<\/li>\n<li><strong><tt>sigma_temperature<\/tt><\/strong>: 1-sigma uncertainty in retrieved temperature in Celsius<\/li>\n<li><strong><tt>sigma_waterVapor<\/tt><\/strong>: 1-sigma uncertainty in retrieved water vapor mixing ratio in g\/kg<\/li>\n<\/ul>\n<\/li>\n<li>Quicklooks\n<ul>\n<li><strong><tt>\\aerioe.skewt.YYYMMDD.HHMM<\/tt><\/strong>: Skew-T plots at ~3-hour intervals<\/li>\n<li><strong><tt>\\aerioe.RH.YYYYMMDD.HHMM<\/tt><\/strong>: Relative humidity time-height cross section<tt><br \/>\n<\/tt><\/li>\n<li><strong><tt>\\aerioe.temperature.YYYYMMDD.HHMM<\/tt><\/strong>: Temperature time-height cross section<tt><br \/>\n<\/tt><\/li>\n<li><strong><tt>\\aerioe.theta.YYYYMMDD.HHMM<\/tt><\/strong>: Potential temperature time-height cross section<tt><br \/>\n<\/tt><\/li>\n<li><strong><tt>\\aerioe.wvmr.YYYYMMDD.HHMM<\/tt><\/strong>: Water vapor mixing ratio time-height cross section<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.temperature.20170529.000005.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1105 size-full\" src=\"http:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.temperature.20170529.000005.png\" alt=\"\" width=\"881\" height=\"329\" srcset=\"https:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.temperature.20170529.000005.png 881w, https:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.temperature.20170529.000005-300x112.png 300w, https:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.temperature.20170529.000005-768x287.png 768w\" sizes=\"auto, (max-width: 881px) 100vw, 881px\" \/><\/a><\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.RH_.20170529.000005.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1104 size-full\" src=\"http:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.RH_.20170529.000005.png\" alt=\"\" width=\"889\" height=\"329\" srcset=\"https:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.RH_.20170529.000005.png 889w, https:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.RH_.20170529.000005-300x111.png 300w, https:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.RH_.20170529.000005-768x284.png 768w\" sizes=\"auto, (max-width: 889px) 100vw, 889px\" \/><\/a><\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.skewt_.20170529.2040.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1106 size-full\" src=\"http:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.skewt_.20170529.2040.png\" alt=\"\" width=\"658\" height=\"621\" srcset=\"https:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.skewt_.20170529.2040.png 658w, https:\/\/www.ssec.wisc.edu\/aeri\/wp-content\/uploads\/sites\/21\/2017\/05\/aerioe.skewt_.20170529.2040-300x283.png 300w\" sizes=\"auto, (max-width: 658px) 100vw, 658px\" \/><\/a><\/p>\n<h3>&nbsp;<\/h3>\n<p>&nbsp;<\/p>\n<h3>Troubleshooting<\/h3>\n<p>In rare cases TPSS may not start correctly and crash.\u00a0 Ensure that a noise filter file is present (see Initialization section above).\u00a0 Reboot the hardware.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Overview The Thermodynamic Profiling Software System is a software package that processes raw AERI data and optional meteorological surface observations in order to produce thermodynamic profiles and uncertainties using an optimal estimation method.\u00a0 The principal components of TPSS are rsync data transfer transfer the raw AERI data (.RNC and .SUM files) from an AERI or [&hellip;]<\/p>\n","protected":false},"author":13,"featured_media":0,"parent":1065,"menu_order":10010,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"footnotes":""},"class_list":["post-1085","page","type-page","status-publish","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.ssec.wisc.edu\/aeri\/wp-json\/wp\/v2\/pages\/1085","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ssec.wisc.edu\/aeri\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.ssec.wisc.edu\/aeri\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.ssec.wisc.edu\/aeri\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ssec.wisc.edu\/aeri\/wp-json\/wp\/v2\/comments?post=1085"}],"version-history":[{"count":21,"href":"https:\/\/www.ssec.wisc.edu\/aeri\/wp-json\/wp\/v2\/pages\/1085\/revisions"}],"predecessor-version":[{"id":1406,"href":"https:\/\/www.ssec.wisc.edu\/aeri\/wp-json\/wp\/v2\/pages\/1085\/revisions\/1406"}],"up":[{"embeddable":true,"href":"https:\/\/www.ssec.wisc.edu\/aeri\/wp-json\/wp\/v2\/pages\/1065"}],"wp:attachment":[{"href":"https:\/\/www.ssec.wisc.edu\/aeri\/wp-json\/wp\/v2\/media?parent=1085"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}