FFMP ARI Configuration Guidance

ARIs and MRMS are not displayable in AWIPS by default with 16.2.1. The ARI data can be downloaded, and FFMP can be configured to display ARIs and MRMS. The instructions for setting this up are in the AWIPS Living Release Notes Google Doc (around row 42 labeled ARI/FFMP and MRMS). Here is a copy of the living release notes. The basic steps:

  1. Downloadi ARI files from the Internet and rename them to include YEZZ88_KWBZ, so the AWIPS grib decoder will recognize them,
  2. Configure FFMPSourceConfig.xml to add source entries for ARI data and MRMS,
    1. The documentation in the SMM is a bit hard to follow and has tripped some people up. The easiest way to add the sources is to back up your SITE-level file, copy the base file to configured, and manually adjust the hydro database names, e.g. hd_ob92oax, for the five VGB (Virtual Gauge Basin) sources. Nobody has been able to add local sources to FFMP prior to 16.2.1 because of limitations in FFMP, so it is extremely unlikely that anyone has local modifications to the FFMPSourceConfig.xml file other than the default hydro database names in the VGBs. If you do, then you can just add them to the file with ARI and MRMS sources. To see what the ARI and MRMS tags look like you can view the 17.1.1 base FFMPSourceConfig.xml file here.
    2. Ensure you have FFMP maps by loading them from the Maps->FFMP Basins->FFMP Small Stream Basins (these have been getting dropped in build updates which causes failures in FFMP customization creating the binary files that map data to basins).
  3. Restart edex-ingestDat on dx3 and dx4 (currently not in release notes). The most efficient way to do this is to "service edex_camel restart ingestDat", so you don't have to bring all edex down.
  4. Monitor edex-ingestDat log files on dx3 or dx4 and watch for any errors in creating the FFMP binary files (templates, sources, and aggrGeom).
    1. e.g. "cd /awips2/edex/logs" and "vim edex-ingestDat-20160701.log" (or "tail -f edex-ingestDat-20160701.log")
  5. Copy renamed files in the manual endpoint. The ingestGrib decoder will process  the grib files into the grid table, and ingestDat will listen for and create basin-averaged FFMP data
    1. A handy command for this in the folder with the files is rename "" YEZZ88_KWBZ. *.grib2
  6. Monitor the ingestDat log file and ensure the data were processed.
    1. e.g. "cd /awips2/edex/logs" and "vim edex-ingestDat-20160701.log" (or "tail -f edex-ingestDat-20160701.log")
    2. If ingestDat doesn't see them, but ingestGrib does, then you may have a problem with your FFMP binary files, and NCF may need to assist you in moving the templates, sources, and aggrGeom binary files out of the way and bouncing ingestDat. It can take an hour to crunch through recreating the binary files (verbose output in log files).
  7. Adjust grid purge rules for the ARI grid data to ensure they don't eventually get purged
    1. change to the site-level override directory for purge rules:
      e.g. cd /awips2/edex/data/utility/common_static/site/OAX/purge (for OAX CWA)
    2. Open the gridPurgeRules.xml file with your editor of choice
    3. Add the following lines to the file.  This rule keeps 1 version of each of the ARI grib files regardless of its reference time. If an update is ever loaded, the older version will be purged. These lines are:
              <rule>
                     <keyValue>ARI</keyValue>
                     <versionsToKeep>1</versionsToKeep>
              </rule>
  8. Adjust FFMP purge. Check if you have any site overrides for ffmpPurgeRules.xml (most sites do NOT), and if you do have an override then you need to ensure you have a sourceName key for ever purgeRuleSet tag:
    1. Open any localization overrides for the ffmpPurgeRules.xml file (IF you have them) with your editor of choice. For example, for a site-level override for OAX:
      e.g. vim /awips2/edex/data/utility/common_static/site/OAX/purge/ffmpPurgeRules.xml
    2. Ensure that a line containing  "<key>sourceName</key>" is present in the file for any purgeRuleSet tags. This line alone helps the purge process to avoid removing archive reference datasets like ARI which might be older than the purge rules allow (there is NO need to manually add an exception for ARI).  And, without this line, FFMP purging may not even function.  The base-level ffmpPurgeRules.xml should correctly contain this line but offices using site (or other) overrides may not have this line in their files.  Update all ffmpPurgeRules.xml overrides with the following syntax for the key definition: 

         <purgeRuleSet>
             <key>sourceName</key>
             ** ... Default or other purge rules go here ... **
         </purgeRuleSet>

  9. Configure  FFMPRunConfix.xml for MRMS using the <product name="MRMS" key="mrms"/> to enable MRMS in FFMP. There is no configuration in this file for ARI.

More ARI Background: ARI data is static gridded data on a CONUS scale. When the data is ingested, it is stored in two separate locations: one in grid and another in FFMP. The data is clipped for a regional scale by default when it is stored as grid data. You can display these clipped grids by using the Product Browser from the CAVE menu (see jobsheet 1 on ARIs page), and this is particularly useful at visualizing ARI artifacts along regional boundaries and other hot spots. FFMP will create its own set of data to display using the FFMP basins map, which will be used to create basin-averaged ARIs to compare with other data in FFMP. So the FFMP ARI maps are smaller than the regional clipped grid because they only cover the area of the FFMP basins.

For questions about FFMP configuration contact Michael.A.Magsig@noaa.gov.