NWS AFWS Precip Gages -- The Automated Flood Warning System rain gages can be located on their map at:

   http://water.weather.gov/afws/

You will need the gage ID for your setup. For example the GetRealtime_Setup.txt line could be:

AFWS-wprn7-pdt; 10910; Precip; Westport Golf Course, NC

Where wprn7 is the gage ID and pdt is my time zone id (Pacific Daylight Time) even though this gage in North Carolina is in the Eastern Time Zone. I'm not sure what happens when PDT turns to PST so I will have to update this soon. I don't know how to get the Lat/Long's of the gages yet but I will update this when I do.

The GetAccess table rsite example line is:

For AFWS sites with stage, then a stage setup could be:

AFWS-dscv2-pdt; 2911; Stage; Unnamed Trib at Dulles Station, VA

where the datatype_id= 2 and parameter_code= Stage.

KISTERS QueryServices - Request Information:  KISTERS database can be accessed by URL like: http://kiwis.grandriver.ca/KiWIS/KiWIS?service=kisters&type=queryServices&request=getTimeseriesValues&datasource=0&format=Ascii

&ts_id=12460042&from=2015-10-23&to=2015-10-24

GetRealtime.exe will replace the needed info in this URL example for a current retrieval.

GetRealtime_setup.txt:

KISTERS; 10049; Rainfall; Frostback Falls, ON

Note that some servers are http: and some https: so the whole server needs to be in table rsite's station_id like:  http://kiwis.grandriver.ca

GetAccess table Rsite:

Note the ts_id goes in parameter_code and http://kiwis.grandriver.ca goes in station_id.

If a username and password needed then the table rsite's paramer code would be: 12460042,myname,mypassword

The Kisters Zulu time will be automatically converted to your PC's local time.

NOAA NEXRAD WSR-88D class radar imagery--These images are provided free on the web and are updated about every 5 minutes:   http://radar.weather.gov/

 New GetNexrad 3.6.0--Iowa State University Mesonet historical images can now be directly loaded for a loop of up to the past 24 hours for Ridge2.  Historical radar images beginning Jan 1, 2012 can be downloaded for 24 hours, 1 day at a time.

Update May 20, 2022 GetNexrad 4.4.1--The NWS has discontinued the N0Q product sent to Iowa Mesonet and has been replaced with N0B. I don't see the point in this N0Q to N0B update but... Update your N0Q png image to N0B png image using GetNexrad.exe. If in your \GetNexrad folder your world files *Q.pgw and *B.pgw look the same you can just rename all your Boundary and Point files *Q.txt to *B.txt. Or you can redo them with LatLongPixels.exe and LatLongPixelsFromFile.exe that have been updated May 20, 2022.

Like this:

NexradBoundaryYUX-10101Q.txt >>> NexradBoundaryYUX-10101B.txt

Iowa Mesonet N0B: https://mesonet.agron.iastate.edu/onsite/news.phtml?id=1431

The new high resolution US mosaics still use the N0Q name.

Typical GetRealtime_setup.txt line for more modern N0Q radar:

NEXRAD-ESX; -10363; Rainfall; Burro Creek Sub 3; 0

Where 0 is for convective dbz to rain.

Table rsite would look like this:

AND NEVER APPLY TIMESHIFT TO RADAR.  I's automatically set to your PC's time and for history daylight savings periods ware adjusted.

Radar Images From File like NOAA Weather and Climate Toolkit KMZ files can be read using these steps. You may want to use GetNexrad first to see what you got and follow the GetNexrad help instructions for KMZ files.

Level II is 0.25 km x 0.5 degree x 0.5 dbz and you would catch hell from ROC for mistating this! This includes both reflectivity and velocities. N0Q averages the level 2 into 1km grids and from my comparisons N0Q may be a slightly better choice for adjusted rainfall due to hitting a gage from 20,000 ft with a 1km vs 0.25km shot.

To read radar images from file:

1) Be sure the radar image folder has the boundary and point file for the station or you can add the boundary and point file location to your GetRealtime_setup.txt file like this:

Radar Boundary File folder=C:\GetBMX\GetRealtime\N0R\

2) Select the radar station(s) from the GetRealtime station list.

3) Check 'Scheduled Batch' and select option 'Read radar from file'

4) Uncheck 'Scheduled Batch'

5) Click 'Start Retrievals' and select radar images to read.

Note that KMZ universal Zulu file times will be converted to your time and adjusted for daylight savings.

If you have more than 4 or 5 days of radar images to select, then you will have to create a file list of the radar png's. I would imagine you could process 1 to 2 million radar images at a time using this file list, about 10 to 20 years but you should break it into start and ending day light savings because GetRealtime only checks the start date.  0_* source png's are already time zone and DST converted so this doesn't matter.

 To create a file list of the radar names:

1) Open a console window.

2) Change directories to the radar folder.

3) Enter the command line: dir 0_* /b > filelist.txt

4) Exit.

Your radar file list name filelist.txt should look like this:

0_ESX_N0Q_201506160907_0.000.png

0_ESX_N0Q_201506160916_0.000.png

0_ESX_N0Q_201506160926_0.000.png

0_ESX_N0Q_201506160936_0.000.png

0_ESX_N0Q_201506160946_0.000.png

Use the 'Scheduled Batch' menu as above and check the 'File List' also and then uncheck 'Scheduled Batch' to watch.  If you need to include multiple paths for your radar images then you can include the full path with each file name but you will have to sort the file list by date yourself.

The rsite table paramter N0Q-Ridge2 has to be correct for realtime downloads. It tells GetRealtime to download N0Q images and Ridge2 means add png instead of gif to image request. Not so (rsite parameter not used) when selecting files on disk or using a file list, everything is determined from the file name. Either saved files, KMZ, and other sources. The point file is then selected based on DSID, ESX, N0Q is expected to be in the same folder as the radar images... unless you use 'Radar Boundary File folder=' in setup. And for huge file list runs close the radar image view window for paint blistering speed.

If your GetRealtime_setup.txt has the 'Radar Boundary File folder=...' then the real-time latest radar image will be saved to this boundary folder so you should point GetNexrad.exe to this boundary folder to view current radar with boundaries. And you can delete your boundary and point files in the GetRealtime.exe folder to clean things up.

The radar imagery provides for computation of area average rainfall amounts for any area in the USA.  This is a new addition and is described at the bottom of this page.

Canada Weather Radar:  The 31 Canadian C band randars have been added and are accessed just like N0AA radar data. The product code is "C0R"  (that's C-zero-R, not C-oh-R) for 0.3 degree base reflectivity (10-minute, 1 km, 2.5 dbz). The radar images are available for the past 48 hours. Historical radar images can be downloaded by hand from http://climate.weather.gc.ca/radar/index_e.html. and GetRealtime can read them from file (you may have to format the filename?).  The Canada radar images also come in higher resolution at the lower reflectivity's for snowfall. If you are interested in constant altitude precip scans called CAPPI then you can use GetNexrad to load those but I would stick to the base 0.3 reflectivity unless ground clutter near the radar site is a problem.

New Canada radar ID's Nov 22, 2020:  RadarSites.txt

Canada Radar has 3 different reporting types:

1) New S band 6 minutes id= Sxx

2) Old C band 10 minutes id=Wxx, Xxx

3) Composite10 minutes id= Sxx, Wxx, Xxx

For composite radar images add '-comp' to the Station like NEXRAD-SFW-comp. One dumb thing about composite images is that they used the same red color for circles as used for the red rain color, not too bright hey! If your radar boundary intersects with these dumb lines then you can manually adjust your point file to omit these dumb lines or write your congressman.

GetRealtime_setup.txt Canadian RAIN example:

NEXRAD-WHK; -10980; Rainfall; Edmonton, AB  

GetRealtime_setup.txt Canadian SNOW example:

NEXRAD-WHK; -10980; Rainfall; Edmonton, AB; 1

GetAccess table 'rsite'GetAccess table 'rsite' setup example:

If you want to mix Canada 10-minute radar with Nexrad 5-minute radar for the same runoff routing time step then add -5min to your setup like this:

NEXRAD-WHK-5min; -10980; Rainfall; Edmonton, AB

Why RAIN and SNOW radar products??? Beats me, but I bet it is similar to choosing Convective or Cool Season Z-R's in the US. So you are living on the cutting edge up there so figure it out yourself.  And yes, GetRealtime does snow accumulation and melts for you lucky guys up there.

Canada Streamflow:  The real-time streamflow and water level data collected at over 1600 stations across Canada has been added to GetRealtime.  I have automated the Wateroffice 'I Agree' button to their Disclaimer Screen that says you must use their real-time stream data with caution at your own risk. In other words it is provisional and subject to revision.  Winter ice in particular can affect water levels and cannot be accounted for in real-time.

Update Mar 14, 2017: Canada full streamflow data no longer supported. Permission denied!!!  Only parameters 46 stage and 47 flow supported for past 3 days.

Update 12/14/2014: The Water Office has changed some things so you may have to use your webbrowser to manually access 1 gage to prime the cookies pump. If the cookie expires you may have to repeat. Unlike everybody else in the water data business, Canada won't let just any taxpaying fool access their data with out consent of their lawyer and doles out cookies to us poor beggars.  Also I believe the flow and G.H. parameter codes have changed to those below.

GetRealtime_setup.txt examples for streamflow and water level:

02HC025; 1984; Flow; HUMBER RIVER AT ELDER MILLS [ON]

02HC025; 2984; Gage Height; HUMBER RIVER AT ELDER MILLS [ON]

GetAccess table 'rsite' setup example:

So far I have found these parameter codes:

46=gage height (M)

47=flow (CMS)

1=air temperature (C)

5=water temperature (C)

18=total rainfall (MM)

19=increment rainfall (MM)

21=conductivity (S)

25=turbidity (JTU)

28=humidity (%)

34=wind direction (deg)

You can convert total rainfall parameter code 18 (datatype 9) to incremental rainfall by using datatype=10 but that can be risky if the total rainfall has diurnal fluctuations.  Wind direction is converted from degrees to 0-4.

The SpotWx forecast models are all now available.

Update Sep 16, 2020: SpotWx may require user to manually save data to html file and enter the file name in setup's formula1 field.  You could SHELL your web browser and then save html to file.

****; **; **; **HRRR Forecast Spotwx Run Manually with Your Web Browser**

SHELL-"C:\Program Files\Internet Explorer\iexplore.exe" "https://spotwx.com/products/grib_index.php?model=hrrr_wrfprsf&lat=29.61&lon=-95.38&tz=America/Chicago&display=table"; 0; Webbrowser; Save Spotwx to File

Check the model types here: https://spotwx.com

Source code 25= Spotwx Forecast RAP, HRRR, NAM, SREF, GFS, HRDPS, RDPS, GDPS, GEPS

RAP and HRRR are updated hourly and will be checked every half hour, the other models will be checked every 6 hours.

GetRealtime_setup.txt:

FORECAST-RAP; -11348; Forecast Rain; Burro Cr; 0; 34.71,-113.17, 6

   or

FORECAST-RAP-10min; -11348; Forecast Rain; Burro Cr; 0; 34.71,-113.17, 6; C:\GetRealWunder\SpotWx Tabular model data.html

The forecast ending hour values will be used to fill the preceeding 5 minute steps, or your duration if added like -10min. The forecast is time shifted to your PC's time zone so no time shift is needed. The ending 34.71,-113.17 are the lat, long for your forecast point.  If you have multiple subs within a few miles use the same lat/long for all the subs so you don't have to download the same forecast multiple times.  The ending 6 is how many hours of the forecast to use. If missing or 0 then all forecast hours are included.

 Because you will probably be writing the forecast to your adjusted radar DSID, no additional rsite table info is needed. If not, you need to have the usual rsite info with anything for parameter_code and station_id.

The datatype_id's are listed in GetRealtime_datatypes.txt:

TMP=17 DPT=16 RH=18 WS=28 WD=19 WG=20 APCP=10 or 11

I don't have a dog in this but RAP is updated hourly, HRRR is always 2 hours behind. Make of it what you want.  In fact I'm so happy with 15-Minute HRRR radar images from Iowa and NWS 7-day QPF that I don't use any of this stuff.  And of course I use my radar Nowcasts to replace hours 1-3.  Actually, the speed of SpotWx compared to downloading all the HRRR images is so much faster that I think I will start using SpotWx HRRR instead.

Your Personal Weather Station or Other Text Files--Text files from your weather station can be added to the Access Database.  The text file must have a date and time and value columns and can be either Space, Comma, SemiColon, or Tab delimited. Below is an example of a Davis weather station day file:

If your date format begins with the day like dd/mm/yy as above then you will need to add a date format to the GetRealtime_setup.txt file base1 field like this: (If your date does not begin with dd and uses '/' and is in column 1 then nothing is needed.)

The Station_ID cell contains 'FILE-n' where n is the data column to write to the database. The Date is column 1, Time is column 2, and so the first data column would be 3... etc. The number and content of the header lines is not used and can be what ever or none at all as long as the first non space character is not numeric.  Also if error retrurns "No data for period" try FILE-2 or 1 less or more than you think and check the values returned are the column you want and adjust FILE-n accordingly.

The GetAccess HDB database table 'rsite' below has the text file name in the 'parameter_code' and the 'station_id' is just 'FILE':

Example for precip accumulation total that resets or not and converts total to increments:

1) put ...\MyData.txt in rtable parameter_code

2) put 9 in rtable datatype_id and use 10xxx as your datatype_site_id.

3) GetRealtime_setup.txt line reads:

FILE-7; 10875; Rainfall; Test; m/d/yyyy

4) Set Days to 20 or whatever to cover your period of interest.

5) Run and now open the created file wunderin.txt with Excel (has no carriage return) to verify what column is being read.

Cumulus from Sandaysoft documentation says it's monthly log file (Aug09log.txt) gets updated about every 10-minutes:

 http://wiki.sandaysoft.com/a/Monthly_log_files

So GetRealtime should be able to read their (;) or (,) delimited file as well... but remember the format is 'dd-mm-yy' or 'dd/mm/yy' and has to be added to the GetRealtime_setup.txt because of this screwy dating convention... just like Davis.

EasyWeather.dat file:

The EasyWeather data file has it's date in the 3rd column so the base1 cell in GetRealtime_setup.txt would look like this:  yyyy-mm-dd, 3

You may find that many of your weather data types are not listed below on the table of datatype_id's. You can select any datatype_id that suits your averaging/cumulation/total needs and then change the unit_name to what ever you would like.  You may need more than one site_id if you still need more datatype_id's or want to use the same datatype_id more than once.

MS Excel data:

Copy from Excel, paste, and save your text file with Notepad. As an example if only a date and 1 value column were being saved then:

Getrealtime_setup.txt:

FILE-1; 1101; flow; Test Using Excel Text File

In table Rsite you need to put the text file name in parameter_code and FILE in station_id just like above.

The GetAccess Excel files DBedit.xls, DBretrieve.xls, and the More Stuff GetFlowRecord.xls can also be used to manually add and retrieve GetAccess database data and can show beginners how to get started with VB database code using Excel.

HEC-DSS files using their MS Excel HEC-DSS Add-In--Write Excel files with HEC-DSS header info for storing to HEC-DSS files using the Excel HEC-DSS AddIn.  You can also write HEC-DSS to GetAccess by using the MS Excel HEC-DSS Add-In to read data into DBedit.xls and then write the data to GetAccess.  Update 1/8/2014--GetRealtime can Put and Get Hec-DSS files directly. See below.

The MS Excel HEC-DSS Add-In can be downloaded for free here: http://www.hec.usace.army.mil/software/hec-dss/excel/

END OF SOURCES

====================================== 

5)      You are now ready to add your new station name and data type to the table Rsite, but first you will need to determine the new station’s Site_id and Datatype_id.  These 2 values will then be used to create a unique Datatype_Site_ID that is stored with each retrieved value.

Site_Id's= 0^00 where 0=area or reach, 00=count in reach.  Site_Id's are 0 to 999.

Datatype_id 1= flow, cfs

Datatype_id 10= rainfall, inches, etc,… (See table of Datatype_id’s below)

Datatype_Site_ID (DSID) = 00^000 where 00=datatype_id and 000=Site_id.

Valid DSID’s are integers in the range -32,768 to 32,767

Example: DSID 10212 = rainfall, area 2, 12th station

Example: DSID 1212 = flow, area 2, 12th station

Example: DSID -1212 = Computed flow, area 2, 12th station

Use negative DSID's for values that are computed by GetRealtime.exe to keep them separated from the web source reported values and also to note it has been computed.

You might use another method for creating the Site_id and associated unique DSID but remember the valid DSID integer range -32,768 to 32,767.

Datatype_id’s are associated with formatting and averaging methods.  If  more than 32 (64 if negatives)datatypes are needed, then you can break up your station into 2 or more site_id's.

If you are retrieving metric values or converting English to Metric then it’s ok to change the unit_names to metric and for that matter even the datatype_name.  Just remember that datatype_id's have certain rounding and averaging methods.  Bearing this in mind, you could make your own data table with completely new parameters and units.  If anyone would like to share their alternative datatype_id table just add it to the comments button below or email me it and I will add it here. 

                        Table of datatype_id’s

Always use 10xxx for rain gages, -10xxx for radar, -11xxx for adjusted radar and 11xxx for snowmelt. These datatype_id's are used by GetNexrad for proper display.

  10xxx = Rainfall

-10xxx = Radar

-11xxx = Adjusted Radar

  11xxx = Snowmelt

To convert total rainfall datatype_id=9 to incremental rainfall use datatype_site_id=10xxx with datatype_id=9.

Important: Datatype_name 'ratio' is reserved for rainfall adjustment and usually associated with datatype_id=31.

Datatype_id's with special associated computations: 19=wind direction, 27=ET, 29=solar/long radiation, 30=runoff, 32=radar image maximum value.

For runoff datatype_id 30 computations, daily datatype_id's 4, -4, and 5 end of day conditions will be written secretly so be aware. Runoff datatype_ID's in rday: datatype_ID 4 = Soil Storage or Initial Loss. datatype_ID -4 = Precip or zero. datatype_ID 5 = Groundwater flow.

Note:  Datatype_id's 10 & 11 rainfall and 27 ET unit values rounded as 0.00000 and hourly's are 0.0000

USGS rounding means:

 0

 0.001 - 0.099

 0.01 - 0.99

 1.0 - 9.9

 10.0 - 99.9

 100 - 999

 1010-999990

6)      Using GetAccess.exe, open the empty database file GetAccessHDB.mdb that has all the data in each table deleted.

7)      As an example we will add the Wunderground example above for Station ID = KLAS and Parameter Code = Dew PointF.  The Dew PointF associated datatype_id = 16.  Our first site_id will be area 1, station 1 or site_id = 101.  Our unique datatype_site_id = 16101.

8)      We are ready to update the table rsite with our new station data.  Click DB Tables.

Select rsite, and then check the Allow Edit check box click GO.  The blue * indicates the row is available for adding new data by typing in fields on that row.

.

Fill the fields as this example then click on the blue * and the data will be stored and a new edit line * added: (use alt-248 on num pad for ° degree symbol)

Let’s add the USGS station example for flow.  Site_id will be area 1, station 2 or 102.  Datatype_id for flow = 1.  The unique datatype_site_id is then 1102.  To add this data we can type at the blue * or we can copy a row from the ExampleRsite table using the Edit Menu button or right click on the selected and the Edit Menu will open.  For now just type in the new site data:

We now have 2 sites with the needed Station ID and Parameter Code for retrieving real-time data using GetRealtime.exe.  GetRealtime.exe stations list now needs revised with just these two new sites before data can be retrieved and stored.  Refer to the GetRealtime section below Adding sites to the GetRealtime station list.

Program Use

GetRealtime.exe

(retrieval and computations of real-time web data)

The hourly and daily data should be updated for the past 7 days using GetRealTime.exe so that the examples can be seen once you read this section on how to that.

Setting the Database Connection String:

If GetRealtime.exe cannot make a connection to the Access database the following message will appear.  Set the connection string to the GetAccessHDB.mdb database using the Connection button on the Select Station form (control button 6 below).

The connection string should look similar to this depending on the path where GetRealtime.exe was installed:

Driver={Microsoft Access Driver (*.mdb)};DBQ=

C:\Program Files (x86)\GetRealtime\GetAccess\GetAccessHDB.mdb;Uid=Admin;Pwd=Me;

Or better yet: 

Provider=Microsoft.Jet.OLEDB.4.0;Data Source=

C:\Program Files (x86)\GetRealtime\GetAccess\GetAccessHDB.mdb;User Id=admin;Password=;

For Excel, the connection string would look like this:

Driver={Microsoft Excel Driver (*.xls)};DBQ=

C:\Program Files (x86)\GetRealtime\GetAccess\GetAccessHDB2.xls

Those using Excel as their database should open the GetAccessHDB2.xls file with Excel and read the 'ReadMe' worksheet to learn more about using Excel as a database.

Description of the controls on the GetRealtime form:

1)      Days—Number of days to retrieve.  1 day means yesterday and today.  The unit values are retrieved and averaged for hourly and daily values.  One daily value for yesterday will be available for storage.

2)      Missing Values Allowed in Day—Percentage of missing unit values allowed in a day before a daily average will not be computed.

3)      Bad Value Check—Percent change in the unit values from one time step to the next before the value is raised as a possible error.  If in batch mode, the value will be listed in the GetRealtime_errorlog.txt file.  If in interactive mode, a message box will appear for how to handle it.

4)      DSID for 1 Station—DatatypeSite_ID, typically not used but can be entered for 1 station retrieval.  Selecting just the 1 station from the station list is easier.

5)      Station Name or range—Displays the selected station or range of stations from the station list if only 1 station or a range of stations is wanted.

6)      Select Station From List button—Displays the station list if only one station is to be retrieved.  For editing the station list see GetRealtime.exe Setup Section above.

7)      Write Dailys—Daily average computed from the retrieved unit values will be stored in the database.

8)      Write Hourly Avg—Hourly average computed from the retrieved unit values will be stored in the database.  The hourly value time stamp will be for the end of the averaging period.  For instance, six 10-minute unit values with the last value reported as 10:04 am will have the hourly time stamp of 10:00 am.

9)      Write Daily MaxMin—Daily Max and Mins of the retrieved unit values will be stored in the database.

10)    Write Unit Values—Retrieved units value will be stored in the database.

Time steps for the unit values retrieved vary:

Wunderground unit time step can range from1-minute, 5-minute, on up to 3 or 4 hour time steps.  5 and 10-minute time steps are the most common.  Airports at Wunderground usually have 30-minute and 1-hour time steps.

The USGS data are typically in 15-minute time steps, but can vary during events.

CalDEC data time steps are 60-minutes.

USCS Snotel data time steps  are 60-minutes.

USBR data can be requested as either 15-minute or 60-minute.  Most USBR meteorological data time steps are available only as 60-minute.

CIMS data time steps are 60-minutes.

11)    Write Unit Shifts—Each shift value and the equation number used will be stored in the database for checking and debugging computation equations if any.

12)    Overwrite Source 5—Each daily, hourly, unit value has a Source field value stored. 1=Wunderground, 2=USGS, 3=CalDEC, 4=USCS Snotel, 5=WRITE PROTECT, 6=USBR, 7=COE, 8=CIMIS.  The database field Source if set to 5 by editing of the database will protect the value from being over written.

13)    Bad Value Check—Check for possible bad values as defined by the Bad Value Check percentage change.

14)    Last Run's Storage--For runoff computations, the last run's starting basin storage space and rainfall sum will be used so that running a month or more is not needed. Now you can just start today if the runoff part of the hydrograph has reached zero. For a 12 hour unit graph's recession this may take a day or 2.

15)    Scheduled Batch—GetRealtime.exe can be run in the background at user defined intervals.  The Wunderground stations will always be retrieved at the interval set.  The other sources only update their data each hour so GetRealtime.exe will only query their stations at half hour intervals.  The first time GetRealtime.exe is run in batch mode all stations will be retrieved for the number of days set.  After the first time, only values for the current day will be retrieved.

Getrealtime.exe can also be ran in Batch Mode from a Desktop shortcut with the shortcut properties Target set as “…\GetRealtime.exe batch”.  Also GetRealtime.exe can be run from the Windows Control Panel’s Scheduled Tasks in the same manner, be sure to include “batch”.

16)    Start Realtime Retrieval button—Start retrieving data.  If the 1 station retrieval box is not checked, then all of the stations on the station list will be retrieved.

17)  Historical button—Retrieve unit values or daily values based on a user set date range instead of the number of days to retrieve.  The periods for the availability of historical unit values can vary for each source and each station.  The Historical Button can be toggled on and off by repeated pressing.

It's important that the MS Access HDB database be routinely 'Compacted and Repaired' to speed up downloads. The size of the MS Access database doesn't seem to matter as much as how often data is being updated. I retrieve about 100 stations every 1/2 hour with 5 minute unit values (Nexrad). I find the down load time will double in about 2 to 3 days so I 'Compact and Repair' using the GetAccess.exe 'Connection' button every day or two... so be smart like me. Look at your 'GetRealtime_errlog.txt' file to see how your download times vary over time.  GetRealtime in Scheduled Batch has a checkbox for 'HDB Compact' which will compact and repair each day at midnight while running in batch mode.

My MS Access database is 200 MB's with about one year's worth of data. Wikipedia says MS Access can handle 1 to 2 GB's. We shall see. One option to reduce the database size is save the database file every year and then delete all the unit values and keep going so historical unit values would be available if ever really wanted in separate files. 

Adding Sites to the GetRealtime Station List:

After you have deleted the example data from all the data base tables, then you need to delete all the sites on the GetRealtime station list as follows:

1)      Make a backup copy of the file GetRealtime_setup.txt for later reference. You will need it for reference!

2)      Open GetRealtime.exe and click on the Select Station from List button:

Click the Edit List button.  Right mouse click on the left column and the edit menu will appear.  Click Delete Record and the selected row will be deleted.  Continue row by row deleting all the records.  The last row cannot be deleted, edit the last row then right mouse click the left column and Add New Record.  Edit the fields like this and click on Save:

Optionally, the setup file GetRealtime_setup.txt in the GetRealtime directory can be edited using Notepad provided with Windows in Start/All Programs/Accessories.

The stations on the list are now ready to be retrieved.

Make sure the DSID for 1 Station check box is not checked.  Click the Start Realtime Retrieval to test the changes to the station list and Access database changes.

The daily average values for yesterday were successfully computed and displayed indicating our changes to the GetRealtime Station List and the Access database were successfully made. Woohoo!!!

If not, recheck the values in the database table Rsite and retry.  If still unsuccessful check the values on the Station List.  The datatype_site_id must match in both the rsite table and station list and the parameter_code and station_id in table Rsite must be valid for that web source.

Change the Days text box to 7 days and repeat the retrieval so GetGraphs can graph the new data.  The final step is to edit the GetGraphs setup to display our 2 new sites.  If you open GetGraphs now you will see only the Web pages will have data.

See Adding Sites to the GetGraphs Setup below.

'Tools' command on the station list window:

The text file GetRealtime_tools.txt can be created for a list of commands that can be shelled. For example the file GetRealtime_tools.txt might read:

Notepad.exe c:/vbget/getxmore/getrealwunder/GetRealtime_errlog.txt

Notepad.exe c:\vbget\getxmore\GetGraphs\GetGraphs_setup.txt

C:\Program Files (x86)\HEC\HEC-RAS\5.0.3\Ras.exe

C:\Program Files (x86)\Microsoft Office\OFFICE11\Excel c:\vbget\runoffIn.txt

When each is selected, the program line is shelled. Shelling a program simply means execute the command line which probably entails opening a new window. The new window remains open until the user closes it or the command self terminates.

Computation Examples on the GetRealtime Station List:

With the 5 equations discussed here along with routing and conditional IF line execution you can imagine how just about any irrigation and reservoir rules and demand scheduling can be handled so Grand Coulee is on the table. Whether it be a short term operations model or long term model with Monte Carlo's or mix and match.

Note that there are TWO ways to go about computing values for storing in your HDB (hydrologic database) database:

1) All parameters are being retrieved at one time and the parameters will not be saved to the HDB database (but of course could be separately). The datatype_site_id would probably have the same site_id as the parameters if they were stored separately.

2) All parameters are already stored in the HDB database and the Station_ID would be the more general method COMPUTE (or COMPUTE-UNIT), COMPUTE-hour, or COMPUTE-Day. This allows for computation from mixed sites and datatype_site_ids into the computation. COMPUTE-2Hour can be used for Gage/Radar ratios. Also COMPUTE-xUnit can be used for input time steps other than 5-minutes. Likewise, COMUTE-xHour and COMPUTE-xDay.

Special Station_ID's for computations discussed below:

1) COMPUTE

2) ROUTE

3) SNOWMELT

4) FILE

5) RUN

6) MRUN

7) NEXRAD

8) COMPUTE-Get

9) COMPUTE-Put

10) SHELL

11) FORECAST

12) LOOKUP

13) IF

COMPUTE and some data_type_id's  also have special built in computations in the HDB table rsite parameter_code:

1) ETshort  (data_type_id 27)

2) ETtall   (data_type_id 27)

3) Solar  (data_type_id 29)

4) Solar long  (same datatype_id 29)

5) Ratio (data_type_id 31) 31 is not a special computation ID. It has no rounding but what is used with the keyword here "Ratio". You can still use datatype_id 31 how you choose.

6) and data_type_id  30 will automatically compute runoff from the same site_id using datatype 10 for precip, or it will use another DSID (10xxx) of your choice for rainfall if you put a DSID in the parameter_code cell.

Now Wizards are available to give you a better idea of how to setup your data_types and datatype_site_id's so be sure to use the setup Wizards.  If there is no Wizard for what you want to do, look at the example setup.txt list that came with the installation and the GetAccess table 'ExampleRsite'.

All these special cases are discussed below in separate sections.

The Station List GetRealtime_setup.txt has fields for adding up to 5 sets of formula expressions, shifts, and bases.  The base of the equation is a value or expression that is evaluated to determine which formula expression will be used.  The shift is available to make changing the equation simpler by including the variable Shift in the equation.

The P1, P2, P3… variables represent the retrieved parameters in the order they were entered in the GetAccess database table Rsite Parameter Code field.

The variable D represents the DateTime value retrieved with the parameters.

The function Julian(D) returns the julian integer day of the year.

The variable N returns the number of parameters with data (missing parameter values are set to zero if N is included in your formula) for that time step.  The variable N is useful for logging database stats or more importantly for computing averages when some gages are missing.

Example of parameter averages with and without missing values:

(P1+P2+P3)/3 returns average or missing if any P is missing.

(P1+P2+P3)/N returns average of non-missing P's.

To fill the last computed value for the future forecast days add 'FILL' to the 'shift1' column like: COMPUTE-Hour; -32307; Fraction; AAC Drop 2 % Ratio Max Flow; 0; FILL; P1/6400

Your formulas (P1^2+P2/P3) are evaluated using VBScript .

VBscript math functions:   http://msdn.microsoft.com/en-us/library/72ca82az.aspx

VBscript other functions:   http://msdn.microsoft.com/en-us/library/3ca8tfek.aspx

To replace a runoff forecast with USGS gage history one could use D and vbscript Now() like this:

COMPUTE; -1299; RUNOFF; Replace Runoff With USGS History; D=Now(); 0; P1; D>=Now(); 0; P2

Where P1 is runoff and P2 is USGS as the table Rsite parameter_codes.

Example of converting Gage Height to Elevation:

05054000; -3710; Elevation; Red River Of The North at Fargo; 0; 0; P1+861.8

Notice the Datatype_Site_ID -3710 is a negative integer to keep calculated values separate from reported values and to point out it was computed.

In the above example the Base1=0 will convert any Gage Height below 0 as 0.

The Shift1=0 is not used.

The Formula1 P1+861.8 adds 861.8 to the retrieved Gage Height and is stored in the database.  The Gage Height retrieved is discarded.

Example of converting Temperature C to Temperature F:

09380000; -7510; Temperature; Colorado R at Lees Ferry; 0; 0; 32+P1*9/5

Base1=0 will convert any retrieved Temp C below 0 to 0 F.  Probably not a good example but ok for this stations water temperature.

Shift1 not used.

Formula1 32+P1*9/5 converts the retrieved Temp C to  Temp F.

Example of converting Temperature C to Temperature F with Shift:

05054000; -7710; Temperature; Red River Of The North at Fargo; -20; 0.5; 32+P1*9/5+shift

Base1=0 will convert any retrieved Temp C below -20 to 0 F.

Shift1=0.5

Formula1 32+P1*9/5+shift converts the retrieved Temp C to  Temp F and increases the value by 0.5 degrees F.

Example of converting Temperature & Dissolved Oxygen to Percent Saturation:

14211010; -13910; Dissolved Oxygen; Clackamas River near Oregon City, Or; 0; 1.015; 100*P2/(shift*(14.55-0.3940*P1+0.00718*P1^2-0.0000611*P1^3))

Base1=0 will convert any P1=TempC retrieved value below 0C  to 0% saturation.

Shift=1.015 is the adjustment for station elevation and is computed as Shift= 0.0000364*ELV+0.000000000563*ELV^2

Formula1=100*P2/(shift*(14.55-0.3940*P1+0.00718*P1^2-0.0000611*P1^3))

Where P1 and P2 represent the Parameter Codes in order in the database Rsite table of:

10,300 where P1=10= water temperature °C and P2=300=D.O. mg/l.

The dissolved oxygen saturation formula example above is a curve fit to USGS Weiss table at 760 mmHg or sea level.

Example of converting Gage Height to Flow:

(note: you could also use 'Lookup' for ratings.)

09419800; -1013; Flow; Las Vegas Wash above Lake Mead;

3.9; .05; 401.8*(P1+shift-3.9)^1.502; D<cdate("2008-11-26"); int(100*(.05+(.10-.05)*(cdate("2008-11-26")-int(D))/(cdate("2008-11-26")-cdate("2008-11-01")))+.5)/100; 401.8*(P1+shift-3.9)^1.502; 4.7; 0.10-0.10*(P1-4.7)/(4.8-4.7); 401.8*(P1+shift-3.9)^1.502; 4.8; +0.00; 602.2*(P1+shift-3.9)^1.502

This conversion uses 4 sets of bases, shifts, and formulas.  Beginning with set 4:

Set 4=  4.8; +0.00; 602.2*(P1+shift-3.9)^1.502

Base4=4.8, should the Gage Height P1 be greater than 4.8 ft the associated shift and formula will be used.

Set 3=  4.7; 0.10-0.10*(P1-4.7)/(4.8-4.7); 401.8*(P1+shift-3.9)^1.502;

Base3=4.7 feet.  Should P1 gage height be >=4.7 ft and <4.8 ft the associated shift and formula will be used.

Shift3= 0.10-0.10*(P1-4.7)/(4.8-4.7) is shifting with stage where the shift will evaluate to 0.10 ft at P1=4.7 ft to 0 ft at P1=4.8 ft.

Formula3=401.8*(P1+shift-3.9)^1.502

Set 2= D<cdate("2008-11-26"); int(100*(.05+(.10-.05)*(cdate("2008-11-26")-int(D))/(cdate("2008-11-26")-cdate("2008-11-01")))+.5)/100; 401.8*(P1+shift-3.9)^1.502;

Base2= D<cdate("2008-11-26").  Should P1 gage height be <4.7 ft and then Base 2 is evaluated.  If the retrieved DateTime<2008-11-26 00:00 AM then the associated shift and formula will be used.

Shift2=  int(100*(.05+(.10-.05)*(cdate("2008-11-26")-int(D))/(cdate("2008-11-26")-cdate("2008-11-01")))+.5)/100 is shifting with time where the shift will evaluate to 0.10 ft on D=Nov 1 decrease to 0.05 ft on D= Nov 26.

Formula2=401.8*(P1+shift-3.9)^1.502

Set1=  3.9; .05; 401.8*(P1+shift-3.9)^1.502;

Base1=3.9 should the Gage Height P1 be less than 4.7 ft and D is greater than Nov 26 then the associated shift and formula will be used.  If the Gage Height P1 is less than 3.9 ft then the computed flow will be 0 cfs.

Shift1= .05

Formula1= 401.8*(P1+shift-3.9)^1.502

Note:  Turn ON the “Write Unit Shifts” on the GetRealtime retrieval to see what shifts are being computed and what formula is being used in the computation.

Example of converting Gate Opening to Flow:

09429000; -1310; Flow; Palo Verde Canal near Blythe, Ca; 81.0; 0.00; 3.17*70*(P1+shift-40.15)^1.5; P2>81; -0.00; 3.17*70*(P1+shift-P2)^1.5; P1>(P3+77); 0; 0.6214*70*P3*(64.32*(P1+shift-P2))^.51

This computation uses 3 sets of bases, shifts, and formulas depending on the flow type as submerged flow, submerged weir, free flowing weir.  Beginning with set 3:

Set 3= P1>(P3+77); 0; 0.6214*70*P3*(64.32*(P1+shift-P2))^.51

Base3= P1>(P3+77) where P1=upstream GH, P3=gate opening.  If Base3 evaluates true, then submerged orifice flow will be used.

Shift3=0

Formula3=0.6214*70*P3*(64.32*(P1+shift-P2))^.51 submerged flow equation.

Set2= P2>81; -0.00; 3.17*70*(P1+shift-P2)^1.5;

Base2= P2>81 where P2=downstream GH and 81 is the elev of the bottom of the gate for submerged weir flow.

Shift2=-0.00.

Formula2=3.17*70*(P1+shift-P2)^1.5 where P1= upstream GH and P2 = downstream GH.

Set1= 81.0; 0.00; 3.17*70*(P1+shift-40.15)^1.5

Base1=81.0 or if P1>81.0 then use free weir flow formula1 else zero flow.

Shift1= 0.00

Formula1= 3.17*70*(P1+shift-40.15)^1.5 where P1=upstream GH.

Example of diurnal Side Inflow from a Sewer Plant:

Compute-unit; -24332; Inflow; Bham Sewer Plant; 0; 0; 30-27.11659*Hour(D)/24+78.76657*(Hour(D)/24)^2-51.46933*(Hour(D)/24)^3

OR

Compute-unit; -24332; Inflow; Bham Sewer Plant; 0; 0; 30-27.11659*(D-int(D))+78.76657*(D-int(D))^2-51.46933*(D-int(D))^3

Here you assume a constant daily average 30 cfs and then vary it +- 3 cfs. The rsite table needs any unit value dsid put in 'parameter_code' and is not used... but you could and change the constant 30 cfs to who knows what.

Example of a pumped waste ON/OFF for high flow dilution:

COMPUTE-unit; 24326; Inflow; Acme Pump Waste; 0; 0; 0; P1>50; 0; 3.5; D>cdate("2016-03-19 13:00"); 0; 0

The datatype_ID = 24 for inflow and 3 sets of forumlas are used. If P1 flow greater than 50 cfs then turn on 3.5 cfs pump. If the date is after March 19 at 13:00 then turn off pump at any flow.

Soil Flood Potential Computation:

Because the floods can occurr with comparatively little rainfall on wet soils I started wondering about CN ranges and what fraction of rainfall would runoff as the CN went up and down...

From SCS CN method Q=(P-.2S)^2/(P+.8S)

For a GetRealtime setup:

COMPUTE-day; -6303; Soil Runoff Fraction; Farley Soil Flood Potential; 0; 0; 0; P2>0.2*P1; 0; 100*(P2-0.2*P1)^2/(P2+0.8*P1)/P2

P1 is S as DSID 4xxx and P2 is P as DSID -4xxx and are automatically saved at the end of each day for each runoff comp. You could use COMPUTE-UNIT if you have set 'Write unit CN P,S' on your basin runoff setup for unit values of P and S to be written (also rain/melt Excess in/hr as 3xxx is written).

Table Rsite would be:

303; -6303; Farley Gage; 4; precip fraction; %; percent; 4303,-4303; COMPUTE; 3; AL; 0;

The idea is to be on your toes with wet soils and just what are wet soils. The forecast rain will be wrong too so don't rely on what the runoff says when your soils are wet.

Example of Computing Evapotranspiration Reference ET with out Solar:

Computation of reference ET is done internally by GetRealtime using methods recommended by the ASCE standard Penman Monteith at an hourly time step or your unit time-step.  Both the tall (alfalfa) and short (grass) computations can be made.  If you are interested in the ASCE methods, the following url will get you started:

The ET coefficients can now be checked and entered by using the 'Wizards' button on the Select Station List. Just click on your station, if present, and then click 'Wizards'. You will see a complete breakout of all your coefficients for editing.

http://www.kimberly.uidaho.edu/water/asceewri/ASCE_Standardized_Ref_ET_Eqn_Phoenix2000.pdf

The solar radiation used in the PM equations is computed internally using regressions of hourly Solar Radiation versus Temperature, Humidity, Zenith Angle, and Elevation in different regions of California. The hourly regressions for a year have R^2 of 0.92 and a standard error of 160 Langley’s/day for the hourly time step.  At the time I didn't know ASCE had a standard method for missing solar too, oh well, they're regressions. Update 5/1/2014--The solar computation has been refined for humidity above 80% and I like it much better.  

MOVRC1; -27407; ET Grass; Bishop, Ca Wunderground; 0; 37.384,-118.422,-120,4183,0.75; P4 this P4 is no longer used after Feb 8, 2022

Base1=0 is not used.

Shift1= 37.384,-118.422,-120,4183,0.75 is made up of 5 parameters separated with a comma that describe the site and are:

1)      Site Latitude=37.384

2)      Site Longitude=-118.422 and is negative for longitude west.

3)      Standard Time Meridian=-120 and is negative for longitude west.  Pacific Time=-120, Mountain Time=-105, Central Time=-90, Eastern Time=-75.

4)      Site Elevation in feet= 4183 feet.

5)      Windspeed adjustment=0.75.  Windspeed used in the PM computations are to be at the 2 meter height.  Standard weather stations make their Windspeed measurements at the 10 meter height so the adjustment for 10 to 2 meter is 0.75.  Wunderground weather stations at airports and Madis stations should use the 0.75 adjustment.  An airport has a Station_ID beginning with K and is 4 letters long.  Madis stations begin with the letter M….  All other stations should use the windspeed adjustment = 1.  The difference between using 1 or 0.75 will normally result in about a 5% difference in ET.

Formula1= P4.  The Parameter_Codes in the database table Rsite is entered in this order:  Use P1 if formula wanted like an adjust P1*1.2.

TemperatureF,Humidity,WindSpeedMPH, Clouds, ETshort

where 'clouds' is optional and is dsid= 26xxx from NWS 7-day forecast cloud-amount.

ETshort is used to compute a grass ETo, ETtall is used to compute an alfalfa ETo.

Example of Computing Evapotranspiration Reference ET with Solar Radiation:

If the weather station reports reliable solar radiation then the reference ET computation can be improved on low solar radiation days by using the reported solar radiation as follows:

MOVRC1; 27407; ET Grass; Bishop, Ca Wunderground; 0; 37.384,-118.422,-120

,4183,0.75;  P5 this P5 is no longer used after Feb 8, 2022 

…note positive DSID of 27407 to distinguish if from the computation at this site with out solar radiation above.

The Shift1 and Base1 are same as above without solar radiation example.

Formula1= P5 (different than P4).  Use P1 if formula wanted like an adjust P1*1.2.

 TemperatureF,Humidity,WindSpeedMPH,SolarRadiationWatts/m^2,ETshort

DSID's would be 17xxx,18xxx,28xxx,26xxx or 29xxx, ETshort

where 26xxx clould cover will compute solar better than blank.

If reported solar is missing or for a forecast period then solar is computed for the missing values. 

Example of Computing Solar Radiation (for Longwave Radiation see SnowMelt below):

 Solar radiation is computed internally using regressions of hourly Solar Radiation versus Temperature, Humidity, Zenith Angle, and Elevation in different regions of California.  The hourly regressions for a year have R^2 of 0.92 and a standard error of 160 Langley’s/day for the hourly time step.

The Solar coefficients can now be checked and entered by using the 'Wizards' button on the Select Station List. Just click on your station, if present, and then click 'Wizards'. You will see a complete breakout of all your coefficients for editing.

KCAHELEN4; -29408; Solar Radiation; Helendale, Ca Wunderground; 0; 34.775,-117.330,-120,2470;  P3 this P3 is no longer used after Feb 8, 2022

Base1=0 is not used.

Shift1=34.775,-117.330,-120,2470 is made up of 4 parameters separated with a comma that describe the site and are:

1)      Site Latitude=34.775.

2)      Site Longitude=-117.330 and is negative for longitude west.

3)      Standard Time Meridian=-120 and is negative for longitude west.  Pacific Time=-120, Mountain Time=-105, Central Time=-90, Eastern Time=-75.

4)      Site Elevation in feet= 2470 feet.

Formula1= P3 no longer used.  Use P1 if formula wanted like an adjust P1*1.2.

The Parameter_Codes in the database table Rsite is entered in this order:

TemperatureF,Humidity,Solar

DSID's would be 17xxx,18xxx,26xxx,Solar

where 26xxx clould cover is optional but will compute solar better than with out.

Also, for HDB computation for Solar, Long Radiation, and Evapotranspiration that require special internal GetRealtime subroutines here is an example GetReatime_setup.txt for them:

Look at the GetAccess table 'exampleRsite' for examples of setups for these computations from the database.  Note the formula Px is not needed unless your were transforming the output with P1.

Recommended Solar, Long Radiation, and Evapotranspiration rsite parameter_codes:

Solar>> 17xxx,18xxx,26xxx,Solar

Long>> 17xxx,18xxx,28xxx,26xxx,Solar long

ET>>    17xxx,18xxx,28xxx,26xxx or 29xxx, ET short or ET tall

The dsid 26xxx is cloud cover and comes from the NWS 7-day forecast of cloud-amount. No history is available so for history the old forecasted values are used.  26xxx and 29xxx can be omitted but not recommended, either 26 or 29 but not both.  Cloud-amount is not a valid parameter if you use a solar input and visa-versa.

Computation of Wind Direction:

Wind direction is retrieved as either text directions such as NNW or in degrees such as 270.  Both are converted internally to a value between 0 to 4 step 0.25 where 0 and 4 both equal direction North, 1=East, 2=South, 3=West, 0.25=NNE, 0.5=NE, 0.75=ENE etc.

Hourly averages and daily averages are computed from the retrieved vectors and weighted with the Wind Speed if available as:

Avg Wind Direction = ATAN2(sum cos*speed, sum sin*speed)

Computation from Database Values (version 2.2.2):

Update 8/20/2012: GetRealtime can get data from other databases in real-time along with your other real-time data. This is done with the station_ID 'COMPUTE-GET' . A file name is entered in the GetAccess parameter code that contains the other database connection string and SQL statement.

GetRealtime_setup.txt line:

COMPUTE-Get; 10035; Rainfall; Las Vegas Rainfall from other database; 0

GetAccess 'rsite' table line:

 35; 10035; Las Vegas Creek above Nowhere, NV; 10; rainfall; inches; inches; GetHourRainVegasCreek.txt; COMPUTE-GET; 1; NV; 0

A filename is entered in the 'rsite' parameter_code that contains the other database connection string and SQL statement. The 'COMPUTE-GET' control file GetHourRainVegasCreek.txt would like this:

 ---------GetHourRainVegasCreek.txt--------------

Las Vegas Creek rainfall from my other database

Put table: rhour

Get database_connection: Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\VBGET\GetRealtime\GetAccessHDB2.mdb;User Id=admin;Password=;

Get sql: SELECT date_time, value FROM rhour WHERE datatype_site_id=1024 AND date_time BETWEEN #DAY1# AND #DAY2# ORDER BY date_time;

END

Your Notes (in control file),

put table is the GetRealtime's GetAccess table rday, rhour, or runit.

database_connection must be a an ODBC database like MS Access or Excel.

Your sql string must return the date_time and value in that order. DAY1 and DAY2 are place holders that are replaced when run.

You can retrieve and compute new values at the same time but I wouldn't because you may want to use GetAccess and GetGraphs for display.

------------------------------------------------

Computations from database values retrieved and stored can be performed in real-time by using the Station ID = COMPUTE-Unit, COMPUTE-Hour, or COMPUTE-Day. For example, to average 3 Wunderground rainfall stations the GetRealtime_setup.txt line could read:

COMPUTE-Hour; 10035; Rainfall; Average Las Vegas Rainfall; 0; 0; (P1+P2+P3)/3

Remember to place the COMPUTE setup line BELOW the stations you will be computing values from.

Hour was used here because Wunderground stations rarely have the same time steps in the unit values table. The GetAccess Rsite table could look like this where the Station_ID = COMPUTE and the Parameter_Code would have the Datatype_Site_Id's to be the P1, P2, and P3 values in your computation, 10030,10031,10032.

If you would like to Lag a parameter so many minutes then the parameter in the table rsite could be 4044Lag60. For example computation of inflow-outflow with storage could be 1043,1044,4044,4044LAG60. And the formula in the station list could be P1-P2-(P3-P4)*12.  Update Feb 28, 2021:  You could use Lag to convert total rainfall to increments but now to convert total rainfall datatype_id=9 to incremental rainfall use datatype_site_id=10xxx with datatype_id=9.

Likewise, to compute runoff from rainfall use Station_ID= COMPUTE-Unit like this:

COMPUTE-Unit; 30755; Runoff; Prairie Creek, Dallas, Tx; 0; 2,2.5,0.15,0.2,9.03; P1

The database table RSITE would have the Parameter_Code= 10753 and Station_Id= COMPUTE.

GetRealtime 2.4.4 allows multiple computations and storage on the same DSID by appending 'Run-1-1-' to the front of the Station_ID, ie 'Run-2-5-Compute-unit' would be stored in the 'm' tables under run_ID=2 and trace=5. See Multiple Scenarios at the bottom of this page.

Program Use

GetGraphs.exe

(displaying the real-time data and web screens)

Setting the Database Connection String:

If GetGraphs.exe cannot make a connection to the Access database the following message will appear:

Notepad will automatically be loaded with the setup file GetGraphs_setup.txt.

Update Mar 14, 2014--For easier page insertions GetGraphs.exe 2.2.4 has removed the leading page numbers in the GetGraphs_setup.txt file as shown below:

The Database Connection String should look similar to this depending on the path where GetAccessHDB.mdb file was installed:

Provider=Microsoft.Jet.OLEDB.4.0;Data Source=

C:\Program Files (x86)\GetRealtime\GetAccess\GetAccessHDB.mdb;User Id=admin;Password=;

For Excel, the connection string would look like this:

Driver={Microsoft Excel Driver (*.xls)};DBQ=

C:\Program Files (x86)\GetRealtime\GetAccess\GetAccessHDB2.xls

Those using Excel as their database should open the GetAccessHDB2.xls file with Excel and read the 'ReadMe' worksheet to learn more about using Excel as a database.

After connecting and loading the real-time data and web screens, then left mouse click on any of the graphs to page through the 15 pages in the provided setup.  Right mouse click on a graph or web screen to bring up a menu for changing the appearance and setting properties of the graphs.

Most of the menu choices will be self evident. Watch Levels and Setup are described below.

The Watch Level example was used to set the Humidity, % Saturation levels that should they be exceeded in the past 24 hours the graph title will start blinking.

The setup menu is used to add and delete pages, graphs, and web screens.

The Datatype_Site_ID for a graph can be scrolled though for any of the data in the GetAccess database.  The Datatype_Site_ID should be left blank when adding a web screen.  A DSID of -1 will be added automatically if blank and a web screen address is assumed for the Station Name.

Changes made to the setup will not be made permanent until Save all to GetGraphs_setup.txt is selected or prompted when quitting.  To quit, simply close the current page screen.

The 'Setup File' button will open the GetGraphs_setup.txt file for direct editing. The 'Access' button will open GetAccess.exe for data edits when needed.  With GetGraphs version 4.1 notes or winter setup lines can be kept below the END statement.

Adding Sites to the GetGraphs Setup:

If  you have deleted and added the two new data sites to the GetAccess data base and GetRealtime Station List then you are ready to delete all the example graphs and web screens in the GetGraphs setup file GetGraphs_setup.txt.  Make a copy of the GetGraphs_setup.txt file for reference if needed later.

Use the right mouse click on a screen to bring up a menu and select Setup, then select Delete Current Page and click Ok.

Continue deleting all pages.

Then use the Setup menu Add to Current Page to add the two new sites that were added to the GetAccess database and GetRealtime Station List examples above.

Use the Setup menu Save all to GetGraphs_setup.txt to save the setup for the 2 new sites.

Now that you are familiar with the steps involved in adding new sites the 3 programs GetAcess, GetRealtime, and GetGraphs you may repeat the process with your sites of interest.

Update 3/8/2012 GetGraphs 2.2.1 can now add a second data series to a graph.

Add your second series dsid like this to your GetGraphs_setup.txt line:

   4 ;-30311,-1313 ;Runoff;Diamond Cr Can, AZ.....etc.

Note that the two dsid's are comma separated, not semicolon. For this example the 2nd series is -1313 and will plot in violet. You have to use Notepad to add it because GetGraphs cannot.  You can now add a 3rd series dsid but then you cannot also use RUN below.

Add M-table Trace: And remember you can use Run-FORECAST-NWS to save forecasts to the 'M' tables for post event analysis of where you went wrong and use GetGraph's menu 'Add Trace' to compare. To view the previous RUN on GetGraphs startup automatically, add ',RUN' to your GetGraphs_setup.txt line for that DSID:

 4 ;-11301 ,-10301 , Run ;Rainfall; Sub1 Adj & UnAdj & Last Forecast; 0 ; .....

This means plot the last run for yesterday from Mtable values for dsid= -11301.

 4 ;-11301 ,-10301 , Run-3 ;Rainfall; Sub1 Adj & UnAdj & Last Forecast; 0 ; .....

This means plot the trace 3 runs ago from Mtable values for dsid= -11301.

Update 6/22/2015: Use the UNITS_OFF on the GetGraphs_setup.txt page line to turn plot HOURLY values on that page. This only matters if "Plot Unit Values= True". This is handy for plotting hourly rainfall and unit value flows. The %MAE'w will then be able to be computed with irratic rain gage time steps.

And if you don't want to FTP a particular page use FTP_OFF on the GetGraphs_setup.txt page line.

Update 7/4/2015: To display flooding color bands you can use a '4' in the frequency GetGraphs_setup.txt column followed by minor, moderate, and major flood stages:

1 ;-2329,2609 ;Stage;Village Cr at 24th St; 0 ; 2 ; 0 ; 9352151 ; 15724527 ; 65280 ; 255 ;12.01;;0; 0 ; 4 ; 8 ; 10 ; 12

In this case I have included the high 'Watch Level' of 12.01', set the frequency column to 4 with 8', 10', and 12' as minor, moderate, and major flood stages. If the Watch Level had been set to 7 below the Minor Level then an 'Action' yellow color band would be displayed below the Minor Level.

Working with Web Screens:

Web Screens are added using the Setup Menu shown above.  The Datatype_Site_ID and Parameter Name should be left blank and the Web Screen URL address entered in the Station Name text box. 

Web Screen Example:

http://waterdata.usgs.gov/nwis/rt

Enter the above example URL into your web browser and this screen is will appear:

The above URL or the same URL taken from the web browser address box could be used in the GetGraphs Setup Menu Station Name text box.

If just the US map picture is wanted, then the scroll bars in the GetGraphs display can be used to center the US map…. Or a better method is to get the URL of just the US Map gif by right mouse clicking the US map in your web browser.  A menu will appear and the Properties can be selected to display the URL Address of just the US map gif:

The gif URL is: http://waterdata.usgs.gov/nwisweb/icons/waterwatch/images/real/us/real.gif

Web Screens can also be files on disk such as pictures or text where the URL is the filename such as C:\mydata\myfile.jpg or C:\mydata\myfile.txt.  Update 12/30/2014: You can use the setup checkbox 'Resize' to fit images to the size of each webbrowser.

When working with web screens it is best NOT to turn on the Setup check box "Allow navigation for this site" if not needed. If it is on then you will need to use the right mouse click menu to turn the page of your web screen. You may also note some web content reacts differently to mouse clicks. When using the right mouse click be sure to take note if it appears. It is sometimes hard to notice in full screen mode.

If you want your web screens to be refreshed at regular intervals like ever 5 or 10 minutes you must have GetRealtime.exe running as a scheduled task downloading at least 1 data site such as a Wunderground temperature. You do not need to include the real-time data graph. Also, Auto Paging must be turned on.

Update 5/6/09—Getrealtime.exe version 1.0.3 has been updated to compute real-time rainfall-runoff from the Wunderground or the other supported rainfall real-time sources.  Below is an example of editing the HDB database table Rsite using GetAccess.exe.  The datatype_id for runoff is 30.

Wunderground rainfall gages may have erratic time steps (especially airports) and so caution should be used for time-steps less than 1 hour.

Quick Overview:  Here are the GetRealtime setup steps for one NEXRAD rainfall boundary (More info on GetNexradHelp):

1) Use GetMapArea.exe to digitize your BoundaryFile as N0Q image pixels (or use LatLongPixelsFromFile if you already have a Lat/Long boundary file.  Think about always digitizing in lat/long instead of pixels and using LatLongPixelsFromFile to generate the pixel file.  Lat/longs can provide boundary KML visualization on Google Earth.  Instead you might try  GoogleKML2Text.exe  to read Google Earth Pro created polygon KML files and convert to text boundary lat-longs for use by LatLongPixelsFromFile.exe but you can still use GetMapArea to load Google Earth with the EPA Waters KMZ units and catchments and use Google Earth's 'Add', 'Polygon' and save as KML.  With Google Earth Pro, if you click on the EPA Waters boundaries, you can save them to KML with out actually digitizing anything... whoo hoo!

NexradBoundaryESX10001Q.txt  (pixel's x,y)

2) Use GetNexrad.exe to convert the BoundaryFile pixels to a PointFile. Inspect the Point file (fill in the missing 1's if needed) and copy both files to your GetRealtime.exe directory.

NEXRADPOINTESX10001Q.txt   (0's and 1's)

3) Add your basin's line to the GetRealtime_setup.txt using GetRealtime.exe.

NEXRAD-ESX; 10001; Rainfall; Village Creek above Nowhere, NV; 0

4) Add your basin's line to the GetAccess table 'rsite' using GetAccess.exe.

1; 10001; Village Creek above Nowhere, NV; 10; rainfall; inches; inches; N0Q-Ridge2; NEXRAD-ESX; 1; NV; 0

5) Run GetRealtime.exe and download the past 2 hours of N0Q radar images for the ESX radar.

Viola! You will see the 2 hours of 5-minute rainfall in your GetAccess database.  The Boundary Setup in more detail is described below here. For areas of sparse radar coverage, blockage, or winter overshoot you can try Iowa Mesonet's composite N0Q of all nearby radars which I term NCQ. You will have to convert your N0Q boundary's to NCQ using LatLongPixelsFromFile.exe or start from scratch. NCQ tends to return much higher reflectivity values than N0Q. Just use NCQ instead of N0Q in the examples above.

THE COOKBOOK:  Concise Steps to a GetRealtime Radar Rainfall Runoff Setup.

GetRealtime.exe uses the SCS Triangular Unit Hydrograph or Linear Reservoir Runoff Hydrograph or Clark synthedic or Dimensionless Unit Hydrographs from USBR Design of Small Dams. Rainfall loss methods are initial loss (or SCS Curve Number CN), constant loss, interception, and percent of basin impervious w/fraction connected. With the Triangle Unit Hydrograph you can custom fit the peak, interflow, and recession by adding 2 additional triangular unit graphs just by setting what fraction of the runoff belongs in each and what recession ratio of time to peak for each.  The Getrealtime.exe setup file is edited using GetReatime.exe as shown below and as follows

Update 9/17/2012--The runoff coefficients can now be checked and entered by using the 'Wizards' button on the Select Station List. Just click on your station, if present, and then click 'Wizards'. You will see a complete breakout of all your coefficients for editing.

Base1=0 optional base flow to add (new 6/22/2012)

Shift 1 contains 5 parameters separated by commas, in this case 1, 0.25, 0.13, 5, 99.3.

These values are as follows:

1.0 is the basin Lag time in hours.

0.25 is the Initial Loss in inches. (or 10-100 SCS Curve Number CN)

0.13 is the Constant Loss in inches/hour.

5 is the Percent of Basin Impervious.

99.3 is the Basin Area in square miles.  

(Optionally, the SCS trianguler unit graph recession ratio may be added and defaults to 1.67 if not included.)

(Also, the rate factor at which the initial loss or CN recovers may be added and defaults to 0.2 if not included.)

Formula1 contains the resulting runoff as P1.

If there is no rainfall the Initial Loss will begin being reset at the rate of 0.2*Constant Loss. In this case the Initial Loss will have returned to it's initial value of 0.25 after 9.6 hours.  Note: This simple idea has been expanded and improved with new updates below.

If there is rainfall the computed and stored runoff will be carried out to the peak discharge. GetGraphs.exe can then display the predicted future runoff hydrograph up to the peak flow that the rainfall up to the last retrieval has produced.  If the computed peak flow will be occurring on a future day like tomorrow because of a large lag time, then that future peak value will be shown as the midnight value of the current day. This way GetGraphs.exe will be able to at least plot what the future peak will be.  Update: GetGraphs allows setting 'Days' and 'Future Days' so you can view more of the future hydrogaph on larger basins.

For basins having significant indirect runoff and need better definition of the recession then you may optionally add 2 additional coefficients to describe the recession in addition to Tp/Tr ratio (1.67). Here is an example of a GetRealtime setup data line for an extended recession.

NEXRAD-DAX-hour; 30425; Runoff; Big Cr nr Groveland, Ca W/adj Rain; 60; 8,0.55,0.10,1.0,16.3,1.67,0.2,0.3,3; P1

Where:

-hour=optional rainfall time-step hour or unit, defaults to unit if omitted (new 6/28/2012)

Base1=60 and optional base flow to add (new 6/22/2012)

8=basin lag time, hours

0.55=initial abstraction, inches

0.10=constant loss rate, inches/hour

1.0=percent impervious, %

16.3=basin area, sq.miles

------optional Tr/Tp and recession values-----

1.67=Tr/Tp

0.2=initial loss recovery rate factor. Recovery rate will be 0.2 * constant loss rate

0.3=fraction of rain excess applied to recession and removed from peak

3=factor used to multiply Tr by

------- 2nd Optional Seepage factors for a base flow (NOT SHOWN)-----

0.2=fraction of rain excess applied to recession and removed from peak

10=factor used to multiply Tr by

As shown in the figure below, version 2.0.1 now has the optional recession Tp located at the same time of the peak triangle. I made this method up and I think it works great... besides I could not figure out what a gamma function is.  The optional recession can allow the recession to recede for days if needed. I don't know how true this is about interflow and baseflow. It's probably more like a quick and easy way to combine storage routing with the runoff unit graph computation so be aware if flow goes into the flood plain you may want to include a Modpul routing for that.

You may wish to use the free GetMapArea (More Stuff) to quickly evaluate the effects of  lag, losses, and percent impervious, especially if  your rainfall gage has a recording streamflow gage for calibration of the rainfall record.  With the new GetRealtime runoff setup wizard, it is even more easier view your parameters, make changes, then click and go to graphically compare effects on the hydrograph.  Calibrating actual runoff is most educational ... just find a USGS streamflow gage and use Wunderground's Wundermap to see if a rain gage is available for honing your runoff skills using GetRealtime or GetMapArea.

Video how to for determining triangular unit graph coefficients:

Simulate SCS Curve Number Loss and add coefficients to Setup file and Database :

Update 1/2/2013--A Linear Reservoir Runoff Hydrograph method has been added. So in addition to the 3-Triangle Unit Graph method, this LRH method allows shaping of the hydrograph also. The Triangle Unit Graph and this Linear Reservoir are the only 2 unit graphs that you can dramatically shape your runoff hydrograph without  changing peak lag time (or by selecting different DGF's below but just a bit).

Linear Reservoir Hydrograph Equation:

Q2 = Q1 exp { −K (T2 − T1) }  +   RainExcess  [ 1 − exp { −K (T2 − T1) } ]

where the K factor can be adjusted to shape your basin's response. This is not quite as versatile as my 3-T unit graph method but it requires no convolution of each excess increment and is quite simple to implement. The current time step runoff is based on the last runoff flow and rainfall excess and should allow erratic time steps even.  So you could easily do runoff in an Excel worksheet sans convolution.  From the equation above you can see it is two competing exponential curves working against each other. One is concave up and one is concave down. Who thinks up this stuff???

I have included a lag time offset so that you can still get the peak at the right time using your usual lag time method of choice but the K shape factor, not the lag time, determines the peak runoff and recession shape.  The shaping K-factor to mimic the SCS dimensionless unit-graph can be calculated as K = 1.129 * lag ^ -1.10391 and lag time offset factor as lagAdj = 0.592 + 0.0281 * lag. You can use GetMapArea.exe to compare it to many unit-graph types and be impressed at how well it does for shorter lag times. And because there is no convolution into the future, this LRH method can pick up right where is left off without having to start the simulation earlier if runoff is still occurring.  Boy, where was I when they passed out this method!

Use the Runoff Wizard for the setup. The Lag and K factor can also be 'What If'ed with the Wizard 'What If' button to see how it shapes the hydrograph... but I'm sticking to the 3-T unit graph myself.  The Tri-Linear Reservoir will also use the triangular unit graphs adjustment factors for combining 3 linear reservoirs.

Update 7/5/2013--Several Dimensionless Unit Hydrographs from the USBR Design of Small Dams or their Flood Hydrology Manual have been added. For larger basins you may find you will need to construct your own dimensionless unit graphs to fit odd complex runoff patterns that even triangular unit graphs cannot handle. A dimensionless unit graph can easily be constructed from a USGS hydrograph. You just take the hydrograph as is or some 1 hourly or 2 hourly interpolation to get around 100 values, remove the base flow, and multiply the hydrograph by 484.36 divided by the sum of the hydrograph values in cfs.  The final DGF ordiantes COUNT at 50% volume cumulation 242.18 for the dgf unit duration as 100/count... not time. See my DGF files for the format. Actually if you did a DGF for any basin with a USGS gage, you wouldn't need my 3 triangle unit graphs. And remember that GetRealtime can compute the base flow to remove in it's wizard's runoff output text file. Everyone should try making their own DGF just to say they have.  And basins with lag times (peak rain to peak flow) greater than 33 hours (5.5*6=33) are getting pretty big and you should think about subdividing.  Forget the 33 hours lag limit if your basin has different interior regions of  odd contributing shapes and losses like urban areas then 6 hours is pretty risky so use your head.

Update 10/7/2014--Clark Unit Graph Method:

The Clark time-area histogram uses the synthetic time-area curve from COE's Hec-1 for an ellipse:

t < 0.5: area = 1.414 * t ^ 1.5

t>= 0.5: area = 1 - 1.414 * (1 - t) ^ 1.5

The Clark time of concentration Tc is your Lag time in hours. Your Clark routing factor C (default 0.75) is used to provide a Muskingum routing travel time K as K=C*Tc. The Clark Unit Graph has Tc/Unit_Duration steps of t=0 to 1 above.

Excess rain is convoluted with the unit graph and then the current runoff value is routed with the Muskingum method using K with X=0. This Clark guy was pretty damn clever. All you provide is 2 inputs, Lag time and C, thus providing your hydrograph timing of the peak and shape of the recession. What a sweet deal!

I thought this was such a sweet deal I thought I could affect hopefully the peak shape or at least the recession beyond just lagging. So I tried maintaining a time-area histogram sum of 1 but varying the 1.5 power above from 0.5 to 2 with power=1 a rectangular basin and 0.5 a figure 8. Not a lot of difference. So then I tried taking the power=1.5 unit graph and resorting it for peak at 1/3 and 2/3 points. Not much difference again. So then I tried a 1.5 power from 0 all the way to 1 with not much difference, so then 1 to zero. Uh oh, that was a mistake because half the runoff vanished. So in conclusion, if you want to reshape a unit graph either use 3 parallel linear reservoirs on smaller basins, 3 triangles on larger, or create you own dimensionless graph because there are no free lunches at Clark's diner. And always start with the SCS dimensionless graph as a start before wasting any more time with foolish ideas. Ok, maybe Clark with only somewhat different recessions with same peak is somewhat helpful. A really smart fellow would subdivide and route but that is blasphemous.  Update 11-7-2014, not so fast. I just put some basins together in the limestone area of northern Alabama with very low dry curve numbers of 40. The runoff has a lot of interflow and recession with smaller peaking. The Clark method came through perfectly but with a routing factor of 6 times the lag time. Hey, that Clark guy really was clever!!!

Update 7/15/2012 V2.3.0 now supports SCS Curve Numbers CN 10-100 with CN recovery when it is not raining.  My new CN recovery method makes possible the use of this popular loss method and because it recovers during dry periods the CN method can now be used in GetRealtime continuous rainfall-runoff simulations.  Seems to work even better than the 'initial loss' method and easier to calibrate (the TR-55 urban manual is a must read also).  Although the CN loss method is an empirical relation between total rainfall and total runoff developed for pretty restrictive conditions, we will simply MAKE IT WORK by using additional loss factors for infiltration into S and return factors for recession flows for S recovery in equation 1 below.  S is treated as an actual physical property as inches of water stored in the soil. Both a soil water return and a groundwater reservoir return can be used.

The CN runoff  *EVENT*  equations for CN 10 to 100:

For continuous simulation when raining, P is increased each time step rainfall amount until S saturation value reached (rare). S is increased by the infiltration amount until S saturation reached .  Infiltration is the time step  rainfall minus computed runoff.  Everything is in inches.

For continuous simulation when it is not raining, P is decreased each time step by the recovery rate based on soil factor and full ET. S recovers by the recovery rate but ET is reduced by S to soil storage range ratio so that saturated soil uses full ET and dry soil has zero ET.

Update 3/18/2015:  You can vary the Initial Abstraction Ia to adjust early runoff. Nothing more than using a different factor in the runoff equation is done, for example Ia=0.05: Q=(P-0.05*S)^2/(P-0.95*S)  In this case, 0.05 initial abstraction is being used instead of original SCS 0.2 with nothing changed about S. These differences are quickly lost after the initial abstraction period, unlike using equivalent CN's that really seem change things.

To simulate vegetation and canopy interception, infiltration (rain-excess; see below) does not begin until P>0.05*S for months April-November and 0.01*S for the other winter months. This helps on very low curve numbers used in forests. You may have to see it to believe it but for CNdry=40, Sdry=18", then 0.9" gets stuck up in the trees, moss, and leaf litter. Another 2.7" infiltrates before runoff begins. Only the 2.7" and above needs recovery due to it's much slower recovery rate.

Update 3/29/2013: An 'Interception Factor' for the growing months has been added so that the 0.05 factor used above can now be set by the user and defaults to 0.03 if not given and 1/3 the factor Dec-Mar (see Runoff Wizard). This interception is included in the CN initial 0.2*S abstraction. This interception can also be used with IL initial loss method but would effect the initial loss so you might want to set the factor to zero.  Here is my guess at  'Infiltration Factors' you might try: 0.05=Forested, 0.03=Grasslands, 0.01=Urban... or maybe they are all 0.05 due to the varying CN used???

And it would be nice for someone to come up with a Recovery Rate versus Curve Number but alas, that will have to be you.  I'm speculating here but this might help estimate soil recovery Factors.  Assume a 0.1 in/hr constant loss rate to go with these recovery factors to get you started.

Conductivity values suggested by Rawls, Brakensiek and Miller (1983)

*Note: See tip #1 below and also K recession index method below.

Loam has a conductivity of 0.134 in/hr, which sounds good for a constant loss rate. Conductivity seems to imply that the recovery factor should be 1.0 to match the saturated constant loss rate which I don't think will work. I have been looking at recovery rate factors of 0.05 to 0.5 times the constant loss rate, but who knows. I thought somebody had figured this all out back in Horton's day. Ah ha! The soil can't be saturated if it is recovering! Geez, sometimes I amaze myself. ... but still begs a solution to the recovery factors.  As long as I am this far out on my limb, I propose the recovery factor to be related to suction head h times conductivity C, for loam is 3.5*0.134 or 0.469. I wanted the recovery rate to be 0.05 so my fudge factor is 0.1 as shown in the above table.  Case solved. I got your science right here! Oye Vey...

Some of my calibrated CN values and recoveries you might try for ungaged basins:

Note: The Arizona basins were so huge that I would not rely on their values.

Update 7/21/2012: Giving this some more thought, the soil initial loss recovery is probably not so much a drainage problem but an Evapotranspiration problem. I still think the above recovery factors are ok for the constant loss (ETo=0.05 to 0.3 in/day versus conductivity 0.1 in/hr ???) shown but the recovery should be patterned as hourly ET. I looked at some summer and winter hourly ETo's I have been computing for a Wunderground Gage near Colby, KS. My ET's were standard ASCE Penman-Monteith reference ET for Tall alfalfa. What is important is the 24 hour distribution of ET. I selected July 16, 2012 for the hourly pattern, about typical. GetRealtime and GetMapArea now calculates the unit time-step recovery based on this distribution. Update, you can now use a nearby ETo record as shown below and is recommended.

Update 7/22/2012: After sleeping on it, I decided the best solution to the initial loss recovery between events was to go back to the constant recovery rate but add to this the mean daily Eto for Colby, KS with the July 16 hourly pattern below.  Here are the Eto hourly factors and the daily mean ETo calculation:

 ETo=0.17+0.12*SIN(0.017214192 * (284 + Date))

7/1/2011 to 6/30/2012 Kansas 1 year total ET: 60 inches.

Hourly Eto Factors (0-23 hr):

"0.010397326,0.005198663,0.003713331,0.002970665,0,0.001485332,0.00037133,0

,0.00965466,0.027478648,0.048644634,0.070181953,0.088377274,0.10731526,0.11548459

,0.122539918,0.107686595,0.093575938,0.083921277,0.059784627,0.030449313,0.008169328

,0.001113999,0.000742666"

Monthly moFactors applied to your KcFactor (see Wizard below): "0.50,0.50,0.63,0.74,0.79,0.84,1.00,1.00,0.84,0.79,0.74,0.50"

To review, there is no need for the user to input any information about ET. Hourly ET will be computed based on the above factors. If the user wishes to improve calibrations, then the first step might be to adjust the 0.8 KcFactor. If further ET refinement is needed then an ETo record can be calculated at a nearby Wundergage. Contrary to what I would have first thought, it probably wont be the wetter areas that need ET refinement but the desert regions where the Kc crop factor for desert plants can take what they get.  And do not forget that the soil recovery ET is again factored by current basin soil storage to simulate stressed conditions.

Update 7/24/2012 I think I got it:

BIG tip of the hat to "A Parsimonious Watershed Model" !!!

(After looking it up, I was into parsimonious when parsimonious wasn't cool.)

The idea here is to keep track of Storage for computing precip loss and runoff as:

Storage = Storage0 + Infiltration - ET - GroundwaterOutflow    (what a novel concept, sheesh!)

Where:

Infiltration = Rainfall - Runoff

and

GWoutflow=ConstantLoss * RecoveryFactor  parameter codes

and

ET and GWoutflow are scaled by:

Factor=(StorageDry - Storage)/(StorageDry - StorageWet)

...but the top soil (RainSum or P above) recovers at full ET and full GWoutflow.

GWoutflow (soil water) contributing to runoff can be reduced by Soilwater_Loss_Factor.

Storage for the SCS CN method is CN Storage and for the Initial Loss Method is Initial Loss. This storage is limited to wet and dry condition values you enter or my automated values:

For CN method, if you do not enter the parameters CNwet and CNdry then:

CNwet=CN/(0.4036+0.0059*CN)

CNdry=CN/(2.334+0.01334*CN)

For Initial Loss method, If you do not enter the parameters ILwet and ILdry then:

ILdry=InitialLoss

ILwet=0 

Add the last parameter code on the end as -1 to create the file 'Runoff.txt' and try graphing the output info to see what is going on. GetMapArea and GetRealtime's runoff wizard have both been updated to easily show graphically the effects of parameter changes.

GetRealtime_setup.txt shift cell parameters examples:

Initial Loss Method:

6.5,1.1,0.1,2.4,90.8,1.0,0.2,0.4,3,0.3,6,-1 <<<auto Loss limits and -1 means create info file 'Runoff.txt' 6.5,1.1,0.1,2.4,90.8,1.0,0.2,0.4,3,0.3,6,1.4,0.5,0.5,-1 <<<has initial loss limits and SoilLossFactor

SCS Curve Number Method:

6.5,70,0.1,2.4,90.8,1.0,0.015,0.4,3,0.3,6,-1 <<<auto CN limits

6.5,70,0.1,2.4,90.8,1.0,0.015,0.4,3,0.3,6,83,60,0.5,-1 <<<has CN limits and SoilLossFactor

Do not forget to remove the -1 to create file 'Runoff.txt' parameter when you do not need it and use the Runoff Wizard for the full monty.

Example of optional runoff coefficients for best results:

Update 1/6/13: You may be wondering where the 'Parsimonious' went with all these options and I do too... At least remember this>>> See the 'Compare DSID' box on the runoff  'What If' screen above. Here is where you can compare your calibration run to the USGS runoff graphically. You enter the USGS DSID here and Hit Cancel if not 'What IF'ing and it will add the hourly USGS graph to your run's graph. This is something not to forget and always remember.

The parsimonious model cited here simulates 2 storages, the unsaturated zone soil moisture using the CN method, and the saturated zone using a constant outflow parameter (but no direct return from the soil). I have combined the 2 into one storage where groundwater outflow is scaled with the amount of CN storage. When storage hits CNdry, there is no groundwater outflow (even less... uh more parsimonious!). Perhaps at a later date I may see the error in my ways and use their 2 storage method instead but for now I like what I see. 

Update 11/23/2012: I have seen the error of my ways and added a groundwater table storage component using the linear reservoir method (see runoff unit graphs above). Although the single soil storage provided by the SCS curve number or Initial Loss method works very well for wetter climates, it was found desirable to add a groundwater table out west and possibly for droughts. Setting the new Groundwater Loss Factor to 1.0 for wetter climates will provide the same results as before. The new groundwater table storage receives channel recharge as 20% (hardly noticeable) of runoff OR as saturated soil when infiltration is above the max of the soil storage range OR as unreturned soil infiltration Soil Loss Factor * Infiltration * soil storage range  Factor above.  The recovery rate and groundwater outflow K-factor is Constant_Loss * Recovery_Factor for the soil above as in/hr.  The daily starting Groundwater Table outflow uses the datatype_ID of 5 and no GetAccess setup is needed unless you wish to view them.  Why groundwater table recharge during runoff and not just above maximum soil storage??? For arid lands, about the only time there is water table recharge is when there is flow in channels, not because of saturated soils (capillary action is always back up for these blood sucking cactus and they don't call it caliche for nothing). And what if you over estimated max soil CN storage, well I just fixed it for you, parsimoniously.  Unreturned soil infiltration in most cases will be far the largest inflow component to the ground water table but you can juggle all these factors to fit and by no means is this a mass balance, but after doing a few basins you may start to see some rhyme and reason to these factors. Let me know if you do. ;-)   So far I have found that the GW Table returns are used for the faster runoff recession and the Unsaturated Soil returns are used for the longer duration base flow recession but I guess you could reverse this (?).  In a snowmelt area with bogs in New York I reversed this and use a very slow 0.05 Ground Water Recovery Adj.  The GW Table returns can also be lagged by setting the GW Lag value in hours for a Muskingum routing of the linear reservoir return with a Muskingum X=0.  The Muskingum routing overcomes the linear reservoirs flashy filling that seems just unavoidable but very acceptable for most fast interflow needs for those not making their own dimensionless unit graph. Just another tool that probably no one needs.

A Groundwater Riparian Factor can be used for seasonal streamloss or even gain by applying this factor to a sine wave (+1 to -1 on Aug 1, 0 on Nov 1) times the unsaturated zone soil (not groundwater table but includes Base Flow if any) returns: Qsoiladj= Qsoil + GRF * Qsoil * Sine(date). Why a sine wave and not adjust based on ET???  Solar radiation is a sine wave peaking on June 21, crop use factors lag it into August, and the Riparian Factor fudges the streamside area and Kc so it is ET.  Update 9/19/2013 now includes the Base Flow in the riparian adjustment.

GetRealtime's flow accountings:

Although "A Parsimonious Watershed Model" cited used a daily time step, it was my intent at first that GetRealtime be used only for real-time 5-minute Nexrad Radar rainfall events of a day to a week in duration. But in actually doing some calibration over a two month period, GetRealtime also would work just as well for a year long or greater period at 5-minute, hourly or daily time steps with much more sophisticated snow melt and ET than proposed by the model cited with out introducing any additional calibration parameters... just wouldn't be as cool as real-time though. I had not planned on it, but the cited study authors make such a good case that I foresee that I will be adding examples for longer duration studies also just to see if it really works so check back often.

To check for issues for long simulations I ran an HOURLY runoff computation for 100 years on my Windows XP Home Edition with 1.28 MB memory. It takes 7 minutes to complete the computations with 328 MB of peak memory use. I also ran a 5-MINUTE runoff computation for 30 years and it took 8 minutes for the computation and another 1 minute to write the hourly values and 5 minutes for unit values to the GetAccess database with 902 MB peak memory.  I tried 200 years of hourly history but something limits the start to 1900 so I tried 200 years into the future and it ran fine with 666 MB of peak memory. So I tried 300 years into the future and used 938 MB ok. 400 years???... finally "Out of string space!"... out there somewhere... beyond the beyond... something is controlling transmission... there is nothing wrong with your computer... you have reached... The Outer Limits!!!  (Actually I did run 500 years but used 1400 MB of virtual memory. Don't get your hopes up, 32-bits can only use 2000 mb.)  I did not know that GetRealtime would run into the future but it does.  It could project future streamflow recession assuming there is no rainfall.  Hmmm... somebody clever enough should look into how to read the NWS's QPF rain forecasts and create a future hourly rain record. I know they do it somehow.  (I have, see below.)

If anyone is actually running 100 year simulations, something to consider for speed is that during EXCESS PERIODS a regular dimensionless UHG will be 3 times faster than a three triangular UHG and a linear reservoir UHG will be 100 times faster than a dimensionless graph due to convolutions... in theory.

After giving my all to get this working, I just learned HEC-1 was updated in 1998 to handle continuous simulations as HEC-HMS... dang!!! Just when I thought I had invented a new mouse trap. Where have I been?!?  But if it's any consolation to your confidence in me, I also just learned while reading up on HEC-HMS, that Leo Beard wrote the original HEC-1 Fortran programs.  Hey Leo!  I worked with and learned from Leo in the mid 80's on flood frequency's but he never let on he wrote HEC-1, although he had Fortran code for everything else to share.  But I see the HEC-HMS Soil Moisture Accounting uses the infamous Green-Ampt; not very parsimonious of them. So give my methods a try and compare. You're going down Leo. ;-)  Don't worry.

Example of a 5-minute calibration for a 32 day period for an urban area with real-time radar adjustment:

Example of a groundwater table with the Base Flow Riparian Factor used to increase streamflow.

Hydrograph Recession Slope (specific yield as RecoveryFactor):

Did you know that you can estimate the SOIL groundwater recharge rate (ConstantLoss*RecoveryFactor) by the hydrograph recession slope and vise a versa, well now you do.  Where this could come in handy is where you know all the streams in your region have a similar base flow recession hydrograph slope measured as days per log cycle on semi-log paper. If you think you know the slope then, knowing the soil type you can get the Carson's Recovery Factor for 0.1 in/hr Constant Loss from the table above. The table below will then give you the CNdry... or any combo of the 3.

For example, your recession slope index is 18 days, knowing the soil type is Silt-Loam B will give a Recovery Factor of 0.17 for Const Loss 0.1 in/hr which is 0.017 CL*RF then from the table below the CNdry=65. If you have more faith in your CNdry than your soil map, then say you think CN_dry is 65 then, your CL*RF is 0.02/constLosss or Recovery Factor=0.02/0.1=0.2.

For the Linear Reservoir method try K factor = −ln (Q2/Q1)

Update 7/25/2013: GetRealtime 3.1.2 includes a variable Lag time that made a very significant improvement in some peak flow comparisons for both bias and %MAE. My variable lag is computed as Lag = Lag0 - PrecipSum / SoilSpace * Lag0 / 2.  For the Initial Loss method Lag = Lag0 - (PrecipSum / 3*InitialLoss) * Lag0 / 2.  This reduces the Lag0 entered for dry conditions to 1/2 Lag0 when saturated. The lag time and all unit graphs are updated on startup soilspace or precipsum and when the computed Lag changes by 5 minutes. So as rainfall increases, soil space is reduced, and the Lag time is reduced. Under dry conditions soilspace increase and Lag time increases. Pretty clever eh!

I highly recommend the variable lag so use it when you first start calibrating your basins. I never have, so when a big peak comes along, I end up checking variable lag and just hope it fixes the lagging peak, so include it when you start out calibrating even smaller peaks. I usually end up increasing my oringal smaller peaks lag time by 20% with variable lag once a monster peak comes along and I have to use it so be smart and be ready already.

Update 8/3/2012 use  your ET record for any Wundergage with Temp, Humidity, and Wind:

To use your own computed ETo from a nearby Wunderground gage simply include the ET datatype_site_id in the table 'rsite' parameter code like 10812,-27811 on your runoff record for either station_id COMPUTE or NEXRAD-ESX as shown below. See the Getrealtime computation examples for setting up your Wunderground ET computation.  Any missing ETo values in your record will use the default values above.

Update 10/13/2013--Added the 'Directly Connected Impervious Fraction' as cascading planes.  This allows the fraction of the % Impervious not directly connected to be removed from the excess and added to infiltration but limited by the Curve Number loss method. A tip of the hat to Ben Urbonas and his "Stormwater Runoff Modeling; Is it as Accurate as We Think?"

From Colorado Urban Drainage and Flood Control District for urban catchments.  The Levels in the graph are for the amount of Low Impact Development added between the impervious areas and collecting channels.  For 38% impervious shows 80% directly connected for minimal low impact development (Level 0).  This was a very important addition and badly needed for urban areas with large %Impervious.

Level 0: No improvements, Beaver Cleaver's neighborhood.

Level 1: The primary intent is to direct runoff from impervious surfaces to flow over grass-covered areas and/or permeable pavement, and to provide sufficient travel time to facilitate the removal of suspended solids before runoff leaves the site, enters a curb and gutter system, or enters another stormwater collection system. Thus, at Level 1, to the extent practical, impervious surfaces are designed to drain over grass buffer strips or other pervious surfaces before reaching a stormwater conveyance system. Houses like mine with clogged gutters.

Level 2: As an enhancement to Level 1, Level 2 replaces solid street curb and gutter systems with no curb or slotted curbing, low-velocity grass-lined swales and pervious street shoulders, including pervious rock-lined swales. Conveyance systems and storm sewer inlets will still be needed to collect runoff at downstream intersections and crossings where stormwater flow rates exceed the capacity of the swales. Small culverts will be needed at street crossings and at individual driveways until inlets are provided to convey the flow to the storm sewer. The primary difference between Levels 1 and 2 is that for Level 2, a pervious conveyance system (i.e., swales) is provided rather than storm sewer. Disconnection of roof drains and other lot-level impervious areas is essentially the same for both Levels 1 and 2.  Town of Bedrock, Fred Flintstone's neighborhood.

Suggested adjusting for Composite CN with unconnected impervious area (TR-55):

%Impervious>30: CNc= CNp+(Pimp/100)*(98-CNp)  not recommended.

%Impervious<30: CNc= CNp+(Pimp/100)*(98-CNp)*(1-0.5R)

where

CNc = composite runoff curve number

CNp = pervious runoff curve number

Pimp = percent imperviousness

R = ratio of unconnected impervious area to total impervious area.

Example:

1/4 acre residential Table 2.2a for Soil B gives 38% impervious and CN=75.

Connected Impervious Area from the Colorado figure above for Level 0 low impact development CF=0.80 for directly connected.

Composite CNc= CNp+(Pimp/100)*(98-CNp)*(1-0.5R) = 75+(38/100)*(98-75)*(1-0.5*0.2) CNc= 82.9

(or CNc=83.7 for %Impervious > 30% but throws out all of Colorado's work.)

Update 12/18/2013: Runoff computations done with datatype_id=30 when run by the number of past days now checks for a recession runoff date on or after the start of computations date and if it is then the starting day is moved to the day of the starting runoff plus 1 day. This insures you are not cutting off a previous recession. The start and end of runoff are now stored in the table 'rupdate'. The earliest starting day for the run is then used for all subsequent routings including the HEC-Ras and HEC-Hms shell routings. Runoff computed by historical dates do not store anything in the table 'rupdate' and do not self adjust the starting date.  If you wish to override or clear the recession dates from the GetAccess table 'rupdate' just use the SQL checkbox and statement 'DELETE * FROM rupdate;'.

As I gain experience re-running past studies I will update any tips I might discover and perhaps add some comparative graphs somewhere. With only the 3 parameters CNdry, ConstantLoss, and RecoveryFactor to play around with it's really not too bad, specially getting started by graphing the daily mean flows for calibration. I have already seen for the CN method that ConstantLoss and the Recovery Factor only move the hydrograph up and down so they are just some what of a base flow gizmo and that only leaves CN. WHAT COULD BE EASIER!!! (Let's see it work first, happy jack.)

Here are some things to remember about the continuous CN method:

0) At startup, S, P, and gwQ are read from HDB.

   datatype_ID 4 = Soil Storage Space S at end of day.

   datatype_ID -4 = Pricip residual P at end of day.

   datatype_ID 5 = Groundwater flow rate gwQ at end of day.

   S is soil SPACE between say CNdry=10 inches and CNwet=1 inch.

   Srange= 10 inch - 1 inch= 9 inch used for soil recovery adjust speed S/Srange.

   Usually CN is only used to set the dry and wet limits of S. CN is not used after that but can be computed for display as CN=1000/(S+10).

Now the rainfall increments loop:

1) ===========WHEN RAINING============

   P=P+Rain increment

   S2 and S8 not changed

   abstraction = P - S2

   qp = (abstr# * abstr#) / (P + S8)

   loss = Rain - (qp - oldqp)

   excess = Rain - loss

   Infiltration= Rain - excess

   S = S - Infil (reduce SPACE)

   Sgw = Sgw + excess * 0.2 (this is magic groundwater made available if needed)

2) ===========WHEN NOT RAINING========

   P is reduced by evap and dGW (both by a factor based on P/Srange for fast recovery).

   S = S + evap (increase space)

   S2 = S * 0.2

   S8 = S * 0.8

3)============ALL THE TIME============

   S = S + dGW (increase space) where dGW is soil rate recession slope factor * S/Srange (slow recovery).

   Send Excess to unit graph.

   Send dGW to base flow

   Send linear reservoir routed Sgw to base flow if wanted.

End of loop

My recipe for getting started at a new site now is:

Go here first, it's much simpler:  http://getmyrealtime.com/HarvardGultchExample.aspx

1--Use SCS dimensionless graph to start.

2--Set the lag time.

3--Adjust Initial CN = Dry CN + 1 to get the first peak.

4--Adjust Recovery Factor to get the next peak.

5--Set the % impervious to get the minor peaks below CN excesses.

6--Set Ground Water Loss Factor=1.

7--Play with the Soil Loss Factor to get the base flow.

8--Lastly try mixing up the Ground Water and Soil Water Loss Factors for base flows and try the Triangle Unit Graph or Linear Reservoir to better shape peaks.

9--Tweak the whole period as the months go by.

And it may help to step back and take a look at all this and  remember where the water went so always remember water is only lost by (a) Runoff and (b) Evapotranspiration :

1-Starting with Rainfall.

2-The CN loss method creates the Runoff.

3-Infiltration is the starting Rainfall minus Runoff (minus what gets stuck in the trees).

4-Infiltration goes into the CN storage.

5-When it stops raining, ET will reduce the CN storage.

6-Now there are no more losses beyond this point.

7-CN storage is reduced at the max rate set by constantLoss*recoveryFactor and returned to runoff.

8-The adjusted rate at which CN storage is returned depends on the current space (Sdry-S)/(Sdry-Swet).

9-The fraction returned in step 7 is set by Soilwater_Loss_Factor. (ok, the fraction not returned is lost unless the GW table is used.)

To start, look up your Soil Type and cover and get a CN. The Wizard will give you a corresponding CNdry. For your first runoff event, hold the CNdry constant and vary the starting CN (this varying CN is what the model will keep track of after the first event based on the recovery factor). If not satisfied, or after more events, adjust CNdry or recoveryFactor to improve peaks, ETfactor to flatten recession, and Soilwater_Loss_Factor to move it all up and down. If groundwater return is giving you a problem, then try the optional GW table.  Remember also that the hydrograph log-linear recession slope is also the constantLoss * recoveryFactor (see above graph).

Calibrations, here is my latest thinking 1/1/2016:

If you got things halfway working then skip steps 1, 2, 3, 4, 5.

1) Set Ground Water Loss Factor to 1.

2) Set Soil Loss Factor to 0.9

3) Set CN value to CNdry+1.

4) Set CNdry and leave CNwet at auto.

5) Try different DGF's a) VillageCr b) LittleTallapoosa c)Maurice d) Tallapoosa. the other dgf's seem about all the same. then Clark, then Linear Reservoirs, then Triangles, then make your own DGF.

 ------------------------------------------

6) Set LONG TERM (a month) recession slope using MOST Important Soil Recovery Factor

7) Set Soil Loss Factor to give longer term flow or raise Base Flow.

8) Set Ground Water Loss factor and speed to get peak shape. Sometimes you may want a very slow gw return (0.05) if boggy and you need your soils drying out faster for CN. This reverses the Soil and GW roles.

9) Here you can go to steps 1-5 or continue below.

--------------------This here is optional------------------

10) Try changing Initial Abstraction (almost like changing CN but easier)

11) Try raising interception factor (.05) for forests.

12) Change %Impervious and Direct Connect for fitting small events.

13) Repeat 10,000 times until hair is gone... then do some thinking now that you see what each factor can do.

The runoff coefficients can now be checked and entered by using the 'Wizards' button on the Select Station List. Just click on your station, if present, and then click 'Wizards'. You will see a complete breakout of all your coefficients for editing.

Wizard CN selections Nov 23, 2020:  I've updated the Runoff Wizard CN automated dry and wet defaults when the CN value is changed. Previously the wet and dry CN's were computed from the entered CN being the middle of the range. Over the years I usually select a CNdry value as CN-1 and enter that in the CNdry text box. A lot of basin within a few days dry out to the CNdry value so that's really the starting CN you want to calibrate for. Also when starting a new basin, uncheck 'Last Run's Storage' to use the middle CN value otherwise the CNdry is used for missing storage. For continous modeling with 'Last Run's Storage checked the middle CN is not used so all this is academic with in a few days. Historical calibrations probably would use CN-1 for dry starting also ('Last Runs' Storage' unchecked).

Old way:

CNdry% = cn! / (2.334 - 0.01334 * cn!)

CNwet% = cn! / (0.4036 + 0.0059 * cn!)

New way:

CNdry% = cn! - 1

CNwet% = 27.542 * Log(CNdry%) - 23.032

Getting all 20 of your CALIBRATED subs back on track (3/19/2015): Note that your base flow rises from the out of the box dry condition with each rain event to eventually match the USGS gage flow. You could uncheck 'Last Run's Storage' to start with wetter conditions of the initial CN and watch the base flow drop to eventually match the USGS gage flow, or try it both ways. But after the first day remember to recheck the 'Last Run's Storage' checkbox. This checkbox is only unchecked for startups and should always be checked there after for continuous modeling. For event modeling it would be unchecked.

The best way to quickly ADJUST ALL SUB'S base flow recessions and get back on track or up to speed is the 'Base Flow Adjust' on the 'Scheduled Batch' menu (uncheck Batch after setting). Try changing the '0.25' precip value to 1.25 and run and compare to the USGS flow and adjust as necessary. So if your adjusted radar or snowmelt really screws up, you can use this 'Base Flow Adjust' to get things back on track and ready for the next event.

Besides 'Base Flow Adjust', another easy way to adjust runoff is to use a factor on the Gage/Radar ratio like 1.3:

COMPUTE-hour; -31251; Ratio; Avg G/R Ratio Oneida Cr; 0; .02,1.2, 5; 1.3*P1/P2

Using a factor to adjust losses was tested and works very well but it screws up the base flow so was not added to GetRealtime code (except snowmelt).

If you would like to save multiple scenarios and variations on any of  the computations you can by simply attaching 'Run-' to the GetRealtime_setup.txt file's 'Station_ID' like 'RUN-COMPUTE-unit'.  Multi scenario computations are described at the bottom of this page.

Radar Rainfall Adjustment

THE VERY MOST IMPORTANT THING OF ALL!!!

I have finally given up thinking radar rainfall could just be used as is for rainfall-runoff computations for sub-regional/local use. I had thought that adjustment of the runoff coefficients would be sufficient to overcome radar random errors and certainly seasonal bias. Wishful thinking. So a 'Radar Adjust Wizard' has been added to help you setup this much needed adjustment in GetRealtime. The above calibration example used this radar basin rainfall adjustment method described here using 2 gages. My Las Vegas Valley radar and runoff study uses single and average ratios for a more in depth look at how hourly radar adjustments improve point rainfall and runoff computation as well as do all of my current radar studies listed on my SiteMap.

The adjustment is based on hourly (or 2hour or daily, forget unit values) ratios of a rainfall Gage or gages and the radar point rainfall at the Gage or gages.

Ratio = (G1+G2+G3...)/N / (R1+R2+R3...)/N

Note that for multiple Gages, the method used here is the ratio Gage_Average / Radar_Average.

Three Gage/Radar ratio methods came to mind when using more than one Gage:

1) (G1+G2) / (R1+R2)   ...Ratio of the sums.

2) (G1+G2)/N / (R1+R2)/N  ...Ratio of the averages.

3) (G1/R1 + G2/R2)/N  ...Average of the ratios.

Methods 1 and 2 are equal when all Gages are available, but method 2 was selected because it was not a trivial task to perform a conditional sum and remove the paired missing gage and radar in real-time. Method 2 will remove the missing Gage in the averaging  setup correctly but alas, both R1 and R2 will be averaged for the denominator. If you have a bad apple Gage, remove it yourself.

Method 3 seemed like the best choice to overcome missing Gages until in actual practice it was found to be averaging the default ratio 1 whenever rainfall was not present and so was rejected.

In your GetRealtime Gages and Radars Averaging setups be sure to use the variable 'N' and not the actual number of gages, ie:   (P1+P2)/N   not   (P1+P2)/2

COMPUTE-Hour; 10818; Rainfall; Watauga Basin Avg of 3 Rain Gages; 0; 0; (P1+P2+P3)/N

And your GetAccess table 'rsite' parameter_code could be 10811,10814,10816 with station_id COMPUTE.

(similarly for your radar points)

COMPUTE-Hour; -10818; Rainfall; Watauga Basin Avg of 3 Radar Points; 0; 0; (P1+P2+P3)/N

And your GetAccess table 'rsite' parameter_code could be -10811,-10814,-10816 with station_id COMPUTE. 

(Try COMPUTE-2Hour and COMPUTE-Day for long distances or winter.)

If (P1+P2)/2 is used then if one Gage is missing, a missing value is computed. How you come up with the Numerator and the Denominator is up to you... actually the Ratio record itself can be up to you, but for real-time computations within GetRealtime the above method seems the best approach so far.

These hourly ratios will then be applied to the 5-minute basin average radar rainfall record. The 'Radar Adjust Wizard' shows you how to construct this ratio setup. It also shows you the setup for adjusting the radar record with this ratio record.

Update 12/21/2012: If you run out of datatype_id 10 for precip/melt then you can now use datatype_id 11 interchangeably with 10 so you may wish to use dsid 11xxx instead of 10xxx to show that it is an adjusted rainfall. The GetNexrad boundary display option 'Show current list values' expects rain gage values to be datatype_id 10, radar to be -10, and adjusted radar to be 11 or -11. GetNexrad will then display the gaged values at the rain gages and adjusted radar for the subbasins.

Update 12/12/2013: Here is a tip for dealing with erractic rain gage records. Say there are 4 Wundergages in your area of interest. You have been using gages 2, 3, and 4 but not 1 for a 3Gage average. Now say gages 3 and 4 start to under report rainfall and you would rather use gages 1 and 2 but not 3 and 4.

Here is the tip, in the GetAccess HDB table 'rsite' parameter code put all 4 gage dsid's in the proper order as P1, P2, P3, P4,...etc.

For the 'rsite' table parameter codes: 10612,10613,10614,10617

P1= 10612

P2= 10613

P3= 10614

P4= 10617

With all 4 dsid's now in 'rsite' you can change the formula in GetRealtime_setup.txt as needed with out changing the table 'rsite'. So to use just rain gages 1 and 2, then the formula is now (P1+P2)/N. If you want to change back to gages 2,3, and 4 then (P2+P3+P4)/N. And don't forget to change your 'rsite' table radar parameters and GetRealtime_setup.txt formula to average the proper radar also.  This update requires GetRealtime.exe as of 12/12/2013.

Update 4/18/2014: You can now use GetGraphs.exe menu 'Gage Off/On' to toggle the use of any of the gages used in GetRealtime's (P1+P2+P3+P4)/N computation. This allows all the gages in the area to remain listed in both 'rsite' and GetRealtime_setup.txt and using GetGraphs menu to turn off offending rain gages and radar at the gages. GetGraphs creates the file 'GetGraphs_OnOff.txt' which lists the offending gages that is read by GetRealtime.exe. GetRealtime assumes the on/off file will be in the GetGraphs folder but if not it will look next where ever your HDB database file is located. The GetGraphs plots will have a yellow background when OFF. Be careful when doing historical studies to remember what gages were historically usable for your historical period and to reset your offenders for current use.  'GetGraphs_OnOff.txt' is re-read every time GetRealtime runs so that even in batch mode any changes will take effect immediately.

To help you evaluate bad rainfall gages you can add a 3 on the end of your GetGraphs_setup.txt line like this:

.....; 65535 ; 16711680 ; 255 ;;;; 2 ; 3

For this example, the first 2 means cumulative plot, and the 3 means you want to display the Mean Absolute Error between the gage and radar. The difference between gage and radar are only calculated when either value is greater than 0.05 in/hr. The n value is the number of errors averaged. To make use of these MAE's you could compare them to each other. If several are near 50% and 1 is 80%, then the 80% gage could be turned off. Likewise though if most are 80% and 1 is 10%, turn off the 10% gage. The gage weighted MAE= Sum G*|G-R| / Sum G*G. You can even use this MAE on your flow graphs to get a 2nd opinion of your eyeballs.

In the following example MAE's, I always know that P8 Nucor Steel is the best rain gage ever so I would turn OFF the gages that strayed from it's MAE:

Remember and NEVER forget that radar adjustment is the most important thing of all. The more thought and inspection of your ratioing scheme the better and cannot be stressed enough. So instead of averaging gages to ratio as the example above, it may be better to subdivide the basin by how many rain gages you have to work with. I'm no expert so you should put more thought into this than I have.

Here are some Ratio tips that I have also included in the 'Wizard Notes':

1) The current hour adjustment will be the previous hours ratio until 30 minutes of radar values are available and the difference between gage time and radar time is 10 minutes or less.

2) Don't use Inversion Suppression when adjusting radar rainfall, use the Ratio's Gage Minimum value of exactly 0.01 near radars and the radar will be zeroed when Gage is zero.

3) Setting a high 'Gage Minimum' of say 0.05 will better insure a viable ratio is computed... but you will have to suppress the inversion layer near radars yourself. The above calibration used 0.01 'Gage Minimum' to suppress the inversion layer and trust to luck for the computed ratios.

4) Using 1 good rain gage for 1,000 square miles is better than half a dozen bad ones.  The assumption made here is that radar is spatially perfect for your region. Given just one of it's point rainfall values, your region of the radar screen is then known... and wishes were fishes.

5) Never use an Wunderground airport (KABC) rainfall record for ratioing because it is impossible to tell which hour the rainfall fell in. Airports reset their rainfall at hh:53, hh:55, or hh:56 and should be avoided.  This is no longer true. Because of need for good data, GetRealtime now makes better use of Airport gages buy assuming hh:5x is the end of the hour and bases the hourly rainfall at this time. Not perfect but may be better than most other rain gages, so I would include NWS Airports when there is not much choice (winter).

6) For radar overshoot in winter you could adjust the radar when 0 by using the HOURLY average of the rain gages and adjust the radar like this:

Get radar:

NEXRAD-BGM; -10301; Radar Rain; Oneida Cr Sub1, NY

Get gage average:

COMPUTE-hour; 10251; Rainfall; Avg Rain Gages Oneida Cr; 0; 0; (P1+P2+P3)/N

Replace radar 5-minute values with gage average/12:

COMPUTE-unit; -10901; Radar Rain; Set Radar 0 to gage; P1=0; 0; P2/12; P1>0; 0; P1

where P1 and P2 are: -10301, 10251 in rtable parameter_codes and radar must be first because 5-minute values. Continue with the radar average and computation of the g/r average ratio.

It was found that in larger basins where a hailstorm can be present and no rainfall at the ratio gage, the default ratio value of 1 was a very very poor default. So using the last computed ratio worked exceptionally well in this particular case; the last computed ratio was 4 hours old.  Only ratios based on rainfall at the gage greater than 0.20 inches will be carried forward.  The ratio will be reset to the default ratio after 6 hours of no radar rainfall at the gage if it was greater than the default or after 1 hour if less than the default and again will probably only matter to large basins.

And again it is imperative to evaluate the rain gage somehow. I only use Wunderground rain gages available free in realtime (parameter_code = dailyrainin and NOT HourlyPrecipIn). One way to evaluate a gage is to compare the total rainfall for the year to date with other nearby gages and usually the best ones have the most rain. But some may have just been down for some reason. Then you should look for gages that have a fast rise and fall because half of them are plugged with leaves. And the ones with the best 5-minute time steps are preferred because getting a good reliable time step seems to require some sort of operator genius who will also properly maintain and calibrate the tipping bucket. And it is nice to have a rain gage located upwind and outside the basin so that a ratioing value will be available instead of using a default as the storm approaches. If you use 1 gage it should be near the basin centroid, and if 2 gages then they should be at opposite ends of the basin. And if no gages are in the area, maybe making a distance vs rainfall for daily values at what gages you can find and just use a daily distance corrected factor?... but do something at least every day! And check a high speed radar loop of your area of interest to check for beam blockage and if so then your rain gages should have some relation to beam blocked areas. The blocked areas will appear as static radial lines or pie slivers of a slightly different color radiating from the radar center as the rapidly moving rain images pass through them and are much more common than you might think. And in winter be sure your rain gages are heated or you will get rainfall everyday for the next week beginning at 8:00am.

I would test and retest this section because... It's the very most important thing of all!!!

Note the order of the 31229,-10230 paramters in table rsite. Do not reverse.

Beam blocked areas can also be revealed when a heavy inversion layer causes ground clutter like this BMX radar image for Birmingham, AL. Of all the places for a radar to be blocked is its city namesake. Birmingham requires a default ratio of 1.5 to overcome the beam blockage.  Ground clutter is removed on the NCQ and A2M radar images.

For comparison, here is Huntsville, HTX unblocked image for the same period:

Update 12/5/2012:

(See the much improved second Nowcast method update 12/16/2015 just below after reading how to set a track here.)

 Use GetNexrad's radar tracking with 'Storm track forecasting' to provide up to 1 to 3 hours lead time based on storm type and tracking speed. Your basin or point of interest is offset upwind to tracking point #1 (Eulerian frame of reference) for rainfall computation where distance and storm speed will compute the nowcast lead time. Write the nowcast unit values to GetAccess for real-time rainfall runoff and GetRealtime will overwrite as true radar basin rainfall comes in. 

I have not had much experience with this yet but some experimenting on how to add multiple forecasts and in which order they overwrite into the GetAccess database should get you going. Meaning if you had a 2 hour radar loop going, you could start a track out at 100 miles, write it's forecast values to the database, then a track at 50 miles, then one at 20 miles... or pick up and move (convective) every 10 minutes so you can see why some experience will help.  You be the judge.  And copy/rename the file RainLoop.txt for later evaluation of these forecasts.  Use GetAccess to add multiple runoffs graphically as you go.  Aint science fun!!!

And to kick it up a notch, add the NWS 6 hour to 72 hour Quantitative Precipitation Forecasts.  Just use GetNexrad's Surface Obs' QPF option to set the start and duration and eyeball your basin's QPF precip and away you go.  (Update: This is old school. Just use FORECAST-NWS in GetRealtime_setup.txt per below.)  The QPF future precip will be reduced for actual radar history and nowcast values.  Your GetAccess database values will be 'source' coded as 13 for real radar values, 14 for tracking nowcast values, and 15 for QPF values.

====================================

One Way to do a GetNexrad Nowcast:

 (Eulerian frame of reference)

1) Using Iowa Mesonet get a 1-hour radar loop going (1-hour can vary) with a single subbasin boundary (not the Show List boundaries) with GetNexrad.

2) Zoom in some to your area and then double click the radar image to clear any mouse clicks.

3) When the radar loop reaches the end of the loop, click the top center text box to start stepping mode.

4) Click on the leading edge of the approaching storm cell for a set point, now back step to the beginning of the loop, and then forward step 2 to 4 steps and click the leading edge of the storm again.

5) With your 2 set points now set, click on the top center text box again to let the loop start looping again.

6) Check that the yellow circle on your tracking marches in lockstep with the leading storm edge. It may take a few tries so repeat steps by double clicking the radar image to clear tracking and start over at step 3.

7) With a nice tracking click the menu 'Overlays' button (not check box), then optionally uncheck the 'Storm track show set points' for better viewing, and check the 'Storm track forecasting' check box.

8) Your subbasin will be displayed at your set point near the start of the loop. Note why you didn't use the first loop step is because the storm would have been halfway through your basin.

9) Note the rainfall 'Sum' on the windows caption. It will show you if this Nowcast is worth using.

10) To use this Nowcast, click the main menu 'Save' button, check 'Save Radar Rainfall Loop', and click OK.

11) On the next menu check 'Send to GetAccess HDB', AND IMPORTANTLY change the 'DataType_Site_Ids' from the radar -10 ID to an adjusted radar ID of -11 or 11 depending on your setup, and click OK.

12) When the loop reaches the end, a message will tell you how many values were sent to the database.

13) Close GetNexrad.exe and open GetRealtime.exe and from the Select Station(s) list, select the -11xxx adjusted radar or NWS forecast station for the sub you used and click the 'Wizard' button.

14) From the 'NWS Forecast and Values Edit Wizard' window, on the lower left corner change the value 6 for Hr's to 2. Don't use 1 until I fix it for 1. Don't use 6 or the NWS forecast cannot update them. Now click the 'Edit 6-Hour Forecast' button and the next 2 hours of forecast will be displayed.

15) Note the date_times and also note the leading 14 and 18 where 14 is you Nowcast and 18 is the NWS forecast. Check the hourly values are what you want. You can edit the values if you like (even skip the whole Nowcast and just edit these).

16) When satisfied, check the 'Write to all forecasts', and click the 'Save' button.

17) That's it, these 2 hours are now marked Source=14 so as not to be overwritten, all -11 dsid forecast are updated for all subs.

18) The radar will overwrite the source 14 as it comes in or you can overwrite manually with the NWS Forecast again by checking 'Overwrite Forecast' on the 'Scheduled Batch' menu and run the NWS forecast stations to replace the Nowcast 14 values with 18 NWS Forecast values.

19) Repeat your Nowcasts or edits as needed for longer storms and practice helps. You don't want to be reading all this when the lights go out.

And if you forget to save the Nowcast as a -11 or 11 forecast dsid, you can use the Forecast Wizard to bring up the -10 radar dsid with source 14 Nowcast values and copy them to your clipboard, retrieve a real 11 forecast, and paste the Nowcast values on the same hours. You could even check your last radar adjustment ratio to adjust these values while your at it.

====================================

Update 12/16/2015: A much better Nowcast has been added to GetRealtime to AUTOMATICALLY provide maybe(?) up to 3 hours of forecast based on the LAST radar image GetRealtime downloads. Unless a low pressure center moves near the basin (update or a squall line forms), the storm speed and direction is pretty constant. So using GetNexrad similar to above you can save your storm track info to a file like:

NowCast_DAX_N0Q.txt that reads:

2015-12-17 19:39 :Date

2015-12-18 01:39 :Expires

0.6264341 :X pixels per minute east

-0.2337041 :Y pixels per minute south

24 MPH ENE 69°

GetRealtime can read the 2 east and south pixel speeds and on the last radar image move each of your subbasins up wind every 5 minutes to compute rainfall at that location (Lagrangian frame of reference). So for a Nowcast of 3 hours then 36 up wind locations of rainfall can be added in less than a second. The nowcasts will have source code 14.  The G/R ratio used for adjusting the nowcast period will be the average of the last 2 ratios last ratio and the default ratio.

You will have to add your GetNexrad storm track folder to the GetRealtime_setup.txt like:

 NowCast Info folder=C:\GETPROJECTS\GETBIGCREEK\GETNEXRAD\

... and on your radar setup line you add 'nowcast 3' like:

NEXRAD-DAX; -10420; Rainfall; Big Creek Basin, Ca; 0; nowcast 3

or save to mtables:

Run-NEXRAD-DAX; -10420; Rainfall; Big Creek Basin, Ca; 0; nowcast 3

AND... to adjust hour 2 values as average of hour 2 and hour 3 use:

NEXRAD-DAX; -10420; Rainfall; Big Creek Basin, Ca; 0; nowcast 3, adjust type 1

Adjusting hour 2 is no longer recommended but you can try it if your hour 2 seems to need it.

Adjust type 2:  To adjust hour 3 values for decay only as proration of adjust factor 0.6 to 0.4 for minutes 120 to 180 if 5-min hour 3 rain count > 0.05" greater than 3.  Assumes any big storm always gets smaller.  Remember we don't have to be good at hour 3, just better than NWS always being 80% low but without predicting doom.

Adjust type 3:  In addition to the adjustment for decay of type 2, type 3 will decay hour 2 only with a prorated 0.8 to 0.6 factor if 5-min hour 2 rain count > 0.05" decreases by more than 3 from last run hour 3.  Hour 1 will decay only with a prorated 1 to 0.7 factor if hour 1 count 3 less than old hour 2.   Hours 2 and 3 nowcasts are probably a fool's errand... so I'm up for it.

Actually I'm still figuing this out and I'll clarify all this eventually... hey it takes fool.  And to see the storm growth counts and differences you should put the main sub basin line first in your radar section of the GetRealtime_setup.txt because only one sub will be reported on. To see these values as counts look at the end of the radar rainfall section text scroll on the GetRealtime view. To view the storm growth count differences when in scheduled batch mode right click the GetRealtime icon in the system tray and select 'View radar' from the popup menu.  To view counts and reductions 'View Storm Count' on the GetRealtime maximized screen or if in batch right click the system tray icon and select 'View Storm Count" there and looks like "0 0 0 NNN" if no counts by hour or reduction. "-4 -3 4 YNY" would have had reductions in hour 1 (-4 diff) and hour 3 (4 count) if your scheduled batch interval was 60 minutes. If 30 minutes the check difference would be -2, 15 minutes or less interval the diff check would be -1. Hour 3 count check is always 4. An hour count of 4 out of 12 being greater than 0.05" usually indicates about 0.5 inches of raw radar rain for that hour.

NEXRAD-DAX; -10420; Rainfall; Big Creek Basin, Ca; 0; nowcast 3, adjust type 2

You can use GetNexrad loops with 'Overlays' checkbox 'Stormtrack from file' to view a field of tracking points based on your nowcast speed and direction to verify other points are consistant or still viable.

That's it. Once your storm track file expires the Nowcasts will no longer be added. And if you have some way to automate storm speed and direction then you can update the above tracking file yourself or shell your program from GetRealtime.  My first time out with this Nowcast I was tracking storm cells continously comming from the southwest. I also saw a frontal line of storms early in the day and thought so what. Then as the front approached, a squall line rapidly descended from the northwest (as general storm cell movement did not change) that my Nowcast was blind too aligning with my track and extremely overestimated the hours 2 and 3 of rainfall as the squall past. So when it really starts raining you better see where it's headed.

This is all I have to show so far: (I've started a gallery of nowcasts here.)

Update 1/3/2016: Automated tracking using NWS winds aloft. Once your manual tracking from above has expired, you can use automated speed and direction from the NWS winds aloft (same Lagrangian frame of reference method above but automated).   I think this is the input to the NWS big kahuna forecast model that updates each hour.

http://rucsoundings.noaa.gov/

The Lat/Long is that of your basin. After watching a few storms and being frustrated with the wind direction I hard wired the altitude for direction at 18000 ft. You don't have to change anything in your WindsAloft file, just know you set your altitude for speed, and direction is now automatically using 18000 ft. Update, optionally, you can add your onw altitude for direction >= to your speed alitude if you don't like 18000' like this in your WindsAloft...txt :

OP40 : title and not used

12000 :Altitude ft msl  speed

17000 :Altitude ft msl direction optional

33.56 :Lat

-86.73 :Long

30R75 :Optional supercell direction/speed adjust or 10R85 for fronts/squalls

Save to your tracking folder (GetNexrad) with radar code and product code:

    WindsAloft_GWX_N0Q.txt

Now the NOAA winds aloft will be checked up to every half hour and the storm track info file NowCast_DAX_N0Q.txt example above will automatically be updated.

If you haven't followed all the steps above, here are the files that need to be in the GetNexrad folder for GetRealtime to automate Nowcasts:

WindsAloft_HGX_N0Q.txt

NowCast_HGX_N0Q.txt

NowCast_HGX_N0Q_RECENT.txt    <<this just needs a title line

The "30R75" Method:

Not long after formation, instead of just "going with the flow", the dynamics affecting most supercell thunderstorms result in a modification of their course to the right of the mean cloud-layer wind. In 1976, R.A. Maddox formulated the 30R75 method as a relatively easy way to calculate the direction of supercell movement. The 30R75 method suggests that a supercell will move 30° to right of the mean wind direction at 75% of the mean wind speed.  

The values in 30R75 can be what ever you wish. I recommend trying 10R85 for fronts and squall lines. In fact, better safe than sorry, try a standard 30R75 as standard practice in the Southeast where super cells and squalls are more often than not.  You don't have to be exact with these speed and directions. You just don't want a squall line sitting right down your wind direction giving 3 hours of rain when maybe 15 minutes is more like it.  You can edit the WindsAloft.txt from the GetRealtime Station List screen radar Wizard button.

The above image is saved with the 'Winds Aloft Graphic' checkbox. I wanted something for GetGraphs to show the status of the Winds for nowcasting. I had forgot that GetGraphs could have just as easily shown the text file with out creating a graphic. Oh well I learned how it can be done if I ever really need to. Also note the line 'Reusing Windsaloft'. This means the NWS OP40 website returned a column of all 99999's for some reason. This happens from time to time. The Nowcast will be based on the last speed and direction available.

If you want to work on historical radar then download the historical radar images and then use GetRealtime 'Read radar from file' on the 'Scheduled Batch' menu and then uncheck 'Scheduled Batch'. Use GetNexrad.exe to create your storm track info file and be sure the expiration date is after the historical radar period.

Like all things about radar, better nowcasting comes thru experience with your basin and getting what you can with what the radar gives you on each and every storm. So Nowcast 1, or 2, or 3 and when to 30R75 it... or do a manual tracking... comes with experience. And as luck would have it, any nowcast is better than nearly all NWS forecasts for the next 3 hours so it's only up from here.  The greastest error in the convective nowcasts seems to be hours out due to random chaotic storm growth and decay. Next largest error is probably guessing at radar rain G/R adjustment and the least error is usually in storm speed and direction. Now we need to experience some really big storms to see how crazy these methods are when it really counts.  And don't embarrass your self like I did by using a winds aloft storm direction that lines righ up with a squall line instead of quickly crossing it.