Select Page

Traditionally the various components of flow analysis including flooding, drought, base-flow, pollutant loading, and duration curves have been examined independently by various analysis methods or software packages. A better approach would be to combine these multiple packages into a single web-tool to improve access. Infrastructure-as-a-Service (IaaS) cloud provides a scalable infrastructure for model implementation, which is a necessity of web services due to the characteristics of web traffic. IaaS centralizes the computational burden and overhead of multiple model runs from local computers to online servers. This paper demonstrates the scalability benefits of the Comprehensive Flow Analysis (CFA) tool in an IaaS environment. The CFA tool is available through the Environmental Risk Assessment Management System (eRAMS) website. eRAMS facilitates GIS data manipulation, visualization, and preparation of input information for models lik CFA. eRAMS uses the Cloud Services Innovation Platform (CSIP) to request runs of the analyses within CFA. CSIP is an IaaS cloud modeling framework designed for executing various environmental models. This paper summarizes a scalability analysis of the analysis methods within CFA using CSIP in a cloud server environment

User Guide

Non-Login Site: www.erams.com/flowanalysis

The non-log-in site allows a user to access and analyze data from USGS’ National Water Information System (NWIS) and U.S. EPA’s STORET/WQX database.
  1. This site begins with the map interface, this tab contains search tools to find a stream monitoring station of interest to analyze.
  2. You can search for a station using keywords in its name or by where it is located on the map. You can also select a buffer radius and search around an area or line or draw a shape on the map to search by.
    • The help button next to the station search options can help explain the searching methods in further detail
  3. Once a station is searched for the results will show up on the left side of the map with icons next to them as well as on the map.
  4. Select the station you wish to analyze from the list or on the map or reset your search and try again.
  5. Once a station is selected a summary balloon will appear on the map, if you click “Flow Analysis Mode” you will launch the CFA interface.
    • The “Site Info.” tab, at the bottom, of the CFA interface gives you a summary of the station and a link to upload your own data files to combine with the public USGS-NWIS or EPA-STORET data
  6. Select the tab for the analysis you wish to perform and provide the necessary inputs for the model.
    • Further help and information about each input is available under the “Help” button
    • When all the inputs are provided, you can run the model by clicking the “Run Model” button
    • Further information about each model is available by the “Model Info.” button
    • Site specific information or access to the data in USGS or EPA is available by the “Site Info.” button which will open a link to the database in a new browser window
    • The “Download Data” button downloads the data for the currently selected model: daily flow, annual peaks, water quality info., etc.
  7. After running the model, a results page with graph, summary, and references will appear
    • Further information about the model and its results are availble from the “Further Model Info.” button
    • The graph image can be saved using the “Save Graph” button
    • The comments section can be filled with information and then the whole result page printed using the “Print Report” button
    • If available a results file can be downloaded using the “Result File” button

Login Site: www.erams.com

The login site has all the features of the non-log-in site, but also allows users to add new stream flow and water quality monitoring sites that are not in the NWIS or STORET databases and upload and analyze data for these new sites in the same manner.
  1. Login to an existing eRAMS account or create a new one
  2. Go to “My Acount” at the top of the page, then click on “Project” on the left hand side
  3. Click “Create Project” and provide a name for it. Select “Flow Analysis” as the “Project Type”. Then click “Okay”
  4. This project is now highlighted in your list of projects as the active project. Click “GIS/Analysis” at the top to go to the map interface to access the tool.
  5. Once on the map interface, you can follow the instructions for the non-login site to access the stream monitoring stations and various flow and water quality analysis tools in CFA.
  • There is an additional option on this version of the tool that is not available on the non-login site which allows you to create stations on the map and upload your own data files to them for analysis by the tool in the same fashion.
  • Click “Add Station” and then select the point on the map where your stream monitoring location is.
    • Once the station is created, you will need to fill in some extra information about the station like name, it’s id, and other.
    • These stations will show up as Orange pin icons on the map when searched for

Further Information

Frequently Asked Questions (FAQs)

Q: Can you compare flow data between two different monitoring stations or do you need to run two different analyses and compare them?

A1: You can select more than one station (hold “ctrl” or “shift” and select another station in the “Station Search Results” list on the left panel) and a button for “Multiple Site Comparison” will appear below the list. Clicking this button will open a different interface to compare daily or instantaneous discharge and flow duration curves for multiple monitoring stations side by side in graphs and summary tables.

A2: You can also simply select a station, open the flow analysis interface and run an analysis. Then select a different station, open another flow analysis interface and run and analysis. Then compare the results windows to each other.

Q: Why can’t I see anything when I go to the map on www.erams.com/flowanalysis?

A: Some internet browsers have issues displaying the maps of eRAMS. eRAMS has been optimized for Mozilla Firefox and Google Chrome and switching to this web browser should correct map display issues.

Q: How do I use data from two different stations in one analysis?

A: Some stream monitoring stations are present in multiple databases due to shifts in responsibility from agencies like the USGS to groups like the Colorado Division of Water Resources (CDWR). Therefore, it is necessary to pull data from multiple databases to get a complete dataset for one stream monitoring station. To address this issue with this tool follow the steps below:

  1. Select the second station and download the desired data
  2. Format this data to the specifications for a user uploaded file for the flow analysis tool
  3. Select the first station and upload this file as user data
  4. Check the box to “Use Uploaded Data” on the analysis input page
  5. Select “Merge Uploaded Data with Public Data” and specify what to use for dates that are included in both datasets

Q: How do I save an interactive graph?

A: The interactive graphs contain a menu (three grey lines) on the top right of the figure which opens an option to save the graph as any of the provided file types.

Q: Why do the static graphs stop at January 1st 1900?

A: A limit of the graphing software used to produce the static graph images is that it cannot handle dates prior to 1900-01-01, so these dates are not included in the graph. However, these dates are still included in all of the statistics and the interactive graphs.

Q: How do the “seasonal” dates work?

A: When a season’s begin and end dates are specified, any data that is after, or equal to, the “begin date” and before, or equal to, the “end date” is included in the season.

For example, if a season is specified to be July 1st through August 20th, July 1st data will be included in the “seasonal” data statistics, but June 30th data will not be included. Additionally, if the “season” is specified as November 1st through March 1st, it will still function properly.

Geospatial

The base layer for the CFA tool is a GIS point layer of stream flow and water quality monitoring locations historically or actively maintained by the Colorado Division of Water Resources (http://www.dwr.state.co.us/), U.S. Geological Survey (http://waterdata.usgs.gov/nwis), or the EPA’s STORET database (http://ofmpub.epa.gov/storpubl/dw_pages.stationcriteria).

Data Creation:

  • The data used in the CFA tool is either auto-extracted from the Colorado Division of Water Resources (CDWR) database, USGS’ NWIS database, the U.S. EPA’s STORET/WQX database. The user is also able to upload and use their data alone or in combination with the database data. The user uploaded data must be in a comma separated value (CSV) file.
  • If you wish to use your own data combined with the available USGS or EPA data, upload your file and then check the “Use Uploaded Data” box on the analysis interface and choose whether you desire to analyze your data only or merge it with available USGS/EPA data and then analyze it.

CSV File Format

    • The first row must contain a label for the columns, for example “date, flow, 00600”
    • The first column must contain dates, in any of the following formats:
  • yyyy-mm-dd
  • yyyy-mm-d
  • yyyy-m-dd
  • yyyy-m-d
  • yyyy/mm/dd
  • yyyy/mm/d
  • yyyy/m/dd
  • yyyy/m/d
  • The remaining columns of the are to be different datasets, with corresponding labels in the first row:
    • If flow data is provided, the label must be “flow” (with units of cfs).
    • If water quality data is provided:
      • The label must to be the 5-digit USGS code (including zeros) for that type of water quality data
      • The water quality data must be in the units of that USGS code
    • If there are dates with no data for one or more columns please put “null” or “n/a” in the column.

Technical Manual with greater detail is available here

Example Outputs

The outputs of CFA vary by the model used but each returns a summary of the inputs, a graphical summary of the analysis data, references to the methods used, and a result file.

Time Series Analysis Results

15-minute Discharge Analysis Results

Stage-Discharge (Rating-Curve) Data Results

Flood Analysis Results

Drought Analysis Results

Baseflow Analysis Results

Flow Duration Curve Analysis Results

Load Duration Curve Analysis Results

LOADEST Analysis Results

API Notes

REST Access

In addition to the interface for the CFA tool, each flow analysis can be accessed using a REST-based protocol. This is because the CFA tool is built into the Cloud Service Innovation Platform, CSIP (Lloyd et al., 2012). An input/output example of this REST-based access to CSIP-CFA is provided below:

Instructions

  1. A REST “get” call to one of the “Endpoints” in the above links for the analyses, will provide an example JavaScript Object Notation (JSON) of the required inputs for the model.
  2. Replace the values of the various inputs with those desired for a specific CDWR, STORET, or USGS stream monitoring station.
  3. A REST “push” call, to the same “Endpoint”, containing the modified JSON object will request a run of the CFA model with the specified inputs
  4. Once finished running, a return JSON will be sent back with a summary of the outputs from the model. An example result JSON is also available at the above links from the CSIP-CFA.

References

  • Andrienko G. and N. Andrienko. 2005. “Visual exploration of the spatial distribution of temporal behaviors”. Proceedings of the Ninth International Conference on Information Visualisation.
  • Arnold, J.G., P.M. Allen, R. Muttiah, & G. Bernhardt. (1995) “Automated base flow separation and recession analysis techniques.” Ground Water. 33(6): 1010-1019
  • Arnold, J.G., and P.M. Allen. (1999) “Automated methods for estimating baseflow and groundwater recharge from streamflow records.” Journal of the American Water Resources Association 35(2): 411-424.
  • Baker, David B., Peter Richards, Timothy T. Loftus, and Jack W. Kramer. 2004. “A New Flashiness Index: Characteristics and Applications to Midwestern Rivers and Streams.” Journal of the American Water Resources Association (JAWRA). April 2004:503-522
  • Cleland, Bruce. (August 2002) “TMDL Development from the ‘Bottom Up’ Part II: Using Duration Curves to Connect the Pieces” National TMDL Science and Policy.
  • Cleland, Bruce. (November 2003) “TMDL Development from the ‘Bottom Up’-Part III: Duration Curves and Wet-Weather Assessments” Example Report.
  • Cleland, Bruce. (August 2007) “An Approach for Using Load Duration Curves in the Development of TMDLs.” National TMDL Science and Policy.
  • DePhilip, M. and T. Moberg. 2010. “Ecosystem flow recommendations of the the Susquehanna River Basin”. The Nature Conservancy. Harrisburg, Pennsylvania.
  • Interagency Advisory Committee on Water Data (IACWD). (1982) “Guidelines for determining flood flow frequency” Bulletin No. 17B (Revised and corrected) Hydrologic Subcommittee Washington, D.C.
  • Hirsch, Robert M., Dennis R. Helsel, Timothy A. Cohn, and Edward J. Gilroy. 1993. “Chapter 17: Statistical Analysis of Hydrologic Data.” The McGraw Hill Handbook of Hydrology. D. R. Maidment, ed., McGraw-Hill New York
  • Lloyd, W., O. David, J. Lyon, K.W. Rojas, J.C. Ascough II, T.R. Green, J.R. Carlson. (2012) “The Cloud Services Innovation Platform-Enabling Service-Based Environmental Modelling Using Infrastructure-as-a-Service Cloud Computing” International Environmental Modelling and Software Society Converence 2012.
  • Runkel, Robert L., Charles G. Crawford, and Timothy A. Cohn. U.S. Department of the Interior. U.S. Geological Survey. (2004) “Chapter A5: Load Estimator (LOADEST): A FORTRAN Program for Estimating Constituent Loads in Streams and Rivers” Techniques and Methods Book 4. Reston, Virginia
  • Salas, Jose D., Chongjin Fu, Antonino Cancelliere, Dony Dustin, Dennis Bode, Andy Pineda, and Esther Vincent. (2005) “Characterizing the Severity and Risk of Drought in the Poudre River, Colorado” Journal of Water Resources Planning and Management 131(5): 383-393.
  • Salas, Jose D., (1993) “Chapter 19: Analysis and Modeling of Hydrologic Time Series” The McGraw Hill Handbook of Hydrology. D.R. Maidment, ed., McGraw-Hill New York.
  • Water Resources Council, Hydrology Committee (WRC-HC) (1967) “A Uniform Technique for Determining Flood Flow Frequencies” Bulletin No. 15, Washington, D.C.