Water Quality Exchange
From Exchange Network Wiki
Contents |
WQX Overview
The Water Quality Exchange (WQX) allows Network partners to share ambient water quality data with the U.S. EPA. WQX represents the next evolution for EPA's STORET model for sharing water quality data. EPA will transition away from a distributed database model and towards the Exchange Network model for the sharing of data. This transition is expected to be fully completed in 2009.
WQX defines the methods, data systems, and Schema by which EPA and State partners compile water quality monitoring data. Its purpose is to provide a seamless collection of monitoring data not restricted by jurisdictional boundaries. WQX consists of two flow parts meeting two separate business needs: a batch update to the STORET Warehouse and the Pacific Northwest Water Quality Exchange (PNW). Partners submit data to the STORET database (EPA’s repository of surface water monitoring data) through a batch update through WQX. The batch update, satisfyiing a regulatory requirement, allows Network partners to share 3 types of ambient water quality data with the EPA: the physical conditions in the environment at the time of site visit; the chemical and bacteriological make-up of the water sampled; and chemical analysis of fish tissue collected. PNW allows publishing of water quality data for states in Region X through a separate flow.
Delivery of monitoring data to STORET is optional, except in rare cases where specific grant language may require it. Many states and tribes have not provided data to STORET, because they did not want or need to use the provided client application for internal data management. The Exchange Network implementation of WQX provides a technology-neutral means of data exchange—primarily to load data into the STORET warehouse. A complementary set of services provides a query interface to data from STORET using the same data format go promote reuse and data comparability. The community of interest for surface water monitoring is large and extremely diverse. It includes data suppliers: Watershed councils, states, tribes, local governments, EPA, and other federal agencies (e.g. all National Park Service monitoring data was until recently hosted by the state of Alaska as a means of getting the data into the STORET database via the Alaska client). It also includes data users: scientific, operational, permitting, and policy analysis staff working for all the community of interest above as well as academia, environmental special interest groups, and the general public.
Data Exchanges within WQX
- physical conditions in the environment at the time of a site visit
- chemical and bacteriological make-up of the water sampled
- chemical analyses of fish tissue collected
- biological taxon abundance data
- toxicity data
- habitat assessment scores and their related metric scores
- biological index scores and their related metric scores
Flow Status
While an Exchange Network data flow for STORET had been a high priority for states since the inception of the Network, WQX came about primarily as an initiative of the Office of Water. The Surface Water Monitoring branch reports that:
- Given budget cuts announced in 2005, the Office of Water could no longer afford to support the STORET client application, which would soon require significant updating; and
- They recognized that the client application was not meeting their needs; as number of states and other monitoring entities had never implemented the client, and were not supplying data to STORET in any form.
The WQX batch update to STORET is in production and use is growing slowly. The STORET warehouse contained about 45 million results prior to the implementation of WQX. That total is already over 75 million, with only a few major partners implementing the exchange. If broadly implemented, WQX should be expected to easily triple EPA’s data holdings. PNW, which is similar to but precedes WQX, is also in production.
Version 1.0 x Version 2.0
WQX versions 1.0 and 2.0 are compatible. A file submitted in V1 can be read by the V2 converter. The only difference is that V2 expanded the types of data it could handle to include two more types, whereas V1 only addresses physical characteristics.
Current Successes
Prior to implementation of the WQX, the primary mechanism for delivering data to STORET was to use an EPA-provided client application locally, and to use the export processes provided with the application to update the STORET warehouse. WQX provides a mechanism for any water monitoring partner to load data into STORET electronically.
Further, coupling the Exchange Network with water monitoring data has created the following opportunities:
- EPA scientists and program managers will have access to substantially more data. The STORET warehouse contained about 45 million results prior to the implementation of WQX. That total is already over 75 million, with only a few major partners implementing the exchange. If broadly implemented, WQX should be expected to easily triple EPA’s data holdings.
- State, tribal, local, and public data uses can more easily share data access tools. A commonly used format for monitoring data means that data access tools (query engines, analytical models, and visualization software) developed anywhere are much more easily applied to data from a different source. For example, EPA’s Office of Water and Region 10 have both developed a number of tools that improve the ability to extract data sets, and graph or map results. Uploading any data to STORET gives immediate access to these tools, but if the data is not supplied to STORET, it is still relatively simple to adapt the tools due to the common format in use.
- Other data collection communities have a lower cost to participate, and can see benefits of adopting a standard. Already, the US Geodetic Survey has agreed to mount a set of services which exactly parallel the WQX services, giving easy access to additional water quantity and other properties. State participants in the pesticides program have indicated an interest in contributing pesticides monitoring data (so that they themselves will have access to complementary data).
- Some grant opportunities may be linked to delivery of data into STORET. Partners may be less redistricted in seeking funding if reporting is routine and inexpensive. The Yurok Tribe reports having decided to implement WQX for this reason. Given the desire to have their own data management system, WQX was the lowest cost way to meet reporting requirements.
- Standard formats and services can make disparate data sets complementary. For example, “gauge” readings for stream height (associated with flow) are a key variable when analyzing pollutant concentrations. This data is usually retrieved from the National Water Information System (NWIS), but before WQX, it was in different formats. Similarly, temperature and flow data are typically strongly correlated, but accessed differently. States may choose to participate in WQX for this reason alone—it eliminates the work needed to establish “common ground” as a precursor to any analysis that includes data from multiple sources.
Future Plans
EPA is currently developing WQX Web that will allow small data users to be able share data via WQX. This tool will allow users to upload 'flat-files' (basically a delimited text file) through CDX Web that will then be submitted to WQX. For more information on WQX Web, please visit the website of U.S. EPA's Office of Water
The EPA currently has a live version of WQX web available at: https://cdx.epa.gov/SSL/CDX/Login.asp?Referer=Registration
There are currently limitations with using WQX web though. WQX does not validate files with the same strict rules that CDX or STORET do. The problem surfaces itself when files are validated in WQX web, then when you go to submit them to CDX they fail because they do not match the schema. WQX web was billed as being much easier to use than WEBsim but because WQX web cannot fully prepare files for submission to STORET, it is a handicapped webtool.
In the next ten years, in may be irrelevant if data is published through a national data warehouse or through another source. Applications may retrieve the desired information from whatever source has published the data, rather than just from STORET. PNW provides an example of how this is already occurring. WQX must determine whether publishing of water quality data necessitates a completely unified warehouse. WQX may also grow into a multimedia system, with more programs (such as BEACHES) already moving their data into WQX and other programs (such as land monitoring) likely to follow.
WQX Background
WQX and Exchange Network Governance Governance resource implications are limited. 106 Grants support the development of WQX. In the Strategic Plan, the following stakeholder responsibilities and flow-specific strategies were identified for the EN governance boards:
CY2007/2008.NOB/NPRG: Develop messaging and transition materials to support state partners transitioning out of STORET.
CY2007/2008-ENLC/NOB: Communicate changes about STORET and the WQX roll-out.
CY2007/2008-NOB/Data Standards: Expand the ESAR data standards to flow biological and habitat data.
| CY07.Milestone | 20% |
| CY08.Milestone | 65% |
| CY09.Milestone | 100% |
| CY10.Target | 100% |
| Production Date | January 2006 |
| Outbound | Y |
| Denominator (# of State partner Flows) | 50 |
Associated Applications
A number of applications have access to WQX data through STORET. Some associated applications include:
- PNW Data Access Client
- Storet.org
