Quality
Assurance
|
Proper
Quality Assurance and Quality Control (QA/QC) protocols are
essential to Lake Access.
We have gone to great lengths to assure the accuracy of our data
-
the following sections describe these measures in detail.
|
|
QA/QC
basically refers to all those things good investigators do to make
sure their measurements are right on (accurate; the absolute true
value), reproducible (precise; consistent), and have a reliable
estimate of their uncertainty. In the regulatory arena, this aspect
of data collection is as crucial to the final outcome of a confrontation
as the numbers themselves. It specifically involves following established
rules in the field and lab to assure that the sample is representative
of the site, free from outside contamination by the sample collector
(no dirty hands touching the water) and that it has been analyzed
following standard QA/QC methods. This typically involves comparing
the sample to a set of known samples for estimating accuracy and
by replicating the measurement to estimate its precision. The U.S.
Environmental Protection Agency has lots to add should you wish
to learn more of the technical aspects of a Quality Assurance Program
(QAP):
Volunteer Monitor's Guide to: Quality Assurance Project Plans
1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands,
Washington, D.C. 20460, USA
|
|
DATA TYPES
There are basically two sets of environmental data that are collected
for Lake Access:
(1) conventional
water quality parameters such as nutrients (N- and P-series
of nutrients), chlorophyll, clarity, fecal coliform bacteria, manual
field profiles for temp, DO, EC, etc. These are based upon traditional
methods where a trained staff person records measurements at different
depths from a sensor lowered over the side of a boat and collects
water from discrete depths that are returned to the lab for analysis.
|
| (2) remotely
sensed and controlled R.U.S.S. (Remote Underwater Sampling System)
units that control the depth and sampling interval of water quality
sondes housing depth, temperature, DO, pH, EC and turbidity probes.
Data may be transmitted via cellular phone/modem to our base computer/website
immediately upon completion of a depth profile, or may be stored on
board the RUSS and downloaded less frequently (each morning, currently)
to save connection costs. |
Conventional
data quality assurance procedures follow guidelines set by the U.S.EPA
(1987; 1989a,b), and APHA (1998). Water chemistry and manual field
profiles are collected by trained staff limnologists and technicians
at both Hennepin Parks (HP under Principal Investigator/Limnologist
John Barten's supervision) and the Natural Resources Research Institute
(NRRI under Co-Principal Investigator / Limnologist Rich Axler). Both
the Hennepin Parks Water Quality Laboratory and the NRRI Central Analytical
Laboratory are certified annually by the Minnesota Department of Health
for Federal Safe Drinking Water Act and Clean Water Act parameters
(Ameel et al. 1993, 1998; Axler and Owen 1994; Archer and Barten 1995,
1996; Barten 1997; MCWD 1997). The certification procedure involves
blind analyses of certified performance standards and an in-depth
site inspection and interview approximately every other year. The
NRRI lab has also been certified over the past decade by the Minnesota
Pollution Control Agency and the Minnesota Department of Natural Resources
for low-level water quality analyses in pristine, acid-sensitive lake
monitoring programs and for sediment contaminant analyses in the St.
Louis River and Upper Mississippi Rivers.
|
RUSS QA/QC is performed at a number of levels. The sensors are either
Hydrolab H20 or YSI 6820 probe/sonde instruments; both HP and NRRI
staff follow the Instrument Manuals for calibration and maintenance
procedures. Our staff also have extensive experience with these calibration
procedures and with their importance in interpreting field data and
distinguishing systematic errors associated with deteriorating, or
bio-fouled probes. Our Lake Access, EMPACT project is a companion
to an earlier NSF-funded Advanced Technology Education project entitled
Water on the Web (WOW)
, now in its third year, that deployed RUSS units on three Minnesota
lakes. In 1998 and 1999 we gained considerable experience in dealing
with problems associated with continuous sensor deployment; the resultant
protocols are included in our Lake Access efforts. Other aspects
of the data management process are discussed in Host et al. (2000a,
2000b).
|
NRRI contributed to the initial development of the RUSS technology.
During the preliminary and early stages of Water on the Web, numerous
tests were conducted in regard to the accuracy and precision of in-situ
data. Since both the YSI and Hydrolab systems are well established
and used for numerous state and federal monitoring programs, the principal
concerns related to the time allowed for sensor equilibration at each
depth . Of all the sensors that we use, dissolved oxygen is most susceptible
to erroneous values from inadequate stabilization- the error being
greatest in regions with steep depth-gradients in DO. Following our
collaborative work on this topic with Apprise
Technologies, Inc., the company subsequently ran a nearly yearlong
experiment in Lake Waco, Texas with Hydrolab,
Inc. comparing RUSS-transmitted data to conventional datalogger
data. The data sets agreed within sensor specifications. Both sensor
companies have internal quality control systems (YSI is ISO14001 registered)
that guarantee the consistent quality of their sensors. Apprise has
worked independently with both companies to integrate these sensor
packages with their RUSS units. As a part of these programs, the RUSS
technology was independently field-tested by both companies and both
YSI and Hydrolab have audited the Apprise facilities for QA/QC compliance.
Apprise has also implemented an internal quality system based on the
ISO9000 system and has been extremely helpful in dealing with problems
that occasionally arise with the Lake Access and WOW
units. A more complete description of our current protocols follows: |
RUSS SENSOR
RESOLUTION & REPORTING LIMITS
On the RUSS unit, the on-board computer processes a user-submitted
instruction sequence, the sensor package is sent to a specified depth,
and a series of feedback corrections are made until the sensors are
stabilized within 0.2 m of the specified depth. Output from the sensors
is monitored to assess when the readings on all parameters have stabilized
to a specified criterion, usually a coefficient of variation <20%
for a running set of 10 consecutive measurements over an interval
of ~1 minute. Dissolved oxygen typically requires the most amount
of time to stabilize on average, in part because of the occurrence
of steeper depth gradients for this parameter. Depending on the site
characteristics and the specific O2-sensor, as much as 3-5 minutes
may be required for complete equilibration. Once stabilized, readings
on all parameters are stored in buffer memory on the on-board computer.
The raw data stream is a simple string of comma-delimited ASCII text
containing a time signature, depth, and parameter values (Table 1).
|
Table 1. Output
from Lake Access RUSS unit on Halsteds Bay, Lake Minnetonka, MN, 6/4/2000.
|
Unit:
EMPT2 site: Halsteds Bay
|
|
Site
|
Sample
|
Sample
|
Depth
|
Temp
|
pH
|
EC
@ 25 C
|
O2
|
O2
|
Turb
|
|
|
Date
|
Time
|
(m)
|
oC
|
|
(uS/cm)
|
(mg/L)
|
(%
sat)
|
(NTU)
|
|
Halsteds
|
06/04/2000
|
00:10:58
|
1
|
18.3
|
8.4
|
406
|
10.0
|
107
|
11
|
|
Halsteds
|
06/04/2000
|
00:11:43
|
2
|
18.3
|
8.4
|
407
|
10.1
|
107
|
6
|
|
Halsteds
|
06/04/2000
|
00:13:34
|
3
|
18.2
|
8.4
|
407
|
10.0
|
106
|
3
|
|
Halsteds
|
06/04/2000
|
00:15:13
|
4
|
17.9
|
8.3
|
410
|
9.1
|
97
|
15
|
|
Halsteds
|
06/04/2000
|
00:17:04
|
5
|
17.6
|
8.2
|
411
|
8.0
|
84
|
5
|
|
Halsteds
|
06/04/2000
|
00:18:55
|
6
|
17.3
|
8.0
|
414
|
6.7
|
70
|
4
|
|
Halsteds
|
06/04/2000
|
00:20:34
|
7
|
16.7
|
7.8
|
419
|
4.9
|
50
|
9
|
|
Halsteds
|
06/04/2000
|
00:22:25
|
8
|
16.3
|
7.6
|
425
|
1.8
|
18
|
14
|
| To date we have
set the reporting limits for RUSS data based on instrument specifications
and prior knowledge of the magnitude of typical field variations.
This information is presented within the RUSS
data section of the Lake Access web site. The resolution, i.e.
the smallest reading shown for a particular parameter is likely to
be considerably lower than the error associated with differences in
time, with depth fluctuations, and with sensor drift and calibration
accuracy. Periodic examination of the RUSS data stream with Apprise
Technologies, Inc. has generally confirmed the estimated accuracy
reported below (Table 2). An important, and greatly underestimated
element of both the Lake Access and WOW projects has been to assess
the accuracy of these data by comparison with approximately biweekly
manual profiles. However, it is likely that the relative precision
of the data between depths within a water column profile and within
a few hours to a day will be better than from week-to-week. |
|
Ideally, if
all of the RUSS sensors behaved according to sensor-manufacturer's
specifications (Table 2) we could simply post the data on the Lake
Access web site and assume it is accurate to these levels. However,
except for temperature, all of the sensors require routine maintenance
and calibration. When using these sensors for manual profiling,
that is, visiting lake sites by boat, we always re-calibrate the
pH, EC and turbidity sensors using individual standard solutions
with known values, and the DO by air calibration. Experience has
taught us that the sensors remain stable during the course of a
sampling day.
|
|
Table
2. Reporting limits for RUSS sensor data (Hydrolab or YSI sensors)
|
|
Depth
(m)
|
Temp
(oC)
|
DO
(mg/L)
|
DO
% saturation
|
pH
|
EC
(uS/cm)
|
Turbidity
(NTUs)
|
|
Resolution
(what is reported by the RUSS sensors)
|
|
±
0.12
|
±
0.1
|
±
0.1
|
±
0.1
|
±
0.1
|
±
1
|
±
1
|
|
Estimated
Accuracy (what we really trust)
|
|
±
0.3
|
±
0.15
|
±
0.2
|
±
2
|
±
0.2
|
±
10
|
±
~3
|
However, when
deployed for continuous operation, as for the RUSS unit, the sensors
are colonized gradually by a biofilm of algae and less noticeably
by bacteria and fungi as well. As this material builds up, its
metabolic
activity interferes with the sensor's ability to accurately sample
the surrounding water. One can easily picture the effect of fine
filaments
of algae wafting intermittently between the electrodes of the EC
sensor or in the light path of the turbidimeter giving seemingly
erratic
values with wide swings as the sensors move up and down. An anomalous
spike in the Ice Lake EC data during July 1998 (see shaded region
in the Surface
Trends for Ice Lake on the WOW site), is a good example of this
effect and is the basis of a lab lesson (Increased Conductivity:
Are Culverts The Culprits? in draft). DO and turbidity
probes are most susceptible to these changes, followed by pH and
EC.
|
SENSOR MAINTENANCE AND CALIBRATION
Lake Access and WOW staff set up the following protocols to minimize
these biofouling and instrument drift effects to quality assure the
RUSS data:
* Clean and re-calibrate sensors frequently (about every 2
weeks) and perform manual profiles with an independent instrument
at the same time
* Compare independent manual profiles with near-simultaneous
RUSS data prior to cleaning (re-calibration). This provides assurance
that the previous period of data is accurate. We calculate test statistics
for each parameter as: |
and
 for each parameter.
They PASS according to rules in Table 2.
|
|
Table
2. Quality Assurance Criteria for RUSS Sensors
|
|
SENSOR
|
RPD
|
DELTA
|
|
Temperature
|
<
5%
|
<
0.2 oC
|
|
DO
|
<
10 %
|
<
0.5 mg O2/L
|
|
EC
|
<
10 %
|
<
5 uS/cm
|
|
pH
|
<
10 %
|
<
0.2 units
|
|
turbidity
|
<
10 %
|
<
5 NTUS
|
If the data "passes", it is considered acceptable for the
previous period. If not, we examine it in the context of our understanding
of the limnology of the individual lake and other data (nutrients,
chlorophyll, trends, etc.) and then either delete it from the database
or allow it to be posted. We have to be careful not to delete anomalous
data that may simply reveal real dynamic changes. The sheer volume
of data (218,720,430,742,644,316,434,172,687,130 values to date) has
been taxing and we lack the resources to always be as current as we
would like. In the interim, data are posted as provisional
. Dates of calibrations and these manual data are posted in the DATA
section of WOW and are available within easily accessible Excel
files - these will soon be posted on the Lake Access site as well.
The three Data Visualization Tools (DVTs) developed for Lake Access
and Water on the Web are also helpful in rapidly displaying the data
in a variety of formats to help identify anomalous data. We are currently
in the process of adding 'calibration date flags' to the control panels
of the Profile Plotter and
Color Mapper DVTs and to the
DxT Profiler to allow the user
to more easily keep track of calibration dates as the data stream
is being viewed.
The first year of WOW, 1998, taught us that we were understaffed for
the frequency of maintenance required for continuous RUSS operation
at Ice Lake and Lake Independence. With an additional three units
being deployed for 1999 and 2000, we set up collaborations with Itasca
Soil and Water Conservation District (for Ice Lake), Hennepin Parks
(for Lake Independence and Lake Minnetonka), the Minnesota Department
of Natural Resources Regional Fisheries for Grindstone Lake, and the
Minnesota Pollution Control Agency staff for the St Louis River site
(still in development as of July 2000). Lake Access and WOW staff
work with these folks to clean and re-calibrate all sensors approximately
every 1-3 weeks depending on the site. The less productive sites (Grindstone
and Ice lakes) generally require less maintenance.
DATA TRANSMISSION AND INITIAL QA SCREENING
The program that imports the RUSS data currently is scheduled to run
every day at 7:30 AM. The RUSS base station software is used to call
each RUSS and download data that has been collected since the last
call. A file containing real-time data (RTD) collected during the
duration of the call is also created. These new profile data and RTD
files are stored on the base station computer as plain ASCII text
files, one file for each day's data. The data files from each site
are stored in a separate directory on the computer. Table 1 (above)
is an sample of an original profile data file created by the RUSS
base station.
The Conversion Process
A program (the importer) is now launched. It reads data files that
have been created or changed since the last time it was run, and converts
the data to the format used by the report generating and data visualization
programs. Additionally, the original data files are copied to the
web server so they are accessible for immediate QA/QC. Profile data
files are copied to http://wow.nrri.umn.edu/data/ and RTD files are
copied to http://wow.nrri.umn.edu/rtd/ .
The importer parses the first line of a new or modified RUSS data
file and tests to make sure that the Unit and Site correspond to what
is expected. If not, an error message is generated and no further
action is taken with this file. This will catch errors that could
occur if, for example, a data file from Halsteds Bay was somehow stored
in the Lake Independence directory. Next, it reads the line containing
the column descriptions, and compares it with what is expected. If
it differs, an error message is generated and no further action is
taken with this file. This will catch errors that could occur if,
for example, a new parameter is being read by the RUSS, but the importer
hasn't been updated to handle the change. Now, each data line is read
and converted to a "Reading". A set of readings is combined to form
a "Profile" in the data base. Specific data is rejected by the importing
program if it is outside these ranges:
|
| temperature |
<
-1 or > 35 oC |
| pH |
<5 or > 10 |
| specific
conductance (EC25) |
<1 or > 600 uS/cm |
| dissolved
oxygen (DO) |
< -1 or > 20 mg/L O2/L |
| DO
% saturation |
<
-5 or > 200 % |
| turbidity |
< -5 or > 100NTU (note: turbidity values between -5 and 0 are
set = 0) |
|
There is no
direct indication in the raw data files of where one profile ends
and the next begins, so the importer applies some heuristics to
decide how to assign readings to profiles. The values listed below
are those in current use, but they can be changed. Since only the
actual time is reported on each data line, the importer assigns
a "scheduled" time to the new profile, using the nearest :00 or
:30 minute time value before the time reported for the first reading
in the profile. Subsequent readings are added to the same profile
provided that:
1) the reading is from a lower depth,
and
2) the reading was taken within 30
minutes of the previous one
When the importer either comes to a line where the reading no longer
qualifies for the current profile or it reaches the end of the data
file, it will add the new profile to the data base provided that:
1) the first reading starts within
3 meters of the surface,
2) there are at least 4 readings in
the profile, and
3) the date is not in the future
Instead it will generate an appropriate error message in the log
file, and disregard the profile. This helps eliminate partial or
invalid profiles that could be caused by RUSS hardware problems.
If it is winter and the RUSS is installed on ice, we set the minimum
upper depth to 1 or 2 meters to minimize the risk of the unit becoming
trapped in the hole through the ice. The data importer then creates
a default reading at 0 meters, listing a temperature of 0 oC, with
all other parameters blank (since we don't know what their true
values are). The time for this reading is set equal to the scheduled
time. The timestamp of each reading is expected to be unique, and
can be used as the key value in a database. There is the possibility
that the first actual reading in the profile could have the same
timestamp as the bogus reading, so the readings in the profile are
checked for duplicate times. If found, 5 seconds are added to the
time of the deeper reading, and the change is noted in the log file.
Sometimes a sensor for a particular parameter at a particular site
will go bad. In this case, the importer program can be customized
to reject that parameter when importing the data from the site.
Data stored in buffer memory is transmitted to the base station
via cellular phone at specified intervals, or at the request of
the user. Standard parity-based error correction techniques are
used to ensure that data were not altered during transmission. At
the base station, a JAVA based application adds the raw data to
a standardized relational database (DBMS) file. For archival purposes
the original ASCII data are stored in a compressed data format (ZIP)
file. The ASCII and DBMS files are periodically downloaded to an
off-site location via File Transfer Protocol (FTP).
FINAL DATA REVIEW & POSTING
At present (July 2000), funding limitations have precluded adherence
to a rigorous schedule for removing the provisional label from RUSS
data. In part this is a due to the need to review ancillary water
chemistry data before making final decisions when the RUSS data
is questionable. All water chemistry data posted on the Lake Access
and WOW sites however, have passed QA/QC prior to being posted,
although this typically takes from 30-60 days after collection.
Despite regular maintenance and calibration schedules, occasional
RUSS data anomalies still occur. To date, they have virtually always
been associated with DO and/or turbidity data although there have
been recurring problems with the pH probe at the WOW Grindstone
Lake site.
The most troublesome anomalies are those that occur within the
calibration window of time, are not flagged by our automated screening
tools
and are not unreasonable values in terms of the range of values
previously measured for that depth stratum and time of year. These
errors have not been trivial to identify and require careful examination
in a complete limnological (lake/watershed/climate) context by
a
professional limnologist. The process is adequately described as
Best Professional Judgement (BPJ). In some cases we have decided
to adjust data by calculating correction factors when there is
accurate
calibration data spanning the period in question and when the results
estimated by interpolation are consistent with the rest of the
data
set. In other cases we have simply rejected the data - omitting
it from the website. Data deletions are summarized and circulated
to all limnological staff and archived in a hidden section of the
Lake Access and WOW websites. The WOW project sends a periodic
e-mail
newsletter providing data updates to all teachers and researchers
using the site for educational or research purposes; you can subscribe
to this newsletter at http://waterontheweb.org/contactus.html.
SUMMARY
The QA/QC of
near-real time remotely collected sensor data has provided challenges
that were not present under traditional sampling regimes. We have
attempted to develop rigorous protocols for each step of the data
aquisition effort, and believe these protocols suit the needs of
projects such as Lake Access and Water on the Web. Nonetheless,
as these technologies become more common in resource management,
future efforts must be directed toward the unique problems posed
by real-time data collection.
ACKNOWLEGEMENTS
RIchard Axler,
Elaine Ruzycki, and Norm Will contributed to the development, testing,
and documentation of these QA/QC protocols.
REFERENCES
Ameel, J.J., Axler, R.P. and Owen, C.J. 1993. Persulfate digestion
for determination of total nitrogen and phosphorus in low-nutrient
waters. Amer. Environ. Labor. October 1993, p.1-11.
Ameel, J., E. Ruzycki and R.P. Axler. 1998. Analytical chemistry
and quality assurance procedures for natural water samples. 6th
edition. Central Analytical Laboratory, NRRI Tech. Rep. NRRI/TR?98/03.
APHA. 1998. Standard methods for the examination of water and wastewater.
American Public Health Association, Washington, D.C.
Archer, A. and J. Barten. 1995. Quality assurance manual. Hennepin
Parks Water Quality Laboratory. September 1995. Hennepin Parks,
3800 County Road 4, Maple Plain, MN 55359.
Archer, A. and J. Barten. 1996. Laboratory Procedures Manual. Hennepin
Parks Water Quality Laboratory. October 1996. Hennepin Parks, 3800
County Road 4, Maple Plain, MN 55359.
Axler, R.P. and C.J. Owen.1994. Fluorometric measurement of chlorophyll
and phaeophytin: Whom should you believe? Lake and Reservoir Management
8:143-151.
Barten, J. 1997. Water quality monitoring plan. Hennepin Parks,
3800 County Road 4, Maple Plain, MN 55359.
EPA. 1987. Handbook of methods for acid deposition studies-Laboratory
analysis for water chemistry. EPA/600/4-87-026
EPA 1989a. Preparing perfect project plans. US EPA Risk Reduction
Engineering Laboratory, Cincinnati, OH, EPA/600/9-89/087.
EPA.1989b. Handbook of methods for acid deposition studies-Field
operations for surface water chemistry. EPA/600/4-89-020.
EPA. 1996. Volunteer
Monitor's Guide to: Quality Assurance Project Plans. EPA 841-B-96-003,
Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460,
USA (http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm)
Host, G., N. Will, R.Axler, C. Owen and B. Munson. 2000a.Interactive
technologies for collecting and visualizing water quality data.
URISA Journal (In Press; refereed: http://wow.nrri.umn.edu/urisa)
Host, G.E. , B. H. Munson, R. P. Axler, C. A. Hagley, G. Merrick
and C. J. Owen. 2000b. Water on the Web: Students
Monitoring Minnesota rivers and lakes over the Internet. AWRA
Spec.Ed. (Dec., 1999). (refereed: www.awra.org/proceedings/www99/w74/index.htm.).
MCWD. 1997. Quality assurance - quality control assessment report.
Lake Minnetonka Monitoring Program 1997. Minnehaha Creek Watershed
District, 2500 Shadywood Road, Excelsior, MN 55331-9578.
|
| |
|
