Public Lab Research note


Evaluating Low-cost Lead Screening Products

by read_holman | April 16, 2019 16:32 | 198 views | 3 comments | #19078 | 198 views | 3 comments | #19078 16 Apr 16:32

Background

A number of products are commercially available that allow for the low-cost screening for the presence of lead.

These are easily found and available online and in department stores. These products are packaged and sold at around the 10-30 dollar price-point. Each package may include one or a handful of tests. They are marketed to consumers — parents, citizens, and the like — who are seeking to better understand whether or not there is lead in their environment. Typically these individuals are seeking to test for the presence of lead in the paint in their home, paint on an old dresser or piece of furniture, dust in a windowsill, or for the presence of lead in their home drinking water. Some people also test for lead on other items, like pottery or dishware.

Currently, you can only buy products to screen for the presence of lead in paint (painted surfaces), dust, and water. There are no low-cost commercial products for the on-site testing of lead in soil.

These low-cost screening products fill an important part of the lead testing marketplace. Other options are much more expensive (from hundreds of dollars) or require working with your local government (which may have a backlog based upon priority).

What is the reliability of these products? What should consumers know about these products before purchasing them? Are there particular cases where they are more reliable? Certain conditions in which they are less reliable? Is there variation across the products, with some better than the others? This note is devoted to exploring these questions.


Note: If you are looking for the various ways to test for lead in your home, these low-cost screening products are only one option. Other options include working through your local government, hiring a contractor, and using a mail-in-service. See the following for a complete list: How to Test for Lead in Your Environment.


How These Products Work (Methods)

These low-cost screeners deploy a colorimetric approach to screening. Low-cost screeners for lead-in-paint tend to require a pen or pens (swabs) that are rubbed onto the surface in question. Those in water use a piece of litmus paper that is set within a cup of the water that is in question.

Whether paint, dust, or water, the concept is the same: The presence of lead would cause a chemical reaction to take place that changes the color of the testing device. This change in color is compared against a standard to determine if there is reason to be concerned or not.

Here are examples of what results look like:

image description

For this product (3M's LeadSwab) which tests paint/surfaces, if it turns red, this indicates the presence of lead. Photo courtesy Jana Martin.

image description

For this product (WaterSafe's Lead Test) which tests water, you compare the two lines that show up. If one is darker than the other, this is supposed to indicate a positive finding. Photo from Amazon user R Young.

Results are subject to interpretation. These pictures show results that are fairly easy to read: The red is very red! The one line is definitely darker than the other! Importantly, these types of tests are not always easy to interpret. Their results are qualitative and subject to user-interpretation. Indeed, in 2007 Consumer Reports and the Consumer Product Safety Commission, conducted the same tests but came up with different results; the difference was explained simply by the interpretation of the colors (the evaluations were of paint/surface screeners).

Results benchmarked against legally allowable limits, not health. These are screening for whether or not there is lead above the regulatory limits. This is particularly applicable to testing in water (the "allowable" amount of lead in paint in miniscule). For example: The EPA's "Action Level" for the lead in home drinking water is 15 parts per billion (ppb). The lead-in-water screening tests claim to compare against this 15ppb standard (source: here and here). However, all lead is bad; there is no safe amount of lead. These are not designed to detect for low-level presence of lead.


Commercial Products Available

Taken together, there are around 12 products commercially available on the market. These are listed below (note: we attempt to be exhaustive; however, our approach to finding these merely relied on our ability to use search engines and navigate the web.)

Most of these products have not been formally evaluated. Those that have been evaluated are noted.

Paint/Surfaces...

  • ESCA Tech, Inc. D-Lead Paint Test Kit. This product was "EPA-recognized" in 2010, but for negative results only; the rate of finding false-positives is 16% (Source PDF))
  • 3M LeadCheck Swabs. This product was "EPA-Recognized" in 2010, but for negative results only; the rate of finding false-positives is 98% (Source PDF)). This is an extremely high false-positive rate.
  • PRO-LAB Lead Surface DIY Kit
  • First Alert LT1 Premium Lead Test Kit
  • ChemSee Detection Kits for Lead

Dust...

  • ESCA Tech, Inc. D-Lead Dust Test Kit

Water...

  • Baldwin Meadows 14-in-1 Drinking Water Test Kit
  • JNW Direct 15-in-1 Test Strips
  • Health Metric Heavy Metals Test
  • Test Assured Lead in Drinking Water Test
  • First Alert Drinking Water Test
  • Silver Lake Research WaterSafe Home Drinking Water Test for Lead

Of note: Two additional products were evaluated by the EPA in 2010. Those products are ANDalyze Lead-in-Paint Test Kit and the Silver Lake Research LeadAVERT Test Kit. However, neither product met the RPP criteria; so neither was "recognized" by the EPA in any way. (Source PDF and Source PDF, respectively). Coincidentally, neither appear to be on the market and available for sale today.


What Does the Government Say About These Products?

The EPA has a formal “Lead Testing Kit Evaluation” website devoted to its Environmental Technology Verification program. This program evaluates exactly these types of products against standards established by the Renovation, Repair, and Painting (RRP) Rule, a rule applicable to paint/surfaces but not for water.

The RRP criteria (simplified) requires that a negative response less than or equal to 5% of the time must be met; and that a positive response less than or equal to 10% must be met. The products that fail to meet either of these RRP criteria thus have high false-negatives and high false-positives. That is: They aren't reliable. A couple of those are noted in the above list.

A couple products are EPA Validated for negative results only. These products are D-LEAD, which is included in our list shown above. And a product manufactured by Hybrivet. Hybribet, in 2011, was acquired by 3M. The 3M LeadCheck, as is the D-Led product, is included in the list above. These products are the only ones that have been "EPA-recognized" though only for their negative results; the rate of false-positives is high.

Indeed, as noted on the above mentioned EPA Lead Testing Kit Evaluation website:

" [The EPA states that] to-date, no product has met the standard for both negative and positive findings per the RRP. "


What Does the Internet Say About These Products?

Consumer-facing outlets

Blogs, media articles, and the like discuss these products in a colloquial way. These are non-conclusive at best and misleading at worst. They generally lack original research, relying instead on other research (or lack thereof) in discussing these products. All articles note that a professional test is the best way to go, or at least a mail-in service such as that provided by your local government.

A few example write-ups:

Government Websites, Programs, and Reviews

EPA evaluations on specific products are listed above in the list of products. A couple other government sites:

Peer-reviewed Literature

An environmental scan of peer-reviewed literature found very little.

Citizen/Community Science

One effort by a member of the Public Lab community yielded the following concluding statement:


Conclusion: What to Make of These Products

Summarizing the above:

  • Generally: There are known issues with the reliability of these types of products: They are prone to giving incorrect results (false-positives and false-negatives); and are subjective to user-interpretation.
  • Of the 7 on-site lead-in-paint screening products we found for sale: Only 4 of them have been formally evaluated (in 2010). Of those, 2 were found to be "EPA-recognized", though only for their negative findings.
  • 3M Lead Check Swab, the most commonly cited and most readily available product on the market, is one of these. With this product, negative findings can generally be trusted. Positive findings, however, cannot. Indeed, the false-positive rate is extremely high at 98%, making this product difficult to justify using even as a screener.
  • We found 1 lead-in-dust product for sale. It has not been formally evaluated.
  • We found at least 6 lead-in-water screening products for sale. None have been formally evaluated.

All this to say: Use with caution. More research is needed.


An Argument for a Formal (Community-Driven) Evaluation?

These conclusions argue for renewed attention to these products by research and governmental institutions.

Lacking that attention, the Public Lab community may conduct an evaluation of products on its own. We won't be so scientific as to equal the EPA's Evaluation Technology Program, but we hope to be rigorous enough in our methodology to be useful. And perhaps to spur more formal evaluations by larger institutions.

If you're interested in this idea, please let us know in the comments below or by emailing read@publiclab.org. We’re seeking input and conversation on this idea (broadly) and on the approach to evaluating these (more specifically).



3 Comments

Hi @read_holman! Sorry to respond slowly here.

I do think we could try for an accessible evaluation. How much would it cost to buy 2x of each of these methods, and what might an evaluation look like?

Also, what would we consider the "standard" test?

Thanks for this listing and idea, Read!

Is this a question? Click here to post it to the Questions page.

Hey @warren - Just realizing I never replied to your comment.

I definitely think a small accessible evaluation (haven't heard that phrase before, but I think I understand what it means) would be useful to the world. I've been thinking about what the design of that could look like, and... well much depends on how rigorous we want our methods to be. For example...

Let's say we bought 10 of each of these. We recruited volunteers to test a home or real place (which we know has lead, or at least we know has a high risk of lead being present). And then we let them use these. We then pay for an official tester so that we know the 'truth' to compare it against... That'd be ideal as we'd be testing these with real users in a real world setting. BUT this would be a lot to coordinate.

A smaller effort would involve something a bit more lab-like and controlled. We have a painted surface and water with known quantities of lead. Finding those things to test against is a rate determining factor here, but is doable. Actually maybe we do this first, and then see if there's momentum building or an argument towards the larger one?

Is this a question? Click here to post it to the Questions page.


Hi @read_holman - just dropping some links in here after chatting!

Very cool rigorous field evaluations of many many air sensors:

http://www.aqmd.gov/aq-spec/evaluations/field

I think I described it as "empirical and egalitarian" haha - they test $100 things and $10,000 things the same way, and even DIY projects off Kickstarter are assessed as "seriously" as a high-end system. I like the idea that nobody gets special treatment.

And the Oil Testing Kit Beta Program I mentioned is here: https://publiclab.org/wiki/oil-testing-kit-beta

I'll send you a copy of the booklet!


Reply to this comment...


Login to comment.

Public Lab is open for anyone and will always be free. By signing up you'll join a diverse group of community researchers and tap into a lot of grassroots expertise.

Sign up