My post Details on the Dirty Dozen on EWG’s Shopper’s Guide to Pesticides™ led me to dive into the 2008 USDA data to see just how contaminated (or not) our produce really is. There’s so much information that it’s a little difficult to work with, but with perseverance and the right software (JMP is the best!*), I was able to re-do the EWG analysis but with the newest available data.
Below you can find my results with a through explanation of what I’ve done and why. The results are posted without all the commentary at Produce Pesticide Rankings which has all of the results and Pesticide Produce Rankings Tables which has comparisons of my results to the EWG results. You can download the original USDA data yourself or check out the Latest PDP Findings of Interest to Consumers.
Concentration and LOD
My first step was to compare the detected Concentration to the Limit of Detection. The LOD seems to have been ignored by EWG. The LOD is the smallest concentration of the chemical you are looking for that will give a positive signal with the method used. Every method/chemical combination has a different LOD that can be found by comparing a blank (no chemical) to smaller and smaller concentrations of the chemical. If the detected concentration is at or below the limit of detection, it does not indicate the chemical is present – which is not the same as saying the chemical is not present. The chemical could be there, but the amount is so small that it can not be detected with the method being used.
Let’s put some numbers on it. There were 1,780,365 tests conducted on 13,381 samples (including fruits, vegetables, fish, nuts, and water), with 33,426 of those tests having a concentration listed (1.88%). Of those, 273 were equal to the LOD leaving 33,153 positive concentrations (1.86%). Not a big difference, but still, it would be incorrect to include the concentrations that are below the LOD. In a lot of experiments a blank is subtracted from the results and I can’t think of a reason why that wouldn’t be appropriate here. So, I created a column of Concentration minus LOD and used these numbers for my calculations.
Units
Most of the tests have a unit of ppm (parts per million), but some are ppb (parts per billion) or ppt (parts per thousand). I converted ppb to ppm (ppb/1000=ppm) and ppt to ppm (ppt*1000=ppm) so all of the average residue values would be in the correct units.
It would be inappropriate to average values with different units. As illustrated by the Alaska Department of Environmental Conservation, ppm is drops per bathtub while ppb is drops per swimming pool!
Comparing the 2008 data with EWG
Because this investigation was inspired by EWG, let’s go through their Spreadsheet column by column to compare the top five values of each. You can find this information in Pesticide Produce Rankings Tables. The 3 types of water tested by USDA top most of the lists in the 2008 data, but since this discussion is on produce, they aren’t included here.
Percent of samples tested with detectable pesticides
This isn’t really a good metric because it doesn’t take into account which of the detected residues are above or below the EPA tolerance level and the EWG numbers don’t take the LOD into account (the numbers I report are all Concentration – LOD), but nonetheless here’s how they stack up.
- % of samples with 1 or more residues: 95.78 Peaches, 95.55 Celery, 95.24 Nectarines, 94.06 Strawberries, 92.75 Catfish.
- % of samples with 2 or more residues: 89.74 Celery, 88.66 Strawberries, 86.04 Peaches, 80.65 Nectarines, 72.22 Blueberries.
- EWG % of samples tested with detectable pesticides: 97.20 Plums, 96.20 Peaches, 95.10 Bell Peppers, 95.00 Celery, 93.60 Apples.
- EWG % of samples with two or more pesticides: 85.70 Peaches, 84.70 Celery, 82.30 Blueberries, 80.60 Bell Peppers, 74.40 Apples.
As you can see, the percentages don’t vary much from the collection of data used by EWG to the 2008 only data. Some of the foods tested in previous years weren’t tested in 2008 (apples, bell peppers).
A lot of the samples for each commodity have 1 residue, fewer have 2, fewer have 3, and so on. For some perspective, consider the percentage of all tests done on all samples for each commodity that had one or more residue.
- % of tests with 1 or more residues: 1.18 Nectarines, 0.92 Collard Greens, 0.90 Summer Squash, 0.83 Kale, 0.79 Almonds.
Average number of pesticides found on a single sample
This is a little more useful than the percent of samples with one or more residues, but not by much, since we’re still leaving out consideration of the EPA tolerance.
- Mean residues detected per sample: 5.15 Celery, 4.94 Strawberries, 3.61 Blueberries, 3.50 Peaches, 2.46 Spinach.
- EWG Average number of pesticides found on a single sample: 3.79 Celery, 3.08 Peaches, 3.00 Blueberries, 2.90 Strawberries, 2.75 Apples.
The USDA lets us know in their Latest PDP Findings of Interest to Consumers that the number of samples with pesticides and number of pesticides per sample doesn’t correlate to pesticides per serving size because the sample sizes were a lot more than a serving. “Sample size ranges from 16 ounces to 5 pounds depending on food tested. For example, for peaches and celery, the sample size is 5 pounds; for strawberries and blueberries is 3 pounds and 1 pound respectively.”
In regards to number of pesticides per sample, the USDA states: “There may be many more pesticides available for use by food producers, but 20 years of testing show that no food has ever been treated with all available pesticides.”
Average amount of pesticides found in ppm
This might be the worst metric of all because it averages pesticides that have very different toxicity levels. One ppm of one pesticide can be very different from one ppm of another pesticide! Still, here’s where we start to see some real differences!
- Mean ppm residue by commodity: 0.8 Potatoes, 0.61 Spinach, 0.37 Rice, 0.35 Nectarines, 0.33 Sweet Potatoes.
- EWG Average ppm of all pesticides found: 1.602 Potatoes, 1.373 Spinach, 1.200 Plums, 1.066 Peaches, 0.906 Red Raspberries.
The EWG shows average ppm of pesticides that are twice what I’ve got from the 2008 data! What’s happening here? One possibility is that EWG didn’t convert the ppt to ppm, but surely they’d notice the different units in the data, so it must be something else. We could have done the averages differently, but that’s unlikely too, it’s just averaging.
The only other thing I can think of is that there were high levels of residues in the past, high enough to skew the overall averages. If this is true, then we have something to celebrate – there have been great reductions in pesticide residues over the years!
Still, this brings up a question: why would the EWG tell people that produce has such high amounts of pesticide residues when produce today actually has much less? If the goal is to tell people what are the safest foods to buy for their families today, why include old data?
The USDA states specifically in their Latest PDP Findings of Interest to Consumers that there have been significant changes over the years, with reduced number of samples with pesticides and reduced ppm of pesticides. Specifically, there have been reductions in the most harmful pesticides as safer alternatives have been approved for use.
Maximum number of pesticides found on a single sample
Again, this metric does not take the EPA tolerances into consideration, and the results are about the same..
- Maximum residues detected per sample: 14 each Strawberries and Celery, 12 Blueberries, 11 Catfish, 10 each Spinach, Collard Greens, and Peaches.
- EWG Maximum number of pesticides found on a single sample: 13 each Blueberries, Strawberries, and Celery, 11 Bell Peppers, and 10 Kale.
.
*Thanks to my husband for explaining that it makes a lot more sense to keep the test data and the sample data in two separate tables that you join when needed based on the sample number. Having all the data in one JMP file is about 8mB which doesn’t work all that well even on a good computer.
Biofortified 
I sort of feel bad that this isn’t more exciting. There’s no “gotcha” no “scary”… it’s just real.
LikeLike
I’d say there’s a “gotcha” here… the EWG isn’t all it’s cracked up to be. I’ve used their cosmetics database before, but now I’m thinking those rankings are totally bogus too. The vast majority of the products they recommend are organic. Hmm…
LikeLike
This is a great post, like the previous one.
The European Food Security Authority (EFSA) recently published its second annual report on pesticide residues in food here , with a press release here. In short:
. Over 70,000 samples from some 200 kinds of food were tested to look for up to 862 different pesticides;
. 96.5 per cent samples conformed to the legal Maximum Residues Levels (MRLs) or, put the other way round, 3.5 per cent exceeded the limits;
. No pesticide residues were detected in 62.1 of the samples;
. Of the 2.062 baby food samples, 76 contained traces of pesticides (note that the MRL is a a more stringent flat 0.01 mg/kg for all residues) and only 4 (0.2%) exceeded the limit, among them two organic ones.
. For some ‘odd’ reason, the data for organic food do not distinguish between samples without detected residues and samples below the MRL; we are therefore to be content with the information that 0.9 per cent of the samples of organic food were above the MRL. However, it seems to me that the tests did not pertain to the pesticides that are autorised in organic agriculture (some of which, although “natural” are known to be a source of concern in terms of public health). Some countries did not provide any data due to – if you believe it – “deficiencies in the data management system implemented at national level”. Those included Austria and Italy, two countries with a large share of agricultural land under organic.
MLRs should not be understood as toxicological safety limits, with the result that any excess would entail a health risk for consumers. On the contrary, MRLs are set far below the levels that could possibly pose a risk. The report states that “even an exceedance of an MRL, does not imply that this is a safety concern …”
To assess consumer risk, EFSA estimated chronic (long-term) exposure to pesticides from major foods that make up the diet of Europeans and acute (short-term) exposure for nine types of crops which were monitored in 2008 as part of the EU coordinated program. As regards long-term exposure, it concluded that none of the evaluated pesticides raised health concerns. For acute exposure, using a worst-case scenario – people eating large portions of foods containing the highest recorded residue levels – EFSA said that for 35 pesticide/commodity combinations a potential risk could occur but only in rare cases.
Overall, and despite the fact that the testing methods improve year on year, the results are better than the previous ones (for example, 3.5 per cent samples over MRL in 2008, as against 4.2 in 2007). Sadly, many media chose to scaremonger and communicate on the fact that a total of 365 pesticide residues had been detected.
Pesticides are used to fight the plant diseases, the insect infestations, etc. which attack our food and can produce, or cause the plants to produce, molecules that are a health hazard for consumers. At the end of the day, the choice for consumers is between, on the one hand, (relatively) cheap food from conventional agriculture that is safe (except in rare cases) in terms of pesticide residues and, on the other, (in certain cases much) more expensive food from organic agriculture that is also safe (but perhaps marginally) in terms of the same pesticide residues. In Europe and using the EFSA results, it is accepting a 3.5 per cent risk of exceedance or paying a high price for a 0.9 per cent risk which comes with a higher, so far unmeasured, risk in terms of “natural” pesticides and a higher risk in terms of natural toxins.
LikeLike
Thanks for this analysis of the EFSA data!
One thought I’ve had with the EPA tolerance levels and same with the MRLs,
if “even an exceedance of an MRL, does not imply that this is a safety concern …”
then why not set the guidelines at the limit where it actually becomes dangerous? I understand that whole 10x safety factor and all that good stuffs (as Joe brings up below) but it seems that these unnecessarily low tolerance levels just serve to confuse individuals and becomes ammunition for groups like EWG when they see amounts that are near the tolerance level.
LikeLike
“then why not set the guidelines at the limit where it actually becomes dangerous?”
Dangerous is a tad too nebulous to be messing with for any toxin – 10x lower is probably good enough, but imo the lower the better (within reason) – setting the limit too close to what is actually dangerous is risky as all hell – imagine the impact (in real terms of people getting sick or whatever) if the MRLs were anywhere near the same values as the point where pesticide concentration poses a viable risk – as soon as you exceed the MRL you’re causing harm – while I don’t normally agree to proposals which err too far on the side of caution I think that in general I’d agree that keeping MRLs well below actual harmful levels is a good idea – it also leaves you open to punish companies/individuals who exceed the MRLs before anyone is actually close to being exposed to harm – which makes the whole industry focus on staying super safe, rather than just safe enough (and thus any errors, while potentially subject to legal action, aren’t actually dangerous unless they are catastrophic in scale)
LikeLike
I understand the value in setting limits overly high, but it leads to so much confusion. For example, the tolerance limit for permethrin residue on spinach is 30ppm. If one finds a residue of 20ppm on a given spinach sample, one might think “that’s so close to 30ppm, so this must be really bad” when the level that’s actually dangerous might be closer to 300ppm or 3000ppm. Perhaps there just needs to be better reporting of what the tolerance levels mean and better discussion of dose from groups like EWG (I wish) so people didn’t freak out about things that are essentially harmless.
LikeLike
The reporting perhaps does need to be better – perhaps a tolerance limit of x ppm safe limit of y ppm reported or suchlike.
I’d still rather that idiots and ideologues be able to confuse issues and have a system that locks down safe levels, than have a system which is easiliy understandable in which levels of pesticides etc can be remotely close to what is dangerous.
LikeLike
Just to give you an idea of how these tolerances are set…the EPA’s minimum allowable amount of pesticide is still a couple thousand times smaller than the minimum level which causes any detectable harm. Just because we can detect pesticides or pesticide residues on food doesn’t necessarily mean we’re ingesting a harmful amount of pesticide.
LikeLike
“(JMP is the best!*)”
Yes. I assume the * denotes only a p value of 0.05 for this statement – seems a tad low, did you have too few reps?
LikeLike
Good one. I just had to publicly thank Brandon for helping me figure out how to deal with the data. I have tried other stats software, though, and JMP is so much better than R or SAS.
LikeLike
As interesting as this topic may be, it’s actually business as usual for the Environmental Working Group.
“EWG … releases “scientific” analyses designed to make the public (especially parents) worry tremendously about tiny amounts of pesticide exposure from fruits and vegetables. *** [Its] agenda (and that of its wealthy financial backers) is “to cripple agribusiness altogether in favor of ‘organic’ alternatives.”
http://activistcash.com/organization_overview.cfm/o/113-environmental-working-group
See also, “Scientists Denounce Scaremongering Activists”, CCF (September 20, 2005), http://www.consumerfreedom.com/news_detail.cfm/h/2886-scientists-denounce-scaremongering-activists
LikeLike
Anastasia–I just saw this, thought you might find it interesting:
Pesticide Politics
By JAMES MCWILLIAMS
http://freakonomics.blogs.nytimes.com/2010/10/20/pesticide-politics/
LikeLike