The Screening Sample Scam

However, a new breed of consultant, usually calling themselves a “Mould Inspector,” or a “Certified Indoor Air Quality” consultant relies exclusively on junk-science, and must ignore an half century of known science and more than a century of known mathematics and statistics, to ply their fear-based trade: Collecting meaningless “mould tests” and “mould samples” and convincing their victim that the “results” can be interpreted.

Although these instant “IAQ certified” inspectors (with the myriad unrecognized “certifications” to which they apply to themselves) have never gained any national or internationally recognized credibility among legitimate science-based communities, the steady decline in their credibility has been hastened by the anti-scientific and fear-mongering antics of their membership (see, for example, this advertisement from a Certified Mold Inspector which reflects the typical technical competence of the sample-collecting “Certified IAQ” consultant). Elsewhere, we have provided general discussions on the technical aspects of “mould testing” and the mould “sampling” typically conducted by these individuals.

However, now faced with mounting criticism, and the public’s increasing awareness of the uselessness of the “tests,” these poorly trained practitioners are changing their game-plan and claiming that although their samples may not be accurate, or precise, they are nevertheless “useful tools” and can be used as “screening samples.” In fact, such samples cannot be used in the manner claimed, and remain entirely meaningless, and merely a means for the charlatan to increase their revenue and impress their victim with a fancy laboratory report, replete with fancy Latin names and mysterious numbers, but otherwise devoid of any actual data.

Defining Terms
Environmental sampling and analysis generally is categorized as “qualitative” or “quantitative” and “subject” or “objective.”

Quantitative Analysis
In “quantitative analysis” a sample is collected and subjected to a method that can specifically identify and quantify one or more items under study (“analytes.”) Quantitatively analyzed samples exhibit high specificity and confidence with regard to the analysis. The information from the laboratory report is then incorporated into a larger decision process known as a “sampling plan” and interpreted only within the context of the parameters of the plan. For example, when an Industrial Hygienist is assessing benzene exposures in a work place, they begin by developing “data quality objectives” and may collect a sample pursuant to a “validated” method such as NIOSH Method 1501. Pursuant to this method, the scientist collects an air sample through a specially designed tube, and subjects the sample to an objective measuring instrument called a “gas chromatograph” for analysis. The instrument is an “objective” measuring device since it will operate in an identical manner regardless of the operator’s personal beliefs – as a machine, it will always produce the same results under the same operating conditions. The method specifically identifies the analyte (benzene), and then measures the mass (amount) of the analyte present. If one took 100 different instruments, and subjected the instruments to the exact same sample, the instruments would exhibit good “reproducibility” and each would positively identify the analyte, and the results from all 100 instruments would fall within predictable margins of error (accuracy and precision). The method has a known precision, accuracy and bias that is maintained and confirmed through a rigorous “quality-assurance/quality-control” program that allows the Industrial Hygienist to express their data with confidence.

If an Industrial Hygienist were to simultaneously collect ten such samples all at once, the Industrial Hygienist would receive from the laboratory ten results that would all be virtually the same; within the known limits of confidence. If the Industrial Hygienist used different labs for the analysis, or resubmit the same ten samples to another lab, the second set of results would exhibit virtually the identical distribution of results as the first set.

Qualitative Measurements
In a qualitative determination, the Industrial Hygienist accepts a compromise in confidence and uses a method that indirectly surmises the identity and/or the concentration of the analyte through an objective process.

Thus, if the Industrial Hygienist (IH) is studying benzene exposures, and he is collecting preliminary (screening) data to help him develop his final data quality objectives, he may select colormetric detection tubes. These are specially designed air sampling tubes through which air is drawn, and the analyte “objectively” reacts with a series of chemicals in the tube, producing a color change. The presence of the color “qualitatively” identifies the analyte, (called a presumptive positive), and the length of the color stain along the tube roughly correlates with the concentration of the analyte in the air.

Even for screening purposes, the IH will still be operating pursuant to predetermined “data quality objectives” (DQOs) by which he will interpret the data in a manner that will recognize the limitations of the device. The detector tubes themselves are manufactured to very strict standards and the accuracy and precision of the tubes are known in advance. The tubes are manufactured and respond according to international standards such as the ANSI/ISEA 104, the British Standard Institution BS5343 and the DIN3382 Standard issued by Deutsches Institute für Normung.

If one were to simultaneously collect ten such samples all at once, the investigator would get ten results back that would all be virtually the same; within the known limits of confidence. If ten separate people collected the same sample, the length of stain would be the same – the method is an objective detection method.

In spite of the known accuracy, precision, and strict standards to which the tubes are manufactured, the “sampling error” associated with the tubes is very high; meaning the results cannot be used to “represent” with confidence the actual anticipated exposure to the analyte in the air.

Therefore, if the IH wishes to use the data as a “screening” device to estimate exposures, then the IH would use the known parameters of the tube’s precision and accuracy and standardized statistical calculations ¹⁴ to figure out the “range” of values the tube reading could represent. The more samples the IH collects for a given study area, the smaller and smaller the sampling error becomes. However, to confidently work out the actual exposure with confidence would take many, many tubes over several days.

Gimmicks We have addressed snake-oil gimmicks such as the "Instascope," the "IAQ Pro 5-Minute Home Mold Test," and the "PRO-LAB® Test Kits" here.

Spore Traps
The type of sampler used by most “certified” mould inspectors is a “spore trap.” (This is also the type of sampler used by a legitimate Industrial Hygienist’s to estimate human exposures to moulds). Spore traps are not “quantitative,” nor are they “qualitative” and neither is the analysis even “objective.” Instead, in a spore trap, air is drawn through a cassette which loses most of the airborne spores below the “cut-size” of the trap, and loses 50% of all spores that have the same size as the “cut-size” (which just happens to be the same size as the most common types of moulds).

After the samples are collected, they are not plugged into a highly precise scientific instrument, rather the sample is looked at by an individual (hopefully a trained individual, hopefully a microbiologist), who huddles around her morning cup of coffee and peers through a microscope at the slide and will ultimately decide to write a report on what she sees. This person is known as the “reader.”

The “reader” is a person who will exclusively use human subjective observations about what they believe they are looking at. At no time is the slide subjected to any kind of objective analysis or highly scientific objective test or inserted into some high-tech instrument. Since the sample is analyzed subjectively, if the “reader” reviews the slide on Monday morning, she will produce one set of laboratory reports, and two days later, if she looks at the same slide again, her subjective assessment will be different, and she will produce an entirely different set of results – even though the slide is exactly the same slide. If the exact same slide is sent to another laboratory, the new reader will produce a completely different set of results; if a third, fourth and fifth reader is used, then each will produce their own unique laboratory report; each identifying a completely different spore count, and each identifying a completely different spore profile¹⁵ – none of which are actually capable of representing the actual spore concentrations in the air from which the sample was collected. In fact, remember that the spore trap is not capable of trapping a representation of the actual spores that are present.

As if these kinds of errors weren’t bad enough, the physics of particle collection conspire with the intrinsic subjective variability of the method to create an almost entire lack of confidence. It is for this reason that the laboratory “results” have no intrinsic meaning outside the constraints of the investigator’s written DQOs. One of those DQOs is to determine how many dozens of spore traps would have to be collected per day to characterize and overcome the sampling and analysis errors associated with the spore trap method. As it turns out, it has been estimated that hundreds of samples would have to be collected over several days from each study area to have sufficient confidence to estimate the exposure to the spores in the air with confidence

Let’s take a real-life look at the kinds of variations seen with these samples. In the photo below, we have presented the experimental apparatus used to evaluate “screening samples.”

In the apparatus, there are three commonly used spore traps; two identical Air-O-Cell samplers and one Andersen N6 sampler. In the study, the apparatus was set up in a variety of locations for study. The sampling devices were all started at exactly the same time, and stopped at exactly the same time. To minimize spatial variability due to stratification and negative bias due to air “scrubbing” by a nearby sampler, the air in each study area was (very) gently mixed prior to and during the collection of the samples. To avoid intra-laboratory error, all of the samples were submitted to the exact same laboratory, and analyzed by the same analysts. Even with these unusual efforts to reduce error, the results of the samples exhibited extreme variability, making the use of the spore trap as a screening method entirely unreliable. In addition to the “collocated” samples, multiple samples were also collected from each study area over the course of several hours, and in some cases, several days.

In the following table, we have summarized the results of the collocated sample data. Trap 1 and Trap 2 are the Air-O-Cell traps and Trap 3 is the Andersen sampler. The three columns on the right identified as “RPD” provide the “relative percent difference” between two data points being compared. Generally speaking, good reproducibility is observed if the RPD is not greater than 15%.

Screening Air Monitoring Data

Trap 1	Trap 2	Trap 3	RPD 1:2	RPD 2:3	RPD1:3
17700	9733	.	58	.	.
690	845	.	20	.	.
576	247	.	80	.	.
453	1200	.	90	.	.
4693	1280	.	114	.	.
1842	683	.	92	.	.
97	151	.	44	.	.
487	333	.	38	.	.
1243	1297	86	4	175	132
177	69	190	88	93	9
926	331	.	95	.	.
4947	432	.	168	.	.
210	1000	.	131	.	.
95333	604	.	197	.	.
164	822	.	133	.	.
33160	5077	.	147	.	.
355	136	.	89	.	.
500	226	91	75	85	150
597	1171	.	65	.	.
2266	744	.	101	.	.
97	25	.	118	.	.
1700000	1000000	.	52	.	.
1700000	1000000	1594	52	199	189
55840	10487	.	137	.	.
1105	474	933	80	65	21
81	29	.	95	.	.
226	323	115	35	95	50
81	194	226	82	15	87
258	53	.	132	.	.
54	9	.	143	.	.
214	38	.	140	.	.
767	302	.	87	.	.
1058	331	.	105	.	.
48	32	.	40	.	.
407	172	.	81	.	.
129	210	80	48	90	35
500	875	.	55	.	.
145	194	.	29	.	.
849	566	226	40	86	114
16	185	181	168	2	130
1274	177	.	151	.	.
387	181	.	73	.	.
427	157	.	92	.	.
181	214	.	17	.	.
484	694	292	36	82	39
193	201	.	4	.	.
81	57	.	35	.	.
419	500	.	18	.	.
242	194	.	22	.	.
260	60	.	125	.	.
309	291	582	6	67	69
1109	818	.	30	.	.
355	296	.	18	.	.
627	216	.	98	.	.
680	186	.	114	.	.
33	180	.	138	.	.
32	5	.	146	.	.
369	17	98	182	141	168
189	71	.	91	.	.
309	228	.	30	.	.
1806	3097	571	53	138	68
1016	726	.	33	.	.
263	333	.	23	.	.
1000	1117	.	11	.	.
161	185	.	14	.	.
444	181	.	84	.	.
926	331	.	95	.	.
123	7	.	178	.	.
403	286	.	34	.	.
22	148	.	148	.	.
147	80	86	59	7	58
177	102	.	54	.	.
915	305	.	100	.	.
81	189	.	80	.	.
189	54	.	111	.	.
194	157	.	21	.	.
568	186	.	101	.	.
162	87	.	60	.	.
946	129	.	152	.
351	203	.	53	.	.

As can be seen from the table, even two exact same samplers located next to each other and operating at the exact same time in the exact same room in the exact same air, and sent to the exact same lab for analysis, exhibited wildly different results. When the actual spore concentrations were estimated from the entire suite of samples for a given study area, none of the above results accurately predicted the actual spore concentration in the room.

What we see from the table, is that it is impossible to determine which of three simultaneous spore traps may actually be used to indicate the spore concentration in a study area, since one cannot with confidence determine which sample is “correct.” Indeed, if the apparatus was operated three minutes after the first set of samples was collected, and then three minutes later, each set of results would be wildly different. If the slides were to be sent to a different laboratory, the results would change again.

Imagine if a “mould inspector” collected a “screening sample” and (just like the real life example given above) the result was 95,333 spores per cubic meter of air, they may conclude the property had a serious mould problem; and yet if we look at the result of the simultaneous collocate (604 spores per cubic meter of air) the conclusion would be completely different – and yet both samples are from the same air, same room, same time, sample sampler and same laboratory.

In our experience, we have never encountered a “mould inspector” or an “Certified IAQ” consultant who understands the limitations of any kind of samples, sampling or spore traps. In our experience, the “mould inspectors” and other consultants running around collecting air samples for moulds are so poorly trained, they actually tell their client that the number that appears on the laboratory report represents the spore count in the house or the room from which the sample was collected. Or they foolishly tell their client, the results are “screening” results… whatever that means. In fact, the number on the laboratory report has no intrinsic meaning; and even the genus or the species that has been indentified in the report may not be correct.

When a legitimate Industrial Hygienist collects a screening sample for a particular parameter, they have foreknowledge of the error and the precision and accuracy of the result. The legitimate Industrial Hygienist knows that when using spore traps, there is absolutely no confidence in the data, and a good Industrial Hygienist can guess a spore concentration of a property within better limits of confidence than seen with a spore trap and a laboratory report.

If it becomes necessary for an actual objective assessment of airborne spore concentrations (something that is virtually never needed during a “mould inspection” of an house or during a legitimate IAQ investigation), the Industrial Hygienist will develop their own written DQOs that will describe the acceptable error, precision, and accuracy, and will determine how many samples will be required to identify the limits of confidence, and how many will be needed to confidently exhibit “representativeness” within objectively stated limits of confidence. This type of sampling will require dozens of samples to be collected over the course of many days – and will therefore be very expensive, typically running into several thousands of dollars. This is why the Department of Health of many states recommends against such sampling. For example the State of Colorado¹⁶ (and California) explicitly tells its citizens:

The Colorado Department of Public Health and Environment does not recommend testing as a first step to determine if you have a mold problem. Reliable air sampling for mold can be expensive and requires expertise and equipment that is not available to the general public.

Since such sampling virtually never provides any information that is not readily available through a visible inspection, knowledgeable Industrial Hygienists (and other legitimate mould assessment personnel) virtually never collect samples. Samples and “mould tests” are almost exclusively the hallmark of a toxic-mould charlatan or a very poorly trained consultant. It is for this reason the US Centers for Disease Control stated: ¹⁷

Other than in a controlled, limited, research setting, sampling for biological agents in the environment cannot be meaningfully interpreted and would not significantly affect relevant decisions regarding remediation, reoccupancy, handling or disposal of waste and debris, worker protection or safety, or public health.

Conclusions

When considering whether to hire a professional for a mould assessment, in your home or business, you should first ask if they are “certified” and next ask if they will collect samples. If the answer to either question is “Yes,” then you know you have probably contacted a poorly trained “toxic-mould-is-gold” consultant and need to look elsewhere to find a legitimate and knowledgeable consultant. The consultant who tries to tell their client (victim) that a spore trap result is a “useful tool” or a “screening sample,” is either trying to convince themselves, or they are trying to bamboozle their victim.

References

1NIOSH Occupational Exposure Sampling Strategy Manual, HEW Publication Number 77-173 (1977)

2Larsen R.I, A Method for Determining Source Reduction Required to Meet Quality Standards JAPCA, 11, 71, 1961

3 Larsen R.I, A New Mathematical Model of Air Pollutant Concentration Averaging Time and Frequency, JAPCA, 19, 24 (1969)

4 Phinney DE, Newman JE, The Precision Associated with the Sampling Frequencies of Total Particulate at Indianapolis, Indiana JAPCA, 22, 9, (1972)

5 Larsen RI, A Mathematical Model for Relating Air Quality Measurements to Air Quality Standards Office of Air Program Publication No. AP-89, U. S. Environmental Protection Agency, Nov. 1971

6 Breslin AJ, Ong, Glauberman H, et al, The Accuracy of Dust Exposure Estimates Obtained from Conventional Air Sampling J AIHA, Vol. 8, pp 56-61, (1967)

7 Sherwood RJ On the Interpretation of Air Sampling for Radioactive Particles Health Physics and Medical Division Atomic Energy Research Establishment, Presented at the AIHA Conference in Philadelphia, 1964 and appearing in its peer reviewed form in J of AIHA Vol. 27, pp 98-109 (1966)

8 Sherwood, RJ The Monitoring of Benzene Exposure by Air Sampling, J AIHA, Vol 32, No 12, pp 840-846, 1971 (DOI:10.1080/0002889718506547R)

9 Jones AR, Brief RS Evaluating Benzene. Exposure J of AIHA Vol. 32 p 610 (1971)

10 Gale HJ, Some Examples of the Application of the Lognormal Distribution in Radiation Protection Ann Occup Hyg Vol 10, No 1 (1967): 39-45. DOI: 10.1093/annhyg/10.1.39

11 Hounam RF An Application of the Lognormal Distribution to Some Air Sampling Results and Recommendations on the Interpretation of Air Sampling Data, AERE-M (HMSO, 1965)

12 P. C. LeClare, A. J. Breslin, and L. D. Y. Ong, Factors Affecting the Accuracy of Average Dust Concentration Measurements” AIHA, Vol. 30 pp 386-393, (1969)

13 Hald A Statistical Theory With Engineering Applications, referenced in Leidel, Busch, and Crouse Exposure Action Measurement Level and Occupational Environmental Variability U.S. Centers for Disease Control, National Institutes of Occupational Safety and Health, Dec. 1975

14 NIOSH Occupational Exposure Sampling Strategy Manual, HEW Publication Number 77-173 (1977)

15 Robertson LD, et al A multi-laboratory comparative study of spore trap analyses Mycologia, 103(1), 2011, pp. 226–231. DOI: 10.3852/10-017

16 Colorado Department of Public Health and Environment, Mold Information Sheet, August 2002

17 The CDC Mold Work Group, National Center for Environmental Health, National Center for Infectious Diseases, National Institute for Occupational Safety and Health, Centers for Disease Control and Prevention, October 2005

This page is a "living discussion;" originally created on May 30, 2012, it will be updated as necessary to reflect known science.

---

To return to our state-of-knowledge mould page, click here.

Visitors to this page generally have an interest in scientific issues. If you are interested in such matters, you may find some of our other discussions interesting.

A discussion concerning myths surrounding duct cleaning, can be found by clicking here.

For a discussion concerning indoor air quality, click here.

For issues surrounding the history and cause of carpal tunnel syndrome click here.

For a discussion concerning indoor radon click here.

For a discussion concerning the myths associated with laboratory fume hood face velocities click here.

For a discussion concerning laboratory fume hood evaluations, click here.

Finally, for a listing of documents associated with the State of Colorado regulations concerning methamphetamine laboratories (meth-labs), click here.

Visit our main page!

Forensic Applications Consulting Technologies, Inc.

Feel free to send Forensic Applications, Inc. an email directly by clicking here