| IMPAIRMENT TESTING- DOES IT WORK?
TABLE OF CONTENTS Introduction Appendices APPENDIX A – EMPLOYERS EXPERIENCE WITH IMPAIRMENT TESTING APPENDIX B – LIST OF COMPANIES THAT HAVE MARKETED IMPAIRMENT TESTING SYSTEMS APPENDIX C – LIST OF EMPLOYERS THAT HAVE USED IMPAIRMENT TESTING APPENDIX D – REASON FOR INITIATING IMPAIRMENT TESTING APPENDIX E – IMPAIRMENT TESTING USERS QUESTIONNAIRE INTRODUCTION Impairment testing is the practice of determining which workers in safety sensitive positions put themselves and others at risk by directly measuring workers’ current fitness for duty. Urine testing, in contrast, attempts to determine which workers have used specific substances known to cause impairment in the relatively recent past. Experts have long recognized that impairment testing has inherent advantages that make it potentially superior to urine testing as a method of improving workplace safety. This study set out to learn whether impairment testing has lived up to this potential in practice. The author collected information from every known user of impairment testing systems that was willing to cooperate. This information indicates that impairment testing improves safety, is accepted by employees, and is generally superior to urine testing. However, the number of respondents is relatively small, and these results must be considered preliminary findings rather than conclusions. EXECUTIVE SUMMARY Experts have long been known that impairment testing has inherent advantages that make it potentially superior to urine testing in both improving safety and protecting employee privacy. Until now, however, no one has known whether existing impairment testing systems actually work well in practice. Despite the fact that impairment testing has now been in public use for over a decade, there has been no effort to examine the experience of employers who have used it. While impairment testing ought to do more to reduce industrial accident rates, no one knows whether or not it actually does. The same is true of impairment testing’s impact on productivity and worker morale. In fact, it is unknown whether or not impairment testing works at all in practice. This has been a serious gap in our collective knowledge about impairment testing that hinders the growth of impairment testing. The objective of this study was to study the experience of employers who have used impairment testing and learn how well it works in practice. We specifically wanted to learn whether it does more to promote safety and productivity than urine testing. Collecting information about the performance of impairment testing proved extremely difficult because the field is so small. Only a handful of companies have ever marketed impairment testing systems and there is no list of their names. However, the Institute conducted an extensive networking program based on our contacts in the field that identified what we believe to be every company that has ever marketed impairment tests. There are only 10 such companies. Of these, only 6 manufactured systems for employers. Three of these 6 are now out of business. This means that there are only 3 companies currently in business that provide impairment testing systems for employers. By contacting these employers, we were able to identify 18 employers who had used impairment testing. Of these, 14 employers participated in our study. One employer that had used impairment testing is now out of business. The remaining 3 employers declined to participate. The primary reason these employers used impairment testing was a desire to improve safety. Almost all of these employers faced serious safety problems if employees came to work impaired. For one, the risk came from handling molten metal. Other employers were fire departments, police departments, construction companies, or private security companies. Most of these employers realized that drug testing had severe limitations and wanted something better. Some employers also wanted to find a less invasive system, although this was generally a secondary motivation. The picture that emerges from their experience is striking:
One must always be cautious when drawing inferences from such a small body of evidence. It would be unjustified to state at this time that impairment testing works in all situations, or that it has been proven to improve industrial safety. But the evidence does show that impairment testing is more than a good idea. Impairment testing systems from several manufacturers have been used by 18 different employers, in a wide variety of industries over a span of 10 years. The experience of these employers demonstrates that impairment testing works in practice, not just in theory. COMPLETE REPORT I. INTRODUCTION Impairment testing has the potential to protect workplace safety far better than drug testing because it indicates:
Impairment testing also appears to be less invasive of employees’ privacy because it:
Moreover, these benefits come at no additional cost. Urine testing requires no initial investment, but there is a charge for each individual test. Impairment testing requires an initial investment in hardware and software. The variable cost per test, however, is virtually nothing. For a very small company, impairment testing might be more expensive. But for a company of any significant size (greater than 100 employees) the money saved by eliminating the cost per test generally exceeds the cost of the initial investment, making impairment testing less expensive than urine testing. This is true, however, only if impairment testing actually works. It is not enough for impairment testing to be superior in theory. For impairment testing to become established it must work in practice. Those who support impairment testing (for financial reasons or reasons of principle) must be able to demonstrate that it functions successfully in the real world. Until now, however, no one has ever systematically gathered information about the field performance of impairment testing, much less analyzed this information to see what conclusions it yields. Without this information, employers who are interested in impairment testing will be reluctant to move forward. This study strived to collect all available information concerning the use of impairment testing in employment situations and determine how well it worked. II. METHODOLOGY The methodology we planned to use was as follows:
return to top of document III. IMPLEMENTATION A. Locating Impairment Testing Companies To identify the companies which market impairment testing systems, we conducted extensive telephone networking with academics, government officials (especially the United States Department of Transportation), and the management of impairment testing companies of which we were aware. This research disclosed that there have been 10 different companies in the field of impairment testing. Of these, only 3 are currently marketing systems for workplace use. They are:
The remaining 7 companies have either gone out of business or do not produce a product for the workplace. A list of all 10 companies that have marketed impairment testing systems can be found in appendix B. B. Identifying Employers Which Have Used Impairment Testing The fact that so many companies which marketed impairment testing systems were no longer in business was a major obstacle. The companies that were no longer active included two of the three largest suppliers, Essex and Performance Factors Inc. (the oldest and largest performance testing company). The three companies still active had too few customers to provide any meaningful information, even if all of them were willing We therefore expanded our search by attempting to identify the customers of impairment testing companies that are no longer in business. We were able to locate the management of two such firms. The first, Essex had completely disbanded, and no business records still existed. With the second company, PFI, we were more fortunate. We learned that the company’s assets had been acquired by an organization named T.A.S.A.L. It’s president, Paul Gregorie, shared the customer records he could locate with us. This entire process disclosed that the number of employers that had experience with impairment testing was far lower than we had assumed. Our exhaustive search, identified only 18 employers that had any experience with impairment testing. (See Appendix C for a list of these employers.) We believe that few, if any, additional impairment testing marketers have ever existed. Having such a short list of employers to work with required changing the information collection process. Pursuing the original plan of sending surveys and asking employers to fill them out would have produced a data set too small to be informative. Instead, we wrote to each employer explaining the project and it’s importance and asking them to participate. We then followed up on each letter by telephone, repeatedly where necessary. This allowed us to secure a participation rate of 83% (15 out of 18 companies). In some cases the employers were willing to talk to us on the telephone, but were not willing to fill out a research survey. In these situations, we conducted telephone interviews. In one case, we traveled to the employer’s place of business and conducted the interview in person. The following is a short summary of the information we received for each company. The entire completed questionnaires (or interview notes) are on file with the National Workrights Institute. Alameda Fire Department (Alameda, California) The Alameda, California fire department used the Bowles-Langley system on an experimental basis for a two month period during 2000. This was motivated by concerns about safety. The fire chief reported that the system performed well and may have increased safety by giving fire fighters better recognition of their fitness for work and how to avoid impairment. The firefighters were initially very skeptical of the system, but they came to accept and value it. Alcoa (Rockdale, Texas) This is an aluminum production facility. They used the PMI impairment testing system for two years as a pre-shift safety screen. They consider the program a success. Accidents went down. Management believes that impairment testing is partially responsible. There is no data or opinion on productivity. The management team that installed impairment testing considers it superior to urine testing. These managers, however, were promoted or transferred over the years, and new management had no personal investment in the system. Asset Protection (Langhorne, Pennsylvania) They are in the business of providing security guards. Asset Protection started using impairment testing in 1996, using the Essex System. Management began testing because they perceived this as something their customers would consider valuable, rather than because they themselves believed it was needed. They test guards only for clients who request this service. They consider the program a success and continue to use it. The nature of the service they provide does not lend itself to measuring safety or productivity, and they have no hard data. California Maritime Academy (San Diego, California) The Academy used impairment testing (Bowles-Langley) on an experimental basis for cadets during sea training. This was a one-time program. The faculty considered impairment testing useful, primarily in helping cadets understand the links between off duty living and their ability to do their jobs safely. The Academy does not have any data. Chamberlain Contractors (Laurel, Maryland) Chamberlain is a construction company. They have been using impairment testing for over 10 years, primarily with operators of heavy equipment. Chamberlain uses the PMI system. Management considers the program highly successful. Accidents have decreased between 50 and 75 percent since the company began using performance testing. Crown Enterprises (Baton Rouge, Louisiana) Crown supplies construction and repair personnel to petrochemical facilities and refineries. They have conducted impairment testing since 1994, utilizing Eye Dynamics. Management considers impairment testing to be part of their drug testing program, along with urine testing. They believe the program has worked, but don’t compare the relative contribution of what they consider two parts of the same program. The employees prefer impairment testing. Eagle County Ambulance District (Edwards, Colorado) This employer provides ambulance services for the county. They used impairment testing from PFI for 4 years and considered it a success. They did not notice any change in their accident rate after beginning impairment testing, but this may be a result of their small size (less than 50 employees). The District also uses urine testing. The employees prefer impairment testing. Management believes impairment testing is superior to urine testing. Despite this success, the District discontinued using impairment testing because it never acquired national acceptance. Metro Machine (Norfolk, Virginia) Metro Machine repairs ships, largely for the military. They started impairment testing in 1993 (using PMI). The goal was to improve safety, particularly by cutting down on lunchtime drinking. While they did not keep records, management believes that accidents were reduced. They are convinced that the amount of lunchtime drinking went down. They also use urine testing, by federal mandate, and believe it is less effective. They discontinued impairment testing in 1997, when sales and profits declined sharply due to a cut in military spending. North Slope Bureau (Barrow, Alaska) North Slope Bureau is a regional authority in northern Alaska. They used impairment testing from PFI in the past, but discontinued using it several years ago. No one in current management remembers how well impairment testing worked or why they stopped using it. Police Department Study. Four metropolitan police departments engaged in a joint evaluation of impairment testing in 1997-1998. They were: Arlington, Virginia; Lowell, Massachusetts; Polk County, Florida; Portland, Oregon. Collectively, these departments employ almost 2,000 officers. The test equipment came from PMI. This program was different than others in that it was voluntary and the results were not used as a pre-shift screen. Instead, researchers attempted to determine if impairment testing results were a reliable indicator of fatigue and a good predictor of accidents. This is of great concern to police departments because non-traditional hours produces great fatigue, and the potential consequences of reduced coordination or judgment can be catastrophic. Researchers investigated fatigue by having officers report the length and quality of their sleep the night before taking the test. Officers who were sleep deprived were considered to be fatigued. Researchers then examined the impairment test scores of both fatigued and non-fatigued officers. It was found that test scores were a good (though not perfect) indicator of fatigue. Researchers then examined the test scores of officers who had accidents or injuries while on duty. Officers whose test scores indicated that they were fatigued were much more likely than other officers to have on-duty accidents or injuries. These results strongly suggest that impairment testing was both accurate and predictive. They indicate that performance impairment would have reduced accidents had it been used as a pre-shift screen. However, since the technology was not used in this fashion, one cannot say that it did reduce accidents. STAR Center (Dania Beach, Florida) STAR is the other major maritime training school. They used Bowles-Langley impairment testing equipment on a temporary basis with cadets in conjunction with the bridge simulator. Their written report did not indicate how well impairment testing worked in practice. Company officials denied our request to discuss their experience in more detail. Supershuttle (Arlington, Virginia) Supershuttle is a transportation company. It began using impairment testing in 1998, primarily due to safety concerns. Supershuttle chose the PMI system. Accidents have decreased since then. Management feels that this is in part because testing has made employees more aware of impairment and its effect on safety. Supershuttle also uses urine testing, and would continue to use it even if they were not required to do so by federal law. Management does not believe, however, that urine testing has reduced accidents, and they consider impairment testing to be superior in promoting safety. Tritech, Inc. (Los Angeles, California) Tritech performs third party construction and repair services. They began using impairment testing from Eye Dynamics in 1994 because they were concerned about the safety risks of workers performing welding and other hazardous tasks on the late evening shift without supervision. The system worked well for them. The company is relatively small. Hence, the accident rate is so low that variations are very difficult to discern. But company officials believe that impairment testing is better that urine testing in protecting safety. In 1995, the nature of their business changed. Instead of performing repairs itself, Tritech began to merely supervise the work of other organizations’ employees. This eliminated their need for testing. R.F. White (Upland, California) White is a petroleum products distributor. It has used Factor 1000 since 1989. Management began using impairment testing because they wanted to test their 14 drivers every day and this was not possible with urine testing. Accidents of all types decreased by over 50% after impairment testing was installed. White also uses urine testing. They are required to do so by Federal law. Management believes both types of tests are valid. R.H. Wyner (Shawmut Mills Division) (West Bridgewater, Massachusetts) Shawmut Mills is a textile company. They began using impairment testing systems from PFI in 1995 because management believed that injuries were mounting due to worker errors. Over time, the number of positive tests for impairment declined 50%, and accidents decreased as well. The company still uses impairment testing. It does not use urine testing because management believes that such testing is invasive and is of little value in preventing accidents because the information usually comes too late. return to top of document IV. ANALYSIS Our original plan was to conduct formal statistical analysis of the data we collected. The small number of employers using performance testing (and the fact that some provided only interviews instead of filling out the entire questionnaire) precluded taking this course. Several trends, however, were so strong that they are meaningful even with such a small sample. These include: A. Overall Success Generally, many users of a new product will conclude that it doesn’t work and that their experiment with the product has been a failure. This is not the case with impairment testing. Of the 15 employers from whom we received information, 13 were able to state whether or not they considered impairment testing to have been a success. Every one of these employers considers their experience to be a success. Not one company said that impairment testing had failed. This is especially noteworthy in light of the fact that these employers used systems from 5 different manufacturers. This conclusion seems at odds with the fact that 7 of the 13 employers that found impairment testing successful no longer use it. However, there were factors other than the product’s performance that affect whether or not a company continued to use it. In the case of Tritech, for example, the nature of the business changed in a manner that eliminated the need for testing. Even more important are management changes. As a bold new idea, impairment testing needs an in-house champion. When management changes transfer the champion to another position, impairment testing is generally terminated regardless of how well it is working. Alcoa is a prime example of this phenomenon. B. Improved Safety Companies that provide impairment testing and privacy advocates perceive its primary advantage as improving safety. The information we collected indicates that this is the view of employers as well. Of the 13 companies that were able to provide their reasons for beginning to use impairment testing, 12 (92%) mentioned safety as a primary goal. For 9 companies, improved safety was the only goal. (One company mentioned both safety and productivity.) A list of reasons given by participating employers for initiating impairment testing can be found in Appendix D. Even the one testing program where the employer did not give improved safety as a goal in adopting impairment testing may be rooted in safety concerns. As discussed, Asset Protection views impairment testing as a marketing device, rather than as a way to increase safety. While management does not know the reason that many of their customers want their security guards tested, it is quite possible that the reason is safety related. Employers who began impairment testing to improve safety were rarely disappointed. Of the 11 companies who were able to report on their safety record, 9 (82%) saw improvements. In some cases, the improvements were dramatic. For example, R.F. White achieved a 67% reduction in motor vehicle accidents and a 64% reduction in workers’ compensation claims. In some cases, management was able to observe how and why the workplace became safer. Donald Fisher, vice president of Metro Machine, noticed that workers were less likely to drink at lunchtime when they knew that they might be tested for impairment before starting work in the afternoon. C. Productivity We anticipated that increased productivity would have been the second major objective of employers who used impairment testing. Logic suggests that, even if an impaired worker did not have an accident, that he or she would be less productive than if they were at their best. This turned out not to be the case. The employers in our sample did not have productivity in mind when trying impairment testing. Only one company (Chamberlain Contractors) listed improved productivity as a goal. Nor did our respondents report increased productivity as a benefit. This is somewhat surprising. As discussed above, our employers generally reported that impairment testing reduced accidents. If workers were able to do their jobs with fewer accidents with impairment testing, how could they not have been more productive? The time saved from reduced accidents alone should have had some effect on productivity. The apparent answer to this puzzle lies in the fact that few, if any, of the employers with whom we spoke attempt to measure productivity. They did not report that productivity had remained the same after the introduction of impairment testing. Rather, they responded that they did not know whether productivity had changed at all. Under these conditions, no inferences can be drawn about the influence of impairment testing upon productivity. What we were not told is also significant. One of the concerns which has hung over impairment testing since its introduction is the “bottleneck problem”. If an employer has a large workforce which all begin work at the same time, and a limited number of testing stations (dictated by the high cost of each station) it might lead to large numbers of employees standing in line for extended periods of time. This would represent both a significant waste of expensive employee time, and a loss of production. In the transportation industry, such bottlenecks could cause schedule disruptions. However, none of the 15 participants in our study even mentioned this phenomenon, much less considered it a major problem. This does not mean that employers can safely ignore the issue. But it does show that bottlenecks can be avoided. D. Privacy Employers’ desire to protect employees’ privacy also turned out to be a motivating factor in the decision to use performance testing. This was something of a surprise. When we began the study, we believed that the desire to improve safety and productivity were the only significant reasons employers choose impairment testing. We, therefore, did not include questions about privacy on our research instrument. However, when we experienced difficulties in getting employers to fill out the questionnaire, we began interviewing them instead. Of the 7 employers we interviewed, 3 spontaneously mentioned that concern for employee privacy was one of the reasons they chose to use impairment testing. None, however, indicated that privacy was their primary concern. E. Employee Reaction Prior to this study, no information was available regarding how employees feel about impairment testing. Privacy advocates hypothesized that employees would like impairment testing because it is not invasive. Nevertheless, we were concerned that employees would resent impairment testing when they failed for reasons having nothing to do with misconduct. In the companies we studied, employee reaction to impairment testing was positive. Of the 11 companies that were able to provide us with input on this point, all but 2 found that employees preferred impairment testing to urine testing. Of the other 2, one employer found no difference in employees’ reaction to the two forms of testing. Only one company found that employees preferred urine testing. This does not mean that employees accepted impairment testing immediately. Often, the initial reaction was skepticism. Acceptance grew, however, as employees found for themselves that the system worked. For example, at Tritech, employees conducted their own unofficial evaluation of the system (taking the test at times when they knew they were, or were not, impaired). After discovering that the impairment testing system accurately reported their condition, the employees accepted it. Nor does it mean that employees like impairment testing. Taking the test may be a great deal like playing a game, but that doesn’t mean that employees enjoy it. In truth, many employees would prefer not to be tested at all. But employees apparently recognize that testing is necessary in safety sensitive situations. And the employees in at least the companies we studied clearly preferred impairment testing over urine testing. F. Comparison to Drug Testing Most of the employers from who we received information had experience with drug testing. Of the 15 companies, 9 had used urine testing at some time. Comparing the relative performance of impairment testing and urine testing was not easy for our respondents. One employer explicitly declined to make any comparison, saying that impairment testing and urine testing were complementary parts of a single program. But where comparisons were made, the results were striking. Of the 8 employers who expressed an opinion, 7 (87%) found impairment testing to be more effective. One factor that was never mentioned in comparing impairment testing and drug testing is cost. The cost structure of impairment testing is quite different than that of drug testing. Impairment testing requires an initial investment in both hardware and software. After this, the variable cost per test is virtually nothing. Drug testing, by contrast, requires no initial investment, but there is a variable cost per test which declines little with increasing quantity. G. Causes of Impairment Our study also gave us a glimpse of the causes of impairment from an employer’s standpoint. Academic research has consistently found that fatigue is the major cause of accidents, and that illegal drug use is among the least common causes. Our input from employers was entirely consistent. Of the 7 employers who responded to this question, 4 reported that fatigue was the most common cause of impairment. The remaining 3 employers named illness, alcohol, and drugs as the most common. H. New Approaches to Testing We began the study with a model of impairment testing as a pre-employment screen. According to this model, workers in safety sensitive jobs would take the test prior to beginning their shift, and those who failed would be assigned to other duties. One employer we interviewed, however, the Alameda Fire Department, used impairment testing in a very different manner. Instead of using testing as a screen, they used it as an educational device. Workers were allowed to start their shift regardless of their test scores. But the workers knew when the system viewed them as impaired. Workers then have the opportunity to observe their own job performance and perhaps observe signs of impairment they would otherwise have missed. This approach also allows workers to modify aspects of their behavior (such as how much they sleep) to avoid being impaired at work. Our study gives no indication of the relative worth of this alternate model. But it is a creative approach with great potential of which we were previously unaware. Table 1- Summary of Findings
return to top of document V. CONCLUSION Few employers have used impairment testing, and information concerning that experience is very limited and extremely difficult to obtain. The available information, however, indicates that impairment testing is not just a better answer on paper, but in practice as well. Employers who have used impairment testing consistently found that it reduced accidents and was accepted by employees. Moreover, these employers consistently found that it was superior to urine testing in achieving both of these objectives. APPENDIX A – EMPLOYERS EXPERIENCE WITH IMPAIRMENT TESTING
(1) “N/A indicates that the question is not applicable to the employer’s business. APPENDIX B – LIST OF COMPANIES THAT HAVE MARKETED IMPAIRMENT TESTING SYSTEMS
return to top of document APPENDIX C – LIST OF EMPLOYERS THAT HAVE USED IMPAIRMENT TESTING
return to top of document APPENDIX D – REASON FOR INITIATING IMPAIRMENT TESTING
return to top of document APPENDIX E – IMPAIRMENT TESTING USERS QUESTIONNAIRE
|