American Community Survey: 2008 Update: Defining Disability

In previous posts, I have looked into the prevalence of disabilities in the United States, as measured especially by the American Community Survey (ACS), especially in 2003. This post takes a first step toward updating the ACS measurement of disabilities, in light of changes made between 2003 and 2008.  The focus here is on the definition of disability under the 2008 ACS.

As of 2008, the ACS 2008 Subject Definitions (p. 38) defined disability as “the restriction in participation that results from a lack of fit between the individual’s functional limitations and the characteristics of the physical and social environment.” The Census Bureau asserts that “disability is not seen as intrinsic to the individual”; nonetheless, the Bureau also identifies “four basic areas of functioning” in which respondents might have “serious difficulty.” In that sense, the language still seems to reflect an impression that disabilities are the individual’s problem, rather than a mutual “lack of fit.” For instance, one might expect a different outcome if respondents were presented with a broader inquiry into functional limitations perceived either by themselves or by others in their lives (e.g., employers, neighbors, family members).

In what the Subject Definitions (p. 38) describe as “a conceptual and empirical break from earlier years,” the 2008 ACS still had six questions about disabilities, but now sought to identify “serious difficulty in four basic areas of functioning: vision, hearing, ambulation, and cognition,” and also included “two questions to identify people with difficulties that might impact their ability to live independently.”

These changes followed from a 2006 American Community Survey Content Test, whose results in the area of disability were written up by Brault and Stern (2007) in an Evaluation Report Covering Disability. The 2006 Content Test was driven by a concern with improved reliability in survey results (Brault & Stern, p. 9), not by a concern with capturing the full numbers of people with disabilities. To the contrary, Brault & Stern admit that some of the changes in the ACS disability questions tend to reduce reported prevalence (p. 8) and that the 2006 Content Test used data collection modes that contained “inherent barriers to collecting disability data” (p. 15). The latter point is important because the regular ACS appears to use essentially the same methods.

In a separate report attached as Appendix B to the Brault and Stern (2007) report (above) (i.e., beginning at page 33 of that PDF), Miller and DeMaio (2006) indicated that, for purposes of consistency with the Americans with Disabilities Act (ADA), an unspecified ACS subcommittee determined that disability would be defined as “a mental or physical impairment that substantially limits at least one major life activity” (p. B-2).  That subcommittee appears to have approached that question by exploring “domains of limitation” or “limitation domains” (pp. B-2, B-3), instead of taking the seemingly obvious step of deciding what counts as a major life activity.

To cite some rather choice examples, Miller and DeMaio did not indicate that the subcommittee considered sleep, sex, or socializing to be major life activities.  All three of those examples seem to involve the Census Bureau’s alleged orientation toward the two-sided question of fit between person and society.  It is society that supplies the people, the stresses, and other key components and determinants of sleep, sex, and socializing.  One would expect, in other words, that these three would be prime candidates for inclusion in the list of major life activities, or as essential contributors thereto.  For instance, there is evidence that sleep deprivation significantly interferes with work and other major daily activities of tens of millions of Americans (Baxter & Kroll-Smith, 2005, p. 39), that it results in substantial numbers of injuries and deaths (e.g., Lyznicki, Doege, Davis, & Williams, 1998), and that one-third of Americans take naps on a typical day (Taylor, 2009).  Although it is presumably not necessary to do so here, one certainly could develop a more extensive argument for sleep, and could make similar arguments for the inclusion of sex, socializing, and other activities that the subcommittee somehow considered not to be "major" for purposes of detecting the existence of a disability.

The subcommittee's decisions in this regard might have been more defensible if the ADA had been construed as taking an interest in major productive activities.  Viewed that way, the subcommittee could reasonably conclude (especially in a traditional industrial economy) that it is all right if workers are tired, as long as they keep working.  Likewise, if industrial-type productivity were the stated focus, attention to sex and socializing would probably seem humorously superfluous.  The full list of impairments that mattered to the subcommittee -- involving vision, hearing, lower body mobility, cognition, activities of daily living (ADLs), instrumental activities of daily living (IADLs), and work (Miller & DeMaio, 2006, p. B-3) would then make sense.  ADLs “generally include self-care type activities such as bathing and dressing,” and IADLs are activities often associated with independent living such as going out alone to shop or visit a doctor’s office” (Brault & Stern, 2007, p. 4), all of which align with an emphasis upon the worker's functionality and his/her ability to remain functional.

Ironically, contrary to the apparent purpose of the ADA, a functional orientation can reinforce stereotyping and exclusionism.  If the majority defines functionality in terms of what the majority can do, then those in the minority will tend to occupy an inferior role.  It will be newsworthy when technology enables them to overcome – say, to run faster than those who began with what was, by definition, a functional advantage (e.g., Christie, 2009).  In reality, there may be many ways (involving e.g., sleep, sex, or socializing) in which many putatively high-functioning people are experiencing profound limitations in major life activities; but that will not be recognized in the ACS until a standard other than industrial-type functionality is used to guide disability determinations.  There would seem to be a number of good, currently relevant candidates for such a standard – ranging from “life, liberty, and the pursuit of happiness” to social capital to adjustment to or satisfaction with life – that could be used alone or in combination for that purpose.

The apparent functional orientation was not the only way in which the 2008 ACS reduced the count of disabilities even below what one might have expected from the subcommittee’s own stated concept. The survey discarded the subcommittee’s broad interest in mental and physical impairments, in favor of very limited kinds of difficulties. The survey’s only questions about mental disabilities were as follows:

18a. Because of a physical, mental, or emotional condition, does this person have serious difficulty concentrating, remembering, or making decisions?

19. Because of a physical, mental, or emotional condition, does this person have difficulty doing errands alone such as visiting a doctor’s office or shopping?

Bizarrely, the 2008 ACS no longer has even the question about work disability (Brault & Stern, 2007, p. 7). That is, there is no question about the presence of a physical, mental, or emotional condition that would limit the kind or amount of work a person can do. So if someone has no problem concentrating etc. (Question 18a), and does not have difficulty doing errands alone (Question 19), it seems s/he would not have a work disability even if s/he spent the workday standing on his/her desk and screaming.

Apparently the question about work disability was deleted because Miller and DeMaio (2006, p. B-32) claimed there was an “excessive extent of misinterpretation and misreporting in this question.” One objection Miller and DeMaio cited had to do with vagueness in the word “work.” Some people reportedly took it to include work around the house. That raised interesting questions in itself, but in any event one version of the test question specified work “at a job or business” (p. B-31). Despite the alleged breadth and complexity of the work construct, Miller and DeMaio devoted only about 1.5 pages to the question of work before recommending its deletion.  (While I did not think that disabilities should be defined exclusively in work-related terms, it did seem obvious that inability to work would be an important indicator of disability.)  That is, Miller and DeMaio did not seriously wrestle with the matter; yet they also did not claim that the question of work-related disability is unimportant.

Bear in mind that Miller and DeMaio (2006), who balked at what they consider excessive vagueness in the work construct, were the same researchers who pointed out that “Respondents’ answers to survey questions are necessarily based on personal experience and perceptions of that experience” (p. B-3), that “it is unfeasible to develop questions that yield a perfect measure of disability” (p. B-4), and that, in connection with a question that was retained in the survey, respondents were being required “to discern a clear line of yes or no in a reality that, for them, was essentially grey and multi-dimensional” (p. B-7). All of these remarks seem to suggest that Miller and DeMaio were generally comfortable with the hard decisions that arise in the process of quantifying experience – including work experience.

Under such circumstances, one might be forgiven for asking whether political considerations during the Bush Administration had any influence upon the choice of researchers and/or the recommendations they made in such regards. In a report on changes to measurement of disability in the 2008 American Community Survey that was produced after Obama’s election, Brault (2009, p. 5) seems to indicate that the previous administration had not prioritized the ACS generally. He says that, “due to budgetary issues, FEFU [i.e., Failed Edit Follow-Up] operations were scaled back from April through September” of 2008.

In his 2009 report, Brault states that the net effect of the changes in questions on the 2008 ACS, including the deletion of the work disability question, resulted in a marked drop in the estimated number of Americans with a disability, from 41.2 million (2007) to 35.9 million (2008). “Given the differences in the questionnaire,” Brault says, “one should not interpret these changes as real differences in the number or percent of people with disabilities” (p. 16).  Brault emphasizes that ACS disability data from 2008 are not comparable with ACS disability data from prior years.

This is not to say that the 2008 ACS is useless for all disability-related purposes. Brault (2008, p. 8) suggests, for example, that the ACS is more likely to capture instances of hearing difficulty than is the Survey for Income and Program Participation (SIPP), because the SIPP excludes hearing difficulties that are corrected by hearing aids. That particular example is not necessarily good or bad in itself, but it does suggest that the ACS may sometimes provide a useful alternate picture on the prevalence of some kinds of disabilities.

There are some other limitations on ACS data.  First, ACS data of all sorts are not yet available for geographic areas with populations of less than 20,000.  In late 2010, according to the Census Bureau, data will be released “for areas as small as census tracts and block groups, nationwide. For very small towns, 2010 will be the first time that ACS data are published. These data products, referred to as 5-year multiyear estimates, will be based on the data accumulated during the 2005-2009 time period.”  The Bureau also says that information on demographic, socioeconomic and housing characteristics will thereafter be available annually for all areas.  These include annual estimates for all areas of 65,000+ population, three-year averages for all areas of 20,000+ population, and five-year averages for census tracts and block groups.  Note also that, starting in 2006, the ACS did begin to include data for the group quarters population.  These data were not available, however, in 15 states that had a total group quarters population of less than 45,000.

Because of the change in the operational definition of disability in the 2008 ACS, the Census Bureau will not be able to provide three-year estimates for disabilities until the 2008-2010 ACS 3-year estimates are released in 2011.  That is, the change in definitions means that users should not compare disability data from 2008 and afterwards to any year before 2008.

Recap So Far

In this series of posts on the prevalence of disabilities in the U.S., I have been moving toward an estimate of disabilities on the local level.  It seemed logical to begin by choosing a national-level survey that would provide local-level data, and then to understand that survey and, on that basis, to interpret its local data.  Preliminarily, the ACS seemed far superior to other federal surveys, in terms of its penetration to the local level.  It did not define disabilities as comprehensively as some other surveys, but it seemed that it might be possible to adjust its reported prevalence rates upwards by some plausible percentage, so as to reach a defensible ballpark figure for actual local disability prevalence.

Unfortunately, this post's critique of the definition of disabilities, and of its operationalization in the 2008 ACS, suggests that the ACS does not provide accurate data on the prevalence of disabilities.  Hence, before conducting further research into federal disability statistics, it presently seems advisable to pursue some other routes.  Several such routes come to mind.  First, there is surely a great deal of relevant third-party literature on the ACS, as distinct from the literature provided by the Census Bureau itself.  Second, there may be good local-level research on the prevalence of disabilities in various types of cities and counties, sufficient to inform an educated guess on prevalence in other similar local areas.  Third, on a more theoretical level, there is no doubt much to learn about the conceptualization of disability -- on, for example, the validity of viewpoints suggested in this post.  Since the conceptualization seemed likely to influence the interpretation of research, my next step in this investigation moved in the conceptual direction.