Notes from "Counting Working-Age People with Disabilities"

Book Discussed

Houtenville, A. J., Stapleton, D. C., Weathers, R. R. II, & Burkhauser, R. V. (Eds.). (2009). Counting working-age people with disabilities: What current data tell us and options for improvement.  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

*  *  *  *  *

I read several chapters of this inexpensive book.  This post presents my notes on (as distinct from a review of) those chapters.

*  *  *  *  *

Preface

I had previously noticed that Cornell had a website with a lot of information on disability statistics.  This preface explains why.  The U.S. Department of Education’s National Institute for Disability and Rehabilitation Research (NIDRR) awarded Cornell a grant for a Rehabilitation, Research, and Training Center (RRTC) (which Cornell called StatsRRTC).This book grew out of a conference on disability statistics research in Washington, DC in October 2006.

*  *  *  *  *

Chapter 1

Stapleton, D. C., Houtenville, A. J., Weathers, R. R. II, & Burkhauser, R. V. (2009). Purpose, overview, and key conclusions.  In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 1-26).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

The title's focus on working-age people follows from an earlier article by Weathers in which he identified the 18-64 age group as being the traditional group of working age, and further pared away the 18-24 group as being in a school-to-work transition stage, and the 62-64 group as being in a work-to-retirement transition stage.  As in that earlier article, this chapter (p. 6) focuses on the group of people aged 25-61.

The authors identify (p. 9) a number of reasons why state-level data on the prevalence of people with disabilities (PWDs) are important -- why, that is, federal data do not capture important local variations.  Some of the reasons include different physical, cultural, economic, and policy environments.  They refer (p. 13) to the National Disability Data System (NDDS), which does not exist in any formal sense (although it should) but can be understood, at present, as the aggregate of a number of disparate data collection and analysis efforts on federal and other levels.  For instance, data from administrative records suggest that only about half of the total number of PWDs estimated by the American Community Survey (ACS) are actually enrolled in federal programs that provide assistance to PWDs (p. 15).

One section of this chapter discusses shortcomings in statistical knowledge about disabilities.  The book has whole chapters that address this and related topics, so I did not read this section in any detail.  I was surprised, though, to see the authors praise the 2008 ACS as having “an improved set of disability questions” (p. 23).  I had not thought that the new set of questions was better, but of course I was not the expert.  So while I was not sure, at this point, that I would read those chapters in their entirety, I was interested to see what this book said about the new ACS.

*  *  *  *  *

Chapter 2

Weathers, R. R., II (2009). The disability data landscape. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 27-68).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

Weathers says that this book will be using concepts based on the International Classification of Functioning, Disability and Health (ICF) published by the World Health Organization (WHO).  The concepts from the ICF are “impairment,” “activity limitation,” “participation restriction,” and “disability.”  An impairment is “a significant deviation or loss in body function or structure” (p. 29).  An activity limitation is “a difficulty that an individual may have in executing activities,” particularly activities of daily living (ADLs) (i.e., activities inside the home, e.g., dressing).  A participation restriction is “an inability to engage in societal activities”; it can be either a work limitation or an instrumental activity of daily living (IADL) (i.e., an activity outside the home, e.g., shopping).  A disability is any one or more of these (e.g., an impairment that is also an activity limitation).  Weathers further divides impairments into sensory (e.g., hearing, seeing), physical (i.e., difficulty performing physical functions), and mental (i.e., difficulty performing mental functions).

Weathers uses these distinctions to talk about what one can learn from five major surveys:  the American Community Survey (ACS); the Community Population Survey (CPS), and especially its Annual Social and EConomic supplement (CPS-ASEC); the 2000 Decennial Census; the National Health Interview Survey (NHIS); and the Survey of Income and Program Participation (SIPP).  Weathers notes that “the disability data landscape is rapidly evolving” (p. 61); for example, he says new disability-related questions are being added to the CPS and to the Behavioral Risk Factor Surveillance System (BRFSS) produced by the Centers for Disease Control (CDC).  There have also been other changes since Weathers wrote this chapter (apparently around 2006), including the elimination of disability questions from the Census (which I therefore don’t discuss here) and the revision of disability questions in the ACS.

In an analysis that may not be entirely current, Weathers traces how each of these surveys operationalizes these concepts.  In the case of mental impairments, for example, the ACS asks about difficulty in learning, remembering, or concentrating because of a physical, mental, or emotional condition lasting at least six months; the CPS-ASEC has no questions; the NHIS asks about sadness, nervousness, worthlessness, etc. over the past 30 days; and the SIPP asks if you have a learning disability, mental retardation, a developmental disability, a problem with confusion or forgetfulness, or any other mental or emotional condition.

Weathers identifies four main kinds of questions that these surveys can be used to answer:  distinguishing subpopulations (e.g., the NHIS and the SIPP ask numerous questions, so you can tell what’s happening with with people who have severe vision disabilities, whereas the ACS doesn’t distinguish different types of impairments (see preceding paragraph)); capturing state and local disability data (only the ACS); capturing long-term trends (especially the CPS and the NHIS); and capturing changes in the circumstances of the same individuals through reinterviewing (especially the SIPP, also the CPS).  The message from this analysis is that the best results come from knowing what each tool can do and being able to use them in combination when necessary.

*  *  *  *  *

Chapter 3

Houtenville, A. J., Potamites, E., Erickson, W. A., & Ruiz-Quintanilla, S. A. (2009). Disability prevalence and demographics. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 69-100).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

The authors say that there is a “generally accepted conclusion that there has been a decline in disability among the elderly,” and there seems to have been no change for the working-age population (aged 25-61) from 1997 to 2000, but there was a sharp rise for the working-age population between 1984 and 1996 (pp. 72-73).  The total increase during that period varied by age group:  18%, for those aged 18-29; 52%, for those 30-39; 46%, for 40-49; and 20%, for 50-59.  This change is theorized to stem from either a change in health or an increase in reporting.

In 2006, working-age disability prevalence by state varied from 9.1% in New Jersey to 21.4% in West Virginia, with a median of 12.6%.  There were fairly strong regional tendencies.  All southern states from New Mexico to West Virginia (except Texas, Georgia, Florida, and Virginia) were in the worst bracket; no other states except Alaska, Maine, and Montana were in that bracket.  The second-worst group was dominated by the other states of the Northwest, from Wyoming westward, and by the midwestern states from Missouri to Pennsylvania (including Michigan, excluding Illinois).  The best rates were California-Nevada, Colorado, the north-central states (including Illinois, excluding North Dakota), and the small states (except Rhode Island and Delaware) from Massachusetts to Maryland.

Working-age disability rates in 2006 varied dramatically by age and race.  All categories of disability (e.g., physical, mental) appeared at least two to three times more frequently among people in the 55-61 group as in those aged 25-34.  Only 6% of Asian-Americans, but 22% of African Americans, reported any disability.

The authors note that differences in socioeconomic status (SES) may explain some of these variations among states and races.  SES can influence lifestyle factors (e.g., smoking, obesity), access to health care, and the kinds of jobs that people have.

*  *  *  *  *

Chapters 4-7:  not covered here

*  *  *  *  *

Chapter 8

Ballou, J., & Markesich, J. (2009). Survey data collection methods. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 265-298).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

Here is the chapter's summary (pp. 290-291):

Recommended Best Practices

Include people with disabilities

PAR [participatory action research] must be considered. Although there is limited research to document the differences in research conducted with and without the participation of people with disabilities, current evidence suggests data quality can be improved by including people with disabilities. Researchers should be vigilant about addressing the need to include people with disabilities in all phases of the survey process.

Use available resources

Surveying Persons with Disabilities: A Source Guide (Markesich, Cashion, and Bleeker 2006) provides a starting point for any disability research project. Although the research included in the collection of sources may not be definitive, these citations provide extensive information related to the methodological issues associated with surveying persons with disabilities and include documentation on approaches that have been used to improve accessibility.

Plan your research

Using the guidelines listed in Table 8.2, researchers must keep in mind the key steps in the process that can impact data quality, particularly for research about and with people who have disabilities. At a minimum, reviewing these guidelines can help in making thoughtful and deliberate decisions about survey methods. In addition, information in this chapter identifies steps in the survey process where particular attention is needed to improve measurement quality.

Train interviewers

Current research identifies what interviewers should know to make sure they have the tools needed to communicate with people who have disabilities. This training should include recognition of types of disabilities, criteria for the selection of proxies, and options that can be used when interviewing people with disabilities, such as alternate wording of questions and qualitative approaches that may differ from interviews with people who do not have disabilities.

Provide documentation

The information presented in Table 8.1 shows what is needed to provide full disclosure of survey methods. It is feasible to provide complete and easily accessible documentation on disability survey information, and doing so has the added benefit of describing how various methods improve survey quality. This documentation is also essential for analysis to assist researchers in evaluating data quality.

Perfecting Best Practices

Meta-analysis of current research

A useful next step would be to conduct a meta-analysis that synthesizes data on similar topics. A systematic analysis of information would identify consistent research results that can be used to set best practice standards with increased confidence and to target the knowledge gaps that require research.

Conduct methodological and experimental research

We described examples of research that is needed to inform a set of best practices for surveying persons with disabilities in our discussion of the steps in the survey process: sampling, questionnaire design, and data collection methods. A goal of the planning group was to establish priorities for future research. This was a tremendous challenge because there are multiple issues that need to be addressed. Information from a meta-analysis could provide guidance on future research priorities.

Educating researchers, both those using data for analysis and those designing surveys to obtain data from and about people with disabilities, will result in improved disability information. One of the major changes needed in disability research is the inclusion of people with disabilities in all phases of the process. Being attentive to the methods used to collect survey information will increase the confidence that the data used for a range of public policy and service provision decisions more accurately represents people with disabilities.

*  *  *  *  *

Chapter 9

Stapleton, D. C., Wittenburg, D. C., & Thornton, C. (2009). Program participants. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 299-352).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

This chapter discusses programs that provide data about “working-age (aged 18-64) participants in the largest federal and federal-state programs that serve people with disabilities, including Social Security Disability Insurance (SSDI), Supplemental Security Income (SSI), Medicare, Medicaid, state vocational rehabilitation (VR) services, and disabled veterans benefits programs” (p. 299).  These are sometimes called “administrative” data sources, as distinct from “survey” data sources.

The authors are particularly interested in efforts to “match” administrative and survey data sources.  Generally, this appears to mean that survey participants agree to give researchers access to their personal files maintained in administrative databases.  Matching expands the amount of information that survey researchers can draw upon to understand groups of participants.  Given the sensitive nature of confidential medical and other administrative records, there are several major restrictions upon researchers’ access to such data.  The Census Bureau has come up with an alternative, known as a “synthetic” data file, in which the individual data points do not correspond with any actual human being, but collectively the data represent the characteristics of the target population.

*  *  *  *  *

Chapter 10

She, P., & Stapleton, D. C. (2009). The group quarters population.  In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 353-380).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

There is the household population, and then there is the nonhousehold population.  The latter includes people who live in institutional group quarters (GQ), noninstitutional GQ, and homeless settings.  The ACS is in the process of becoming the main source of information on disabilities among the nonhousehold population.  That population is believed to have disabilities at far higher rates than the household population. 

The authors use the 2000 Census and three surveys of prison and jail inmates:  the Survey of Inmates of Local Jails (SILJ), the Survey of Inmates of State Correctional Facilities (SISCF), and the Survey of Inmates of Federal Correctional Facilities (SIFCF).  The authors welcome the recent expansion of the ACS to include the GQ population, but note that it “does not contain the wealth of information that can be found in other surveys of the household population” (p. 374).

In the institutional GQ, the authors observe that, up through 2000, there was a gradual decline in the percentage of the general population that lives in nursing homes, and a rapid rise in the share of the population that consists of people (especially young men) in correctional facilities.  The latter phenomenon seems posed to halt if not reverse, given current budget difficulties in many governmental entities. Nonetheless, the draining of people with (especially mental) disabilities from the household population into the nonhousehold population, especially into correctional facilities, may artificially depress the reported rate of disabilities in the household population:  “It is possible that growth in the incarceration of young adult males helps to substantially explain the decline in disability prevalence for young males” (p. 372).

*  *  *  *  *

Chapter 11

Stapleton, D. C., Livermore, G. A., & She, P. (2009). Options for improving disability data collection. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 381-418).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

The differences in time horizons and other features of the major surveys (see the last paragraph under Chapter 2, above) means that they can complement one another if they are asking the same questions.  To this end, the ACS questions are now used also by the CPS and NHIS.  But there is generally a tradeoff between large sample sizes (as in the ACS, which covers large numbers of people and therefore can provide estimates down to the county level) and the amount of information collected per person.  That is, the ACS questions do not capture the same amount of detail as some of the others (e.g., SIPP):  “One particular concern is that the ACS might fail to identify many people with significant psychiatric conditions” (p. 391).  Moreover, there are typically not enough people with a particular health condition to provide much detail from a statistical perspective.

According to the authors, “The surveys that provide the most in-depth information about people with disabilities are those that are conducted very infrequently or have only been conducted once” (p. 387).  In particular, “The NHIS Disability Supplement (NHIS-D) represents the most ambitious effort to date to collect a wide range of disability-relevant information from a large, nationally representative sample of people with disabilities of all ages.  The survey was conducted in two phases in 1994 and 1995.  The data are now more than a decade old, and the survey has not been repeated” (p. 388).

The authors advocate including the ACS questions in all federal surveys:  “In 1977, the [Office of Management and Budget] mandated the use of a standardized set of questions on race and ethnicity in all federal data collection.  A similar mandate for those at risk for disability now seems justified and would be welcomed by many users of disability data and statistics” (p. 392).  The reason is to provide comparability among surveys.  If, for example, the SIPP included the ACS questions, both would show the same prevalence of disabilities, but the ACS would then be supplemented with the greater data and somewhat longitudinal advantages of the SIPP.  That is, researchers would have much more insight into the characteristics of that 10% or 12% of the population that is identified as having a disability.  Presumably it would also be possible to speculate, at least, about county-level disability details (e.g., the numbers of people having a certain disability in a certain county) by interpreting SIPP data in light of ACS county-level data.

The authors advocate a number of other improvements to disability data collection, including stronger longitudinal data collection (especially in the SIPP), better matching of administrative and survey data, and better researcher access to matched records.  The authors want to see periodic disability supplements to existing surveys, periodic surveys of specific subpopulations, and periodic national disability surveys like the NHIS-D.  The top priorities, they say, are the inclusion of ACS questions in all federal surveys and the strengthening of longitudinal and administrative data (p. 410).