Disability Prevalence -- Where Are We?

This post summarizes the general flow of my posts on disabilities over the past half-year.  I still have a few more posts in draft form, and I'll be wrapping those up shortly, but this is a good point at which to sketch out the picture as it has developed in this blog.

This post originated as an e-mail message to a researcher who seemed potentially interested in looking into data on disabilities.  I wanted to summarize, for him, the questions I have been studying.  As the message grew longer and began to cite my other blog posts, I realized that I should probably just put it up on the blog and refer him to it.  So what was going to be an e-mail message has now become the following paragraphs.

*  *  *  *  *

Let me describe the situation in general terms, and see if there are particular aspects of it that seem to have the best potential for further investigation from a research perspective.  I tend to be somewhat philosophically oriented, so my apologies in advance if it takes me a while to reach the ground.

What the Question Is

The general question is, how many people have disabilities?  There is a definitional aspect to that question.  Speaking strictly from my own perspective, I have posted some blog entries about such matters.  This definitional question matters in the sense that, if we make the circle too small, we deny relevant assistance to people who need it.  An example, in the area of mental disability, is a person who does not qualify for an official psychiatric diagnosis but nonetheless experiences obvious difficulty.

That general question is operationalized in various surveys.  At this level, we move from the purely conceptual to a mix of the hypothetical and the actual.  Elsewhere, I have cited references to a so-called National Disability Data System (NDDS).  The NDDS itself does not exist formally; the concept is that it exists in effect, through the data provided by actual research efforts.   This is still an academic's discussion; at this level we are kicking around various ways of going at the question of disability prevalence.

From there, we move to a more concrete level.  This is the level at which politicians and the public are given specific numbers.  They may not be the right numbers, but that's what footnotes are for.  Most notably, the American Community Survey (ACS) replaces the decennial census; that is, questions about disabilities have disappeared from the latter because the prevalence of disabilities is now being estimated rather than counted.  The ACS is in the process of replacing the census, for this purpose, down to the local level.

Why It Matters

As you can see, I have been trying to get a grasp on what we think we know, and why we think we know it.  But why does it matter?  Why should we care about the prevalence of disabilities?  There seem to be two ways to answer that.

The National Perspective

On one hand, we can approach the issue from a national perspective.  As the posts describe, the disability-related questions on the ACS have been modified, in the last few years, for purposes of improved reliability.  All well and good; but in the process, the estimate of people with disabilities dropped by some 15%.  Meanwhile, more sensitive measures (e.g., the SIPP) have the potential (but, alas, not the financial backing) to show a significantly higher rate.

The nation has a profound interest, budgetary and otherwise, in knowing whether the number of people with disabilities is 38 million or, instead, 53 million (to cite one alternate figure that I have encountered).  A million people here, a million people there, and pretty soon we're talking about real people.  The estimates are definitionally driven, of course, but that's the point:  how much higher does the prevalence rate go if the researcher uses a different, but comparably respectable, definition?

For example:  in an interesting book, Bagenstos contends that the Americans with Disabilities Act (ADA) has been developed in the direction of treating disability as a minority-rights kind of issue.  This, he says, has had the advantage of drawing upon the legacy of civil rights movements of the 1960s and 1970s, thus giving disability rights advocates a certain automatic sense of legitimacy.  The drawback has been that such movements invite opposition from those whom they exclude, particularly if the latter are expected to pay for adjustments to rectify perceived wrongs.

The alternative, Bagenstos says, is to treat the condition in question -- disability, in this case -- as a universal issue, something in which everyone partakes, or is at risk of partaking, through various forms of inability and imperfection.  In this approach, disability prevalence can be calculated without segregating "people with disabilities" into their own conceptual ghetto.  If disability is treated as something that anyone is capable of experiencing, like chickenpox or the flu, but if only a fraction of the population is likely to experience it at any particular point, what is that fraction?

Good question.  But how can we answer it?  The budgetary infeasibility of extending the SIPP to localities across the entire nation, on a par with the ACS, demonstrates that national disability prevalence estimation is presently stuck in a rather absurd place.  Because of its lack of local foundation, it can be gerrymandered, from a desk in D.C., to add or drop five or ten million people here and there, for reasons of statistical or budgetary convenience.  The nation, and the disability community, need something better than that.  The following suggestion illustrates a national alternative on the local level.

The Local Perspective

The relatively narrow operationalization of disability in the ACS is obviously problematic.  If its national estimate of disabilities is on the conservative side, its local estimate will tend to be so as well.

That seems reasonable enough.  But putting it that way highlights a bigger problem.  The idea seems to be that the best way to know whether my neighbor has a disability is to wait for the latest ACS to be completed; wait for the local-level ACS data to be compiled by someone in Washington; adjust that local number upwards by a fudge factor due to the conservative bias of the ACS; and then calculate my neighbor's odds.

Faced with that kind of logic, practical decisionmakers and advocates say, in effect, "Research be damned."  They aren't going to plumb the intricacies of the Supreme Court's latest interpretations of the Americans with Disabilities Act, and they aren't going to invest the time required for a clear understanding of the ACS.  They're going to rely, instead, on what they've heard and what they believe, supplemented by the occasional citation to some source or other.

Suppose we began, instead, from present experience.  Suppose, for example, that I cannot walk to work.  Research on the benefits of outdoor exposure suggests that this state of affairs will tend to make me less happy than I would be if I could walk to work.  The reason for this impairment of my subjective well-being may not be crucial:  it may not matter, for that purpose, whether I can't walk to work because I have no legs or, instead, because the streets between here and there are dangerous for pedestrians.  Either way, I can't do it.

The focus, in that example, is upon achieving a certain outcome.  Outcome-oriented disability estimation is, in essence, the language of actual local life.  The mayor finds that 39% of her constituents are furious about the state of the roads.  They are experiencing some transportation-related disability.  The fact that 3% of constituents are furious about the state of the sidewalks may be politically trivial, but a transportation-related disability nonetheless exists there as well.

Improved accessibility will often be politically infeasible if the public impression is that we are trying to spend a fortune on curb cuts for a small number of people in wheelchairs who never use the sidewalks anyway.  Rather than ask for special handouts, a more defensible view of transportation-related disability would focus on getting the roads and sidewalks into shape for people on foot, in wheelchairs, and in cars.  Infrastructure is essential.  Everyone needs effective transportation.

Summary

Disability has been defined in different ways.  A cursory review suggests that American law is presently oriented toward treating a disability as a flaw in the individual.  Hence, instruments like the ACS look for vision impairments and other personal characteristics that prevent people from functioning like everybody else.  The social model of disability is incorporated only in the limited sense that some survey questions acknowledge, in various ways, that disability may entail mismatch between person and society; yet even that acknowledgement inevitably brings the focus back to the individual.

That approach to disability has the potential to get everyone bogged down in mutual recrimination, with the familiar old vocabularies of "handouts" versus "privilege," and "normal people" versus "the oppressed."  An approach that could be more readily calculated on the local level, and more consistent and politically supportable on the national level, would focus upon desired life outcomes.

Using transportation as a particularly important disability-related outcome, one can ask how many people are not able to get where they need to go within a reasonable amount of time, in a reasonable manner, at reasonable cost.  There are many kinds of transportation-related disabilities in this sense.  Here are some examples:

• People who are disabled from independent transportation because they are under the control of others.  Examples include children and prison inmates.  
• Those whose health precludes independent travel -- hospital inpatients, for instance, and nursing home residents.
• People may also be economically disabled from utilizing independent transportation:  for instance, they may not have money for a car or even for bus fare, assuming there is a bus line near them.
• People whose obligations prevent independent travel:  people have to stay at work, or have to stay near a certain location to be available for work, or have to stay home with the kids or with a sick relative.
• Social disability precluding independent transportation.  People are stared at and harassed if they are someplace where, in effect, they don't belong.  This can include kids in the vicinity of a bully, women who are out late alone, bicyclists on a busy street, individuals of an unfamiliar or unwelcome race, and people who dress funny or act funny.

The purpose of such an investigation would be to provide an alternative perspective that would be more immediately familiar to the public and more responsive to actual human experiences of disability.  The idea is that, for whatever reason, some people can't get where they need to go.

Needless to say, this post does not purport to address the gamut of disability-related concerns and issues.  Indeed, it is precisely not that sort of thing.  What I have observed, in my half-year of exposure to disability-related matters, is that the cerebral model of disability -- the one that begins with abstract, individual-oriented definitions and works its way down to concrete application -- is not really very practical.

It tentatively seems that it would be more useful, marketable, and appropriate to treat disability as a matter of sociopersonal constraints that everyone experiences in various forms, and to focus especially upon those global, national, state, and/or local conditions that most profoundly impair the achievement of the most important outcomes.  This approach would still prioritize many individual impairments, but would do so as a matter of an investment in society's future rather than as a handout to a person who has managed to become privileged in the eyes of the law.

Notes from "Counting Working-Age People with Disabilities"

Book Discussed

Houtenville, A. J., Stapleton, D. C., Weathers, R. R. II, & Burkhauser, R. V. (Eds.). (2009). Counting working-age people with disabilities: What current data tell us and options for improvement.  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

*  *  *  *  *

I read several chapters of this inexpensive book.  This post presents my notes on (as distinct from a review of) those chapters.

*  *  *  *  *

Preface

I had previously noticed that Cornell had a website with a lot of information on disability statistics.  This preface explains why.  The U.S. Department of Education’s National Institute for Disability and Rehabilitation Research (NIDRR) awarded Cornell a grant for a Rehabilitation, Research, and Training Center (RRTC) (which Cornell called StatsRRTC).This book grew out of a conference on disability statistics research in Washington, DC in October 2006.

*  *  *  *  *

Chapter 1

Stapleton, D. C., Houtenville, A. J., Weathers, R. R. II, & Burkhauser, R. V. (2009). Purpose, overview, and key conclusions.  In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 1-26).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

The title's focus on working-age people follows from an earlier article by Weathers in which he identified the 18-64 age group as being the traditional group of working age, and further pared away the 18-24 group as being in a school-to-work transition stage, and the 62-64 group as being in a work-to-retirement transition stage.  As in that earlier article, this chapter (p. 6) focuses on the group of people aged 25-61.

The authors identify (p. 9) a number of reasons why state-level data on the prevalence of people with disabilities (PWDs) are important -- why, that is, federal data do not capture important local variations.  Some of the reasons include different physical, cultural, economic, and policy environments.  They refer (p. 13) to the National Disability Data System (NDDS), which does not exist in any formal sense (although it should) but can be understood, at present, as the aggregate of a number of disparate data collection and analysis efforts on federal and other levels.  For instance, data from administrative records suggest that only about half of the total number of PWDs estimated by the American Community Survey (ACS) are actually enrolled in federal programs that provide assistance to PWDs (p. 15).

One section of this chapter discusses shortcomings in statistical knowledge about disabilities.  The book has whole chapters that address this and related topics, so I did not read this section in any detail.  I was surprised, though, to see the authors praise the 2008 ACS as having “an improved set of disability questions” (p. 23).  I had not thought that the new set of questions was better, but of course I was not the expert.  So while I was not sure, at this point, that I would read those chapters in their entirety, I was interested to see what this book said about the new ACS.

*  *  *  *  *

Chapter 2

Weathers, R. R., II (2009). The disability data landscape. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 27-68).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

Weathers says that this book will be using concepts based on the International Classification of Functioning, Disability and Health (ICF) published by the World Health Organization (WHO).  The concepts from the ICF are “impairment,” “activity limitation,” “participation restriction,” and “disability.”  An impairment is “a significant deviation or loss in body function or structure” (p. 29).  An activity limitation is “a difficulty that an individual may have in executing activities,” particularly activities of daily living (ADLs) (i.e., activities inside the home, e.g., dressing).  A participation restriction is “an inability to engage in societal activities”; it can be either a work limitation or an instrumental activity of daily living (IADL) (i.e., an activity outside the home, e.g., shopping).  A disability is any one or more of these (e.g., an impairment that is also an activity limitation).  Weathers further divides impairments into sensory (e.g., hearing, seeing), physical (i.e., difficulty performing physical functions), and mental (i.e., difficulty performing mental functions).

Weathers uses these distinctions to talk about what one can learn from five major surveys:  the American Community Survey (ACS); the Community Population Survey (CPS), and especially its Annual Social and EConomic supplement (CPS-ASEC); the 2000 Decennial Census; the National Health Interview Survey (NHIS); and the Survey of Income and Program Participation (SIPP).  Weathers notes that “the disability data landscape is rapidly evolving” (p. 61); for example, he says new disability-related questions are being added to the CPS and to the Behavioral Risk Factor Surveillance System (BRFSS) produced by the Centers for Disease Control (CDC).  There have also been other changes since Weathers wrote this chapter (apparently around 2006), including the elimination of disability questions from the Census (which I therefore don’t discuss here) and the revision of disability questions in the ACS.

In an analysis that may not be entirely current, Weathers traces how each of these surveys operationalizes these concepts.  In the case of mental impairments, for example, the ACS asks about difficulty in learning, remembering, or concentrating because of a physical, mental, or emotional condition lasting at least six months; the CPS-ASEC has no questions; the NHIS asks about sadness, nervousness, worthlessness, etc. over the past 30 days; and the SIPP asks if you have a learning disability, mental retardation, a developmental disability, a problem with confusion or forgetfulness, or any other mental or emotional condition.

Weathers identifies four main kinds of questions that these surveys can be used to answer:  distinguishing subpopulations (e.g., the NHIS and the SIPP ask numerous questions, so you can tell what’s happening with with people who have severe vision disabilities, whereas the ACS doesn’t distinguish different types of impairments (see preceding paragraph)); capturing state and local disability data (only the ACS); capturing long-term trends (especially the CPS and the NHIS); and capturing changes in the circumstances of the same individuals through reinterviewing (especially the SIPP, also the CPS).  The message from this analysis is that the best results come from knowing what each tool can do and being able to use them in combination when necessary.

*  *  *  *  *

Chapter 3

Houtenville, A. J., Potamites, E., Erickson, W. A., & Ruiz-Quintanilla, S. A. (2009). Disability prevalence and demographics. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 69-100).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

The authors say that there is a “generally accepted conclusion that there has been a decline in disability among the elderly,” and there seems to have been no change for the working-age population (aged 25-61) from 1997 to 2000, but there was a sharp rise for the working-age population between 1984 and 1996 (pp. 72-73).  The total increase during that period varied by age group:  18%, for those aged 18-29; 52%, for those 30-39; 46%, for 40-49; and 20%, for 50-59.  This change is theorized to stem from either a change in health or an increase in reporting.

In 2006, working-age disability prevalence by state varied from 9.1% in New Jersey to 21.4% in West Virginia, with a median of 12.6%.  There were fairly strong regional tendencies.  All southern states from New Mexico to West Virginia (except Texas, Georgia, Florida, and Virginia) were in the worst bracket; no other states except Alaska, Maine, and Montana were in that bracket.  The second-worst group was dominated by the other states of the Northwest, from Wyoming westward, and by the midwestern states from Missouri to Pennsylvania (including Michigan, excluding Illinois).  The best rates were California-Nevada, Colorado, the north-central states (including Illinois, excluding North Dakota), and the small states (except Rhode Island and Delaware) from Massachusetts to Maryland.

Working-age disability rates in 2006 varied dramatically by age and race.  All categories of disability (e.g., physical, mental) appeared at least two to three times more frequently among people in the 55-61 group as in those aged 25-34.  Only 6% of Asian-Americans, but 22% of African Americans, reported any disability.

The authors note that differences in socioeconomic status (SES) may explain some of these variations among states and races.  SES can influence lifestyle factors (e.g., smoking, obesity), access to health care, and the kinds of jobs that people have.

*  *  *  *  *

Chapters 4-7:  not covered here

*  *  *  *  *

Chapter 8

Ballou, J., & Markesich, J. (2009). Survey data collection methods. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 265-298).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

Here is the chapter's summary (pp. 290-291):

Recommended Best Practices

Include people with disabilities

PAR [participatory action research] must be considered. Although there is limited research to document the differences in research conducted with and without the participation of people with disabilities, current evidence suggests data quality can be improved by including people with disabilities. Researchers should be vigilant about addressing the need to include people with disabilities in all phases of the survey process.

Use available resources

Surveying Persons with Disabilities: A Source Guide (Markesich, Cashion, and Bleeker 2006) provides a starting point for any disability research project. Although the research included in the collection of sources may not be definitive, these citations provide extensive information related to the methodological issues associated with surveying persons with disabilities and include documentation on approaches that have been used to improve accessibility.

Plan your research

Using the guidelines listed in Table 8.2, researchers must keep in mind the key steps in the process that can impact data quality, particularly for research about and with people who have disabilities. At a minimum, reviewing these guidelines can help in making thoughtful and deliberate decisions about survey methods. In addition, information in this chapter identifies steps in the survey process where particular attention is needed to improve measurement quality.

Train interviewers

Current research identifies what interviewers should know to make sure they have the tools needed to communicate with people who have disabilities. This training should include recognition of types of disabilities, criteria for the selection of proxies, and options that can be used when interviewing people with disabilities, such as alternate wording of questions and qualitative approaches that may differ from interviews with people who do not have disabilities.

Provide documentation

The information presented in Table 8.1 shows what is needed to provide full disclosure of survey methods. It is feasible to provide complete and easily accessible documentation on disability survey information, and doing so has the added benefit of describing how various methods improve survey quality. This documentation is also essential for analysis to assist researchers in evaluating data quality.

Perfecting Best Practices

Meta-analysis of current research

A useful next step would be to conduct a meta-analysis that synthesizes data on similar topics. A systematic analysis of information would identify consistent research results that can be used to set best practice standards with increased confidence and to target the knowledge gaps that require research.

Conduct methodological and experimental research

We described examples of research that is needed to inform a set of best practices for surveying persons with disabilities in our discussion of the steps in the survey process: sampling, questionnaire design, and data collection methods. A goal of the planning group was to establish priorities for future research. This was a tremendous challenge because there are multiple issues that need to be addressed. Information from a meta-analysis could provide guidance on future research priorities.

Educating researchers, both those using data for analysis and those designing surveys to obtain data from and about people with disabilities, will result in improved disability information. One of the major changes needed in disability research is the inclusion of people with disabilities in all phases of the process. Being attentive to the methods used to collect survey information will increase the confidence that the data used for a range of public policy and service provision decisions more accurately represents people with disabilities.

*  *  *  *  *

Chapter 9

Stapleton, D. C., Wittenburg, D. C., & Thornton, C. (2009). Program participants. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 299-352).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

This chapter discusses programs that provide data about “working-age (aged 18-64) participants in the largest federal and federal-state programs that serve people with disabilities, including Social Security Disability Insurance (SSDI), Supplemental Security Income (SSI), Medicare, Medicaid, state vocational rehabilitation (VR) services, and disabled veterans benefits programs” (p. 299).  These are sometimes called “administrative” data sources, as distinct from “survey” data sources.

The authors are particularly interested in efforts to “match” administrative and survey data sources.  Generally, this appears to mean that survey participants agree to give researchers access to their personal files maintained in administrative databases.  Matching expands the amount of information that survey researchers can draw upon to understand groups of participants.  Given the sensitive nature of confidential medical and other administrative records, there are several major restrictions upon researchers’ access to such data.  The Census Bureau has come up with an alternative, known as a “synthetic” data file, in which the individual data points do not correspond with any actual human being, but collectively the data represent the characteristics of the target population.

*  *  *  *  *

Chapter 10

She, P., & Stapleton, D. C. (2009). The group quarters population.  In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 353-380).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

There is the household population, and then there is the nonhousehold population.  The latter includes people who live in institutional group quarters (GQ), noninstitutional GQ, and homeless settings.  The ACS is in the process of becoming the main source of information on disabilities among the nonhousehold population.  That population is believed to have disabilities at far higher rates than the household population. 

The authors use the 2000 Census and three surveys of prison and jail inmates:  the Survey of Inmates of Local Jails (SILJ), the Survey of Inmates of State Correctional Facilities (SISCF), and the Survey of Inmates of Federal Correctional Facilities (SIFCF).  The authors welcome the recent expansion of the ACS to include the GQ population, but note that it “does not contain the wealth of information that can be found in other surveys of the household population” (p. 374).

In the institutional GQ, the authors observe that, up through 2000, there was a gradual decline in the percentage of the general population that lives in nursing homes, and a rapid rise in the share of the population that consists of people (especially young men) in correctional facilities.  The latter phenomenon seems posed to halt if not reverse, given current budget difficulties in many governmental entities. Nonetheless, the draining of people with (especially mental) disabilities from the household population into the nonhousehold population, especially into correctional facilities, may artificially depress the reported rate of disabilities in the household population:  “It is possible that growth in the incarceration of young adult males helps to substantially explain the decline in disability prevalence for young males” (p. 372).

*  *  *  *  *

Chapter 11

Stapleton, D. C., Livermore, G. A., & She, P. (2009). Options for improving disability data collection. In A. J. Houtenville, D. C. Stapleton, R. R. Weathers, II, & R. V. Burkhauser (Eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 381-418).  Kalamazoo:  W. E. Upjohn Institute for Employment Research.

The differences in time horizons and other features of the major surveys (see the last paragraph under Chapter 2, above) means that they can complement one another if they are asking the same questions.  To this end, the ACS questions are now used also by the CPS and NHIS.  But there is generally a tradeoff between large sample sizes (as in the ACS, which covers large numbers of people and therefore can provide estimates down to the county level) and the amount of information collected per person.  That is, the ACS questions do not capture the same amount of detail as some of the others (e.g., SIPP):  “One particular concern is that the ACS might fail to identify many people with significant psychiatric conditions” (p. 391).  Moreover, there are typically not enough people with a particular health condition to provide much detail from a statistical perspective.

According to the authors, “The surveys that provide the most in-depth information about people with disabilities are those that are conducted very infrequently or have only been conducted once” (p. 387).  In particular, “The NHIS Disability Supplement (NHIS-D) represents the most ambitious effort to date to collect a wide range of disability-relevant information from a large, nationally representative sample of people with disabilities of all ages.  The survey was conducted in two phases in 1994 and 1995.  The data are now more than a decade old, and the survey has not been repeated” (p. 388).

The authors advocate including the ACS questions in all federal surveys:  “In 1977, the [Office of Management and Budget] mandated the use of a standardized set of questions on race and ethnicity in all federal data collection.  A similar mandate for those at risk for disability now seems justified and would be welcomed by many users of disability data and statistics” (p. 392).  The reason is to provide comparability among surveys.  If, for example, the SIPP included the ACS questions, both would show the same prevalence of disabilities, but the ACS would then be supplemented with the greater data and somewhat longitudinal advantages of the SIPP.  That is, researchers would have much more insight into the characteristics of that 10% or 12% of the population that is identified as having a disability.  Presumably it would also be possible to speculate, at least, about county-level disability details (e.g., the numbers of people having a certain disability in a certain county) by interpreting SIPP data in light of ACS county-level data.

The authors advocate a number of other improvements to disability data collection, including stronger longitudinal data collection (especially in the SIPP), better matching of administrative and survey data, and better researcher access to matched records.  The authors want to see periodic disability supplements to existing surveys, periodic surveys of specific subpopulations, and periodic national disability surveys like the NHIS-D.  The top priorities, they say, are the inclusion of ACS questions in all federal surveys and the strengthening of longitudinal and administrative data (p. 410).

Survey of Income and Program Participation (SIPP): Core Questions on Disability

In an exploration of Census Bureau statistics on disabilities, I wanted to examine the questions that were asked in the most recent panel of the Survey of Income and Program Participation (SIPP).  A SIPP Data page at the website of the National Bureau of Economic Research (NBER) seemed to indicate that results from the 2008 panel were only starting to come out.  Similarly, at the Bureau's own SIPP website, a 2008 Panel SIPP Data Product Schedule indicated that, as of March 2010, Waves 1-4 of the 2008 Panel were to have been completed, but only the Wave 1 data were actually supposed to be available.

The Census Bureau indicates that the SIPP consists of core content plus topical modules.  At present, the most recent core content shown on the Bureau's Core Content webpage is that of the 2004 Panel.  That is, the 2008 questionnaire did not appear there; this seemed to be one of several Bureau webpages that seemed not to be maintained currently.  I went back to that NBER webpage, went down to the 2008 Panel listings, and saw that there was not yet a PDF file listed on the 2008 Wave 1 Core row.  I took a look into the 08w1.zip file listed next to it, but when I unzipped it I saw that it contained only a .dat file suitable for opening in a statistics program like SPSS.

I went looking for the 2008 Panel questionnaire.  On the Bureau's SIPP User Guide webpage, I found this statement:

We are currently updating the Third Edition, 2001. The revised chapters include information on the 2001 Panel, 2004 Panel and current up-to-date information on the 2008 Panel. All chapters are currently being revised.

I opened the SIPP Users' Guide, Third Edition, 2001 PDF.  It seemed to contain a lot of useful information, but not an actual copy of any questionnaires.  This was not surprising; the 2004 questionnaires, with instructions and all, had been several hundred pages long.  I found another page, described as the SIPP FTP page, and it offered several files related to the 2008 Panel -- some data files, and also some technical text files -- but no copies of the 2008 questionnaire.  I found the reason in a footnote (p. 16, n. 11) to a Census Bureau Research Report by Moore et al. (2009), "The 2008 Survey of Income and Program Participation Event History Calendar Field Test:  Study Design and Initial Results."  That footnote referred to the "automated" SIPP questionnaire.  This explained the odd format I had seen in the 2004 questionnaires:  they had apparently been attempting to replicate, on paper, the different decisions that a computer program would make, regarding which questions to ask next, based on what the respondent said.

Revising my search, I found a reference to AttA - W1 Core Items Booklet.pdf in a RegInfo.gov webpage.  RegInfo.gov, apparently maintained by the Regulatory Information Service Center, turned out to be a repository of materials currently under review by the Office of Information and Regulatory Affairs (OIRA) within the Office of Management and Budget (OMB).  In length and format, the W1 Core Items Booklet was very similar to the 2004 questionnaires that I had downloaded earlier (above).  Unfortunately, it was dated August 22, 2007, which meant that it might not have presented the final set of questions that were actually posed to the 2008 Panel.  I searched for "W1 Core Items Booklet" and found another entry at RegInfo.gov with what appeared to be the same PDF document.  On a webpage at GovPulse.us, which appeared to be a website set up by three people who wanted to make the Federal Register more searchable, I found an indication that the Census Bureau was requesting OMB review of the Wave 6 topical modules for the 2008 Panel, with this statement:  "The core SIPP and reinterview instruments were cleared under Authorization No. 0607-0944."  A search for that authorization led to a different RegInfo.gov page, bearing the appealing certification date of January 28, 2010.  This webpage indicated that the relevant Information Collection Review (ICR) pertained to something known as the SIPP 2008 Panel Survey Instruments; and when I clicked on that link, I found myself back at the original RegInfo.gov webpage.  It seemed, in short, that the W1 Core Items Booklet dated August 22, 2007 seemed to be the final 2008 Panel Core Questionnaire.

I searched that 2008 Panel Core Questionnaire for references to "disability" and its cognates.  I found a number of such references.  I found no references to "impair" or its cognates in this Core Questionnaire, however, and the references to disability tended to be using accepted terminology rather than defining it -- referring, for instance, to "disability compensation," "disability insurance," and "disability benefits."  Because the SIPP is oriented toward income and benefits that people receive, it appears that it may be possible at least to use the data from its Core Questionnaire to make estimates about the portion of its target population for which disability affects the ability to work.  Whether that would prove necessary seemed to depend on what I could learn from SIPP's disability-related topical questionnaires.

Survey of Income and Program Participation (SIPP): Topical Questions on Disability

My search for disability statistics led me to the 314-page 2008 Panel Core Questionnaire for the U.S. Census Bureau's Survey of Income and Program Participation (SIPP).  While the Core Questionnaire did have some disability-related questions, those questions tended not to ask about the nature or extent of a disability; the concern was, rather, with whether the disability interfered with (or was perhaps the source of) the respondent's income and benefits.  It seemed that disability-oriented topical questions posed to the SIPP's 2008 Panel of respondents would shed more light on the nature and extent of disabilities experienced by people in the United States.

As with the Core Questionnaires, the Census Bureau's SIPP website did not provide ready links to its Topical Module Questionnaires after 2004.  Fortunately, the 2008 Panel SIPP Data Product Schedule found on one of the pages in that website indicated that data collection for the Work Disability History module was scheduled to take place as part of Wave 2, in January through April 2009, and that the corresponding data products were expected to be released in April 2010.  Data collection for the other disability-related Topical Modules, Functional Limitations and Disability for adults and children, was scheduled to take place as part of Wave 6, in summer 2010, with data products being released in September 2011.

By pleasant coincidence, the 2008 Panel Wave 6 Topical Modules seemed to be included on the same RegInfo.gov page from which I had previously obtained the 2008 Panel Core Questionnaire, so it was easy to get the Wave 6 modules.  The link provided on that page produced an 82-page Wave 6 Topical Modules Items Booklet, in which the Functional Limitations and Disability section ran from pages 31 to 53.  A preliminary look at some of these questions confirmed that they were not at all limited to income production, as had been the case with disability-related questions in the Core Questionnaire.

The remaining step was to find the Wave 2 Topical Modules Booklet that would hopefully include the Work Disability History module.  A search led to a Federal Register entry that cited the same OMB Control Number 0607-0944 that had applied to the 2008 Panel Core Questionnaire.  A search for that number led to a different RegInfo.gov page, where I downloaded a 2008 Panel Wave 2 Topical Modules Items Booklet, in which the Work Disability Topical Module appeared on pages 1 through 5 (PDF pages 3 through 7).  As the "work disability" moniker suggests, the questions in this module appeared to be oriented toward disabilities that interfered with work.

My ultimate purpose, in this pursuit, was to become informed on defensible ways of estimating the numbers of persons with disabilities in a particular location.  The page counts (above) told me that I was likely to get a more extensive treatment of disability concepts in the Wave 6 Topical Module, and it also seemed that that module's questions would take a broader perspective on the meaning of disability.  In a subsequent post, I look more closely into that module.

Survey of Income and Program Participation (SIPP): An Introduction

The Survey of Income and Program Participation has its roots in the War on Poverty of the Johnson Administration in the late 1960s.  It was hard to find good data on welfare and income for poor people.  To meet this need, the U.S. Department of Health, Education, and Welfare (now HHS) created the Income Survey Development Program (ISDP) in 1975.  According to Citro and Scholz (2009, pp. 18-19), the transition from ISDP to the new SIPP survey proceeded in fits and starts, narrowly surviving cancellation until 1984, when the first SIPP panel of about 21,000 households finally began.

From 1985 through 1993, new panels were introduced every February, on a rotating basis.  Within each panel, households were followed through multiple four-month waves (Citro & Scholz, p. 22).  That is, the sample was divided into four groups, and all of the households in a given group were interviewed once every four months, also on a rotating basis.  After a major redesign effort beginning in 1993, panels were interviewed every four months over a four-year period, and one panel was completed before another was introduced.  After 1993, then, panels were introduced in 1996, 2001, 2004, and 2008 (pp. 23-24, 29).

SIPP seems to have been plagued, throughout its existence, by irregular and inadequate political support and funding.  In 2006, the Bush Administration proposed to cut its budget from $44 million to $9.2 million, and to focus more than half of the latter figure on the creation of a new data collection program called the Dynamics of Economic Well-being System (DEWS) (p. 28).  DEWS would rely upon administrative sources of data, rather than a survey, to update and complement decennial census data.  Congress was persuaded that SIPP was important, however, and instead cut its 2007 budget to $33 million in 2007 (p. 29).  DEWS was downscaled, and now seems to survive as the "re-engineered SIPP."

In the 2004 and 2008 panels, it was planned that households would be followed for as many as 12 waves.  These panels used 51,400 households in 2004 and 45,000 in 2008 (pp. 25, 29).  Budgetary and resource restrictions forced a significant cutback in the numbers of households and waves in later years of the 2004 panel.  The 2008 panel now seems to have sufficient funding to carry through to completion.  Data from the 2008 panel will apparently become available in 2012.  The next panel is planned for 2013, structured as a three- or four-year series of annual interviews.

In their book, Citro and Scholz provide extensive information on strengths and weaknesses of SIPP, recommendations for improving data quality and using administrative records effectively, potential innovations in data collection, and other matters.  The general sense is that these authors consider the SIPP flawed but capable of being greatly improved, and in any event irreplaceable as a practical matter.