During the past few months, we’ve been conducting major field quality assurance operations. The last two I’ve blogged about in the past – the vacant/delete check and the field verification check. Both of these are last chances to include cases that may have been incorrectly or incompletely measured.

All our data collection is now done. I am pleased to report that we remain on schedule as we have gathered a complete picture of the nation.

Our follow up is important to ensuring an accurate census, and I want to thank the American public again for their patience. At this point, we have attempted to record the population characteristics of every household, and sought a resolution on every address.

We are now entering the data processing phase. All of the questionnaires we received, either directly from respondents or from in-person follow up, must be converted into an electronic format, processed, and then tabulated, in order to provide the state population totals necessary for apportionment.

As mandated by law, the national and state population counts must be delivered to the President of the United States by December 31, 2010. This will be the first release of any data from the 2010 Census. We have much to do to get ready for this.

Please submit any questions pertaining to this post to ask.census.gov

Comparing the census counts to alternatives is one way to get a sense of how good the census is; if all alternative ways of estimating the size of the population yield about the same conclusions, we feel good about the results. The last post on quality described “demographic analysis” as an alternative way of measuring the population of the United States.

This post describes the post-enumeration sample survey approach to measuring the population. (This decade’s version of the post-enumeration survey is called “Census Coverage Measurement.”) Post-enumeration sample surveys have been part of US censuses for some time. For 2010 a survey will be used to estimate the number of persons missed by the census as well as those erroneously enumerated (e.g., duplicates and visitors from other countries).

A post-enumeration survey draws two samples – one from the full population in complete ignorance of whether sample cases were covered in the census; another from the census address universe. After a survey is done, each address and each person enumerated within the address are carefully matched to the census file, as a way of determining which cases were captured by both the census and the survey or by only one of the methods. From this matching operation, estimates of the misses and erroneous enumerations are made.

Just like demographic analysis, if the post-enumeration sample survey achieves its ideal form, it offers completely accurate estimates of differential undercount, the tendency for some populations to be covered by the census less well than others. However, the ideal post-enumeration survey is never achieved.

In an ideal post-enumeration sample survey,

a) The likelihood that a sample person is measured in the survey is completely independent of the likelihood that the person is measured in the census. (More loosely stated, those who were covered by the census and those not covered by the census have the same probability of being measured in the survey.)

Problem: This independence is not fully achievable.

Fix: The field staff working on the post-enumeration survey are different from those working on the census, and use different materials. Overlap between the operations is kept to a minimum. This means that the operations are kept independent, but if those who are reluctant to respond in a census are also reluctant to respond in the survey, the problem remains.

b) The probability of being captured in the census is the same for all persons and for those in the survey all persons have the same probability of being measured.

Problem: The assumption is violated in the census and the survey; the probabilities of capture range widely among people with different life styles (e.g., very mobile young singles who live by themselves vs. nuclear families).

Fix: Group people with similar characteristics who share similar probabilities of being captured in the census; use statistical models to reduce the effect of violating this assumption.

c) The respondent to the survey correctly reports his or her April 1, 2010, residence and household composition.

Problem: The survey interviewing begins in mid-August, 2010; some persons, especially people who have moved, may have difficulty recalling their April 1 residence status.

Fix: Additional fieldwork and statistical models can be used to mitigate the effect of incomplete information and reporting errors. Further, an auxiliary study has been mounted to estimate what difficulties persons have in accurately reporting their April 1 residence.

d) The survey operation collects the information requested completely.

Problem: Not all persons in the sample survey will be contacted or agree to participate; those who don’t may have distinctive characteristics on geography or person-level attributes.

Fix: We try to contact proxies, neighbors or others who know the nonrespondent cases. Statistical models will also be used in an attempt to remove the nonresponse errors from the sample survey.

Finally, an inherent weakness of a post-enumeration survey is that it is based on just a sample of the population, not the total population. This means that all estimates from it are subject to instability due to sampling variability. This, however, can be measured, just as we are accustomed to seeing in surveys as “margins of error” or the sampling error figures.

As this admittedly high-level description itself demonstrates, the statistical complexity of the post-enumeration survey is high; the technical expertise required to construct the estimates and evaluate them is considerable. Whenever possible, the violations of assumptions in the estimates of undercount will be themselves investigated, but no one involved believes that perfection will be attained. Instead, as with demographic analysis, we will openly document the strengths and weaknesses of the estimates from the post-enumeration survey, so that all can form their judgments about the utility of the estimates to evaluate the census.

We will not have the statistical results of the post-enumeration survey until 2012, so we have to wait awhile to compare this alternative way of measuring the population to the 2010 census. Stay tuned!

As I noted in an earlier post, we are now running two field operations as part of the quality improvement phase of the 2010 Census – the vacant/delete check (double checking the status of units of a variety of types, and conducting interviews for supplemental households added to our list too late to include them in mailout or earlier operations) and the field verification check (where we check the addresses of some units that used a “be counted” form or called the Telephone Questionnaire Assistance line to respond).

On the vacant/delete check, we are very nearly complete (above 99%) and on schedule. We’re finding that about 26% of the 8.5 million units are occupied. Most of those, we suspect, are those supplemental addresses that we added into this operations based on new postal delivery units or updates from local officials. This operation is somewhat over budget, but given savings in other areas, we can absorb this overage.

On the “field verification” operation, we have completed about 95% of the work. This too is on schedule.

Please submit any questions pertaining to this post to ask.census.gov

The point of the first three posts on census quality was to note that there is no known “truth” with regard to the population of the United States. We have different tools, each of which gives us a different look at the population at different points in time, but each of which has some weakness.

This post is a short description of the method of “demographic analysis.” It provides estimates of population counts at the national level only. It provides separate estimates by single years of age, separately for males and females, separately for two race groups (Black and nonblack).

It does not provide separate estimates for each state (and thus is not helpful in providing a different set of reapportionment figures); it does not provide estimates for the 8 million different census blocks (and thus does not permit different redistricting estimates).

Just like the census, demographic analysis has a simple ideal. If, for every year, we could count all the births in the country, subtract all the deaths, add all immigrants to the country, subtract all emigrants, then we could have a perfect accounting of the total population and how it has changed across time, and from one census to the next.

If we could know the gender of births, and the age and gender of deaths, immigrants, and emigrants, we could produce counts by males and females separately by age. If we could do the same for race groups, we could produce counts by different race groups. This is essentially what we try to do, but, as with the census, we can’t achieve the ideal. Each of the major components of demographic analysis has some problem in practice and need a fix:

1) Because of the quality of the historical data on births, for the age groups 65 years and older, we use counts from Medicare enrollment by age, gender, and race.

a. Problem: not all those 65 and older are enrolled in Medicare. Fix: use survey data on medical insurance coverage to estimate the number not enrolled.

b. Problem: Medicare uses different race measurement than Census. Fix: Supplement the Medicare data with data from other sources and in some cases assume race measurement is equivalent.

2) For those under 65 years of age:

a. Problem: underegistration of births in older age groups (e.g., 6% of Black births in 1950). Fix: use estimates of the completeness of birth registration from birth registration research, combined with professional judgment, to adjust birth counts

b. Problem: different race measurement on birth and death records than on Census. Fix: examine patterns of racial identification in the last census and use professional judgment to assign groups only to “black” and “nonblack” race categories.

c. Problem: no direct count of components of international migration. Fix: use a combination of survey and other data sources and professional judgment to estimate international migration

Every component that involves some professional judgment is subject to professional debate. Some of the debates concern very small categories, which do not individually affect the national figures too much.

Increasingly, the component generating the most controversy is the count of international migrants by race, age, and gender groups. There is little consensus among demographers about how large those groups are.

Most professionals believe that the strongest features of demographic analysis come from its consistency over different age by gender groups. When the estimates of these group sizes systematically vary from the counts based on a census, many believe that is evidence of weaknesses in the census, not in the demographic analysis. Few professionals would attempt to adjust the census counts based on these comparisons, however, given the problems reviewed above.

To be transparent with the American public, in early December 2010, we will release a preliminary range of estimates by age, gender, and race, developed using the demographic methods above. This range of estimates can be compared to the total population count we release in late December from the 2010 Census.

Later, in the Spring of 2011, the range of estimates by age, race, and gender can be compared to the 2010 Census counts. Then, we may be able to improve our assumptions about components to the demographic analysis estimates, and then release an improved set of demographic analysis numbers sometime in 2011.

I expect in late December, when we release the 2010 Census counts and compare them to the demographic analysis range of estimates, journalists will ask which one is correct. As you can see from the description above, I won’t be able to answer that question. Each method has strengths and weaknesses for the various statistics of interest.

Please submit any questions pertaining to this post to ask.census.gov

There have been many reasonable critiques of the costs of the decennial census. At many public events I have shared my own judgment that we must reduce the costs of the next census. While the spending that occurred before 2009 had a variety of complicated causes, I am rather proud of what the collective Census Bureau staff has accomplished on cost controls since I arrived in July 2009.

Today, we announced that we will return to the Treasury not less than $1.6 billion of our budgeted funds for 2010. That’s about 22% of the total amount Congress gave us to do the job this year. While we’re not finished with the operations of the 2010 Census, we’re confident we can finish up a professional job without calling on that $1.6 billion.

How did this happen?

A big part of the 2010 Census advertising campaign delivered the message that completing and mailing back the form saved us taxpayers large amounts of money. The American people really came through, exceeding our estimates on the mail return of questionnaires. This made the costs of the Nonresponse Followup phase much lower than we were prepared to spend.

In addition, the temporary census workers we hired, in this time of high unemployment rates, were just spectacular. They put in the hours; they worked more efficiently than we were expected; they made our field processes go smoothly.

To be fair, about $800 million of the savings was due to good fortune. That’s how much we set aside for contingencies such as hurricanes, floods, earthquakes, or a massive H1N1 flu pandemic. Each of these events might have made it more difficult to hire folks, or to gain cooperation of the public. Those things and other bad events did not happen, so we never had to tap our contingency funds.

We also saved hundreds of millions due to good management practices by the career-staff leaders of the 2010 Census. We had a net savings of $150 million from our other operations, such as the enumeration of Group Quarters and special count of American Indian Reservations, which all came in under budget.

One of the burdens I felt most deeply over the past year was trying to be a wise steward of the taxpayers’ money, in this time of fiscal suffering across the nation. My colleagues at the Census Bureau did a superb job at achieving these cost efficiencies. We should all salute the nearly 1 million people who served their country in such an efficient manner.

Please submit any questions pertaining to this post to ask.census.gov