Non-parametric Rank Tests for independence in opinion surveys

Mayer Alvo, Philip Yu, K. Lam

Nonparametric rank tests for independence between two characteristics are commonly used in many social opinion surveys. When both characteristics are ordinal in nature, tests based on rank correlations such as those due to Spearman and Kendall are often used. The case where some ties exist has already been considered whereas Alvo and Cabilio (1995) have studied the case when there are missing values but no ties in the record. However, it frequently happens that the survey data may contain simultaneously many tied observations and/or many missing values. A naive approach is to simply discard the missing observations and then to make use of the rank correlations adjusted for ties. This approach would be less powerful as it does not fully utilize the information associated with the incomplete data set. In this article, we generalize Alvo and Cabilio's notion of distance between two rankings to incorporate tied and missing observations, and define new test statistics based on the Spearman and Kendall rank correlation coefficients. We determine the asymptotic distribution of the Spearman test statistic and compare its efficiency with the corresponding statistic based on the naive approach. The proposed test is then applied to a real data set collected from an opinion survey conducted in Hong Kong.

Regression Estimators for the 2001 Canadian Census

Mike Bankier, Statistics Canada

In the 2001 Canadian Census of Population, calibration or regression estimation was used to calculate a single set of household level weights to be used for all Census estimates based on a 1 in 5 national sample of more than two million households. Because many auxiliary variables were available, only a subset of them could be used. Otherwise, some of the weights would have been smaller than one or even negative. A forward selection procedure was used to discard auxiliary variables which caused weights to be smaller than zero or which caused a large condition number for the calibration weight matrix being inverted. Also, two calibration adjustments were done to achieve close agreement between auxiliary population counts and estimates for small areas. Prior to 2001, the projection Generalized Regression (GREG) Estimator was used and the weights were required to be greater than 0. For the 2001 Census, a switch was made to a pseudo-optimal regresssion estimator which kept more auxiliary variables while, at the same time, required that the weights be one or more.

Analysis Issues for Data from Complex Surveys

David A. Binder and Georgia R. Roberts, Statistics Canada

Fitting models to data is a common practice in statistical applications. The properties of the estimates obtained using "traditional" fitting methods are generally well known under certain model assumptions. However, one key implicit assumption is that the sample design used to collect the data is not informative. We describe what this means when applied to complex survey data and discuss methods of analysis that are valid whether or not the designs are informative. Emphasis is placed on target parameters motivated from a model and on the assumptions about what is random.

A Nonparametric Regression Smoother for Nonnegative Data

Yogendra P. Chaubey, Concordia University,
Pranab K. Sen, University of North Carolina
Xiaowen Zhou, Concordia University

Recently, there has been a lot of development in nonparametric regression using kernel method, local linear smoothing, splines and so on. Some of these methods seem to work well, however, there are no clear winners. Moreover, these methods are general purpose methods and there may be scope for better methods in specific situations. For example, when the observations are known to be nonnegative there is no reason to use a symmetric kernel, which may unnecessarily put mass over some negative values. Such situations are common in survival studies. Motivated by such problems, Chaubey and Sen (2001) provide an alternative method of smoothing the multivariate survival function and study its asymptotic properties along with that of the derived
probability density function. In this paper, we consider use of this estimator in proposing estimators of continuous functionals of the joint survival function. In this process we derive an alternative smooth estimator of the regression function E(Y|X) where X is a vector of p independent variables, assuming non-negative support for all variables. Properties of the derived estimator are theoretically and numerically investigated.

Jiahua Chen, University of Waterloo

If a population contains many zero values and the sample size is not very large, the central limit
theorem based confidence intervals for the population mean may have poor coverage prob-abilities. This problem is substantially reduced by constructing parametric likelihood ratio intervals when an appropriate mixture model can be found. However, in the context of
survey sampling, a general preference is to make minimal assumption about the population.
We have therefore investigated the coverage properties of nonparametric empirical likelihood
confidence intervals for the population mean. Under a variety of hypothetical populations, the empirical likelihood intervals often outperformed parametric likelihood intervals by having
larger lower bounds, or more balanced coverage rates. We have also used a real data set to illustrate the empirical likelihood method.

This is joint work with Shun-Yi Chen and J.N.K. Rao.

Composite Estimation in Small Area Inference

Gauri Sankar Datta, University of Georgia

Composite estimators are very popular for estimation of small area means. Such estimators are obtained by taking a weighted average of a model-based synthetic estimator and a traditional survey estimator or direct estimator. Two important models in small area estimation are the Fay-Herriot model and the nested error regression model. Empirical best linear unbiased prediction (EBLUP) method is widely used in developing suitable composite estimators of small area means based on these models. EBLUP estimator of a small area mean is obtained by replacing the variance parameters in the BLUP estimator of a small area mean. The BLUP estimator is a weighted average of the synthetic estimator and the direct estimator, the weight attached to the latter is proportional to the model error variance. However, often for many data sets the model error variance, and hence the weight to the direct estimator, is estimated to be zero. Consequently, the EBLUP completely ignores the survey regression estimator/direct estimator. Completely ignoring the direct survey estimator is not desirable, especially, if the direct estimator is based on a large small area sample and if the model does not fit adequately.

In this talk, we will consider composite estimators based on weights which are either known or are obtained from other consideration. We will obtain approximate mean squared error of prediction of these estimators. Based on a composite estimator and an estimated measure of uncertainty in estimating a small area mean, we will develop an approximate confidence interval for the small area mean. The resulting confidence interval will be calibrated to achieve the nominal coverage probability to a greater degree of accuracy.

Inference Based on Quadratic-Form Test Statistics Computed from Complex
Survey Data

John Eltinge, Bureau of Labor Statistics

Multivariate analyses of complex survey data are often based on quadratic form test statistics, e.g., , where is a -dimensional parameter vector of interest; is an estimator that is approximately unbiased for under specified design or model conditions; is a smooth function of , is the value of under a particular null hypothesis; and is a symmetric positive definite matrix. In some cases (e.g., customary Wald tests), equals the inverse of an estimator of the covariance matrix of the approximate distribution of under specified design or model conditions. In other cases (e.g., first- and second-order Rao-Scott adjusted tests), M is based on other approximations related to the covariance structure of .

This paper considers the large-sample properties of inference methods based on for cases in which the matrix is relatively stable. Three issues receive principal attention.

1. Approximations for the distribution of under a specified null hypothesis, and under moderate deviations from this null hypothesis.
   
   
2. Use of (a) to evaluate approximate power curves for the tests associated with (a).
   
   
3. Related efficiency measures computed form the approximate volumes of test-inversion-based confidence sets.
   

Building on previous literature on misspecification effect matrices, generalized design effects and Rao-Scott-type adjustments, results for (a)-(c) are expressed in terms of the eigenvalues and eigenvectors of the matrix , where is the symmetric square root of the covariance matrix of the approximate distribution of . The paper closes with illustrations of the general ideas in (a)-(c) for several specific cases, including test statistics based on, respectively, (i) first- and second-order Rao-Scott adjusted tests; (ii) matrices computed from multivariate generalized variance functions; and (iii) matrices computed from stratum collapse or other approximations to the true sample design.

Beginning a Foundation for an Empirical Science of Missing Data in Surveys:
A Review of Contributions by J.N.K. Rao

Robert E. Fay, U.S. Census Bureau

Sample surveys encounter both unit and item nonresponse for a variety of reasons - when the interviewer or organization is unable to contact the respondent, when the respondent refuses, when the respondent is unable to understand a question or recall the information, when the interviewer fails to ask a question or record the answer, or other circumstances. Nonresponse is generally but not exclusively a consequence of human actions, and the study of nonresponse arguably should be largely a social science. A few researchers have begun to make such connections, but this work remains at early stages.

This review will describe a possible future in which survey researchers approach missing data in surveys as a scientific problem, and communicate their results in the standard language of science. The previous, current, and ongoing methodological work of statisticians to develop statistical methods will hopefully be an important part of this emerging science. J.N.K. Rao and his collaborators have made numerous methodological contributions to the statistical analysis of missing data. This review will survey his work and sketch its possible contributions to a future empirical science of missing data in surveys.

Survey Regression Estimation

Wayne A. Fuller and Mingue Park

The basic properties of regression estimators, including variance estimation, are reviewed. The role of models and the construction of design consistent estimators with desirable model properties are discussed. Some practical aspects associated with the construction of regression weights are considered.

Some Issues Arising in the Use of Sampling in the Legal Setting

Michael D. Sinclair and Joseph L. Gastwirth

Statisticians know that inferences from properly conducted sample surveys are a reliable and cost-effective method of obtaining information about the population of interest. Although it took a while before courts accepted samples, today sample survey evidence is used in cases concerned with trademark infringement, auditing and accounting to determine the accuracy of tax collections or whether expenditures were in compliance with a statute and copyright law. Statisticians using samples based on complex designs may need to demonstrate that these "newer" methods satisfy legal standards for admissibility. One case where the "experts" disagree on the reliability of an estimate derived from such a design will be discussed in depth. A simulation study is described that clarifies the issues involved. An alternative design, which appears to be more efficient, is proposed and investigated.

Small Area Estimation Based on NEF-QVF Models and Survey Weights

Malay Ghosh, University of Florida & Tapabrata Maiti, Iowa State University

The paper proposes small area estimators based on natural exponential family (NEF) quadratic variance function (QVF) models. Morris (1982,1983) characterized NEF-QVF distributions and studied many of their properties. We propose pseudo empirical best linear unbiased estimators of small area means based on these models when the basic data consist of survey-weighted estimators of these means, area-specific covariates and certain summary measures involving the weights. We provide also explicit approximate mean squared errors of these estimators in the spirit of Prasad and Rao (1990), and these estimators can be readily evaluated. We illustrate our methodology by estimating the proportion of poor children in the age group 5-17 for the different counties in one of the states in the US.

The Trade-off Theory for Sampling and Design.

A. Sam Hedayat, University of Illinois at Chicago

There is an intimate connection between the theory of binary proper block designs and survey sampling without replacement. The theory of trade-off developed originally for block design can be modified and updated so that it can be used for survey sampling. In this talk we shall present the latest news for trade-off theory related to block designs. Then, we shall show their implications and impacts on survey sampling.

Double Sampling: Sampling and Estimation

M.A. Hidiroglou and W. Jocelyn
Business Survey Methods Division, Statistics Canada

The theory of double sampling usually supposes that one large master sample is selected and a sub-sample fully contained in the master sample is taken. Such a sample design is known as two-phase sampling. In this paper, we refer to this sampling method as nested double sampling. The sample of first phase provides auxiliary information (x) at a relatively low cost, whereas the information of interest is collected via the second phase sample. The first-phase data can be used in several ways: (a) to stratify the second phase sample; (b) to improve estimation through regression modelling; (c) to sub-sample a set of non-answering units. Rao (1973) studied double nested sampling in the context of sampling and estimation for a single variable of interest.

It should be noted that it is not necessary that one of the sample be contained in the other or even be selected using the same frame. The case of non-nested double sampling is slightly discussed in classic sampling books such as Des Raj (1968) or Cochran (1977).

Several surveys at Statistics Canada use double sampling. In this article, we present some theory for allocating a sample in this context, as well as the ensuing estimation, given that we have more than one variable of interest. Examples of sample design used at Statistical Canada illustrate the use of this theory.

Mean Squared Error of Empirical Predictor

Jiming Jiang, University of California, Davis

The term empirical predictor refers to a two-stage predictor of a mixed effect, linear or nonlinear. In the first stage, a predictor is obtained but it involves unknown parameters, thus in the second stage, the unknown parameters are replaced by their estimators. In the context of small area estimation, Prasad and Rao (1990) proposed a method based on Taylor series expansion for estimating the mean squared errors (MSE) of empirical best linear unbiased predictor (EBLUP). The method is suitable for a special class of normal mixed linear models. In this talk I consider extensions of Prasad-Rao' approach in two directions. The first extension is to estimation of MSE of EBLUP in general mixed linear models, including mixed ANOVA models and longitudinal models. The second extension is to estimation of MSE of empirical best predictor in generalized linear mixed models for small area estimation.

This talk is based on joint work with Kalyan Das, Partha Lahiri and J.N.K. Rao.

Methods for Variance Estimation in Surveys with Imputed Data

Graham Kalton, J. Michael Brick, and Jae Kwang Kim

The standard methods for estimating the variances of survey estimates based on complex sample designs employ either a Taylor Series approximation or some form of replication, such as balanced repeated replication or jackknife repeated replication. However, these methods underestimate the true variance of a survey estimate when some of the data used in the estimate are imputed. Several different methods have been developed to overcome this problem, including adjusted jackknife replication methods, a fractional imputation method, model-assisted methods, and multiple imputation. This paper deals mainly with variance estimation when missing values have been imputed using some form of hot deck imputation. It reviews the various variance estimation methods with a focus on the missing data models underlying the methods.

Clarifying Some Issues in the Analysis of Survey Data

Phil Kott, USDA/NASS/RDD

The literature offers two distinct reasons for incorporating sample weights into the estimation of linear regression coefficients from a model-based point of view. Either the sample design is informative or the model is incomplete. The traditional sample-weighted least-squares estimator can be improved upon even when the sample design is informative, but not when the standard linear model fails and needs to be extended.

It is often assumed that the realized sample derives from a two-phase process. In the first phase, the finite population is drawn from a hypothetical superpopulation via simple random (cluster) sampling. In the second phase, the actual sample is drawn from the finite population. Many think that the standard practice of treating the sample as if it was drawn with replacement from the finite population is (roughly) equivalent to the full two-phase process. That is not always the case.

Bias in the Epidemiological Indicators of Risk due to
Record Linkage Errors in Cohort Mortality Studies

R. Mallick, Carleton University
D. Krewski, University of Ottawa
J.M. Zielinski, Health Canada

The advent of computerized record linkage methodology has facilitated the conduct of cohort mortality studies in which exposure data in one database are electronically linked with mortality data from another database. Epidemiological indicators of risk such as standardized mortality ratios or relative risk regression model parameters can be subject to bias and additional variability in the presence of linkage errors. In this paper, we review recent analytical results on bias and additional variations in standardized mortality ratios and logistic regression model parameters in large samples due to linkage errors. A simulation study based on data from the National Dose Registry of Canada is then used to examine bias and variation in these indicators in small samples.

Estimation of Mean Wages for Small-areas: an EBLUP Approach

M. Sverchkov, U.S. Bureau of Labor Statistics and User Technology Associates, Inc.
P. Lahiri, University of Nebraska, Lincoln and University of Maryland at College Park

We use data from the National Compensation Survey conducted by the U.S. Bureau of Labor Statistics to estimate mean wages for many cells obtained by classifying industries and geographic regions. The direct survey estimator of mean wage for a particular cell is typically unreliable because of small sample size in the cell. In order to improve on the direct survey estimator, we consider a three-level hierarchical model which connects the cells and use an empirical best linear unbiased prediction (EBLUP) method to estimate the mean wages for the cells. Based on a test that we developed, it turns out that the sampling weights can be ignored for our estimation purposes. A jackknife method is used to estimate the MSE of our proposed EBLUP.

Theoretical Foundations of the Generalised Weight Share Method

Pierre Lavallée, Statistics Canada and Jean-Claude Deville

To select the samples needed for social or economic surveys, it is useful to have sampling frames, i.e. lists of units intended to provide a way to reach desired target populations. Unfortunately, it happens that one does not have a list containing the desired collection units, but rather another list of units linked in a certain way to the list of collection units. One can speak therefore of two populations U^A and U^B linked to each other, where one wants to produce an estimate for U^B. Unfortunately, a sampling frame is only available for U^A. It can then be considered to select a sample s^A from U^A in order to produce an estimate for U^B by using the correspondence existing between the two populations. This can be designated by indirect sampling.

Estimation for a target population U^B surveyed by indirect sampling can constitute a big challenge, in particular if the links between the units of the two populations are not one-to-one. The problem comes especially from the difficulty to associate a selection probability, or an estimation weight, to the surveyed units of the target population. In order to solve this type of estimation problem, the Generalised Weight Share Method (GWSM) has been developed by Lavallée (1995) and Lavallée (2001). The GWSM provides an estimation weight for every surveyed unit of the target population U^B. Basically, this estimation weight corresponds to a weighted average of the survey weights of the units of the sample s^A.

The GWSM possesses certain desirable properties, such as unbiasedness for totals. Under mild conditions, unbiasedness is kept whatever the linkage existing between the populations U^A and U^B. This affects however the precision of the estimates produced by the GWSM.

The purpose of this paper is to describe the theoretical foundations underlying the GWSM. This will enable us to explain, for example, the unbiasedness of the method and what types of linkage produce the best precision. First, we will give an overview of the GWSM as it has been described in Lavallée (1995) and Lavallée (2001). Second, we will reformulate the GWSM in a more theoretical framework that will use, for instance, matrix notation. The use of matrix notation for the GWSM has previously been presented by Deville (1998). Third, we will use this theoretical framework to state some general properties associated to the GWSM. For example, we will study the effect of various typical matrices of links between U^A and U^B on the precision of the estimates obtained from the GWSM. Another example is the study of the effect of these typical matrices of links on the transitivity of the GWSM. Transitivity is to go from the population U^A to the target population U^B, through an intermediate population U^C.

Survival Analysis Based on Survey Data

Jerry Lawless, University of Waterloo

As longitudinal surveys in which panels of individuals are followed over a period of months or years have become common, so has the use of survival analysis methodology with survey data. This talk will review issues associated with analytic inference for survival or duration times when the data arise from a complex survey. Methodology for parametric and for semiparametric models will be described. Examples will be drawn from Canadian and U.S. national longitudinal surveys.

Estimation Problems in Multiple Frame Surveys

Sharon Lohr, Arizona State University

In a multiple frame survey, independent probability samples are taken from each of several sampling frames that together cover a target population. Multiple frame surveys can achieve
the same accuracy as a single frame survey, but with much lower costs. In this talk, we discuss some of Professor Rao's contributions in multiple frame survey theory and practice. We also (in joint work with Professor Rao) examine extensions of the Skinner/Rao (1996) pseudo-maximum-likelihood estimator to more than two frames.

A Comparison of Data Augmentation Algorithms for Small Area Empirical Bayes Estimation of Exponential Family Parameters in Hierarchical Generalized Linear Models with Random Effects

Patrick J. Farrell, Carleton University,
Brenda MacGibbon, Université du Québec à Montréal,
Thomas J. Tomberlin, Concordia University

The basic idea for the small area estimation problem studied here consists of incorporating into a generalized linear model nested random effects which reflect the complex structure of the data in a multistage sample design. However, as compared to the ordinary linear regression model, it is not feasible to obtain a closed form expression for the posterior distribution of the parameters. The approximation most used to avoid the intractable numerical integration is that originally
proposed by Laird (1978). This method, which uses the EM algorithm, has been used previously for logistic regression with random intercepts by Stiratelli, Laird and Ware (1984) and for small area estimation of proportions by Dempster and Tomberlin (1980), MacGibbon and Tomberlin (1989), Farrell, MacGibbon and Tomberlin (1994, 1997a, 1997b). Essentially, the posterior is expressed as a multivariate normal distribution having its mean at the estimated mode and covariance matrix equal to the inverse of the information matrix evaluated at this mode. Here, inspired by the work of Gu and Li (1998) on stochastic approximation computing techniques, we also study a stochastic simulation method to approximate this posterior first by assuming normality and secondly in a more general approximation by a rejection/acceptance sampling method. The results from these three methods are compared in a Monte Carlo study on simulated data from a two-stage sample design. A real data set is also presented.

Pseudo-Coordinated Bootstrap Method for Inference from Two Overlapping Sample Surveys

Harold Mantel and Owen Phillips, Statistics Canada

This paper deals with the problem of variance estimation for the difference of two cross-sectional estimates from different points in time based on partially overlapping samples. This situation may arise, for example, in the context of a rotating panel survey, or when a longitudinal panel of persons is supplemented by cohabitants or by extra sample for cross-sectional estimation. The partial overlap of the samples induces correlation in the cross-sectional estimates that must be accounted for in the variance estimation procedures. One approach to this problem is to use a coordinated bootstrap method where, for clusters that are common to the two cross-sectional samples by design, the same bootstrap subsamples of clusters are used for each time point. Roberts, Kovacevic, Mantel and Phillips (2001) proposed a pseudo-coordinated bootstrap method that can be used when cross-sectional bootstrap weights were produced independently for each of the two time points. In this paper we propose variations and refinements of the pseudo-coordinated bootstrap method, and compare them numerically using data from Statistics Canada’s Survey of Labour and Income Dynamics.