npcdensbw computes a conbandwidth object for
estimating the conditional density of a \(p+q\)-variate kernel
density estimator defined over mixed continuous and discrete
(unordered, ordered) data using either the normal-reference
rule-of-thumb, likelihood cross-validation, or least-squares cross
validation using the method of Hall, Racine, and Li (2004).
npcdensbw(...)# S3 method for formula
npcdensbw(formula, data, subset, na.action, call, ...)
# S3 method for NULL
npcdensbw(xdat = stop("data 'xdat' missing"),
ydat = stop("data 'ydat' missing"),
bws, ...)
# S3 method for conbandwidth
npcdensbw(xdat = stop("data 'xdat' missing"),
ydat = stop("data 'ydat' missing"),
bws,
bandwidth.compute = TRUE,
nmulti,
remin = TRUE,
itmax = 10000,
ftol = 1.490116e-07,
tol = 1.490116e-04,
small = 1.490116e-05,
memfac = 500,
lbc.dir = 0.5,
dfc.dir = 3,
cfac.dir = 2.5*(3.0-sqrt(5)),
initc.dir = 1.0,
lbd.dir = 0.1,
hbd.dir = 1,
dfac.dir = 0.25*(3.0-sqrt(5)),
initd.dir = 1.0,
lbc.init = 0.1,
hbc.init = 2.0,
cfac.init = 0.5,
lbd.init = 0.1,
hbd.init = 0.9,
dfac.init = 0.375,
scale.init.categorical.sample = FALSE,
...)
# S3 method for default
npcdensbw(xdat = stop("data 'xdat' missing"),
ydat = stop("data 'ydat' missing"),
bws,
bandwidth.compute = TRUE,
nmulti,
remin,
itmax,
ftol,
tol,
small,
memfac,
lbc.dir,
dfc.dir,
cfac.dir,
initc.dir,
lbd.dir,
hbd.dir,
dfac.dir,
initd.dir,
lbc.init,
hbc.init,
cfac.init,
lbd.init,
hbd.init,
dfac.init,
scale.init.categorical.sample,
bwmethod,
bwscaling,
bwtype,
cxkertype,
cxkerorder,
cykertype,
cykerorder,
uxkertype,
uykertype,
oxkertype,
oykertype,
...)
npcdensbw returns a conbandwidth object, with the
following components:
bandwidth(s), scale factor(s) or nearest neighbours for the
explanatory data, xdat
bandwidth(s), scale factor(s) or nearest neighbours for the
dependent data, ydat
objective function value at minimum
if bwtype is set to fixed, an object containing
bandwidths (or scale factors if bwscaling = TRUE) is
returned. If it is set to generalized_nn or adaptive_nn,
then instead the \(k\)th nearest neighbors are returned for the
continuous variables while the discrete kernel bandwidths are returned
for the discrete variables.
The functions predict, summary and plot support
objects of type conbandwidth.
a symbolic description of variables on which bandwidth selection is to be performed. The details of constructing a formula are described below.
an optional data frame, list or environment (or object
coercible to a data frame by as.data.frame) containing the variables
in the model. If not found in data, the variables are taken from
environment(formula), typically the environment from which the
function is called.
an optional vector specifying a subset of observations to be used in the fitting process.
a function which indicates what should happen when the data contain
NAs. The default is set by the na.action setting of options, and is
na.fail if that is unset. The (recommended) default is
na.omit.
the original function call. This is passed internally by
np when a bandwidth search has been implied by a call to
another function. It is not recommended that the user set this.
a \(p\)-variate data frame of explanatory data on which bandwidth selection will be performed. The data types may be continuous, discrete (unordered and ordered factors), or some combination thereof.
a \(q\)-variate data frame of dependent data on which bandwidth selection will be performed. The data types may be continuous, discrete (unordered and ordered factors), or some combination thereof.
a bandwidth specification. This can be set as a conbandwidth
object returned from a previous invocation, or as a \(p+q\)-vector
of bandwidths, with each element \(i\) up to \(i=q\)
corresponding to the bandwidth for column \(i\) in ydat,
and each element \(i\) from \(i=q+1\) to \(i=p+q\)
corresponding to the bandwidth for column \(i-q\) in
xdat. In either case, the bandwidth supplied will serve as a
starting point in the numerical search for optimal bandwidths. If
specified as a vector, then additional arguments will need to be
supplied as necessary to specify the bandwidth type, kernel types,
selection methods, and so on. This can be left unset.
additional arguments supplied to specify the bandwidth type, kernel types, selection methods, and so on, detailed below.
which method to use to select
bandwidths. cv.ml specifies likelihood cross-validation,
cv.ls specifies least-squares cross-validation, and
normal-reference just computes the ‘rule-of-thumb’
bandwidth \(h_j\) using the standard formula \(h_j = 1.06
\sigma_j n^{-1/(2P+l)}\),
where \(\sigma_j\) is an adaptive measure of spread of
the \(j\)th continuous variable defined as min(standard deviation,
mean absolute deviation/1.4826, interquartile range/1.349), \(n\)
the number of observations, \(P\) the order of the kernel, and
\(l\) the number of continuous variables. Note that when there
exist factors and the normal-reference rule is used, there is zero
smoothing of the factors. Defaults to cv.ml.
a logical value that when set to TRUE the
supplied bandwidths are interpreted as ‘scale factors’
(\(c_j\)), otherwise when the value is FALSE they are
interpreted as ‘raw bandwidths’ (\(h_j\) for continuous data
types, \(\lambda_j\) for discrete data types). For
continuous data types, \(c_j\) and \(h_j\) are
related by the formula \(h_j = c_j \sigma_j n^{-1/(2P+l)}\), where \(\sigma_j\) is an
adaptive measure of spread of continuous variable \(j\) defined as
min(standard deviation, mean absolute deviation/1.4826,
interquartile range/1.349), \(n\) the number of observations,
\(P\) the order of the kernel, and \(l\) the number of
continuous variables. For discrete data types, \(c_j\) and
\(h_j\) are related by the formula \(h_j =
c_jn^{-2/(2P+l)}\), where here
\(j\) denotes discrete variable \(j\). Defaults to
FALSE.
character string used for the continuous variable bandwidth type,
specifying the type of bandwidth to compute and return in the
conbandwidth object. Defaults to fixed. Option
summary:
fixed: compute fixed bandwidths
generalized_nn: compute generalized nearest neighbors
adaptive_nn: compute adaptive nearest neighbors
a logical value which specifies whether to do a numerical search for
bandwidths or not. If set to FALSE, a conbandwidth object
will be returned with bandwidths set to those specified
in bws. Defaults to TRUE.
character string used to specify the continuous kernel type for
xdat. Can be set as gaussian,
epanechnikov, or uniform. Defaults to gaussian.
numeric value specifying kernel order for
xdat (one of
(2,4,6,8)). Kernel order specified along with a
uniform continuous kernel type will be ignored. Defaults to
2.
character string used to specify the continuous kernel type for
ydat.
Can be set as gaussian, epanechnikov, or
uniform. Defaults to gaussian.
numeric value specifying kernel order for
ydat (one of
(2,4,6,8)). Kernel order specified along with a
uniform continuous kernel type will be ignored. Defaults to
2.
character string used to specify the unordered categorical
kernel type. Can be set as aitchisonaitken or
liracine. Defaults to aitchisonaitken.
character string used to specify the unordered categorical
kernel type. Can be set as aitchisonaitken or liracine.
character string used to specify the ordered categorical
kernel type. Can be set as wangvanryzin or
liracine. Defaults to liracine.
character string used to specify the ordered categorical
kernel type. Can be set as wangvanryzin or liracine.
integer number of times to restart the process of finding extrema of the cross-validation function from different (random) initial points
a logical value which when set as TRUE the search routine
restarts from located minima for a minor gain in accuracy. Defaults
to TRUE.
integer number of iterations before failure in the numerical
optimization routine. Defaults to 10000.
fractional tolerance on the value of the cross-validation function
evaluated at located minima (of order the machine precision or
perhaps slightly larger so as not to be diddled by
roundoff). Defaults to 1.490116e-07
(1.0e+01*sqrt(.Machine$double.eps)).
tolerance on the position of located minima of the cross-validation
function (tol should generally be no smaller than the square root of
your machine's floating point precision). Defaults to
1.490116e-04 (1.0e+04*sqrt(.Machine$double.eps)).
a small number used to bracket a minimum (it is hopeless to ask for
a bracketing interval of width less than sqrt(epsilon) times its
central value, a fractional width of only about 10-04 (single
precision) or 3x10-8 (double precision)). Defaults to small
= 1.490116e-05 (1.0e+03*sqrt(.Machine$double.eps)).
lower bound, chi-square
degrees of freedom, stretch factor, and initial non-random values
for direction set search for Powell's algorithm for numeric
variables. See Details
lower bound, upper bound, stretch factor, and initial non-random values for direction set search for Powell's algorithm for categorical variables. See Details
lower bound, upper bound, and
non-random initial values for scale factors for numeric
variables for Powell's algorithm. See Details
lower bound, upper bound, and non-random initial values for scale factors for categorical variables for Powell's algorithm. See Details
a logical value that when set
to TRUE scales lbd.dir, hbd.dir,
dfac.dir, and initd.dir by \(n^{-2/(2P+l)}\),
\(n\) the number of observations, \(P\) the order of the
kernel, and \(l\) the number of numeric variables. See
Details
The algorithm to compute the least-squares objective function uses a block-based algorithm to eliminate or minimize redundant kernel evaluations. Due to memory, hardware and software constraints, a maximum block size must be imposed by the algorithm. This block size is roughly equal to memfac*10^5 elements. Empirical tests on modern hardware find that a memfac of 500 performs well. If you experience out of memory errors, or strange behaviour for large data sets (>100k elements) setting memfac to a lower value may fix the problem.
Tristen Hayfield tristen.hayfield@gmail.com, Jeffrey S. Racine racinej@mcmaster.ca
If you are using data of mixed types, then it is advisable to use the
data.frame function to construct your input data and not
cbind, since cbind will typically not work as
intended on mixed data types and will coerce the data to the same
type.
Caution: multivariate data-driven bandwidth selection methods are, by
their nature, computationally intensive. Virtually all methods
require dropping the \(i\)th observation from the data set, computing an
object, repeating this for all observations in the sample, then
averaging each of these leave-one-out estimates for a given
value of the bandwidth vector, and only then repeating this a large
number of times in order to conduct multivariate numerical
minimization/maximization. Furthermore, due to the potential for local
minima/maxima, restarting this procedure a large number of times may
often be necessary. This can be frustrating for users possessing
large datasets. For exploratory purposes, you may wish to override the
default search tolerances, say, setting ftol=.01 and tol=.01 and
conduct multistarting (the default is to restart min(5, ncol(xdat,ydat))
times) as is done for a number of examples. Once the procedure
terminates, you can restart search with default tolerances using those
bandwidths obtained from the less rigorous search (i.e., set
bws=bw on subsequent calls to this routine where bw is
the initial bandwidth object). A version of this package using the
Rmpi wrapper is under development that allows one to deploy
this software in a clustered computing environment to facilitate
computation involving large datasets.
npcdensbw implements a variety of methods for choosing
bandwidths for multivariate distributions (\(p+q\)-variate) defined
over a set of possibly continuous and/or discrete (unordered, ordered)
data. The approach is based on Li and Racine (2004) who employ
‘generalized product kernels’ that admit a mix of continuous
and discrete data types.
The cross-validation methods employ multivariate numerical search algorithms (direction set (Powell's) methods in multidimensions).
Bandwidths can (and will) differ for each variable which is, of course, desirable.
Three classes of kernel estimators for the continuous data types are available: fixed, adaptive nearest-neighbor, and generalized nearest-neighbor. Adaptive nearest-neighbor bandwidths change with each sample realization in the set, \(x_i\), when estimating the density at the point \(x\). Generalized nearest-neighbor bandwidths change with the point at which the density is estimated, \(x\). Fixed bandwidths are constant over the support of \(x\).
npcdensbw may be invoked either with a formula-like
symbolic
description of variables on which bandwidth selection is to be
performed or through a simpler interface whereby data is passed
directly to the function via the xdat and ydat
parameters. Use of these two interfaces is mutually exclusive.
Data contained in the data frames xdat and ydat may be a
mix of continuous (default), unordered discrete (to be specified in
the data frames using factor), and ordered discrete (to be
specified in the data frames using ordered). Data can be
entered in an arbitrary order and data types will be detected
automatically by the routine (see np for details).
Data for which bandwidths are to be estimated may be specified
symbolically. A typical description has the form dependent data
~ explanatory data,
where dependent data and explanatory data are both
series of variables specified by name, separated by
the separation character '+'. For example, y1 + y2 ~ x1 + x2
specifies that the bandwidths for the joint distribution of variables
y1 and y2 conditioned on x1 and x2 are to
be estimated. See below for further examples.
A variety of kernels may be specified by the user. Kernels implemented for continuous data types include the second, fourth, sixth, and eighth order Gaussian and Epanechnikov kernels, and the uniform kernel. Unordered discrete data types use a variation on Aitchison and Aitken's (1976) kernel, while ordered data types use a variation of the Wang and van Ryzin (1981) kernel.
The optimizer invoked for search is Powell's conjugate direction
method which requires the setting of (non-random) initial values and
search directions for bandwidths, and, when restarting, random values
for successive invocations. Bandwidths for numeric variables
are scaled by robust measures of spread, the sample size, and the
number of numeric variables where appropriate. Two sets of
parameters for bandwidths for numeric can be modified, those
for initial values for the parameters themselves, and those for the
directions taken (Powell's algorithm does not involve explicit
computation of the function's gradient). The default values are set by
considering search performance for a variety of difficult test cases
and simulated cases. We highly recommend restarting search a large
number of times to avoid the presence of local minima (achieved by
modifying nmulti). Further refinement for difficult cases can
be achieved by modifying these sets of parameters. However, these
parameters are intended more for the authors of the package to enable
‘tuning’ for various methods rather than for the user themselves.
Aitchison, J. and C.G.G. Aitken (1976), “Multivariate binary discrimination by the kernel method,” Biometrika, 63, 413-420.
Hall, P. and J.S. Racine and Q. Li (2004), “Cross-validation and the estimation of conditional probability densities,” Journal of the American Statistical Association, 99, 1015-1026.
Li, Q. and J.S. Racine (2007), Nonparametric Econometrics: Theory and Practice, Princeton University Press.
Pagan, A. and A. Ullah (1999), Nonparametric Econometrics, Cambridge University Press.
Scott, D.W. (1992), Multivariate Density Estimation. Theory, Practice and Visualization, New York: Wiley.
Silverman, B.W. (1986), Density Estimation, London: Chapman and Hall.
Wang, M.C. and J. van Ryzin (1981), “A class of smooth estimators for discrete distributions,” Biometrika, 68, 301-309.
if (FALSE) {
# EXAMPLE 1 (INTERFACE=FORMULA): For this example, we compute the
# likelihood cross-validated bandwidths (default) using a second-order
# Gaussian kernel (default). Note - this may take a minute or two
# depending on the speed of your computer.
data("Italy")
attach(Italy)
bw <- npcdensbw(formula=gdp~ordered(year))
# The object bw can be used for further estimation using
# npcdens(), plotting using plot() etc. Entering the name of
# the object provides useful summary information, and names() will also
# provide useful information.
summary(bw)
# Note - see the example for npudensbw() for multiple illustrations
# of how to change the kernel function, kernel order, and so forth.
detach(Italy)
# EXAMPLE 1 (INTERFACE=DATA FRAME): For this example, we compute the
# likelihood cross-validated bandwidths (default) using a second-order
# Gaussian kernel (default). Note - this may take a minute or two
# depending on the speed of your computer.
data("Italy")
attach(Italy)
bw <- npcdensbw(xdat=ordered(year), ydat=gdp)
# The object bw can be used for further estimation using
# npcdens(), plotting using plot() etc. Entering the name of
# the object provides useful summary information, and names() will also
# provide useful information.
summary(bw)
# Note - see the example for npudensbw() for multiple illustrations
# of how to change the kernel function, kernel order, and so forth.
detach(Italy)
}
Run the code above in your browser using DataLab