BootCI

Description

BootCI returns a 95% confidence interval using bootstrap statistics. To determine the CI it bootstraps a statistic and then uses either the plain percentile method or the bias-corrected and accelerated (BCa) method, which is more accurate for skewed sample distributions but is computationally more costly.

The user must pass a lambda function that is used to calculate the statistic for each bootstrap sample. The user has the option of using a smoothed bootstrap if the statistic is “lumpy” (eg an order statistic such as the median).

Syntax

BootCI(sample, fn, [bca=FALSE], [B=2000], [bw=0]) where

• sample is the original sample from which bootstrap samples are created. The data can be univariate (row/column vector) or multivariate (matrix, with rows as observations).
• fn is a lambda function, such as LAMBDA(x, AVERAGE(x)), that returns a single, scalar value: the statistic that is being bootstrapped.
• bca is a boolean that determines whether the BCa method is used to correct the CI boundaries. The default is FALSE meaning the plain percentile method is used.
• B is the number of bootstrap statistics to calculate. Since CIs are concerned with the tails of a distribution, the default B = 2000 is considered the minimum for reasonable accuracy using the percentile method. For the BCa method with skewed distributions the CI may reach further into the tail so it is recommended to use more bootstraps; 5000+ is often recommended.
• bw is the bandwidth to use for sampling a KDE of the sample instead of the raw sample (ie, to perform a smoothed bootstrap). The default is bw = 0 meaning that an unsmoothed bootstrap is performed. If a non-zero value is used, its meaning is the same on all KDE-based functions; a value of bw = 1 means the Silverman-recommended bandwidth is used. This argument is ignored if sample is multivariate.

The function returns two values as a column vector: the lower and upper boundaries of the CI. This is a volatile function.

Increasing B, setting bca = TRUE, and using smoothed bootstraps all increase the computational cost, so consider turning off automatic calculations or changing the result of this calculation to a value (after appropriate documentation of course).

Example

The example below compares the bootstrap CIs with non-bootstrap versions for a dataset ($N = 260$; range named sample in the workbook) with significant large outliers. A few notes:

• the non-bootstrap CIs for the mean and trimmed mean are parametric CIs: they assume a Gaussian distribution for the sample statistic
• both CIs for the median are non-parametric: the non-bootstrap version uses the inverse binomial distribution and is slightly conservative (and the boundaries are required to be in the original sample)
• the default percentile CIs are used with the bootstrap. Feel free to experiment with the BCa method using the attached Excel file
• Note that I used Excel’s TRIMMEAN function rather than truncating the tmean function. This was done for computational efficiency but one must remember to double the desired (one-tail) trimming level since Excel uses non-standard notation for the trimmed mean, giving the two-tailed trimming proportion
• For the bootstrap median CI, a smoothed bootstrap was used with bw = 1

You might be wondering: what’s all the fuss? After all the bootstrap CIs are not that much different than the non-bootstrap CIs. The reason is that the bootstrap CI is non-parametric and makes no distributional assumptions about your population. And there will not always be a theoretical or parametric version of the CI available.

Download: BootCI.xlsx

Source Code

Helpers used: BCaPercentiles, BootObs

BootCI calls helper function BCaPercentiles if the user wants a BCa CI; its source is on its own page (linked above). Note that Excel’s LET function is lazy; the variable bcaResult will not be calculated unless needed downstream (ie by the last statement, if variable bca evaluates to TRUE). Thus computation of the BCa CI, which is computationally a little more expensive, doesn’t happen unless the user requests it.

BootCI = LAMBDA(
    sample,
    fn,
    [bca],     // use the BCa method instead of the plain percentile method (default FALSE)
    [B],       // number of bootstrap samples (default 2000)
    [bw],      // smoothing bandwidth factor for the underlying Bootstrap draws (default 0 = unsmoothed)
    LET(
        B, IF(ISOMITTED(B), 2000, B),
        bca, IF(ISOMITTED(bca), FALSE, bca),
        bw, IF(ISOMITTED(bw), 0, bw),
        stats, Bootstrap(sample, fn, B, bw),
        percentileResult, VSTACK(PERCENTILE.INC(stats, 0.025), PERCENTILE.INC(stats, 0.975)),
        bcaResult, IFERROR(
            LET(
                X, BootObs(sample),       // same shape fn sees throughout Bootstrap/BootSample: matrix as-is, or a flattened column vector
                n, ROWS(X),
                thetaHat, fn(X),           // statistic on the full original sample
                idx, SEQUENCE(n),
                // leave-one-out jackknife, one extra fn call per observation (per row, for matrix input)
                thetaJack, MAP(idx, LAMBDA(i, fn(FILTER(X, idx <> i)))),
                BCaPercentiles(stats, thetaHat, thetaJack)
            ),
            "BCa error: bootstrap distribution or jackknife estimates too degenerate to compute a BCa interval (try a larger B)"
        ),
        IF(bca, bcaResult, percentileResult)
    )
);