Use this guide to connect probability, estimation, regression, time series and statistical learning. Each concept explains a mathematical rule or modeling decision, resolves an original example and identifies a specific error to avoid. Work through the foundations first, then use the modeling sections to compare assumptions and interpret results.
Probability models and stochastic processes
1. Conditional probability and Bayes' rule
Conditional probability changes the population under consideration. Bayes' rule reverses conditioning: P(A given B) equals P(B given A)P(A)/P(B), provided P(B) is positive. Compute the denominator across all possible groups; an event can be relatively common within a small group without making that group the most likely source.
Worked example: High-risk policies form 20% of a portfolio. Claim probabilities are 10% for high-risk and 2% otherwise. Given a claim, the high-risk probability is 0.02/(0.02 + 0.016) = 5/9.
Mistake to avoid: Treating the probability of a claim given high risk as the probability of high risk given a claim.
Context reference: Casualty Actuarial Society
2. Means, variances and covariance in sums
Expectations add whenever they exist, regardless of independence. Variances require covariance terms: Var(X + Y) = Var(X) + Var(Y) + 2Cov(X,Y). Independence implies zero covariance when the relevant moments exist, but zero covariance alone does not generally establish independence. Keep variance in squared units.
Worked example: Two losses have means 4 and 7, variances 9 and 16, and covariance 3. Their total has mean 11 and variance 9 + 16 + 6 = 31.
Mistake to avoid: Adding standard deviations, or dropping covariance merely because the losses belong to different categories.
Context reference: Casualty Actuarial Society
3. The law of total expectation
The law of total expectation averages conditional means using the probabilities of the conditioning groups: E(X) = E[E(X given Z)]. It applies to discrete groups and continuous conditioning variables. The weights describe the population being modeled, so changes in group composition can change the overall mean even when group means stay constant.
Worked example: One policy group represents 25% of exposures and averages six claims per hundred exposures; the remainder averages two. The portfolio average is 0.25 × 6 + 0.75 × 2 = 3.
Mistake to avoid: Taking an unweighted average of group means when the groups have unequal exposure.
Context reference: Casualty Actuarial Society
4. Within-group and between-group variance
Total variance separates into E[Var(X given Z)] plus Var[E(X given Z)]. The first term measures variation within groups; the second measures differences between group means. A mixture can therefore be substantially more variable than its average conditional variance, even when each component is relatively stable.
Worked example: Two equally likely groups have means 2 and 6 and variances 1 and 3. Average conditional variance is 2; variance of the conditional means is 4. Total variance is 6.
Mistake to avoid: Using only the average group variance and overlooking variation caused by different group means.
Context reference: Casualty Actuarial Society
5. Transforming a continuous random variable
For a monotone transformation Y = g(X), transform the support and use the inverse transformation to derive the distribution. The density includes the absolute derivative of the inverse. A cumulative-distribution calculation is often safer because it forces you to identify exactly which values of X satisfy Y ≤ y.
Worked example: If X is uniform on (0,2) and Y = X², then F_Y(y) = √y/2 for 0 < y < 4. Differentiating gives f_Y(y) = 1/(4√y).
Mistake to avoid: Substituting the inverse into the original density without multiplying by the inverse derivative.
Context reference: Casualty Actuarial Society
6. Poisson counts and exposure
A Poisson count with mean μ has probability P(N = k) = exp(−μ)μ^k/k! and variance μ. In a homogeneous Poisson process, μ equals the event rate multiplied by interval length. The model also assumes independent counts over disjoint intervals; matching a mean alone does not establish a suitable process model.
Worked example: At a rate of four events per hour, a half-hour interval has mean 2. Its probability of no events is exp(−2), approximately 0.1353.
Mistake to avoid: Using the hourly rate directly as the mean for an interval of a different length.
Context reference: Casualty Actuarial Society
7. Exponential waiting times and memorylessness
In a homogeneous Poisson process with rate λ, the waiting time to the next event is exponential, with survival probability exp(−λt). Memorylessness means P(T > s + t given T > s) = P(T > t). This property belongs to this particular waiting-time model and should not be assumed for arbitrary durations.
Worked example: With two events per hour, the probability of waiting another half-hour after already waiting one hour is exp(−1), approximately 0.3679.
Mistake to avoid: Subtracting elapsed time from the mean waiting time as though the remaining wait were deterministic.
Context reference: Casualty Actuarial Society
8. Poisson thinning and superposition
Independently classifying each event of a Poisson process with constant probability p creates a Poisson subprocess with rate pλ. The complementary subprocess is independent under these assumptions. Conversely, adding independent Poisson processes adds their rates. Dependence or history-dependent classification can invalidate these conclusions.
Worked example: A process produces ten reports daily. Independent tagging sends 30% to review, giving review rate 3 and ordinary rate 7. The probability of no review reports in a day is exp(−3).
Mistake to avoid: Applying independent thinning when tagging depends on recent events or shared conditions.
Context reference: Casualty Actuarial Society
9. Random sums and aggregate losses
For S = X₁ + ⋯ + X_N, assume independent, identically distributed severities that are also independent of N. Then E(S) = E(N)E(X), and Var(S) = E(N)Var(X) + Var(N)[E(X)]². The second term captures count uncertainty; using a fixed count removes that source of variation.
Worked example: If N is Poisson with mean 3 and severity has mean 100 and variance 400, aggregate mean is 300 and variance is 3 × 400 + 3 × 10,000 = 31,200.
Mistake to avoid: Multiplying severity variance by expected count while omitting count variability.
Context reference: Casualty Actuarial Society
10. Markov transitions over multiple steps
A time-homogeneous Markov chain uses the current state to determine the next-state distribution. Its transition matrix P has nonnegative entries and rows summing to one. Two-step probabilities come from P², which sums over every possible intermediate state. A row probability vector evolves as π_next = π_current P.
Worked example: For P = [[0.8, 0.2], [0.3, 0.7]], the probability of returning to state A after two steps from A is 0.8² + 0.2 × 0.3 = 0.70.
Mistake to avoid: Squaring a single transition probability and ignoring paths through other states.
Context reference: Casualty Actuarial Society
11. Stationary distributions and convergence
A stationary distribution satisfies πP = π and has nonnegative components summing to one. It describes a distribution unchanged by one transition. For a finite irreducible chain, it is unique; aperiodicity additionally supports convergence from arbitrary initial distributions. Stationarity and convergence are related but distinct properties.
Worked example: For transitions A to B of 0.2 and B to A of 0.3, balance gives 0.2π_A = 0.3π_B. With π_A + π_B = 1, the stationary distribution is (0.6, 0.4).
Mistake to avoid: Assuming that solving the stationary equations automatically proves convergence for a periodic chain.
Context reference: Casualty Actuarial Society
12. Survival functions and hazard rates
For a nonnegative continuous duration, the survival function is S(t) = P(T > t), and the hazard is h(t) = f(t)/S(t) where S(t) is positive. The cumulative hazard H(t) gives S(t) = exp[−H(t)]. Hazard is an instantaneous conditional rate, so it is not itself an interval probability.
Worked example: If h(t) = 0.2t for t ≥ 0, then H(t) = 0.1t². Survival to time 2 is exp(−0.4), approximately 0.6703.
Mistake to avoid: Interpreting a hazard value of 0.4 as a 40% probability over an unspecified interval.
Context reference: Casualty Actuarial Society
Statistical inference and parameter estimation
13. Likelihood as a function of the parameter
A likelihood treats observed data as fixed and varies the unknown parameter. For independent observations, multiply their probability masses or densities; log-likelihood turns this product into a sum. Parameter-free factors may be omitted when locating a maximum, but they can matter when comparing differently defined data likelihoods.
Worked example: Three independent Bernoulli observations are 1, 0 and 1. Their likelihood is p²(1 − p), and log-likelihood is 2 log p + log(1 − p).
Mistake to avoid: Treating likelihood as a probability distribution over parameter values without specifying a prior.
Context reference: Casualty Actuarial Society
14. Maximum likelihood and boundary solutions
Maximum likelihood selects the parameter value that maximizes the likelihood within its allowed parameter space. Solving a derivative equation produces candidates, not an automatic answer: inspect curvature, boundaries and existence. Boundary estimates can require different inferential treatment from regular interior estimates.
Worked example: For the Bernoulli data 1, 0, 1, setting 2/p − 1/(1 − p) to zero gives p = 2/3. If all observations are successes, the maximum occurs at p = 1.
Mistake to avoid: Rejecting a legitimate boundary maximum because the derivative has no interior zero.
Context reference: Casualty Actuarial Society
15. Method-of-moments estimation
Method of moments equates sample moments to their population counterparts and solves for the parameters. Use as many independent moment equations as unknown parameters, and check that the solution lies in the parameter space. These estimators can be convenient without having the same efficiency as maximum likelihood estimators.
Worked example: An exponential distribution with rate λ has mean 1/λ. If the sample mean duration is 2.5, the moment estimate is λ = 1/2.5 = 0.4.
Mistake to avoid: Confusing an exponential rate parameter with its scale, which is the reciprocal.
Context reference: Casualty Actuarial Society
16. Bias, variance and mean squared error
For an estimator T of θ, bias is E(T) − θ. Mean squared error decomposes into Var(T) plus squared bias. Unbiasedness therefore does not guarantee the smallest estimation error; a modestly biased estimator can perform better if its variance is sufficiently lower. Compare estimators on the same target and sampling basis.
Worked example: Estimator A is unbiased with variance 9. Estimator B has bias 1 and variance 4. Their mean squared errors are 9 and 5, respectively.
Mistake to avoid: Choosing an estimator solely because its bias is zero.
Context reference: Casualty Actuarial Society
17. Why sample variance uses n minus one
For independent, identically distributed observations with finite variance, the sample variance using denominator n − 1 is unbiased for the population variance. Estimating the mean consumes one degree of freedom. The denominator n can arise in maximum likelihood estimation under a normal model, so identify the estimator being requested.
Worked example: For observations 2, 4 and 6, the mean is 4 and squared deviations sum to 8. The unbiased sample variance is 8/2 = 4; the normal-model maximum likelihood estimate is 8/3.
Mistake to avoid: Switching denominators without distinguishing unbiased estimation from maximum likelihood.
Context reference: Casualty Actuarial Society
18. Sufficient statistics and factorization
A statistic is sufficient when it retains all sample information about the parameter under the specified model. The factorization criterion writes the likelihood as a parameter-dependent function of the statistic times a parameter-free function of the data. Sufficiency depends on the model; it does not mean retaining every feature of the observations.
Worked example: For independent Poisson observations with common mean λ, likelihood is exp(−nλ)λ^(sum x_i)/product(x_i!). Thus the sample sum is sufficient for λ.
Mistake to avoid: Assuming that a sufficient statistic for one distribution remains sufficient after changing the model.
Context reference: Casualty Actuarial Society
19. Fisher information and estimation precision
Under suitable regularity conditions, Fisher information measures the expected curvature of the log-likelihood. Independent observations contribute additive information. A regular maximum likelihood estimator has approximate variance equal to the inverse total information for large samples. This approximation needs caution near boundaries or when the model violates regularity conditions.
Worked example: For n Bernoulli observations, information is n/[p(1 − p)]. At p = 0.25 and n = 100, approximate estimator variance is 0.25 × 0.75/100 = 0.001875.
Mistake to avoid: Using single-observation information as though it were the information in the entire sample.
Context reference: Casualty Actuarial Society
20. The delta method for transformed estimates
The delta method approximates the variance of g(T) by [g′(θ)]²Var(T) when T is concentrated near θ and g is smooth. It translates uncertainty to a transformed scale using a local linear approximation. Strong curvature, small samples or estimates near singularities can make the approximation unreliable.
Worked example: An estimate of a positive mean is 4 with standard error 0.2. For its logarithm, the approximate standard error is 0.2/4 = 0.05.
Mistake to avoid: Transforming interval endpoints and transforming standard errors as if these were the same operation.
Context reference: Casualty Actuarial Society
21. Confidence intervals and their target
A confidence interval is constructed by a procedure with specified repeated-sampling coverage. For a normal population mean with known standard deviation σ, use sample mean ± z × σ/√n. The interval concerns the population mean, rather than the spread of individual observations or the uncertainty of a future outcome.
Worked example: With sample mean 20, known σ = 6 and n = 36, a 95% interval uses standard error 1 and multiplier approximately 1.96, giving (18.04, 21.96).
Mistake to avoid: Claiming that 95% of individual observations lie within a confidence interval for the mean.
Context reference: Casualty Actuarial Society
22. P-values and hypothesis-test decisions
A p-value is the probability, under the null model, of results at least as incompatible with the null as the observed result, using the chosen statistic. It is not the probability that the null hypothesis is true. Tail selection must follow the alternative hypothesis rather than the observed direction of the data.
Worked example: For a standard normal test statistic z = 2, a two-sided p-value is approximately 0.0455. A prespecified upper-tailed test instead gives approximately 0.0228.
Mistake to avoid: Halving a two-sided p-value after seeing the data to create a more favorable one-sided result.
Context reference: Casualty Actuarial Society
23. Type I error, Type II error and power
Type I error rejects a true null; Type II error fails to reject under a specified alternative. Power is one minus the Type II error probability and depends on the actual alternative, variability, sample size and rejection rule. Lowering the significance level generally reduces power when other conditions stay fixed.
Worked example: Reject a null mean of 0 when the sample mean exceeds 1.645, with standard error 1. If the true mean is 2, power is P(Z > −0.355), approximately 0.639.
Mistake to avoid: Describing power without specifying the alternative value at which it is evaluated.
Context reference: Casualty Actuarial Society
24. Bayesian updating with a conjugate prior
Bayesian inference combines prior information with the likelihood to form a posterior distribution. A Beta(a,b) prior for a Bernoulli probability becomes Beta(a + successes, b + failures). Its posterior mean is a weighted combination of the prior mean and observed proportion, with weights determined by their respective counts.
Worked example: A Beta(2,3) prior and four successes in ten trials produce Beta(6,9). The posterior mean is 6/15 = 0.4.
Mistake to avoid: Adding the total trial count to both prior parameters instead of separating successes and failures.
Context reference: Casualty Actuarial Society
Linear and generalized linear models
25. Least squares and projection
Ordinary least squares minimizes the sum of squared residuals. With a full-rank design matrix, the solution satisfies X′Xβ_hat = X′y. Fitted values are the projection of the response onto the columns of X, so residuals are orthogonal to those columns. Including an intercept makes the residuals sum to zero.
Worked example: For points (0,1), (1,2) and (2,4), the fitted line is y_hat = 5/6 + 1.5x. Residuals are 1/6, −1/3 and 1/6, which sum to zero.
Mistake to avoid: Assuming residuals must sum to zero when the fitted model has no intercept.
Context reference: Casualty Actuarial Society
26. Categorical predictors and reference levels
With an intercept, a categorical predictor having k levels typically uses k − 1 indicator columns. The omitted level is the reference, and each coefficient compares another level with it, holding other predictors fixed. Changing the reference changes coefficient representation while preserving fitted values for the same model.
Worked example: With group A as reference, an intercept of 100 and coefficients B = 20 and C = −10 predict means 100, 120 and 90 for A, B and C.
Mistake to avoid: Including every level indicator plus an intercept and expecting uniquely identifiable coefficients.
Context reference: Casualty Actuarial Society
27. Interactions and conditional effects
An interaction allows one predictor's effect to depend on another predictor. In y_hat = β₀ + β₁x + β₂z + β₃xz, the slope for x is β₁ + β₃z. Main effects must therefore be interpreted at the other variable's reference value, which may become more meaningful after centering.
Worked example: For y_hat = 5 + 2x + 3z + 4xz with binary z, the x slope is 2 when z = 0 and 6 when z = 1.
Mistake to avoid: Reporting β₁ as a common slope across all values of z despite including an interaction.
Context reference: Casualty Actuarial Society
28. Regression assumptions and what they justify
A correctly specified conditional mean and errors with conditional mean zero support unbiased least-squares coefficients under standard design conditions. Homoscedastic, uncorrelated errors justify conventional variance formulas. Normal errors additionally support exact normal-model t and F inference; they are not required merely to calculate the least-squares fit.
Worked example: Suppose residual spread rises with policy exposure. The fitted coefficients can still target the conditional mean correctly, but conventional homoscedastic standard errors may be inappropriate.
Mistake to avoid: Treating any failure of normality as proof that least-squares coefficients cannot estimate the mean relationship.
Context reference: Casualty Actuarial Society
29. Leverage, residuals and influence
Leverage measures how unusual an observation's predictor configuration is relative to the fitted design. A residual measures disagreement between its response and the fitted value. Influence concerns how much the fitted model changes when the observation is altered or removed. High leverage can create influence, but does not guarantee it.
Worked example: An observation at x = 20 when most x values lie between 0 and 2 has high leverage. If it lies exactly on the fitted line, its residual is zero.
Mistake to avoid: Calling every high-leverage observation an influential outlier without examining its response and effect on the fit.
Context reference: Casualty Actuarial Society
30. Exponential-family mean and variance structure
A generalized linear model combines an exponential-family response distribution, a linear predictor and a link function. In a common representation, mean is b′(θ) and variance is φb″(θ). The distribution therefore specifies a mean-variance relationship, rather than allowing arbitrary constant residual variance.
Worked example: For a Poisson response, b(θ) = exp(θ) and φ = 1. Both mean and variance equal exp(θ), so larger fitted means imply larger conditional variance.
Mistake to avoid: Applying constant-variance normal-model reasoning to every generalized linear model.
Context reference: Casualty Actuarial Society
31. Links and inverse links
A link function maps the conditional mean μ to the linear predictor η = Xβ. Predictions on the response scale require the inverse link. For a log link, μ = exp(η), which guarantees positive means and makes coefficient changes multiplicative. The link acts on the mean, not necessarily on each observed response.
Worked example: A log-link model has η = 1 + 0.2x. Moving x from 0 to 1 multiplies the predicted mean by exp(0.2), approximately 1.221.
Mistake to avoid: Interpreting 0.2 as an additive increase of 0.2 response units.
Context reference: Casualty Actuarial Society
32. Logistic coefficients and odds
Logistic regression models log[p/(1 − p)] as a linear predictor. Exponentiating a coefficient gives an odds ratio for a one-unit predictor change, holding other predictors fixed. Because probability is a nonlinear transformation of odds, the corresponding probability change depends on the starting probability.
Worked example: Starting probability 0.2 gives odds 0.25. A coefficient log 2 doubles the odds to 0.5, producing probability 0.5/1.5 = 1/3.
Mistake to avoid: Interpreting an odds ratio of 2 as doubling the probability.
Context reference: Casualty Actuarial Society
33. Poisson regression with exposure offsets
For counts observed over differing exposures, a log-exposure offset expresses μ_i = exposure_i × exp(X_iβ). Its coefficient is fixed at one, encoding proportionality between exposure and expected count. Estimated coefficients describe changes in the modeled rate. Exposure must be positive and measured on a consistent basis.
Worked example: A fitted annual rate of 0.08 claims per policy-year and exposure of 250 policy-years imply expected count 250 × 0.08 = 20.
Mistake to avoid: Treating log exposure as an ordinary estimated predictor when proportional exposure adjustment is the intended assumption.
Context reference: Casualty Actuarial Society
34. Gamma models for positive responses
A Gamma generalized linear model is appropriate only for strictly positive continuous responses under its assumed distribution. Its variance function is proportional to μ², allowing variability to grow with the mean. A log link keeps predicted means positive, but the log link is a modeling choice rather than the Gamma canonical link.
Worked example: With dispersion φ = 0.25 and fitted mean 100, conditional variance is 0.25 × 100² = 2,500 and standard deviation is 50.
Mistake to avoid: Fitting a standard Gamma response model to exact zeros without a separate, justified modeling treatment.
Context reference: Casualty Actuarial Society
35. Deviance and nested-model comparisons
Deviance compares fitted likelihood with a saturated model that fits observations individually. For nested models using the same observations and a suitable likelihood, a deviance reduction can support a likelihood-ratio comparison. A chi-square approximation needs regularity conditions and appropriate dispersion treatment; it is not automatic for every fitted model.
Worked example: Nested Poisson models have deviances 120 and 112, and the larger model adds two parameters. The difference is 8; its approximate chi-square p-value with two degrees of freedom is 0.0183.
Mistake to avoid: Subtracting deviances from models fitted to different datasets and treating the result as a valid nested comparison.
Context reference: Casualty Actuarial Society
36. Overdispersion in count models
A Poisson model requires conditional variance to equal conditional mean. Excess variability can arise from omitted heterogeneity, dependence or an unsuitable count distribution. Pearson residuals can help diagnose dispersion, but a dispersion adjustment does not repair an incorrect mean structure or dependence. Diagnose the source before selecting an alternative model.
Worked example: A Pearson statistic of 180 on 90 residual degrees of freedom gives estimated dispersion 2. A quasi-Poisson adjustment would multiply conventional standard errors by √2, approximately 1.414.
Mistake to avoid: Using an overall sample variance above the sample mean as conclusive evidence of conditional overdispersion.
Context reference: Casualty Actuarial Society
Time series structure and forecasting
37. Weak stationarity
A weakly stationary process has a constant finite mean, a constant finite variance and covariance depending only on lag. Stationarity allows observations separated by the same lag to share a covariance structure across time. A trend or changing variance can violate it even when a plot appears locally stable.
Worked example: For X_t = 5 + ε_t with independent mean-zero errors of variance 4, the mean is 5, variance is 4 and covariance at every nonzero lag is zero.
Mistake to avoid: Calling a process stationary solely because its observations remain within a visually narrow range.
Context reference: Casualty Actuarial Society
38. Autocorrelation and white noise
Autocorrelation at lag k is covariance at that lag divided by the common variance for a stationary process. Weak white noise has constant mean, constant variance and zero autocovariance at nonzero lags. Zero autocorrelation does not generally imply independence; Gaussian assumptions provide stronger conclusions than second-moment conditions alone.
Worked example: If lag-one covariance is 3 and variance is 12, lag-one autocorrelation is 0.25. This indicates linear association between adjacent observations.
Mistake to avoid: Interpreting zero autocorrelation as proof that no nonlinear temporal dependence exists.
Context reference: Casualty Actuarial Society
39. Stationary autoregressive models
An AR(1) model is X_t = c + φX_(t−1) + ε_t. With white-noise innovations and |φ| < 1, its stationary mean is c/(1 − φ), variance is σ²_ε/(1 − φ²), and lag-k autocorrelation is φ^k. Distinguish the intercept c from the long-run mean.
Worked example: With c = 2, φ = 0.5 and innovation variance 3, the stationary mean is 4 and variance is 3/0.75 = 4.
Mistake to avoid: Reporting the autoregressive intercept as the unconditional mean.
Context reference: Casualty Actuarial Society
40. Moving-average models and sign conventions
For MA(1), X_t = μ + ε_t + θε_(t−1), variance is σ²_ε(1 + θ²), lag-one autocorrelation is θ/(1 + θ²), and higher-lag autocorrelations are zero. Some texts use a minus sign before θ, which changes signs. Invertibility conventionally selects |θ| < 1 for this representation.
Worked example: With θ = 0.5 and innovation variance 4, variance is 5 and lag-one autocorrelation is 0.5/1.25 = 0.4.
Mistake to avoid: Memorizing an autocorrelation sign without checking how the moving-average equation defines θ.
Context reference: Casualty Actuarial Society
41. Distinguishing AR and MA signatures
For ideal stationary AR(p) models, the partial autocorrelation function cuts off after lag p while the autocorrelation function generally tails off. For MA(q), the autocorrelation cuts off after q while partial autocorrelation generally tails off. Sample plots contain estimation noise, so these signatures suggest candidate structures rather than proving an order.
Worked example: A sample autocorrelation plot with prominent lags one and two followed by small values suggests considering MA(2), then checking fitted residuals and competing models.
Mistake to avoid: Selecting model order from a single isolated sample correlation without considering sampling variation.
Context reference: Casualty Actuarial Society
42. Differencing trends and seasonal structure
First differencing forms ΔX_t = X_t − X_(t−1), while seasonal differencing forms X_t − X_(t−s). These transformations can remove particular nonstationary components, but excessive differencing can introduce unnecessary dependence. Differencing changes the forecasted quantity, so forecasts must be accumulated appropriately to return to the original scale.
Worked example: Levels 100, 105, 109 and 116 yield first differences 5, 4 and 7. A forecast next difference of 6 implies next level 116 + 6 = 122.
Mistake to avoid: Reporting a forecast change as the forecast level.
Context reference: Casualty Actuarial Society
43. Multi-step forecasting and innovation uncertainty
For stationary AR(1), the h-step forecast is μ + φ^h(X_t − μ). With known parameters, forecast-error variance is σ²_ε times the sum of φ^(2j) for j from zero to h − 1. Forecasts approach the long-run mean while uncertainty accumulates; estimated parameters add further uncertainty.
Worked example: With μ = 10, φ = 0.5, X_t = 14 and innovation variance 4, the two-step forecast is 11 and forecast-error variance is 4(1 + 0.25) = 5.
Mistake to avoid: Using the one-step innovation variance unchanged for every forecasting horizon.
Context reference: Casualty Actuarial Society
44. Residual checks and chronological validation
A time-series model should leave residuals without systematic temporal structure relevant to its assumptions. Examine residual autocorrelation and changing spread, and evaluate forecasts using information available before each forecast origin. Rolling-origin validation respects time order and can reveal deterioration that random train-test splits conceal.
Worked example: Fit through December, forecast January, then refit through January and forecast February. Average the resulting forecast losses to assess repeated one-step predictive performance.
Mistake to avoid: Training on observations that occur after the forecasted period through a random split.
Context reference: Casualty Actuarial Society
Statistical learning and classification
45. Prediction error and the bias-variance tradeoff
Under squared-error loss with mean-zero outcome noise, expected prediction error decomposes into irreducible noise, squared prediction bias and prediction variance. Greater flexibility can reduce bias while increasing variance across training samples. The relevant tradeoff concerns new observations, rather than how closely a model reproduces its training data.
Worked example: Model A has squared bias 4, prediction variance 1 and noise variance 2, totaling 7. Model B has components 1, 5 and 2, totaling 8.
Mistake to avoid: Choosing the more flexible model solely because it has lower training error.
Context reference: Casualty Actuarial Society
46. Data leakage through preprocessing
Validation data must remain outside every training-dependent operation, including imputation, scaling, feature selection and tuning. Learn preprocessing rules on the training portion and apply those fitted rules to held-out observations. Leakage can occur without using outcome labels if future or held-out information influences the fitted workflow.
Worked example: Training values have mean 10 and standard deviation 2. A held-out value of 14 becomes (14 − 10)/2 = 2 using training scaling parameters.
Mistake to avoid: Computing scaling statistics on the entire dataset before cross-validation.
Context reference: Casualty Actuarial Society
47. Cross-validation and model selection
K-fold cross-validation repeatedly fits on K − 1 folds and evaluates on the remaining fold. Compare candidates using identical folds and a loss suited to the prediction task. Reusing the same cross-validation results for extensive tuning and final performance claims can introduce selection optimism; separate evaluation or nested validation can address it.
Worked example: Two models have three equally sized fold losses of (2,3,4) and (3,3,2). Their mean losses are 3 and 8/3, so the second performs better under this comparison.
Mistake to avoid: Comparing models on different partitions and attributing every loss difference to model quality.
Context reference: Casualty Actuarial Society
48. Nearest-neighbor prediction
K-nearest-neighbor methods predict from training observations closest to the query under a chosen distance. Regression commonly averages their responses; classification commonly uses class proportions. Small K produces local, variable predictions, while larger K smooths more broadly. Predictor scales strongly affect distance and should reflect the intended notion of similarity.
Worked example: The three nearest training responses are 8, 10 and 15. A three-neighbor regression prediction is (8 + 10 + 15)/3 = 11.
Mistake to avoid: Including the query's known response among its neighbors when evaluating a held-out prediction.
Context reference: Casualty Actuarial Society
49. Linear and quadratic discriminant analysis
Linear discriminant analysis models class-conditional Gaussian predictors with a shared covariance matrix, producing linear decision boundaries. Quadratic discriminant analysis allows separate covariance matrices and generally produces quadratic boundaries. QDA estimates more parameters, so its flexibility may increase estimation variance when class samples are limited.
Worked example: For equally likely one-dimensional classes with means 0 and 2 and shared variance 1, LDA assigns an observation to the second class when x > 1.
Mistake to avoid: Assuming that separate class variances always improve out-of-sample classification.
Context reference: Casualty Actuarial Society
50. Classification thresholds and error costs
A predicted probability becomes a class decision only after selecting a threshold. With calibrated probability p, false-positive cost C_FP, false-negative cost C_FN and zero cost for correct decisions, predict positive when p > C_FP/(C_FP + C_FN). The rule assumes these costs capture the decision consequences and that no additional constraints apply.
Worked example: If a false positive costs 1 and a false negative costs 4, the threshold is 1/5 = 0.2. A probability of 0.3 therefore leads to a positive decision.
Mistake to avoid: Using 0.5 automatically when the two error costs differ.
Context reference: Casualty Actuarial Society
51. Confusion matrices and class imbalance
Sensitivity is TP/(TP + FN), specificity is TN/(TN + FP), and precision is TP/(TP + FP). Each conditions on a different population. Accuracy can look strong when the majority class dominates, while precision changes with prevalence even if sensitivity and specificity remain constant.
Worked example: With TP = 30, FN = 10, FP = 20 and TN = 940, sensitivity is 75%, specificity is 940/960, and precision is 60%. Accuracy is 97%.
Mistake to avoid: Using high accuracy to conclude that a model identifies the minority class well.
Context reference: Casualty Actuarial Society
52. Decision-tree splits and leaf predictions
A regression tree partitions predictor space and predicts the average training response within each leaf. A squared-error split is judged by the reduction in within-node squared deviations. Classification trees instead use criteria such as Gini impurity or entropy. Evaluate the weighted child impurity, not an unweighted average of differently sized children.
Worked example: Responses 1, 2, 8 and 9 have mean 5 and squared-error total 50. Splitting into (1,2) and (8,9) gives total 1, a reduction of 49.
Mistake to avoid: Choosing a split because one child is very pure while ignoring the other child's size and impurity.
Context reference: Casualty Actuarial Society
53. Bagging and random forests
Bagging averages models fitted to bootstrap samples, often reducing variance for unstable learners such as deep trees. Random forests additionally restrict candidate predictors at splits to reduce correlation among trees. Averaging is more effective when errors are less correlated; increasing the number of trees cannot eliminate systematic bias shared by all trees.
Worked example: For 100 equally variable predictions with variance 9 and pairwise correlation 0.2, average variance is 9[0.2 + 0.8/100] = 1.872.
Mistake to avoid: Assuming that averaging many highly correlated trees removes nearly all predictive uncertainty.
Context reference: Casualty Actuarial Society
54. Boosting as sequential loss reduction
Boosting builds an additive model sequentially, with each new learner addressing errors under a specified loss. For squared-error gradient boosting, the negative gradient is the current residual. Learning rate controls update size, while tree complexity and iteration count control flexibility. Classification losses generally yield gradients different from ordinary numeric residuals.
Worked example: A current prediction is 10, and a fitted residual learner predicts 4. With learning rate 0.1, the updated prediction is 10 + 0.1 × 4 = 10.4.
Mistake to avoid: Adding each learner at full strength when the fitted procedure specifies shrinkage.
Context reference: Casualty Actuarial Society
Regularization and dimension reduction
55. Ridge regression and correlated predictors
Ridge regression minimizes squared error plus λ times the sum of squared slope coefficients, typically leaving the intercept unpenalized. Positive λ shrinks coefficients and stabilizes estimation when predictors are correlated. Ridge usually retains nonzero coefficients, so its main purpose is controlled estimation and prediction rather than automatic variable selection.
Worked example: With centered data, one predictor has sum x² = 10 and sum xy = 30. Under objective SSE + λβ² with λ = 5, the ridge coefficient is 30/(10 + 5) = 2.
Mistake to avoid: Expecting ridge to set unimportant coefficients exactly to zero.
Context reference: Casualty Actuarial Society
56. Lasso and exact coefficient zeros
Lasso uses an absolute-value penalty, whose corner at zero can produce exactly zero coefficients. It combines shrinkage with variable selection, but correlated predictors can make the selected set unstable. Numerical formulas depend on how the loss and penalty are scaled, so read the objective before applying a threshold.
Worked example: For the one-coefficient objective 0.5(β − 3)² + 2|β|, soft thresholding gives β = 1. If the unpenalized coefficient were 1 instead, the solution would be zero.
Mistake to avoid: Using a memorized lasso threshold without checking the objective's scaling.
Context reference: Casualty Actuarial Society
57. Elastic net and combined penalties
Elastic net combines absolute-value and squared-coefficient penalties. The absolute component can create zeros, while the squared component helps stabilize correlated predictors. A mixing parameter controls their balance, but software conventions differ. Interpret the actual objective and compare methods with consistent scaling and validation.
Worked example: Under penalty λ[α sum |β_j| + (1 − α) sum β_j²/2], α = 1 gives lasso and α = 0 gives ridge. Setting α = 0.4 retains both penalty components.
Mistake to avoid: Assuming the same mixing-parameter convention without checking how a method defines its penalty.
Context reference: Casualty Actuarial Society
58. Standardization and penalty fairness
Coefficient penalties depend on predictor units. Standardizing predictors using training means and standard deviations makes a common penalty operate on comparable one-standard-deviation effects. Store these transformations for prediction and convert coefficients back when interpreting original units. Standardization does not establish that every predictor deserves equal substantive importance.
Worked example: If z = (x − 100)/20 and its coefficient is 6, the original-scale slope is 6/20 = 0.3. The intercept must also absorb the centering adjustment.
Mistake to avoid: Changing predictor units while keeping the same penalty and expecting unchanged fitted predictions.
Context reference: Casualty Actuarial Society
59. Choosing shrinkage along a regularization path
A regularization path fits models across penalty strengths. Select λ by predictive performance within the training workflow, rather than by training fit or a preferred coefficient size. A one-standard-error rule can favor greater regularization among candidates near the minimum estimated validation loss; it is a selection heuristic, not a significance test.
Worked example: The minimum mean validation loss is 10 with estimated standard error 0.5. A more regularized model with mean loss 10.3 qualifies under a one-standard-error rule using the 10.5 cutoff.
Mistake to avoid: Choosing λ on the final test set and then presenting that same test error as independent evaluation.
Context reference: Casualty Actuarial Society
60. Principal components and partial least squares
Principal component regression fits the response using selected directions of high predictor variance; constructing those directions does not use the response. Partial least squares uses response information when constructing predictive directions. Neither method guarantees that a small component set preserves every important signal, and response-informed components must be recomputed within each validation training fold.
Worked example: A high-variance predictor is unrelated to the response, while a low-variance predictor carries the signal. Keeping only the first principal component can discard the useful direction.
Mistake to avoid: Assuming that the direction explaining the most predictor variance must be the best direction for prediction.
Context reference: Casualty Actuarial Society
Sources
Source context:
All study guides