What Is Econometrics? Step-by-Step Guide with Examples

Econometrics Definition: Explained Theory and Numerical Examples

Many economists think that economics theory must be backed up with empirical evidence. Theories must be tested and proven with real world data to validate the theory. Data is essential for making a statistical or mathematical proof of an economic theory, or for predicting future economic changes. 

Econometrics means “economic measurement”. It combines economic theory, mathematics, and statistics to measure relationships between economic variables, test theories, and forecast the future. Economic theory usually tells us the direction of a relationship (for example, a higher price lowers demand). Econometrics quantifies the effect and provides the evidence behind it. Standard textbook treatments include Gujarati and Porter (2009), Wooldridge (2020), Stock and Watson (2019), and Greene (2018); full references are listed at the end of this article.

That is where econometrics fits in: econometrics is a vital tool that merges economics, mathematics and statistics to test theories and quantify future forecasts.

Its three main goals are:

• Estimation: finding numerical values, such as “each extra PKR 1,000 of income raises consumption by PKR 725”.
• Hypothesis testing: checking whether a relationship is real or just luck in the sample.
• Forecasting: predicting inflation, GDP, exchange rate or demand.

The Econometric Methodology in 6 Steps

1. State the theory: consumption depends on income (Keynesian consumption function).
2. Specify the model: Y = β₀ + β₁X + u.
3. Collect data: surveys, SBP, PBS or World Bank.
4. Estimate the parameters: usually with Ordinary Least Squares (OLS).
5. Test and diagnose: t-test, F-test, R², and checks for model problems.
6. Use the model: policy analysis, prediction and decision making.

Econometrics?

A theory may say that “higher income raises consumption”, but only econometrics tells you by how much, how sure you can be, and whether the result will hold tomorrow. In this guide, we cover the full subject, from basic theory to regression, hypothesis testing, diagnostic tests, and time series, and every concept is backed by a small worked numerical example that you can repeat with a calculator or Excel.

Three Types of Data

Statistical Foundations You Need First

Econometrics is built on a few statistical ideas. If these are clear, everything later becomes easier. We use the same ten-household data from Example 1 below (income X and consumption Y), where X̄ = 55 and Ȳ = 44.2.

Variance, Covariance and Correlation

Variance measures how spread out a variable is: s² = Σ(X − X̄)² / (n − 1). For income, s² = 5000 / 9 = 555.6 and the standard deviation is √555.6 = 23.57. For consumption, Σ(Y − Ȳ)² = 2659.6, so s² = 295.5 and the standard deviation is 17.19.

Covariance shows whether two variables move together: Cov(X, Y) = Σ(X − X̄)(Y − Ȳ) / (n − 1) = 3625 / 9 = 402.8. It is positive, so income and consumption rise together. But its size depends on the units, so we standardise it into the correlation coefficient:

r = Cov(X, Y) / (s_X × s_Y) = 402.8 / (23.57 × 17.19) = 0.994

Correlation always lies between −1 and +1. A value of 0.994 means a very strong positive linear relationship. Notice that r² = 0.994² = 0.988, which is exactly the R² we will compute in Example 1. In simple regression, R² is just the squared correlation.

The Logic of Hypothesis Testing

1. State the hypotheses: the null H₀ (usually “no effect”, β = 0) and the alternative H₁.
2. Choose a significance level α, commonly 5% (0.05).
3. Compute a test statistic such as t = estimate / standard error.
4. Compare it with the critical value, or compare the p-value with α.
5. Decide: reject H₀ if |t| is above the critical value, or if p < α.

A Type I error means rejecting a true null (a false alarm, with probability α). A Type II error means failing to reject a false null (a missed effect). The power of a test is the probability of correctly detecting a real effect; it rises with sample size.

Numerical illustration: in Example 2 below, household size has t = 2.15 with 7 degrees of freedom. The two-tailed p-value is about 0.069. Since 0.069 is above 0.05 we cannot reject H₀ at 5%, but since it is below 0.10 we can reject it at 10%. The p-value therefore tells you how strong the evidence is, not just yes or no.

What Makes a Good Estimator?

• Unbiased: on average, across many samples, the estimator equals the true value.
• Efficient: among unbiased estimators it has the smallest variance.
• Consistent: as the sample grows, the estimate converges to the true value.

The Gauss–Markov theorem says OLS is the most efficient among all linear unbiased estimators when the classical assumptions hold, which is why it is called BLUE (Gujarati and Porter, 2009; Wooldridge, 2020).

The Simple Linear Regression Model

The simplest econometric model has one explanatory variable:

Y_i = β₀ + β₁X_i + u_i

• Y is the dependent variable (what we want to explain), here consumption.
• X is the independent variable (the explanatory factor), here income.
• β₀ is the intercept: the value of Y when X is zero.
• β₁ is the slope: the change in Y for a one-unit change in X.
• u is the error term: everything else that affects Y but is not in the model.

Since we never know the true β values, we estimate them from a sample. The estimated line is written Ŷ = b₀ + b₁X.

What Does OLS Do?

Ordinary Least Squares picks the line that makes the sum of squared residuals as small as possible. A residual is the gap between actual Y and the value predicted by the line. Squaring stops positive and negative gaps from cancelling each other. The formulas are:

b₁ = Σ(X − X̄)(Y − Ȳ) / Σ(X − X̄)²     b₀ = Ȳ − b₁X̄

Numerical Example 1: Estimating the Consumption Function

Ten households report monthly income (X) and monthly consumption (Y), both in thousand PKR. The means are X̄ = 550 ÷ 10 = 55 and Ȳ = 442 ÷ 10 = 44.2.

Step 1: Slope and Intercept

b₁ = 3625 / 5000 = 0.725

b₀ = 44.2 − (0.725 × 55) = 44.2 − 39.875 = 4.325

Ŷ = 4.325 + 0.725X

Interpretation: if monthly income rises by PKR 1,000, consumption rises by about PKR 725 on average. The number 0.725 is the Marginal Propensity to Consume (MPC), so the household saves the remaining 27.5%. The intercept of 4.325 is the “autonomous consumption” predicted at zero income.

Step 2: Fitted Values and Residuals

The residual sum of squares is RSS = 31.475. Notice the residuals add up to zero, which is always true in OLS when the model has an intercept.

Step 3: How Good Is the Fit? (R²)

R² tells us what share of the variation in Y is explained by the model. First compute the Total Sum of Squares, TSS = Σ(Y − Ȳ)² = 2659.6. Then:

ESS = TSS − RSS = 2659.6 − 31.475 = 2628.125

R² = ESS / TSS = 2628.125 / 2659.6 = 0.988

So about 98.8% of the variation in consumption is explained by income. This is very high because the data is artificial. With real household data an R² of 0.3 to 0.6 is already considered good, so do not judge a model by R² alone.

Step 4: Standard Errors and the t-Test

Is the slope truly different from zero, or could it be sampling luck? We test H₀: β₁ = 0 against H₁: β₁ ≠ 0.

σ² = RSS / (n − 2) = 31.475 / 8 = 3.934

SE(b₁) = √(σ² / Σ(X−X̄)²) = √(3.934 / 5000) = 0.0281

t = b₁ / SE(b₁) = 0.725 / 0.0281 = 25.85

With n − 2 = 8 degrees of freedom, the 5% two-tailed critical value is 2.306. Since 25.85 is far larger, we reject H₀: income significantly affects consumption. The same method gives SE(b₀) = 1.665 and t = 2.60, so the intercept is also significant at 5%.

Step 5: Confidence Interval

95% CI for β₁ = 0.725 ± (2.306 × 0.0281) = 0.660 to 0.790

We are 95% confident the true MPC lies between 0.66 and 0.79.

Step 6: The F-Test and Prediction

In simple regression the F-statistic equals the square of t: F = 25.85² = 668, far above the 5% critical value F(1, 8) = 5.32, so the model as a whole is significant.

Prediction: for a household earning PKR 60 thousand, Ŷ = 4.325 + 0.725 × 60 = 47.825, meaning about PKR 47,825 of consumption.

The Classical Assumptions (Why OLS Works)

OLS gives the Best Linear Unbiased Estimators (BLUE), as stated by the Gauss–Markov theorem, only when these assumptions hold:

1. The model is linear in parameters.
2. The errors have a zero mean.
3. Homoskedasticity: the error variance is constant.
4. No autocorrelation: errors are not correlated with each other.
5. No perfect multicollinearity: no explanatory variable is an exact combination of others.
6. Exogeneity: the errors are not correlated with the explanatory variables.

When an assumption fails, estimates can become biased or their standard errors unreliable. The next sections show how to detect each problem.

OLS in Matrix Form

Software never uses the one-variable formulas. For any number of regressors it solves the compact matrix formula:

b = (X′X)⁻¹ X′y     Var(b) = σ² (X′X)⁻¹

Numerical Example: A Five-Observation Regression

Let X = 1, 2, 3, 4, 5 and Y = 2, 4, 5, 4, 5. The X matrix has a column of ones (for the intercept) next to the X values. Then:

• X′X = [[5, 15], [15, 55]] (5 observations, ΣX = 15, ΣX² = 55)
• X′y = [20, 66] (ΣY = 20, ΣXY = 66)
• Determinant = 5 × 55 − 15 × 15 = 50, so (X′X)⁻¹ = (1/50) × [[55, −15], [−15, 5]]

b₀ = (55 × 20 − 15 × 66)/50 = 110/50 = 2.2    b₁ = (−15 × 20 + 5 × 66)/50 = 30/50 = 0.6

You can check the slope with the scalar formula: Σ(X − 3)(Y − 4) / Σ(X − 3)² = 6 / 10 = 0.6. The matrix method gives the same answer and extends unchanged to ten regressors. The diagonal of σ²(X′X)⁻¹ gives the variances of the coefficients, from which standard errors and t-values come.

Multiple Regression

Real outcomes depend on many factors. Multiple regression adds more variables: Y = β₀ + β₁X₁ + β₂X₂ + u. Each slope now measures the effect of one variable holding the others constant.

Numerical Example 2: Adding Household Size

We add household size (Z) to the same ten households, with Z = 3, 4, 3, 5, 4, 6, 5, 6, 5, 7. Solving the normal equations (in practice with Excel, Stata, R or Python) gives:

Ŷ = 0.811 + 0.657X + 1.511Z

Reading the result: holding household size constant, PKR 1,000 more income raises consumption by about PKR 657. Holding income constant, one extra family member adds about PKR 1,511. Notice that the income coefficient dropped from 0.725 to 0.657. This happens because richer households in our sample are also larger, and the simple model was giving income credit for the effect of household size.

R² vs Adjusted R²

R² never falls when you add a variable, even a useless one. Adjusted R² penalises extra variables:

Adj. R² = 1 − (1 − R²)(n − 1)/(n − k − 1)

For the multiple model, R² = 0.9929 and Adjusted R² = 1 − (0.0071 × 9/7) = 0.9908. For the simple model the adjusted value was 0.9867. Adjusted R² rose, so adding household size improved the model.

Prediction: for income = 60 and household size = 4, Ŷ = 0.811 + 0.657(60) + 1.511(4) = 46.28.

Hypothesis Tests in Multiple Regression

Testing Several Coefficients Together: The Joint F-Test

A t-test checks one coefficient. To test whether one or more variables together matter, compare a restricted model (without them) to an unrestricted model (with them):

F = [(RSS_R − RSS_U) / q] / [RSS_U / (n − k − 1)]

Here q is the number of restrictions. In Example 2 we test H₀: the household size coefficient is zero (q = 1). The restricted model is the simple regression with RSS_R = 31.475, and the unrestricted model has RSS_U = 18.968 with n − k − 1 = 7.

F = [(31.475 − 18.968) / 1] / [18.968 / 7] = 12.507 / 2.710 = 4.62

The 5% critical value of F(1, 7) is 5.59. Since 4.62 is smaller, we cannot reject H₀. Notice that 4.62 is exactly the square of the t-value of household size (2.148² = 4.62): for a single restriction the F-test and the two-sided t-test are the same test. The F-test becomes truly useful when testing two or more coefficients jointly.

Omitted Variable Bias

Leaving out a relevant variable that is correlated with an included one biases the included coefficient. The size of the bias is:

Bias = β₂ × δ

where β₂ is the true effect of the omitted variable and δ is the slope from regressing the omitted variable on the included one. In our data, household size (Z) regressed on income (X) gives δ = 0.045. The true effect of Z is β₂ = 1.511, so the bias is 1.511 × 0.045 = 0.068. That is exactly why the simple model gave b₁ = 0.725 instead of the multiple-regression value 0.657 (0.657 + 0.068 = 0.725).

The opposite error, including an irrelevant variable, does not bias the estimates but makes them less precise by inflating standard errors. When theory says a variable matters, it is safer to include it.

Choosing Between Models: AIC and BIC

Adjusted R² is one guide. Information criteria (Akaike, 1974; Schwarz, 1978) go further by balancing fit against the number of parameters k. For OLS: AIC = n ln(RSS/n) + 2k and BIC = n ln(RSS/n) + k ln(n). Lower values are better.

Both criteria prefer the multiple model, consistent with the adjusted R². BIC punishes extra variables more heavily than AIC, so when the two disagree BIC favours the smaller model.

Dummy Variables: Putting Categories into a Model

Qualitative factors like gender, education level, urban/rural or a policy change can enter a regression as dummy variables that take the value 1 or 0.

Numerical Example 3: The Degree Premium

Six employees earn these monthly salaries (thousand PKR). Three hold a BS degree (D = 1): 80, 90, 100. Three do not (D = 0): 50, 60, 70. Regress salary on D:

Salary = β₀ + β₁D + u

• Mean salary without a degree = (50 + 60 + 70)/3 = 60, so b₀ = 60.
• Mean salary with a degree = (80 + 90 + 100)/3 = 90, so b₀ + b₁ = 90 and b₁ = 30.

Employees with a degree earn PKR 30,000 more on average, in this sample. A dummy coefficient always measures the gap between the category and the base group. Remember to leave one category out (the base), otherwise you fall into the “dummy variable trap”, which is perfect multicollinearity.

Functional Forms: Elasticity with Log-Log Models

Economists often want percentage effects rather than unit effects. In a log-log model, ln Y = α + β ln X, the slope β is directly the elasticity: the percentage change in Y for a 1% change in X.

Taking natural logs of our ten households and re-estimating gives ln Y = 0.117 + 0.918 ln X. So a 1% rise in income raises consumption by about 0.92%. As a cross-check, the linear model’s elasticity at the means is b₁ × (X̄/Ȳ) = 0.725 × (55/44.2) = 0.90, which is very close.

Three Major Problems in Econometrics (and How to Detect Them)

1. Multicollinearity

It occurs when explanatory variables are highly correlated with each other. Estimates stay unbiased, but standard errors inflate and t-values become unreliable. The standard detector is the Variance Inflation Factor:

VIF = 1 / (1 − R²_j)

In Example 2, income and household size have a correlation of r = 0.806. With only two regressors, R²_j = r² = 0.649, so VIF = 1 / (1 − 0.649) = 2.85. A common rule says VIF above 10 (some use 5) signals a serious problem, so ours is acceptable. Remedies include dropping or combining variables, or collecting more data.

2. Heteroskedasticity

The error variance changes with X. A typical case is spending: rich households vary far more in what they spend than poor ones. OLS stays unbiased, but standard errors are wrong, so t-tests mislead. A simple Breusch–Pagan test (Breusch and Pagan, 1979) works like this:

1. Run the original regression and save the residuals.
2. Square them (e²) and regress e² on X.
3. Compute the test statistic n × R² from this auxiliary regression. It follows a chi-square distribution with 1 degree of freedom (here).

For Example 1 the auxiliary regression gives R² = 0.0755, so the statistic is 10 × 0.0755 = 0.755. The 5% critical chi-square value is 3.84. Since 0.755 is smaller, we do not reject constant variance. If it was rejected, the usual fix is robust (White) standard errors or a log transformation. With just 10 observations the test has little power, so treat this as a demonstration of the method.

3. Autocorrelation

Common in time series: today’s error is related to yesterday’s. A recession year’s shock spills into the next year. The classic detector is the Durbin–Watson (DW) statistic (Durbin and Watson, 1950):

DW = Σ(e_t − e_t−1)² / Σe_t²     ≈ 2(1 − ρ)

DW is near 2 when there is no autocorrelation, near 0 for strong positive autocorrelation and near 4 for strong negative autocorrelation.

Numerical Example 4: Detecting Autocorrelation

An output index for 12 years is regressed on a time trend (t = 1 to 12). The estimated trend line is Ŷ = 96.42 + 3.755t.

From the residuals: Σ(e_t − e_t−1)² = 73.17 and Σe_t² = 91.10, so:

DW = 73.17 / 91.10 = 0.80

For n = 12 and one regressor at 5% significance, the Durbin–Watson table (extended by Savin and White, 1977) gives d_L = 0.971 and d_U = 1.331. Because 0.80 is below d_L, we reject the no-autocorrelation hypothesis: there is positive autocorrelation (ρ ≈ 1 − 0.80/2 = 0.60). Notice the residuals come in runs of the same sign (negative, then positive, then negative), which is the visual clue. Remedies include adding lagged variables, the Cochrane–Orcutt procedure (Cochrane and Orcutt, 1949), first-differencing, or Newey–West robust standard errors (Newey and West, 1987).

More Specification and Diagnostic Tests

Normality of Errors: The Jarque–Bera Test

The t and F tests assume normally distributed errors in small samples. The Jarque–Bera test (Jarque and Bera, 1987) uses the skewness (S) and kurtosis (K) of the residuals:

JB = (n/6) × [S² + (K − 3)² / 4]

For the residuals of Example 1, S = 0.534 and K = 1.85. Then JB = (10/6) × [0.534² + (1.85 − 3)²/4] = 1.667 × [0.285 + 0.331] = 1.03. Under H₀ (normal errors) JB follows a chi-square distribution with 2 degrees of freedom, whose 5% critical value is 5.99. Since 1.03 is smaller, we do not reject normality. With only 10 observations this test is weak, so treat it as a method demonstration.

Functional Form: The RESET Test

Ramsey’s (1969) RESET test checks whether the linear form is adequate. You add the squared (and cubed) fitted values to the regression and use an F-test on them. If they are significant, the model likely misses a nonlinearity or an omitted variable, and you should try logs, squares or extra regressors.

Structural Breaks: The Chow Test

Does a relationship stay stable over time? A policy change or crisis may change the slope. The Chow test (Chow, 1960) compares one pooled regression against two separate regressions on each sub-period:

F = [(RSS_P − RSS₁ − RSS₂) / k] / [(RSS₁ + RSS₂) / (n − 2k)]

Take a 12-year output index: 100, 103, 106, 110, 112, 115 | 121, 127, 132, 139, 144, 150, and test a break after year 6 (k = 2 parameters).

F = [(70.160 − 0.819 − 0.800) / 2] / [(0.819 + 0.800) / 8] = 34.27 / 0.2024 = 169.3

The 5% critical value of F(2, 8) is 4.46, so we strongly reject stability: the growth rate jumped from about 3.0 to 5.8 index points a year after year 6.

Fixing the Problems: Robust Errors, WLS and GLS

• Robust (White or HC) standard errors (White, 1980) keep the OLS coefficients but correct the standard errors for heteroskedasticity. They are the default choice in applied work.
• Newey–West (HAC) standard errors correct for both heteroskedasticity and autocorrelation.
• Weighted Least Squares (WLS) gives less weight to noisy observations. If Var(u) is proportional to X², divide every term of the model by X and run OLS on the transformed model, which restores constant variance.
• Generalised Least Squares (GLS), including Cochrane–Orcutt and Prais–Winsten, handles autocorrelation by transforming the data with the estimated ρ.
• Measurement error: if an explanatory variable is measured with random noise, its OLS coefficient is biased toward zero (attenuation bias). Instrumental variables are the standard remedy.

Outliers and Influential Observations

One extreme household can pull the whole regression line. Inspect scatter plots and residual plots, and use measures such as Cook’s distance and leverage to find influential points. Do not simply delete them: investigate whether they are data-entry errors or genuine cases, and report results with and without them.

Time Series Econometrics

Macroeconomic data such as inflation, GDP and exchange rates arrive over time, and time series have special features: trends, seasonality, and persistence.

Stationarity

A series is stationary when its mean, variance and autocovariance do not change over time. Non-stationary series (like a rising price index) can produce spurious regression: a high R² and a significant t-value between two variables that have nothing to do with each other, simply because both trend upward. This is why we test for stationarity before regressing time series.

Numerical Example 5: An AR(1) Model for Inflation

An autoregressive model of order one says today’s value depends on yesterday’s: π_t = c + φπ_t−1 + ε_t. Take 14 quarters of hypothetical inflation (%):

Regressing π_t on π_t−1 using the 13 available pairs gives:

π̂_t = 5.652 + 0.459 π_t−1    (SE of φ = 0.212)

• Persistence: since |φ| = 0.459 is below 1, the series is stationary and shocks fade out gradually.
• Long-run mean: c / (1 − φ) = 5.652 / (1 − 0.459) = 10.45%.
• Forecast: the last observation is 10.6%, so the next quarter forecast is 5.652 + 0.459 × 10.6 = 10.52%.

Numerical Example 6: The Dickey–Fuller Unit Root Test

To test for non-stationarity formally, estimate Δy_t = α + γy_t−1 + ε_t and test H₀: γ = 0 (unit root, non-stationary) against H₁: γ < 0 (stationary). Consider a hypothetical price index:

The regression gives γ = −0.0235 with a t-statistic of −0.29. Important: this t-value is not compared with the normal t-table. The Dickey–Fuller (Dickey and Fuller, 1979) 5% critical value (with intercept; more complete tables are in MacKinnon, 1996) is about −2.86 for large samples, and even more negative for small ones. Because −0.29 is much higher than −2.86, we fail to reject H₀: the series has a unit root. The usual remedy is to difference the series (work with Δy, which here is the growth in the index) and test again. Such a series is called integrated of order one, I(1).

Cointegration and the Error Correction Model

Two non-stationary I(1) series can still have a stable long-run relationship, for example consumption and income. If the residuals of their regression are stationary, the variables are cointegrated (tested by the Engle and Granger, 1987, or Johansen, 1991, procedures). Then the Error Correction Model (ECM) links short-run changes to the deviation from the long-run equilibrium, and the “speed of adjustment” coefficient tells how quickly the gap closes.

Differencing and Random Walks with Drift

The price index in Example 6 is non-stationary, but its first differences are well behaved: 1.5, 2.3, −0.7, 2.1, 2.7, −0.5, 2.2, 2.7, −0.5, 2.4, 2.7, −0.8, 2.7. Their average is 18.8 / 13 = 1.45 index points per quarter. A simple “random walk with drift” forecast for the next quarter is therefore 118.8 + 1.45 = 120.25.

ACF, PACF and the ARIMA (Box–Jenkins) Method

ARIMA(p, d, q) combines p autoregressive terms, d differences and q moving-average terms. The Box–Jenkins method (Box and Jenkins, 1970) has four steps:

1. Identify: make the series stationary by differencing d times, then use the autocorrelation function (ACF) and partial autocorrelation function (PACF) to choose p and q. A PACF that cuts off after lag p suggests AR(p); an ACF that cuts off after lag q suggests MA(q).
2. Estimate the parameters.
3. Diagnose: the residuals should look like white noise (for example, a Ljung–Box test that does not reject).
4. Forecast and compare alternative models using AIC or BIC.

Our inflation AR(1) in Example 5 is simply an ARIMA(1, 0, 0). Seasonal data (such as monthly CPI, where food prices jump around Ramadan and harvest time) can be handled with seasonal dummies, seasonal differencing or a seasonal ARIMA (SARIMA).

Measuring Forecast Accuracy

A model should be judged on forecasts it has not seen. Suppose the actual values are 100, 110, 105, 120 and the forecasts are 98, 114, 102, 121, giving errors of 2, −4, 3 and −1.

RMSE punishes large errors more than MAE does. MAPE is easy to explain to non-technical readers, but it breaks down when actual values are near zero.

Granger Causality

Variable X “Granger-causes” Y (Granger, 1969) if past values of X help predict Y beyond what past values of Y already do. It is a test of predictive content, not true cause and effect. The test is an F-test comparing a model of Y on its own lags (restricted) with a model that also includes lags of X (unrestricted).

Numerical illustration: with n = 40 observations, suppose RSS_R = 120, RSS_U = 90, two lags of X are added (q = 2) and the unrestricted model has 5 parameters. Then F = [(120 − 90)/2] / [90/35] = 15 / 2.571 = 5.83. The 5% critical value of F(2, 35) is about 3.27, so we reject H₀ and conclude X Granger-causes Y. A typical use is testing whether money supply growth helps predict inflation.

Vector Autoregression (VAR)

When several variables influence each other, such as inflation, interest rate and exchange rate, a VAR (Sims, 1980) treats each as a function of lags of all of them. VARs are used for forecasting, Granger tests, impulse response functions (how a shock to the policy rate spreads over time) and variance decomposition. If the variables are I(1) and cointegrated, the VAR is written in error-correction form (VECM).

Volatility: ARCH and GARCH Models

Financial and macro series often show volatility clustering: calm periods and turbulent periods. ARCH and GARCH models (Engle, 1982; Bollerslev, 1986) let the variance itself change over time. The GARCH(1,1) model is:

σ²_t = ω + α ε²_t−1 + β σ²_t−1

With ω = 0.1, α = 0.1, β = 0.8, yesterday’s squared shock ε² = 4 and yesterday’s variance σ² = 2, today’s variance is 0.1 + 0.1 × 4 + 0.8 × 2 = 2.1. The long-run variance is ω / (1 − α − β) = 0.1 / 0.1 = 1.0, so volatility will gradually fall back toward 1 if no new shocks arrive. These models are common for stock-market (for example, KSE-100) and exchange-rate returns.

Maximum Likelihood Estimation: The Idea Behind Logit and Probit

OLS minimizes squared errors. Maximum Likelihood Estimation (MLE) instead chooses the parameter values that make the observed data most probable. It is the engine behind logit, probit, GARCH and many other models.

Numerical Example: Estimating a Probability

A bank approves 7 of 10 loan applications. If p is the approval probability, the log-likelihood is ln L(p) = 7 ln p + 3 ln(1 − p).

• At p = 0.7: ln L = 7 × (−0.3567) + 3 × (−1.2040) = −6.109.
• At p = 0.5: ln L = 10 × (−0.6931) = −6.931.

The likelihood is higher at p = 0.7, and calculus confirms the maximum occurs at p̂ = 7/10 = 0.7. We can also run a likelihood ratio (LR) test of H₀: p = 0.5. LR = 2 × (−6.109 − (−6.931)) = 1.65, which is below the chi-square critical value 3.84 (1 degree of freedom), so with only 10 applications we cannot reject that p = 0.5. LR tests are used to compare nested logit, probit and time-series models.

MLE estimators are consistent and asymptotically efficient, but they need a correctly specified distribution and a reasonably large sample.

Models for Binary Outcomes: Logit and Probit

When the dependent variable is yes/no (loan approved or not, employed or not), a straight-line OLS model can predict probabilities below 0 or above 1. Logit and Probit models solve this by using an S-shaped curve. In the logit model the log of the odds is linear: ln(p / (1 − p)) = β₀ + β₁X.

Numerical Example 7: Reading a Logit Result

Suppose a model of loan approval gives ln(odds) = −1 + 0.8 × (years of banking relationship).

• Odds ratio: e^0.8 = 2.23. Each extra year of relationship multiplies the odds of approval by about 2.23.
• Probability at 2 years: z = −1 + 0.8 × 2 = 0.6, so p = 1 / (1 + e^−0.6) = 1 / (1 + 0.549) = 0.646, or a 64.6% chance of approval.

Logit coefficients are not marginal effects. The effect on probability depends on where you are on the curve, so software reports “average marginal effects”.

Other Limited-Dependent-Variable Models

• Ordered logit/probit: for ranked outcomes such as low, medium and high income.
• Multinomial logit: for unordered choices such as the mode of transport.
• Tobit (Tobin, 1958): when the outcome is censored, for example spending that is zero for many households.
• Poisson and negative binomial: for count data such as the number of bank branches or patents.

Panel Data Models

Panel data follows the same units (countries, firms, banks, households) over several periods. It lets you control for unobserved differences that stay constant over time, such as management quality or geography, which cross-section data cannot do.

Pooled OLS, Fixed Effects and Random Effects

• Pooled OLS ignores the panel structure and treats all observations alike. It is biased if unit-specific effects are correlated with X.
• Fixed Effects (FE) gives every unit its own intercept, by subtracting each unit’s mean from its variables (the “within” transformation).
• Random Effects (RE) treats unit effects as random draws. It is more efficient than FE but needs the effects to be uncorrelated with X.

Numerical Example: Why Fixed Effects Matter

Two firms are observed for three years each. Both have the same true slope of 2 (Y = α + 2X), but Firm B has a higher baseline (α = 6 against α = 1 for Firm A) and also uses more input.

Pooled OLS on all six points gives a slope of 3.29, which is wrong because it confuses the baseline gap between the firms with the effect of X. Fixed effects uses the within-firm deviations: slope = Σ(x̃ỹ) / Σ(x̃²) = 8 / 4 = 2.00, the true value. The firm intercepts are recovered as α_A = 5 − 2 × 2 = 1 and α_B = 16 − 2 × 5 = 6.

Hausman test (Hausman, 1978): to choose between FE and RE, compare their coefficients. A significant Hausman statistic means the unit effects are correlated with X, so you should use fixed effects. Panel models are also widely used for dynamic questions (with lagged dependent variables, estimated by GMM; see Hansen, 1982, and Arellano and Bond, 1991) and for bank-level and cross-country studies.

Endogeneity, Instrumental Variables and Simultaneous Equations

A variable is endogenous when it is correlated with the error term, which makes OLS biased and inconsistent. There are three main causes: omitted variables (such as ability in a wage regression), measurement error, and simultaneity (when X and Y determine each other, like price and quantity).

Instrumental Variables (IV) and 2SLS

An instrument Z must satisfy two conditions: relevance (Z is correlated with X) and exogeneity (Z affects Y only through X and is not correlated with the error). Two-Stage Least Squares (2SLS) works in two steps: first regress X on Z and save the fitted values; then regress Y on those fitted values.

Numerical Example: Returns to Schooling

We want the effect of schooling on wages, but ability affects both, so OLS is biased. Suppose living near a college (Z = 1) raises schooling, and that distance affects wages only through schooling. In a sample, people near a college have average schooling of 14 years and average wage of PKR 62 thousand; those far away have 12 years and PKR 50 thousand.

β_IV = (Ȳ_Z=1 − Ȳ_Z=0) / (X̄_Z=1 − X̄_Z=0) = (62 − 50) / (14 − 12) = 6

Each extra year of schooling raises the monthly wage by about PKR 6,000. This is the Wald form of the IV estimator. In practice, always test the instrument’s strength (a first-stage F-statistic above about 10 is the usual rule) and, when you have more instruments than needed, run an over-identification test (Sargan or Hansen). The Hausman test can also tell whether OLS and IV differ significantly.

Simultaneous Equation Models

Demand and supply are the classic case: price and quantity are determined together, so a regression of quantity on price mixes both curves. Such models are written as structural equations, and each equation must be identified. The order condition says an equation is identified if the number of exogenous variables excluded from it is at least the number of endogenous variables included on its right-hand side. Identified equations are estimated by 2SLS or 3SLS, never by plain OLS.

Causal Inference: Difference-in-Differences

Regression shows association; causal designs try to isolate a cause. The simplest is difference-in-differences (DiD) (see Card and Krueger, 1994; Angrist and Pischke, 2009): compare how a treated group changed with how a similar untreated group changed over the same period.

Suppose a district receives a microfinance programme (treated) while a similar district does not (control). Average monthly household income, in thousand PKR:

The control group shows that income would have risen by about 7 anyway (inflation, growth). The programme’s effect is the extra 8. In regression form, Y = β₀ + β₁Treated + β₂After + β₃(Treated × After) + u, and β₃ is the DiD estimate. The key assumption is parallel trends: without the programme, both districts would have followed the same trend. Other designs include regression discontinuity (comparing people just above and below a cutoff) and randomised controlled trials.

Applications of Econometrics in Pakistan

Here are typical research questions in a Pakistani context, the model that fits each, and where the data comes from:

Because many Pakistani macro series are short (often 30 to 40 annual observations), unit root tests have low power and results can be sensitive to specification. Always report robustness checks.

How to Report Your Results Professionally

A Standard Regression Table

Report coefficients with standard errors (or t-values) in brackets, significance stars, the number of observations, R² or adjusted R², and the main diagnostic tests. For example, from Example 2: Consumption = 0.811 + 0.657X + 1.511Z, with standard errors 2.141, 0.039 and 0.704, n = 10 and adjusted R² = 0.991.

How to Write the Interpretation

A good sentence states the size, the unit, the direction, the condition and the uncertainty. For example: “Holding household size constant, an extra PKR 1,000 of monthly income is associated with about PKR 657 more monthly consumption (95% confidence interval 0.56 to 0.75 in thousand rupees).” Avoid saying “causes” unless you have a causal design.

A Practical Checklist

1. Define the research question and the economic theory behind it.
2. Clean the data: units, missing values, outliers, definitions.
3. Plot the data before estimating anything.
4. For time series, test for unit roots and decide between levels, differences or cointegration.
5. Estimate the model and check coefficient signs against theory.
6. Run diagnostic tests: VIF, heteroskedasticity, autocorrelation, normality, RESET and stability.
7. Use robust standard errors where needed and check alternative specifications.
8. Report results transparently, including limitations.

Which Software Should You Use?

The simple regression in Example 1 can be reproduced in Excel with the function =SLOPE(Y range, X range) and =INTERCEPT(Y range, X range), or =LINEST for the full output.

Common Mistakes Beginners Make

• Confusing correlation with causation. A strong association does not prove that X causes Y.
• Chasing R². A high R² with a wrong model structure is worthless; spurious regressions have very high R².
• Ignoring diagnostics. Always check multicollinearity, heteroskedasticity and autocorrelation.
• Regressing non-stationary series directly. Test for unit roots first.
• Confusing statistical and economic significance. A tiny coefficient can be “significant” with a huge sample but meaningless for policy.
• Extrapolating far outside the data. A line fitted on incomes of 25 to 95 says little about an income of 500.

Quick Reference: Which Test for What?

Frequently Asked Questions

What is the difference between econometrics and statistics?

Statistics provides general tools for analysing data. Econometrics adapts those tools to economic data and problems, where controlled experiments are rare and economic theory guides the model.

What is the difference between R² and adjusted R²?

R² measures the share of variation in the dependent variable that the model explains, and it never falls when you add variables. Adjusted R² corrects for the number of variables, so it can fall if a new variable adds nothing useful.

What does a p-value mean?

It is the probability of getting a result at least as extreme as yours if the null hypothesis were true. A p-value below 0.05 means we reject the null hypothesis at the 5% level.

What is OLS in simple words?

Ordinary Least Squares is a method that draws the line through your data that makes the total of squared gaps (residuals) as small as possible. Under the classical assumptions it gives the best linear unbiased estimates.

What does a high R² tell us?

It tells us the model fits the sample data closely, but it does not prove the model is correct or that X causes Y. Spurious regressions between two trending series can also show a very high R².

Why do we test for stationarity?

Regressing non-stationary series on each other can give a spurious relationship, with a high R² and significant t-values that mean nothing. Tests such as Dickey–Fuller tell us whether we need to difference the data first.

Where can I get real Pakistani data to practise?

Use SBP EasyData for monetary, banking, exchange rate and external sector series, the Pakistan Bureau of Statistics (PBS) for CPI inflation, trade and labour force statistics, and World Bank Open Data for GDP and development indicators. FRED is also useful for international series. Direct links are listed in the Sources section at the end of this article.

How much maths do I need to learn econometrics?

Basic algebra, averages and variance are enough to start, plus a little calculus and matrix algebra for advanced topics. Software does the heavy computation, so understanding how to interpret results matters most.

What is the difference between fixed effects and random effects?

Fixed effects give each unit its own intercept and remove anything constant within a unit, so they are safe when unit effects may correlate with the explanatory variables. Random effects assume no such correlation and are more efficient when that assumption holds. The Hausman test helps decide.

Can I do econometrics in Excel?

Yes, for learning. Excel’s Data Analysis ToolPak and the LINEST function handle simple and multiple regression. For time series tests, panel data and robust standard errors, use EViews, Stata, R or Python.

How many observations do I need?

There is no single rule, but more is better. A common guide is at least 10 to 15 observations per explanatory variable for cross-section regressions, and at least 30 to 50 time-series observations for unit root and cointegration work. Very small samples, like the ten-household examples here, are only for demonstration.

Conclusion

The logic of econometrics is: Develop a model based on economic theory, estimate the model using the least-squares method, test the estimates, examine the assumptions, and trust the estimates for policy or forecasting purposes.

We have noted that in our example the MPC is 0.725 with a narrow confidence interval, that a second variable can alter the first coefficient, that a degree costs PKR 30 thousand of a toy sample, and that time series should require additional care when calculating the coefficient of MPC by testing them for stationarity and unit root.

The most effective method of learning is to perform these examples by yourself in Excel and then attempt with real-life data provided by SBP EasyData or Pakistan Bureau of Statistics (PBS). Begin with simple regression, then introduce some diagnostics, and eventually progress to time series.

References

In-text citations in this article follow the author–year style. Textbooks and the original papers behind each test or model are listed below in alphabetical order.

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723.

Angrist, J. D., & Pischke, J.-S. (2009). Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press.

Arellano, M., & Bond, S. (1991). Some tests of specification for panel data: Monte Carlo evidence and an application to employment equations. Review of Economic Studies, 58(2), 277–297.

Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 31(3), 307–327.

Box, G. E. P., & Jenkins, G. M. (1970). Time Series Analysis: Forecasting and Control. Holden-Day.

Breusch, T. S., & Pagan, A. R. (1979). A simple test for heteroscedasticity and random coefficient variation. Econometrica, 47(5), 1287–1294.

Card, D., & Krueger, A. B. (1994). Minimum wages and employment: A case study of the fast-food industry in New Jersey and Pennsylvania. American Economic Review, 84(4), 772–793.

Chow, G. C. (1960). Tests of equality between sets of coefficients in two linear regressions. Econometrica, 28(3), 591–605.

Cochrane, D., & Orcutt, G. H. (1949). Application of least squares regression to relationships containing auto-correlated error terms. Journal of the American Statistical Association, 44(245), 32–61.

Dickey, D. A., & Fuller, W. A. (1979). Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association, 74(366), 427–431.

Durbin, J., & Watson, G. S. (1950). Testing for serial correlation in least squares regression: I. Biometrika, 37(3–4), 409–428.

Engle, R. F. (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica, 50(4), 987–1007.

Engle, R. F., & Granger, C. W. J. (1987). Co-integration and error correction: Representation, estimation, and testing. Econometrica, 55(2), 251–276.

Granger, C. W. J. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3), 424–438.

Greene, W. H. (2018). Econometric Analysis (8th ed.). Pearson.

Gujarati, D. N., & Porter, D. C. (2009). Basic Econometrics (5th ed.). McGraw-Hill.

Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators. Econometrica, 50(4), 1029–1054.

Hausman, J. A. (1978). Specification tests in econometrics. Econometrica, 46(6), 1251–1271.

Jarque, C. M., & Bera, A. K. (1987). A test for normality of observations and regression residuals. International Statistical Review, 55(2), 163–172.

Johansen, S. (1991). Estimation and hypothesis testing of cointegration vectors in Gaussian vector autoregressive models. Econometrica, 59(6), 1551–1580.

MacKinnon, J. G. (1996). Numerical distribution functions for unit root and cointegration tests. Journal of Applied Econometrics, 11(6), 601–618.

Newey, W. K., & West, K. D. (1987). A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica, 55(3), 703–708.

Ramsey, J. B. (1969). Tests for specification errors in classical linear least-squares regression analysis. Journal of the Royal Statistical Society: Series B, 31(2), 350–371.

Savin, N. E., & White, K. J. (1977). The Durbin–Watson test for serial correlation with extreme sample sizes or many regressors. Econometrica, 45(8), 1989–1996.

Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461–464.

Sims, C. A. (1980). Macroeconomics and reality. Econometrica, 48(1), 1–48.

Stock, J. H., & Watson, M. W. (2019). Introduction to Econometrics (4th ed.). Pearson.

Tobin, J. (1958). Estimation of relationships for limited dependent variables. Econometrica, 26(1), 24–36.

White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48(4), 817–838.

Wooldridge, J. M. (2020). Introductory Econometrics: A Modern Approach (7th ed.). Cengage Learning.

Data Sources, Software and Further Reading

The external links below are for real data, software and background reading. Please check each website’s own terms of use before reusing its data.

Data Sources

• SBP EasyData: monetary, banking, exchange rate and external sector time series for Pakistan.
• Pakistan Bureau of Statistics (PBS): CPI inflation and price statistics, trade, labour force and national accounts.
• World Bank Open Data: GDP and development indicators for Pakistan and other countries.
• FRED: free economic time series from the Federal Reserve Bank of St. Louis.

Software

• R and Python statsmodels: free tools to repeat every example in this article.
• Stata and EViews: paid packages widely used in universities and research.

Background Reading

• Encyclopaedia Britannica: Econometrics
• Wikipedia: Econometrics