| Raj Chetty, John N. Friedman, and Jonah E. Rockoff
This appendix provides a detailed step-by-step guide to estimating teacher value-added (VA) in the presence of drift, a common issue in educational data. The method involves three main steps:
1. **Residualization of Test Scores**: Student test scores are residualized with respect to controls using an OLS regression with teacher fixed effects. This results in residuals that are used for further analysis.
2. **Estimation of Variance Components**: The individual-level variance of residual test scores is estimated, and the total variance of the residuals is calculated, accounting for the degrees of freedom correction. Data are collapsed to the classroom level, and precision-weighted averages are constructed for each classroom. Covariances between mean scores across years within teacher are estimated, and covariances for lags greater than 7 are set to a default value.
3. **Construction of VA Estimates**: VA estimates for each teacher are constructed using a best linear predictor, incorporating data from all other years to increase precision. The weights on scores are determined based on the precision of classroom means and the number of students in each class.
The appendix also introduces the concept of "teacher-level bias," which is the systematic misprediction of a teacher's performance when estimation error in VA vanishes. This bias is relevant for ensuring equitable treatment of teachers. The relationship between teacher-level bias and forecast bias is discussed, highlighting that forecast-unbiased VA estimates can still be biased at the teacher level if the teacher-level bias is negatively correlated with true value-added.
Additionally, the appendix outlines a matching algorithm used to link school district data to tax records, ensuring accurate and de-identified data for analysis. The algorithm matches individuals based on date of birth, gender, last name, and other variables, with a focus on minimizing selection bias.
Finally, the appendix assesses the unconditional sorting of students to teachers based on observable characteristics, finding that better students are slightly more likely to be assigned to better teachers, but the overall impact is small. The appendix also provides a quasi-experimental estimator of forecast bias, showing that OLS estimation can identify the degree of forecast bias under certain assumptions.This appendix provides a detailed step-by-step guide to estimating teacher value-added (VA) in the presence of drift, a common issue in educational data. The method involves three main steps:
1. **Residualization of Test Scores**: Student test scores are residualized with respect to controls using an OLS regression with teacher fixed effects. This results in residuals that are used for further analysis.
2. **Estimation of Variance Components**: The individual-level variance of residual test scores is estimated, and the total variance of the residuals is calculated, accounting for the degrees of freedom correction. Data are collapsed to the classroom level, and precision-weighted averages are constructed for each classroom. Covariances between mean scores across years within teacher are estimated, and covariances for lags greater than 7 are set to a default value.
3. **Construction of VA Estimates**: VA estimates for each teacher are constructed using a best linear predictor, incorporating data from all other years to increase precision. The weights on scores are determined based on the precision of classroom means and the number of students in each class.
The appendix also introduces the concept of "teacher-level bias," which is the systematic misprediction of a teacher's performance when estimation error in VA vanishes. This bias is relevant for ensuring equitable treatment of teachers. The relationship between teacher-level bias and forecast bias is discussed, highlighting that forecast-unbiased VA estimates can still be biased at the teacher level if the teacher-level bias is negatively correlated with true value-added.
Additionally, the appendix outlines a matching algorithm used to link school district data to tax records, ensuring accurate and de-identified data for analysis. The algorithm matches individuals based on date of birth, gender, last name, and other variables, with a focus on minimizing selection bias.
Finally, the appendix assesses the unconditional sorting of students to teachers based on observable characteristics, finding that better students are slightly more likely to be assigned to better teachers, but the overall impact is small. The appendix also provides a quasi-experimental estimator of forecast bias, showing that OLS estimation can identify the degree of forecast bias under certain assumptions.