Causal Inference Using Propensity Score Method
Introduction
The detrimental effect of peer victimization on academic performance has been well-documented in both educational and psychological research (e.g., Juvonen & Graham, 2014; Nakamoto & Schwartz, 2010). Students who are bullied by their peers are more likely to exhibit lower academic achievement, decreased classroom engagement, and greater school avoidance (e.g., Espelage et al., 2013; Wang et al., 2009). However, drawing causal conclusions from these associations remains challenging. Randomized controlled trials are not feasible due to ethical constraints, and observational studies are prone to confounding and selection bias. Propensity score methods offer a promising solution by approximating the conditions of randomized experiments (Rosenbaum & Rubin, 1983; Austin, 2011). Yet, an added complexity arises when the exposure of interest—peer victimization—is not directly observed but inferred from multiple indicators. In such cases, latent class analysis can be used to identify subgroups of students with distinct victimization profiles (Lanza et al., 2003; Nylund et al., 2007). This study integrates these methodological advances to examine how peer victimization patterns affect academic outcomes. Specifically, it aims to:
- Identify latent classes of peer victimization based on students’ self-reported experiences.
- Apply propensity score weighting to adjust for a rich set of confounders.
- Evaluate the impact of latent class exposure on students’ performance in mathematics and science.
Methods
Data Source
This study utilized data from the 2015 Trends in International Mathematics and Science Study (TIMSS). The U.S. TIMSS sample comprised 10,221 students, representing over 3.8 million eighth graders nationwide. The dataset included comprehensive background questionnaires capturing students’ school experiences, socio-demographic characteristics, and contextual factors.
Data Preparation
The data was cleaned by removing 121 cases with missing peer victimization responses. Key variables were selected, including victimization items, plausible value scores for math and science, and covariates (e.g., age, gender, SES). Categorical variables were recoded as factors. Following Solberg & Olweus (2003), peer victimization items were recoded into two categories based on frequency: 1 = “Never” or “A few times a year”, and 2 = “Once or twice a month” or “At least once a week”. The analytic sample included students with complete responses on all peer victimization items (10,100). A listwise deletion would reduce the sample size to 8384.Overall, less than 1% of the individual data points across the dataset are missing (see plot below).
Measures
Peer Victimization
Peer victimization was assessed using 9 items.Each item asked students to report how often other students from their school: Made fun of me, Left me out of things, Spread lies about me, Stole something from me, Hit or hurt me, Forced me to do things I didn’t want to, Shared embarrassing information about me, Posted embarrassing information online and Threatened me. They responded using a four-point scale: “Never,” “A few times a year,” “Once or twice a month,” and “At least once a week.” Based on Solberg and Olweus (2003), responses were dichotomized to distinguish between students with no/minimal exposure and those with frequent exposure.
During data screening, items “Posted embarrassing information online” and “Threatened me” were found to have insufficient response variation—no students endorsed experiencing these behaviors at the higher-frequency thresholds. Therefore, these two items were excluded from the latent class analysis. The final analysis was conducted using the remaining seven items.
Confounders
The following covariates were included as potential confounders in the propensity score model: Age, gender, race/ethnicity, language spoken at home, School composition, percent of English learners, percent of low-income students, income level of neighborhood, public/private school status, total hours of instruction, school emphasis on academic success, school discipline climate, SES, instructional resources for math and science, and School belonging.
Academic Performance
The first plausible value scores in mathematics and science were used as outcome variables. These scores are scaled with a mean of 500 and standard deviation of 100.
Analysis
Latent Class Analysis
To identify subgroups of students with distinct experiences of peer victimization, we estimated latent class models ranging from 2 to 7 classes using poLCA package. Model selection was guided by theoretical interpretability and statistical criteria such as AIC, BIC, ABIC, and CAIC. Fit indices improved markedly from 2 to 3 classes, with BIC reaching its minimum at the 3-class model (see Figure below). While AIC continued to decline slightly with additional classes, gains were marginal and came at the cost of parsimony. Consequently, a 3-class model was selected as the optimal solution.
The three latent classes are:
- Non-victimized (74%): Students in this group had low probabilities of endorsing any form of victimization.
- Sometimes Victimized (21%): Students in this class showed moderate endorsement of items related to verbal and relational victimization (e.g., “Left Out”, “Spread Lies”, “Made Fun”).
- Victimized (5%): Students in this group exhibited high probabilities of experiencing multiple types of victimization, particularly more severe forms such as “Hit or Hurt,” “Stole From Me,” and “Forced Actions.”
The figure below shows item-response probabilities across the three latent classes. In the Non-victimized group, the likelihood of endorsing any victimization item was minimal-typically below 0.25. In contrast, the Victimized class demonstrated consistently high probabilities (often exceeding 0.75) across nearly all indicators. The Sometimes Victimized class displayed intermediate probabilities, particularly for relational and reputational harm such as being left out or having lies spread about them.
Propensity Score Analysis
To estimate the causal effect of peer victimization on academic performance in math and science, we implemented a multinomial propensity score weighting approach using the twang package. The exposure was students’ latent class membership: Victimized, Sometimes Victimized, and Nonvictimized. Propensity scores were estimated using a generalized boosted model with 5,000 trees. The model was optimized using both the effect size mean (es.mean) and the Kolmogorov-Smirnov mean (ks.mean) as stopping criteria.
Covariate Balance
As shown in Figure 3, covariate balance was assessed across all pairwise class comparisons. Prior to weighting, substantial imbalance was observed for multiple variables—particularly school belonging, SES, and instructional resources. After weighting, covariate balance was substantially improved across all comparisons. Standardized mean differences for nearly all covariates were reduced to below 0.1.
Survey Design and Weights
To appropriately estimate the causal effects of peer victimization on academic outcomes, we incorporated both propensity score weights and survey design features into our analytic framework. We derived inverse probability of treatment weights from the previously estimated multinomial propensity score model. These weights adjust for baseline differences across victimization groups. To enhance the generalizability of the results, we integrated TIMSS’s complex sampling weights. Then we created a survey design object to enable accurate estimation of standard errors and population-level inferences.
Analysis of Treatment Effects
To estimate the impact of peer victimization on academic outcomes, weighted linear regression models were fitted separately for mathematics and science performance. The models used latent class membership as the key predictor, with the non-victimized group serving as the reference category.
The plot below presents the regression estimates and confidence intervals for each victimization group relative to the non-victimized reference, across both academic domains:
Math Performance (blue, circles):
Science Performance (orange, squares)
Students classified as victimized had significantly lower scores in both math and science compared to their non-victimized peers.
Their math scores were, on average, approximately 25–30 points lower.
Their science scores showed a similarly negative pattern, with an estimated decrease of nearly 30 points.
The sometimes victimized group also demonstrated lower academic performance, but the magnitude of these differences was more modest i.e, Math and science scores for this group were estimated to be 10–15 points lower on average than the non-victimized group.
All estimates were statistically significant based on robust standard errors derived from the survey-weighted regression models.
The results confirm a graded relationship between peer victimization and academic performance, with increasing exposure to victimization associated with progressively worse outcomes in both math and science. The visual representation in Figure 5 highlights this trend, showing the most substantial performance decline among the victimized subgroup. These findings provide strong empirical support for the hypothesis that peer victimization undermines cognitive and academic functioning, even after adjusting for a wide range of potential confounders.
Insights
This study offers evidence that peer victimization may negatively impact academic performance. The findings suggest that even occasional victimization is associated with lower achievement, underscoring the importance of early identification and support. These results indicate that peer victimization is not merely a social-emotional issue, but a potential academic risk factor.
Policy Implications
- Comprehensive Screening: Establish systematic, school-wide screening protocols to detect all levels of peer victimization, from occasional incidents to chronic patterns.
- Tiered Intervention Framework: Adopt a multi-tiered system of support (MTSS) that aligns intervention intensity with students’ victimization risk profiles, ensuring resources are allocated efficiently and equitably.
- Integrated Academic Recovery: Combine academic support services—such as tutoring and enrichment—with anti-bullying efforts to address the full impact of victimization on learning outcomes.
- Data-Driven Decision-Making: Incorporate peer victimization indicators into early warning systems and MTSS data dashboards to proactively identify at-risk students and guide timely intervention
Conclusion
Peer victimization has a measurable, negative effect on math and science outcomes. Schools and districts must act to identify, intervene, and support impacted students—both socially and academically.
References
Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46(3), 399–424. https://doi.org/10.1080/00273171.2011.568786
Espelage, D. L., Hong, J. S., Rao, M. A., & Low, S. (2013). Associations between peer victimization and academic performance. Theory Into Practice, 52(4), 233–240. https://doi.org/10.1080/00405841.2013.829727
Juvonen, J., & Graham, S. (2014). Bullying in schools: The power of bullies and the plight of victims. Annual Review of Psychology, 65, 159–185. https://doi.org/10.1146/annurev-psych-010213-115030
Lanza, S. T., Flaherty, B. P., & Collins, L. M. (2003). Latent class and latent transition analysis. In J. A. Schinka & W. F. Velicer (Eds.), Handbook of psychology: Volume 2. Research methods in psychology (pp. 663–685). Wiley.
Nakamoto, J., & Schwartz, D. (2010). Is peer victimization associated with academic achievement? A meta-analytic review. Social Development, 19(2), 221–242. https://doi.org/10.1111/j.1467-9507.2009.00539.x
Nylund, K. L., Bellmore, A., Nishina, A., & Graham, S. (2007). Subtypes, severity, and structural stability of peer victimization: What does latent class analysis say? Child Development, 78(6), 1706–1722. https://doi.org/10.1111/j.1467-8624.2007.01097.x
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41
Wang, J., Iannotti, R. J., & Nansel, T. R. (2009). School bullying among adolescents in the United States: Physical, verbal, relational, and cyber. Journal of Adolescent Health, 45(4), 368–375. https://doi.org/10.1016/j.jadohealth.2009.03.021