Comparison of School Leavers' Outcomes using Bayesian Analysis with R and brms
In Victoria, Australia, the State Government conducts an annual survey called the On Track Survey to gauge post-High School outcomes. The latest data, available under CC BY 4.0 and covering several years up to 2021, shows a notable disparity in educational attainment between Government, Catholic, and Independent school sectors.
The dataset, consisting of 157 Government schools, 57 Independent schools, and 50 Catholic schools listed, is analyzed using Bayesian analysis methodologies and the R package brms. This approach allows for a probabilistic understanding of the differences in proportional outcomes between sectors, taking into account confounding factors such as socioeconomic status, prior achievement, and sex.
The analysis demonstrates how to model proportional data using a Beta likelihood function and performs Bayesian ANOVA to explore the differences in proportional outcomes between sectors. The tidybayes::compare_levels() function is used to compute the difference in posterior means between sectors for each outcome.
Key findings reveal substantial differences in higher educational attainment (Bachelor or Higher Degree) rates across sectors and states. Victorian males and females have above-average rates of bachelor’s attainment compared to other states, indicating a positive overall trend in Victoria. However, there is marked variation across sectors, with Independent schools showing higher rates of higher education achievement.
The differences by school sector, sex, and state are "very large and significant," implying that the school sector attended is a major factor influencing post-high school outcomes. In Victoria, for example, Independent schools generally show higher rates of higher education achievement compared to Catholic and Government schools.
To further quantify these differences, a comparison of two models is made using the Bayesian LOO estimate of the expected log pointwise predictive density (elpd_loo). Model m3b, which does not have a pooled φ term, has a higher expected logpoint wise leave one out value, indicating better predictive accuracy on unseen data.
A posterior predictive check is performed to confirm the superiority of model m3b. The results show an 89% posterior probability that the difference between Catholic and Government student undergraduate enrolment is between 5.6% and 15.7% with a mean of 10.7%. Additionally, the difference between Independent and Government student undergraduate enrolment is between 17.8% and 27% with a mean of 22.5%. These differences are substantial, and there's a 100% chance that these differences are not zero.
The dataset, presented in a wide report format, requires hard coding to relabel vectors and create a tidy data frame. The analysis uses Beta likelihood regression to model the distribution of proportions by outcome and sector.
In conclusion, the data reveals a significant difference in post-high school outcomes between Government, Catholic, and Independent schools in Victoria, with Independent schools generally showing higher rates of progressing to bachelor or higher qualifications. These findings have policy implications, as they highlight the need for targeted interventions to address the educational disparities between school sectors.
[1] Source: [Link to the original study]
- The analysis of the On Track Survey data in Victoria, Australia suggests that Independent schools have higher rates of higher education achievement, indicating a significant difference in education-and-self-development outcomes compared to Catholic and Government schools.
- online-education platforms may find it beneficial to focus on catering to students from Government schools, as the data indicates that Independent schools are already showing higher rates of progression to bachelor or higher qualifications, potentially leaving a gap for further learning opportunities for students from Government and Catholic schools.
