We examine (a) cognitive skills, (b) socio-emotional skills, (c) institutional learning enviroments (e.g., school), and (d) how the interplay of these factors affects educational outcomes and returns to education. To this end, we draw on representative data (e.g., from large-scale assessments) and capitalize on multivariate statistical models and integrative data analysis (e.g., meta-analysis). In doing so, we aim at contributing to evidence based educational practices and policies.
|Dr. Gesine Fuchs||Psychometric Data Quality of Standard Based Proficiency Tests|
|Andrea Hasl||The Interplay between Cognitive and Non-Cognitive Skills in Education|
|Dr. Lena Keller||BIG-GENDER: Big Data Meta-Analyses of Gender Differences in Students' Achievement and Achievement Motivation Based on Large-Scale Assessments|
|Dr. Julia Kretschmann||Analyses of Causal Effects in Educational Settings with Observational Data|
|Sophie Stallasch||Multilevel Design Parameters and Effect Size Benchmarks for Students' Competencies|
MULTI-DES: Multilevel Design Parameters and Effect Size Benchmarks for Students’ Competencies
Project partners: Prof. Cordula Artelt and Prof. Oliver Lüdtke
The advancement of educational research as well as the development of evidence-based educational practices and policies that aim to foster students' competencies hinges on the availability of rigorous research and its effective communication to a broad audience. Particularly, educational intervention studies need to be conducted in ecological valid settings to probe whether the interventions actually work in the field. Such studies are often designed as so-called group-randomized or cluster-randomized trials where entire groups (e.g., entire schools) are randomly assigned to experimental conditions. To ensure sufficient statistical power several design parameters are essential to determine the sample size (i.e., number of students, classes, and schools) of group-randomized trials. Design parameters comprise estimates of between-school and between-class differences (in terms of intraclass correlations) as well as the amount of variance explained by covariates (e.g., pretest scores) at the individual, class or school level. Moreover, to effectively communicate effects of educational interventions on students´ competencies requires empirical effect size benchmarks that serve as vital aids to interpret empirical results. Crucially, both design parameters and effect size benchmarks should correspond to the target population. The present project is the first to rigorously examine design parameters and effect size benchmarks that are targeted to the multilevel framework of Germany´s school system. To this end, we will conduct four studies that capitalize on data from three German longitudinal large-scale assessments: the National Educational Panel Study (NEPS), the longitudinal extension of the German year 2003 cycle of PISA (PISA-I+), and the Assessment of Student Achievements in German and English as a Foreign Language (DESI). These large-scale assessments cover a wide range of students´ competencies and age groups (i.e., students from grade levels 1 to 12). More specifically, Study 1 will examine design parameters for two-level designs (students nested within schools) and three-level designs (students nested within classes, classes nested within schools) for the general student population as well as for different school types. Study 2 investigates how various covariates (i.e., pretest scores, sociodemographic characteristics, basic cognitive functions), their combination, and the time-lag between pre- and posttest affect the precision/statistical power of group-randomized trials for two-level and three-level designs. Study 3 examines academic-growth as effect size benchmark for the general student population and different school types. Finally, Study 4 investigates performance gaps between weak and average schools as effect size benchmarks. MULTI-DES is funded by the German Research Foundation (DFG).
BIG-GENDER: Big Data Meta-Analyses of Gender Differences in Students' Achievement and Achievement Motivation Based on Large-Scale Assessments
Project partner: Prof. Franzis Preckel
An essential requirement for any scientific and political discourse on gender differences in school (and beyond) is a reliable body of empirical knowledge on the nature, size, variability, and moderating factors of these differences. This knowledge is highly relevant for at least three reasons: It can be used (a) to learn about gender differences before university entry as plausible antecedents of still existing gender gaps in academic fields, (b) to provide scientific evidence that can help dispel the persistent stereotypes (e.g., that only boys can excel in mathematics) that may discourage girls from pursuing careers in science, technology, engineering, and mathematics (STEM), and (c) to identify target points for evidence-based decision making in educational policy (e.g., boys from families with low socioeconomic status [SES]). The main goal of this meta-analytic big data project is therefore to provide highly robust and widely generalizable knowledge on cross-national gender differences in students’ achievement and achievement motivation (concerning means and variances). To this end, we will meta-analyze individual student data from 999 representative student samples from 112 different countries/economic regions (total N > 4 million) participating in 24 cycles of international large-scale assessments covering the period from 1995 to 2015: the Trends in International Mathematics and Science Study (TIMSS; Grades 4, 8, and 12), the Progress in International Reading Literacy Study (PIRLS; Grade 4), and the Programme for International Student Assessment (PISA; 15-year-olds). This project will be the first to quantitatively synthesize this wealth of data with meta-analytic methods. Specifically, we will conduct three domain-specific meta-analyses to examine gender differences (concerning mean levels and variability) in achievement and achievement motivation in mathematics, science, and reading, respectively. We will study students’ age and SES, the selectivity of the sample (e.g., the bottom 10% or the top 5% of the achievement distributions), sociocultural indicators of gender equality, and historical changes as moderators of gender differences. Further, we will conduct one meta-analysis to examine gender differences in achievement and motivational profiles in multiple domains among three groups of top-performing students who belong to the top 5% in mathematics, science, or reading in their respective countries. To sum up, our project will provide novel insights into cross-national, temporal, and age-related trends concerning gender differences in the general student population and among top-performing students, as well as on the complex interactions between gender, the selectivity of the sample, SES, and sociocultural indicators of gender equality. BIG-GENDER is funded by the German Research Foundation (DFG).