Publications

Publications

Filter:

Filter results Close
  • expanded
  • expanded
  • expanded
  • expanded
  • expanded
  • expanded
  • expanded
  • expanded
  • expanded
  • expanded
  • expanded
  • expanded
Guirong Li
Journal Articles

Early Childhood Development and Parental Training Interventions in Rural China: A Systematic Review and Meta-Analysis

Dorien Emmers, Qi Jiang, Hao Xue, Yue Zhang, Yunting Zhang, Yingxue Zhao, Bin Liu, Sarah-Eve Dill, Yiwei Qian, Nele Warrinnier, Hannah Johnstone, Jianhua Cai, Xiaoli Wang, Lei Wang, Renfu Luo, Guirong Li, Jiajia Xu, Ming Liu, Yaqing Huang, Wenjie Shan, Zhihui Li, Yu Zhang, Sean Sylvia, Yue Ma, Alexis Medina, Scott Rozelle
BMJ Global Health , 2021
Introduction: Inadequate care during early childhood can lead to long-term deficits in skills. Parenting programmes that encourage investment in young children are a promising tool for improving early development outcomes and long-term opportunities in low-income and middle-income regions, such as rural China. Methods: We conducted a systematic review and a meta-analysis to investigate the prevalence of early developmental delays and stimulating parenting practices as well as the effect of parental training programmes on child development outcomes in rural China. We obtained data in English from EconPapers, PubMed, PsycARTICLES, Cochrane Library, Web of Science and Scopus (Elsevier) and in Chinese from China National Knowledge Infrastructure, Wanfang Data and VIP Information. We conducted frequentist meta-analyses of aggregate data and estimated random-effects meta-regressions. Certainty of evidence was rated according to the Grading of Recommendations Assessment, Development and Evaluation approach. Results: We identified 19 observational studies on the prevalence of developmental delays and stimulating parenting practices for children under 5 years of age (n=19 762) and ten studies on the impact of parental training programmes on early child development (n=13 766). Children’s risk of cognitive, language and social-emotional delays in the rural study sites (covering 14 provinces mostly in Central and Western China) was 45%, 46%, and 36%, respectively. Parental training programmes had a positive impact on child cognition, language and social-emotional development. Conclusion: There is evidence to suggest that early developmental delay and the absence of stimulating parenting practices (ie, reading, storytelling and singing with children) may be prevalent across rural, low-income and middle-income regions in Central and Western China. Results support the effectiveness of parental training programmes to improve early development by encouraging parental engagement.
Show body
Journal Articles

Skill Levels and Gains in University STEM Education in China, India, Russia, and the United States

Prashant Loyalka, Ou Lydia Liu, Guirong Li, Elena Kardanova, Igor Chirikov, Shangfeng Hu, Ningning Yu, Liping Ma, Fei Guo, Tara Beteille, Namrata Tognatta, Lin Gu, Guangming Ling, Denis Federiakin, Huan Wang, Saurabh Khanna, Ashutosh Bhuradia, Zhaolei Shi, Yanyan Li
Nature Human Behavior , 2021
Universities contribute to economic growth and national competitiveness by equipping students with higher-order thinking and academic skills. Despite large investments in university science, technology, engineering and mathematics (STEM) education, little is known about how the skills of STEM undergraduates compare across countries and by institutional selectivity. Here, we provide direct evidence on these issues by collecting and analysing longitudinal data on tens of thousands of computer science and electrical engineering students in China, India, Russia and the United States. We find stark differences in skill levels and gains among countries and by institutional selectivity. Compared with the United States, students in China, India and Russia do not gain critical thinking skills over four years. Furthermore, while students in India and Russia gain academic skills during the first two years, students in China do not. These gaps in skill levels and gains provide insights into the global competitiveness of STEM university students across nations and institutional types.
Show body
Journal Articles

The Impacts of Highly Resourced Vocational Schools on Student Outcomes in China

Guirong Li, Jiajia Xu, Liying Li, Zhaolei Shi, Hongmei Yi, James Chu, Elena Kardanova, Yanyan Li, Prashant Loyalka, Scott Rozelle
China & World Economy , 2020
Policymakers in developing countries have prioritized the mass expansion of vocational education and training (VET). Evidence suggests, however, that the quality of VET can be poor. One possible reason given by policymakers for this is a lack of resources per student. The goal of this study is to examine whether the quality of VET in developing countries increases by investing greater resources per student. To achieve this goal, we examine the impacts of attending model schools (which have far more resources per student) compared with non-model schools (which have fewer resources) on a range of student cognitive, non-cognitive, and behavioral outcomes. Using representative data from a survey of approximately 12,000 VET students from China, multivariate regression and propensity score matching analyses show that there are no significant benefits, in terms of student outcomes, from attending model vocational high schools, despite their substantially greater resources.
Show body
Journal Articles

Examining Mode Effects for an Adapted Chinese Critical Thinking Assessment

Lin Gu, Guangming Ling, Ou Lydia Liu, Zhitong Yang, Guirong Li, Elena Kardanova, Prashant Loyalka
Assessment & Evaluation in Higher Education , 2020
We examine the effects of computer-based versus paper-based assessment of critical thinking skills, adapted from English (in the U.S.) to Chinese. Using data collected based on a random assignment between the two modes in multiple Chinese colleges, we investigate mode effects from multiple perspectives: mean scores, measurement precision, item functioning (i.e. item difficulty and discrimination), response behavior (i.e. test completion and item omission), and user perceptions. Our findings shed light on assessment and item properties that could be the sources of mode effects. At the test level, we find that the computer-based test is more difficult and more speeded than the paper-based test. We speculate that these differences are attributable to the test’s structure, its high demands on reading, and test-taking flexibility afforded under the paper testing mode. Item-level evaluation allows us to identify item characteristics that are prone to mode effects, including targeted cognitive skill, response type, and the amount of adaptation between modes. Implications for test design are discussed, and actionable design suggestions are offered with the goal of minimizing mode effect.
Show body
Working Papers

Education and EdTech during COVID-19: Evidence from a Large-Scale Survey during School Closures in China

Guirong Li, Xinwu Zhang, Delei Liu, Hao Xue, Derek Hu, Oliver Lee, Chris Rilling, Yue Ma, Cody Abbey, Robert Fairlie, Prashant Loyalka, Scott Rozelle
2020
In response to the COVID-19 epidemic, many education systems have relied on distance learning and educational technologies to an unprecedented degree. However, rigorous empirical research on the impacts on learning under these conditions is still scarce. We present the first large-scale, quantitative evidence detailing how school closures affected education in China. The data set includes households and teachers of 4,360 rural and urban primary school students. We find that although the majority of students engaged in distance education, many households encountered difficulties including barriers to learning (such as access to appropriate digital devices and study spaces), curricular delays, and costs to parents equivalent to about two months of income. We also find significant disparities across rural and urban households.
Show body
Journal Articles

Large-Scale International Assessments of Learning Outcomes: Balancing the Interests of Multiple Stakeholders

Guirong Li, Irina Shcheglova, Ashutosh Bhuradia, Yanyan Li, Prashant Loyalka, Olivia Zhou, Shangfeng Hu, Ningning Yu, Liping Ma, Fei Guo, Igor Chirikov
Journal of Higher Education Policy and Management , 2020
The demand for large-scale assessments in higher education, especially at an international scale, is growing. A major challenge of conducting these assessments, however, is that they require understanding and balancing the interests of multiple stakeholders (government officials, university administrators, and students) and also overcoming potential unwillingness of these stakeholders to participate. In this paper, we take the experience of the Study of Undergraduate Performance (SUPER) in conducting a large-scale international assessment as a case study. We discuss ways in which we mitigated perceived risks, built trust, and provided incentives to ensure the successful engagement of stakeholders during the study’s implementation.
Show body
Journal Articles

Academic Achievement and Mental Health of Left-behind Children in Rural China

Lei Wang, Yaojia Zheng, Guirong Li, Yanyan Li, Zhenni Fang, Cody Abbey, Scott Rozelle
China Agricultural Economic Review , 2019
Purpose – China’s rapid pace of urbanization has resulted in millions of rural residents migrating from rural areas to urban areas for better job opportunities. Due to economic pressures and the nature of China’s demographic policies, many of these migrants have been forced to leave their children with relatives – typically paternal grandparents – at home in the countryside. Thus, while income for most migrant families has risen, a major unintended consequence of this labor movement has been the emergence of a potentially vulnerable sub-population of left-behind children (LBCs). The purpose of this paper is to examine the impacts of parental migration on both the academic performance and mental health of LBCs. Design/methodology/approach – Longitudinal data were drawn from three waves of a panel survey that . followed the same students and their families – including their migration behavior (i.e. whether both parents, one parent, no parent migrated) – between 2015 and 2016. The survey covers more than 33,000 students in one province of central China. The authors apply a student fixed-effects model that controls for both observable and unobservable confounding variables to explicate the causal effects of parental migration on the academic and mental health outcomes for LBC. The authors also employ these methods to test whether these effects differ by the type of migration or by gender of the child.
Findings – The authors found no overall impact of parental migration on either academic performance or mental health of LBCs, regardless of the type of migration behavior. The authors did find, however, that when the authors examined heterogeneous effects by gender (which was possible due to the large sample size), parental migration resulted in significantly higher anxiety levels for left-behind girls. The results suggest that parental migration affects left-behind boys and girls differently and that policymakers should take a more tailored approach to addressing the problems faced by LBCs.
Show body
Working Papers

EdTech for Equity in China: Can Technology Improve Teaching for Millions of Rural Students?

Cody Abbey, Yue Ma, Guirong Li, Matthew Boswell, Claire Cheng, Robert Fairlie, Oliver Lee, Prashant Loyalka, Andrew Mi, Evan Peng, Scott Rozelle, Adrian Sun, Andy Zeng, Jenny Zhao
2019

Previous literature suggests subpar teaching is a primary reason why rural Chinese students lag behind academically. We initiate an investigation into the potential of educational technology (EdTech) to increase teaching quality in rural China. First, we discuss why conventional approaches of improving teaching in remote schools are infeasible in China’s context, referring to past research. We then explore the capacity of technology-assisted instruction to improve academic performance by examining previous empirical analyses. Third, we show that China is not limited by the resource constraints of other developing countries due to substantial policy support and a thriving EdTech industry. Finally, we identify potential implementation-related challenges based on the results of a preliminary qualitative survey of pilots of EdTech interventions. With this paper, we lay the foundation for a long-term research investigation into whether EdTech can narrow China’s education gap.

Show body
Journal Articles

The Prevalence of Parent-Teacher Interaction in Developing Countries and its Effect on Student Outcomes

Guirong Li, Millie Lin, Chengfang Liu, Angela Johnson, Yanyan Li, Prashant Loyalka
Teaching and Teacher Education , 2019

Empirical evidence from developed countries supports the idea that parent-teacher interaction is high and improves student outcomes. The evidence from developing countries is, however, decidedly mixed. Using longitudinal data from nearly 6000 students and their 600 teachers in rural China, we show the prevalence of parent-teacher interaction is generally much lower than that of developed countries. We also show parent-teacher interaction, when it exists, can have positive effects on raising academic achievement and reducing learning anxiety. We demonstrate that the prevalence and effectiveness of parent-teacher interaction in a developing country context varies considerably due to both demand-side and supply-side factors.

Show body
Journal Articles

Stuck in Place? A Field Experiment on the Effects of Reputational Information on Student Evaluations

James Chu, Guirong Li, Prashant Loyalka, Chengfang Liu, Leonardo Rosa, Yanyan Li
Social Forces , 2019

Studies suggest that students’ prior performance can shape subsequent teacher evaluations, but the magnitude of reputational effects and their implications for educational inequality remain unclear. Existing scholarship presents two major perspectives that exist in tension: do teachers primarily use reputational information as a temporary signal that is subsequently updated in response to actual student performance? Or do teachers primarily use reputational information as a filter that biases perception of subsequent evidence, thus crystallizing student reputations and keeping previously poor-performing students stuck in place? In a field experiment, we recruited a random sample of 832 junior high school teachers from the second-most populous province of China to grade a sequence of four essays written by the same student, and we randomly assign both the academic reputation of the student and the quality of the essays produced. We find that (1) reputational information influences how teachers grade, (2) teachers rely on negative information more heavily than positive information, and (3) negative reputations are crystallized by a single behavioral confirmation. These results suggest that students can escape their prior reputations, but to do so, they must contradict them immediately, with a single confirmation sufficient to crystallize a negative reputation.

Show body
Journal Articles

Does Teacher Training Actually Work? Evidence from a Large-Scare Randomized Evaluation of a National Teacher Training Program

Prashant Loyalka, Anna Popova, Guirong Li, Zhaolei Shi
American Economic Journal: Applied Economics , 2019
Despite massive investments in teacher professional development (PD) programs in developing countries, there is little evidence on their effectiveness. We present results of a large-scale, randomized evaluation of a national PD program in China in which teachers were randomized to receive PD; PD plus follow-up; PD plus evaluation of the command of PD content; or no PD. Precise estimates indicate PD and associated interventions failed to improve teacher and student outcomes after one year. A detailed analysis of the causal chain shows teachers find PD content to be overly theoretical, and PD delivery too rote and passive, to be useful.
Show body
Journal Articles

Computer Science Skills Across China, India, Russia, and the United States

Prashant Loyalka, Ou Lydina Liu, Guirong Li, Igor Chirikov, Elena Kardanova, Lin Gu, Guangming Ling, Ningning Yu, Fei Guo, Liping Ma, Shangfeng Hu, Angela Sun Johnson, Ashutosh Bhuradia, Saurabh Khanna, Isak Froumin, Jinghuan Shi, Pradeep Kumar Choudhury, Tara Beteille, Francisco Marmolejo, Namrata Tognatta
Proceedings of the National Academy of Sciences of the United States of America , 2019

We assess and compare computer science skills among final-year computer science undergraduates (seniors) in four major economic and political powers that produce approximately half of the science, technology, engineering, and mathematics graduates in the world. We find that seniors in the United States substantially outperform seniors in China, India, and Russia by 0.76–0.88 SDs and score comparably with seniors in elite institutions in these countries. Seniors in elite institutions in the United States further outperform seniors in elite institutions in China, India, and Russia by ∼0.85 SDs. The skills advantage of the United States is not because it has a large proportion of high-scoring international students. Finally, males score consistently but only moderately higher (0.16–0.41 SDs) than females within all four countries.

Show body
Journal Articles

Stereotype Threat and Educational Tracking: A Field Experiment in Chinese Vocational High Schools

James Chu, Prashant Loyalka, Guirong Li, Liya Gao, Yao Song
Socius , 2018

Educational tracks create differential expectations of student ability, raising concerns that the negative stereotypes associated with lower tracks might threaten student performance. The authors test this concern by drawing on a field experiment enrolling 11,624 Chinese vocational high school students, half of whom were randomly primed about their tracks before taking technical skill and math exams. As in almost all countries, Chinese students are sorted between vocational and academic tracks, and vocational students are stereotyped as having poor academic abilities. Priming had no effect on technical skills and, contrary to hypotheses, modestly improved math performance. In exploring multiple interpretations, the authors highlight how vocational tracking may crystallize stereotypes but simultaneously diminishes stereotype threat by removing academic performance as a central measure of merit. Taken together, the study implies that reminding students about their vocational or academic identities is unlikely to further contribute to achievement gaps by educational track.

Show body
Journal Articles

Assessing the Quality of Upper-Secondary Vocational Education and Training: Evidence from China

Hongmei Yi, Guirong Li, Liying Li, Prashant Loyalka, Linxiu Zhang, Jiajia Xu, Elena Kardanova, Henry Shi, James Chu
Comparative Education Review , 2018
An increasing number of policymakers in developing countries have made the mass expansion of upper-secondary vocational education and training (VET) a top priority. The goal of this study is to examine whether VET fulfills the objectove of building skills and abilities along multiple dimensions and further identify which school-level factors help vocational students build these skills and abilities. To fulfill this goal, we analyzed representative, longitudinal data that we collected on more than 12,000 students from 118 schools in once province of central China. First, descriptive analysis shows approximately 90% of VET students do not make any gains in vocational or general skills. In addition, negative behaviors (misbehavior in the classroom, anti-social behavior, and other risky behaviors) are highly prevalent among VET students. A nontrivial proportion of student internships also fail to meet minimum government requirements for student safety and well-being. Perhaps as a result of these outcomes, more than 60% of students express dissatisfaction with their VET programs, as evidenced by eitehr self-reports or dropping out. Finally, using a multi-level model, we find that school inputs (such as school size, teacher qualifications, and per pupil expenditure) are not correlated with vocational and general skill at the end of the school year, or student dropout in the academic year.
Show body
Journal Articles

Assessing College Critical Thinking: Preliminary Results from the Chinese HEIghten® Critical Thinking Assessment

Ou Lydia Liu, Amy Shaw, Lin Gu, Guirong Li, Shangfeng Hu, Ningning Yu , Liping Ma, Changqing Xu, Fei Guo, Qi Su, Elena Kardanovaj, Igor Chirikov, Jinghuan Shi, Zhaolei Shi, Huan Wang, Prishant Loyalka
Higher Education Research & Development , 2017
Assessing student learning outcomes has become a global trend in higher education. In this paper, we report on the validation of the Chinese HEIghten® Critical Thinking assessment with a nationally representative sample of Electrical Engineering and Computer Science students from 35 institutions in China. Key findings suggest that there was a test delivery mode effect favoring the paper tests over the online tests. In general, the psychometric quality of the items was satisfactory for low-stakes, group-level uses but there were a few items with low discrimination that awaits further investigation. The relationships between test scores and various external variables such as college entrance examination scores, university elite status and student perceptions of the test were as expected. We conclude with speculations on the key findings and discussion of directions for future research.
Show body
Working Papers

Unpacking Teacher Professional Development

Guirong Li, Chengfang Liu
Working Paper , 2017

Despite massive investments in teacher professional development (PD) programs in developing countries, there is little evidence on their effectiveness. We present the results of a large-scale, randomized evaluation of a high-profile PD program in China, in which teachers were randomized to receive PD; PD plus follow-up; PD plus evaluation of their command of the PD content; or no PD. Precise estimates indicate that PD and associated interventions failed to improve teacher and student outcomes.

Show body
Journal Articles

Developing Instruments to Assess and Compare the Quality of Engineering Education: the Case of China and Russia

Elena Kardanova, Prashant Loyalka, Igor Chirikov, Lydia Liu, Guirong Li, Huan Wang, Ekaterina Enchikova, Henry Shi, Natalie Johnson
Assessment and Evaluation in Higher Education , 2016
Relatively little is known about differences in the quality of engineering education within and across countries because of the lack of valid instruments that allow for the assessment and comparison of engineering students’ skill gains. The purpose of our study is to develop and validate instruments that can be used to compare student skill gains in mathematics and physics courses in undergraduate engineering programmes across countries. The approach includes procedures to establish construct validity and other necessary psychometric properties. Drawing on data collected from over 24 engineering experts and 3600 engineering students across Russia and China, we establish that it is possible to develop valid, equitable and cross-nationally comparable instruments that can assess and compare skill gains.
Show body
Journal Articles

Appendices to “Developing Instruments to Assess and Compare the Quality of Engineering Education: the Case of China and Russia”

Igor Chirikov, Elena Kardanova, Lydia Liu, Guirong Li, Huan Wang, Ekaterina Enchikova, Henry Shi, Natalie Johnson
Assessment and Evaluation in Higher Education , 2016

Appendices include:

  • Appendix A. Analytical approach: technical details

  • Appendix B. The results of the psychometric analysis for grades 1 and 3 physics tests 

  • Appendix C. The Content Areas of the Math and Physics Tests

  • Appendix D. Selection of Majors

  • Appendix E. Chinese Curricular Standards 

  • Appendix F: Test Item Selection

Show body
Journal Articles

Developing Instruments to Assess and Compare the Quality of Engineering Education: The Case of China and Russia

Elena Kardanova, Prashant Loyalka, Igor Chirikov, Lydia Liu, Guirong Li, Huan Wang, Ekaterina Enchikova, Henry Shi, Natalie Johnson
Assessment & Evaluation in Higher Education , 2016
Relatively little is known about differences in the quality of engineering education within and across countries because of the lack of valid instruments that allow for the assessment and comparison of engineering students’ skill gains. The purpose of our study is to develop and validate instruments that can be used to compare student skill gains in mathematics and physics courses in undergraduate engineering programmes across countries. The approach includes procedures to establish construct validity and other necessary psychometric properties. Drawing on data collected from over 24 engineering experts and 3600 engineering students across Russia and China, we establish that it is possible to develop valid, equitable and cross-nationally comparable instruments that can assess and compare skill gains.
Show body
Journal Articles

The Impact of Teacher Credentials on Student Achievement in China

Jessica Hsiaochieh Chu, Prashant Loyalka, James Chu, Qinghe Qu, Yaojiang Shi, Guirong Li, Scott Rozelle
China Economic Review , 2015
Teacher quality is an important factor in improving student achievement. As such, policymakers have constructed a number of different credentials to identify high quality teachers. Unfortunately, few of the credentials used in developing countries have been validated (in terms of whether teachers holding such credentials actually improve student achievement). In this study, we employ a student-fixed effects model to estimate the impact of teacher credentials on student achievement in the context of the biggest education system in the world: China. We find that having a teacher with the highest rank (a credential based on annual assessments by local administrators) has positive impacts on student achievement relative to having a teacher who has not achieved the highest rank. We further find that teacher rank has heterogeneous impacts, benefiting economically poor students more than non-poor students. However, other credentials (whether the teacher attended college or held teaching awards) have no impact on student achievement. 
Show body