MLDS Center Research Series
Upcoming Research Series:
Title: Problem, Research, Action: Poverty Measurement Transition in Baltimore City Public Schools
Presenters: Romona Carrico & Chris Wohn – Baltimore City Public School System; Amir François – Maryland State Department of Education
When: October 4, 2018 12:30-1:30 pm
Where: University of Maryland School of Social Work (525 W. Redwood St) Room 3E04
This presentation will cover the methodology of the longitudinal and historical poverty analysis and subsequent school-level and student subgroup analyses using data from Baltimore City Public Schools. The second part of the presentation will discuss how the Office of Achievement and Accountability (OAA) in Baltimore City Public Schools assessed the impact of the change in the poverty measurement process on school-level poverty rates using a multivariate prediction model.
Romona C. Carrico is a Data Analyst for the Title I Services in the Office of Achievement and Accountability at Baltimore City Public Schools. Her work includes analyzing data to inform and support decision making pertaining to Title I schools. She began her career at the Caribbean Community (CARICOM) Secretariat, analyzing international merchandise trade data to support the Regional Integration process. Romona holds a B.Sc. in Development and Economics with upper second class honors from the University of London, lead College London School of Economics and Political Science (LSE).
Christopher Wohn is the interim director of research in the Office of Achievement and Accountability in Baltimore City Public Schools. In this role, Dr. Wohn assists the District with analysis of assessment data, early warning indicators of student success, and the evaluation of school programs. He has also served as a data science consultant for several companies and organizations around the DMV region. Prior to his work on the research team, Dr. Wohn was a teacher for three years in a Baltimore City High School where he taught chemistry, physics, and the Project Lead the Way (PLTW) biomedical sciences curriculum. He received his teaching certificate through the Baltimore City Teaching Residency (BCTR) program in 2013. He has a B.S. in Biochemistry from the University of Maryland in College Park and a M.D. from the University of Maryland Medical School.
Amir François is a Senior Research and Data Specialist in the Office of Research and Strategic Data Use at the Maryland State Department of Education (MSDE). Amir began his career as an independent school middle grades science teaching fellow and next worked as a public charter school elementary math associate teacher in Boston, MA. After obtaining his Doctorate degree, he worked as a postdoctoral fellow in the Department of Mental Health at the Johns Hopkins Bloomberg School of Public Health in the Center for School-Based Violence Prevention studying school climate effects on intercultural competence and the prevention of adolescent alcohol and drug use. Before joining MSDE, Amir worked as Program Evaluator II - Title I in the Office of Achievement and Accountability in Baltimore City Public Schools. While there, Amir’s work included evaluating the effectiveness of federal funding within the district and supporting district and school staff in evaluating the effectiveness of programming in Title I schools. Amir holds a B.A. in Philosophy-Neuroscience-Psychology from Washington University in St. Louis and a Ph.D. in Applied Developmental Science and Educational Psychology from the Curry School of Education at the University of Virginia.
Concluded MLDS Center Research Series
Note: Please use Mozilla Firefox or Internet Explorer browsers for viewing the following attached videos as the required plugins for the video presentation may not work with Google Chrome browser.
- collapse All
- show All
Year - 2018
Presenters: Bess A. Rose, Dawnsha R. Mushonga, and Angela K. Henneberger
Topic: The Relationship Between Poverty and Long-Term Student Outcomes: Disentangling the Effects of Individual and School Poverty
Presentation Abstract: The MLDS Center is examining the effects of school-level concentrated poverty and individual student poverty on outcomes such as high school graduation, entry and persistence in post-secondary education, entry into the workforce, and wages earned. From previous research, we know that both individual poverty and school-level poverty are significant barriers to educational success, but the relative impact of these factors and how they interact is unclear. We used statewide longitudinal data to examine the relative impact and interaction of student- and school-level poverty on long-term outcomes. Preliminary findings suggest that defining poverty based on students’ status at a single point in time, rather than considering their history of poverty, may lead to underestimating poverty’s effects. Findings also suggest that while both student poverty and school-level concentrations of poverty have a significant and negative effect on students’ outcomes, the effect of school-level poverty is considerably larger than that of individual poverty alone.
Presenters: Dr. Nolan G. Pope
Topic: The Multidimensional Impact of Teachers on Students
Presentation Abstract: For decades, policymakers and researchers have used value-added models that rely solely on student test scores to measure teacher quality. However, since teaching ability is multidimensional, test-score value-added measures of teacher quality may not fully capture the impact of teachers on students. In this talk, Dr. Pope will present research using test-score and non-test-score measures of student achievement and behavior from over a million students in the Los Angeles Unified School District to estimate multiple dimensions of teacher quality. Results indicate that test-score and non- test-score measures of teacher quality are only weakly correlated, and that both measures of teacher quality affect students’ performance in high school. Results from a simulation study removing teachers based on both dimensions of teacher quality show improvement in most long-term student outcomes by over 50 percent compared to removal of teachers using test scores alone. The long-term effects of teachers in later grades are larger than in earlier grades and that performance in core elementary school subjects matters more for long-term outcomes than other subjects.
Presenters: Heath Witzen, Research Fellow, Maryland Longitudinal Data System Center
Topic: The Effect of High School Career and Technical Education on Postsecondary Enrollment and Early Career Wages
Presentation Abstract: Career and Technical Education (CTE) has become a topic of considerable policy-making interest as a way of providing specialized education and expanding the number of career pathways available to high schools students. This research examines the effect of CTE program completion during high school on postsecondary outcomes, including college enrollment and workforce wages. Using propensity score matching, this research uses MLDS data to estimate a causal effect of CTE on postsecondary enrollment and wages up to six years after high school graduation.
Year - 2017
Presenters: Dr. David Blazar, University of Maryland College Park
Topic: Validating Teacher Effects on Students’ Attitudes and Behaviors: Evidence from Random Assignment of Teachers to Students
Presentation Abstract: There is growing interest among researchers, policymakers, and practitioners in identifying teachers who are skilled at improving student outcomes beyond test scores, and in turn building longitudinal data systems that track these measures. However, questions remain about the validity of these teacher effect estimates. Leveraging the random assignment of teachers to classes, Dr. Blazer’s research has found that teachers have causal effects on their students’ self-reported behavior in class, self-efficacy in math, and happiness in class that is similar in magnitude to effects on math test scores. Weak correlations between teacher effects on different student outcomes indicate that these measures capture unique skills that teachers bring to the classroom. Teacher effects calculated in non-experimental data are related to these same outcomes following random assignment, revealing that they contain important information content on teachers. However, for some non-experimental teacher effect estimates, large and potentially important degrees of bias remain. These results suggest that researchers and policymakers should proceed with caution when using these measures. They likely are more appropriate for low-stakes decisions, such as matching teachers to professional development, than for high-stakes personnel decisions and accountability.
Presenters: Dr. Angela K. Henneberger, Ph.D., Research Director, MLDS Center; Mr. Heath Witzen, Research Fellow at the MLDS Center
Topic: Applying Causal Inference Techniques to Strengthen Dual Enrollment Program Evaluation in Maryland
Presentation Abstract: Maryland Longitudinal Data System (MLDS) Center is legislatively required to submit an annual report on dual enrollment program participation (simultaneous enrollment in both a public high school and college). Programs like dual enrollment are often difficult to evaluate causally due to the absence of randomization. Modern causal inference techniques, such as propensity score methods, can be used to strengthen causal inferences in the absence of randomization in education sciences. We will apply this approach to data from the MLDS to evaluate dual enrollment program participation in high school. Outcomes examined will include college enrollment and persistence.
Presenters: Laura M. Stapleton and Yating Zheng, University of Maryland, College Park
Topic: Value Added Modeling and Alternate Approaches for Ranking Institutions
Presentation Abstract: One of the research questions on the MLDS research agenda is “Which 4-year institutions are graduating students most effectively and in the timeliest fashion?” Typically, an analysis referred to as value-added modeling (VAM) is used to address this type of question. However, VAM has been criticized in its use to rank institutions and teachers. In this talk, value-added modeling will be defined and its assumptions delineated. We will propose an alternate approach, one that utilizes propensity score methods to rank institutions within a set of institutions with some similar student characteristics (e.g., high school attendance). Workforce outcomes will be examined for students within institutions. A proof-of-concept analysis will be presented and the audience will be asked for feedback and ideas.
Presenters: Amber Bloomfield, Ph.D.
Topic: Investigating the “Brain-Drain” Phenomenon in Maryland
Presentation Abstract: Upon graduation from secondary school, students may follow one of three possible paths. They may enroll in a post-secondary educational institution within their state of residence; they may enroll in a post-secondary educational institution outside their state; or they may go on to activities other than post-secondary education, such as entering the workforce or military service. An ongoing goal for many states is to increase the proportion of in-state high school students who go on to attend in-state post-secondary educational institutions. A major driving force behind this goal is the concern that students who attend out-of-state institutions are less likely to return to seek employment in their original state of residence following graduation, and thus are less likely to contribute to the state’s economy and workforce quality. This occurrence is popularly known as “brain-drain.” There is as yet little evidence to evaluate how substantial this type of “brain-drain” may be. We address this gap in knowledge using data from the Maryland Longitudinal Data System (MLDS) to investigate employment information for Maryland public high school graduates who go on to attend an in-state post-secondary educational institution versus an out-of-state institution.
Presenters: Heath Witzen
Topic: Need-based Grant Aid in Higher Education: The Effects of the Howard P. Rawlings Educational Assistance (EA) Grant on Financial Aid, Academic Persistence, and Working while in School
Presentation Abstract: This presentation will showcase preliminary findings on the effects of the Howard P. Rawlings Educational Assistance (EA) Grant, the State of Maryland’s largest need-based grant program, on several student outcomes. Data from the Maryland Longitudinal Data System (MLDS) will be used to examine the strict eligibility cutoff for the grant as an identification strategy. A regression discontinuity approach will be used to investigate how receipt of the grant affects a student’s overall financial aid package, including grants from other sources and student loans. Using earnings data, it also examines whether students respond to grant aid by earning less in wages. Finally, the research estimates the effect of the EA grant on postsecondary persistence and degree receipt, and whether these effects differ by student income and high school characteristics.
Presenters: Angela K. Henneberger, Ph.D., Research Director, MLDSC
Topic: Dual Enrollment in Maryland: Highlights from the 2016 Dual Enrollment Report
Presentation Abstract: This presentation will highlight findings from the Maryland Longitudinal Data System Center’s annual report on dually enrolled students, which is a requirement of the College and Career Readiness and College Completion Act (CCR-CCA) of 2013. As defined in Education Article §18-14A-01, Annotated Code of Maryland, a dually enrolled student is a student who is enrolled in both a secondary school (high school) and an institution of higher education (college) in Maryland. Data from the Maryland Longitudinal Data System (MLDS) were used to identify students who (1) had overlapping enrollment dates in a Maryland public high school and a Maryland college and (2) were enrolled in the college for at least 30 days. Findings on the rates and trends in dual enrollment, the courses in which students were dually enrolled, and the college enrollment outcomes of dually enrolled students will be presented.
Year - 2016
Presenters: Terry V. Shaw and Susan Klumpner
Topic: Postsecondary College and Workforce Outcomes for Baltimore City Graduates
Presentation Abstract: Data from the Maryland Longitudinal Data System (MLDS) were used to examine the wages earned and industries employing Maryland high school diploma earners, certificate of completion earners, diploma earners via the GED program, and high school non-completers. For the population of students who did not enroll in college, wage outcomes were calculated using two methods: (1) annualized wages and (2) four-quarter wages. Annualized wages were calculated by dividing the total amount of wages earned in the year by the number of quarters worked and multiplying by four. This method retains all individuals who earned any amount of wages during the year. Four-quarter wages were calculated by summing all four-quarters of wages. This method retains only individuals who had wages for all four quarters within the year. Results using the two methods will be compared and discussed.
Presenters: Rachel E. Durham and Faith Connolly
Topic: Postsecondary College and Workforce Outcomes for Baltimore City Graduates
Presentation Abstract: Baltimore’s Promise (BP) is a city-wide partnership of public agencies, foundations, businesses and non-profit organizations that seeks to coordinate resources to promote better outcomes for Baltimore’s children and young adults. The Maryland Longitudinal Data System (MLDS) is one important partner to BP, as it collects and links critical data that can speak to many of its target goals. From the Baltimore Education Research Consortium, another research partner to BP, Drs. Durham and Connolly will share preliminary findings from analyses of MLDS data on college enrollment, persistence, degree completion, workforce attachment and earnings of Baltimore graduates from the Classes of 2008 through 2015. While their presentation will raise many more questions than it can answer in one hour, the authors hope to use these baseline findings to generate compelling future avenues for research on the well-being of Baltimore youth. Please come be a part of this conversation!
Presenters: Dr. Angela K. Henneberger, Ph.D., Research Director, MLDS Center
Topic: Remedial Coursework in Maryland Colleges: Examining High School Predictors and College Outcomes
Presentation Abstract: Data from the Maryland Longitudinal Data System (MLDS) will be used to examine the percentage of Maryland high school graduates who are assessed to need remedial coursework upon entering Maryland colleges. High school predictors of needing remedial coursework, such as demographic characteristics, attendance, and high school assessments scores, will be examined. Additionally, college outcomes, including degree completion and time to degree, will be examined. Policy implications and directions for future research will be discussed.
Presenters: Xiaying Zheng, Michael E. Woolley, and Laura M. Stapleton
Topic: STEM Workforce Preparation and Retention in Maryland
Presentation Abstract: This presentation will begin with an overview of what the STEM education and workforce includes and the current state of that workforce. Then analyses of the data housed in the Maryland Longitudinal Data System Center (MLDSC) will be used to detail first year workforce wage outcomes for postsecondary students in Maryland who earned degrees in STEM related disciplines. Those degree earners included graduates from the 2008-2009 to the 2012-2013 academic years. The retention patterns of those postsecondary STEM graduates will also be presented.
Presenters: Mathew Uretsky, MSW, MPH, MLDS Research Branch
Topic: High School Graduation in Context: An Investigation into the Individual and School-level Factors Related to On-Time Graduation in Baltimore City
Presentation Abstract: Over the past decade national data have described consistent improvements in reducing dropout and increasing graduation rates culminating in a record high four-year graduation rate of 81.4% in 2013 (Kena et al., 2015). This has been attributed in large part to reductions in disparities for minority students and reductions in dropout rates for students in large urban school districts such as Baltimore (DiPaoli et al., 2015). Although this reduction in dropouts is a positive trend, there is an increasing group of students who persist into and through four years of high school without earning a diploma (Kena et al., 2015; Maryland State Department of Education [MSDE], 2015). This talk explores this growing population of students, with the goal of informing policy, programming, and practice in order to promote on-time graduation, preparation for the workforce, and post-secondary readiness.
Presenters: Angela K. Henneberger, Ph.D. , MLDS Research Branch
Topic: Dual Enrollment in Maryland: Using Data from the Maryland Longitudinal Data System to Examine Trends, Demographics, and Outcomes
Presentation Abstract: Dual enrollment, which in Maryland refers to enrollment in a secondary school in the State and an institution of higher education in the State, has the potential to improve student educational access and outcomes. The primary aim of this presentation is to detail what is known about dual enrollment in Maryland using cross-agency data from the Maryland Longitudinal Data System. Longitudinal trends in the number of students dually enrolled, the characteristics of students dually enrolled, and the postsecondary enrollment outcomes of students dually enrolled will be presented. A discussion will focus on implications for future research and policy.
Year - 2015
Presenters: Dr. Laura Stapleton, MLDS Research Branch
Topic: Expanding MLDS Data Access and Research Capacity with Synthetic Data Sets
Presentation Abstract: The Center has an obligation to make data accessible to researchers, policy makers, and stakeholders. However, a constellation of state and federal confidentiality laws and policies including the statute that created the MLDSC, limits the ability of the Center to provide even de-identified raw data sets to researchers outside of the Center staff and officially affiliated researchers. As part of the 2015 State Longitudinal Data Systems Grants recently announced by the U.S. Department of Education, the State of Maryland has been awarded approximately $2.6 million to create a synthetic data system. In this talk, we will describe what synthetic data are and the challenges that we will face in the creation of the system. In this talk, the presenters will describe what synthetic data are and the challenges that they will face in the creation of the system. They will also identify considerations surrounding the creation of the system that require input from possible end users from the research community. This Research Series presentation and the discussion to follow initiate this first step in system creation.
Presenters: Mr. Daniel McNeish, Research Analyst, MLDS Research Branch
Topic: Clustered Data - Are Multilevel Models Really Necessary?
Presentation Abstract: In educational research, multilevel models (MLMs) are a popular method for modeling data that come from a clustered or nested structure (e.g., students clustered within schools). Although other methods for clustered data exist, an informal literature search in a recent study found that MLMs are employed in published articles by an overwhelming margin of over 15 to 1 to other clustered data methods. This talk explores how the seeming knee-jerk reaction to apply MLMs to clustered data in education, while well-intentioned, may, at best, overcomplicate many analyses and, at worst, yield estimates that are not congruent with researchers’ expectations. Alternative methods to account for clustering that make fewer assumptions and are simpler to implement are discussed as are broad implications of relying too heavily on MLMs.
Presenters: Alison Preston, M.Ed., Student Fellow with the MLDSC Research Team
Topic: Visual Representations of Data - Review and Recommendations
Presentation Abstract: Creating quality visuals is an essential part of data analysis and publication, especially in education, due to the diversity of the intended audience and the need to effectively communicate accurate information. Research has shown that stakeholders identify with graphics that are familiar, easy to interpret, and relevant to their needs. Therefore, it is important to consider each individual graphic as well as the publication as a whole and make conscientious choices about the data representation. Elements of successful graphics (graphs and tables) include the selection of an appropriate format, color scheme, scale, and style. These elements will be discussed together with examples that demonstrate their impact on readability and interpretability. Once developed, individual graphics can be assembled into larger graphic displays, or dashboards, with similar attention paid to design elements and interpretability. Overall, the characteristics of the audience and the purpose of the content play a crucial role in the development of visuals for publication.
Presenters: Dr. Faith Connolly, Executive Director of the Baltimore Education Research Consortium; Dr. Rachel Durham, Assistant Professor, School of Education, JHU and Research Co-Director for the Baltimore Education Research Consortium
Topic: Post Secondary Enrollment and Assessed Need for Developmental Coursework
Presentation Abstract: This presentation considers the meaning of college readiness from both the district and college perspective. In the past decade, numerous actors in educational research, policy, and practice have converged upon a seemingly practical definition of college readiness that hinges upon a student’s lack of need for postsecondary remedial education. For good or for bad, this conception has become the de facto working definition of college readiness. BERC examined local Baltimore colleges’ assessment of remediation need and high school performance for the Baltimore City graduating Class of 2011. We focus on the four local institutions Baltimore City graduates attend: the Community College of Baltimore County (CCBC), Baltimore City Community College (BCCC), Morgan State University and Coppin State University.
Presenters: Ms. Treva Stack, Researcher and Statistician, Jacob France Institute; Dr. David Stevens, Executive Director, Jacob France Institute; Dr. Ting Zhang, Associate Director and Researcher, Jacob France Institute
Topic: The Business Case for Sustained Investment in the MLDS Center’s Research Capability
Presentation Abstract: The presenters will use nearly three decades of longitudinal information assembled from multiple administrative data sources to illustrate research challenge and solution lessons learned from a JFI cohort study conducted under a Workforce Data Quality Initiative sub-award from the Department of Labor, Licensing and Regulation.
Continuing proliferation of on-line earnings ‘outcome’ data products that lie beyond the control of SLDS programs nationwide pose at least two threats to these programs—absorption of staff time to understand data quality and interpretation nuances, and investment in explanation of the relevance of these nuances to diverse stakeholders.
A basic theme of the messages is that the L in MLDS Center initiatives matters. The value of the Center’s contributions will grow over time, but only if continuity of the capability occurs.
Year - 2014
Presenters: Dan McNeish is a Research Analyst at MLDSC; Laura M. Stapleton is Associate Professor in the Department of Human Development and Quantitative Methodology and Associate Director of Research with the MLDSC; Alison Preston is also a doctoral student in the Measurement, Statistics and Evaluation program and works on the MLDSC Research Team.
Topic: Strategies for Missing Data in Education Research
Presentation Abstract: In educational research and in large databases alike, missing data can be a pervasive challenge for estimating statistical models appropriately. Simply ignoring or removing observations with missing values can introduce bias, sometimes of large magnitude, into statistical analyses, resulting in inappropriate conclusions which can adversely affect policy decisions. This research talk will highlight the challenges that missing data present, why deleting or ignoring observations with missing values is problematic, and will introduce modern statistical methods (specifically, full information maximum likelihood and multiple imputation) that can be used to address missing data to produce unbiased estimates.
Presenters: Robert Croninger, Ph.D.; MJ Bishop, Ed.D.; Corey Shdaimah, Ph.D.
Topic: Online Education: Research, Theory, and Practice
Presentation Abstract: Delivering education online is both an important and rapidly growing method to provide education services. This panel discussion will include three panelists, one who has been synthesizing the available research on the trajectory and current state of online education, one who is a conceptual thinker and trainer in the area of the use of technology and the delivery of education online, and finally a practitioner applying online strategies to deliver graduate level postsecondary education coursework. Each panelist will give a 10-15 minute presentation at the beginning, the rest of the time will be for questions and discussion among the panelists and the panel members.
Presenters: Dr. Elisa Klein, Associate Professor, College of Education, University of Maryland College Park; Dr. Gail Sunderman, Research Scientist and Director, Maryland Equity Project, University of Maryland College Park; Dr. Lisa Berlin, Associate Professor, School of Social Work, University of Maryland Baltimore; Dr. Brenda Jones-Harden, Associate Professor, College of Education, University of Maryland College Park; Panel Facilitator: Dr. Michael Woolley, Associate Director, MLSDC and Associate Professor, School of Social Work, University of Maryland Baltimore.
Topic: Research, Policy and Practice in Early Education and Childcare
Presentation Abstract: The panel members bring extensive expertise and experience across the areas of research, policy, and practice in Maryland and beyond. The panelists will each have 10-15 minutes to present their research and practice experience, offer their policy input and insights, and suggest areas for research attention by the MLDS Center going forward. After some discussion among the panel members, the discussion will be opened up to attendees to ask the panel questions.
Presenters: Dr. Terry Shaw, Assistant Professor at the University of Maryland School of Social Work; Dr. Jill Farrell, Research Assistant Professor at the University of Maryland School of Social Work
Topic: Linking Information to Enhance Knowledge: Maryland's Multi-agency Data Collaborative
Presentation Abstract: The LINKs Data Collaborative was recently formed at the University of Maryland using data from Maryland's Department of Juvenile Services, Department of Human Resources, and the Evidence-Based Practice Center. LINKs provides a valuable case study on how a linked data system can facilitate comprehensive, data-driven, evidence-based decision making in Maryland. Presenters will give an overview of LINKs, the participating agencies, and present preliminary findings. This session should be instructive on the future research possibilities and outcomes of the MLDS Center.
Panel: Michael E. Woolley, Ph.D., Associate Professor, UM Baltimore, School of Social Work and Associate Director of the Research and Policy Services Branch of the MLDS Center (Moderator); Marvin Titus, Ph.D., Associate Professor of Higher Education, UM College Park, Department of Counseling, Higher Education and Special Education; Donni Turner, Director of Policy, Maryland Department of Labor, Licensing, & Regulation; Katherine Oliver, Assistant State Superintendent, Career & College Readiness, Maryland State Department Of Education; Jason Perkins-Cohen, Executive Director, Job Opportunities Task Force and member of the MLDS Governing Board; June Streckfus, Executive Director, Maryland Business Roundtable for Education.
Topic: The March MLDSC Research Series will be a panel discussion about Labor Force issues.
Presentation Abstract: The focus of this panel will be important issues facing Maryland and specifically what kind of questions should the MLDS Center be examining with respect to the educational preparation, entry, equity and retention of individuals in the Maryland Labor Force. We have a very knowledgeable set of 5 panelists who bring a wide range of experience and perspectives on these vital issues. Each panelist will have an opportunity to speak about the issues she or he sees as important, then the panel will have an opportunity to ask each other questions, and finally attendees will have the opportunity to ask the panel questions.
Year - 2013
Presenters: Dr. Laura Stapleton, MLDS Research Branch
Topic: Casual Inference in Education Policy Research
Presenters: Michael E Woolley, MSW, DCSW, Ph.D.
Topic: Examining Group Differences in Statistical Analysis of School and Workforce Outcomes
- collapse All
- show All