Student Reports

Spring 2012

Professors Douglas J. Besharov and Douglas Call

In Spring 2012, students worked with clients at the Baltimore City Health Department, Baltimore City Public Schools, Folger Shakespeare Library, Montgomery County Department of Health and Human Services, Montgomery County CountyStat, Montgomery County Public Schools, Montgomery County Office of Community Partnerships, Montgomery County Office of Legislative Oversight, U.S. Department of Education, U.S. Department of Health and Human Services, U.S. Government Accountability Office, and the U.S. National Endowment for the Arts. Students performed a wide variety of analyses, including cost-benefit analysis, cost-effectiveness analysis, performance measure development, process evaluation, and meta-assessment.

The following are the final projects prepared by the students. They are listed in order of clients.

Note: The associated files are best viewed using Adobe Reader. To get the software, go to

Municipal Government

Baltimore City Health Department

1. A Case Study Approach to Finding Lessons Learned on Health in all Policies for Healthy Baltimore 2015. The Baltimore City Health Department (BCHD) has requested lessons learned in other jurisdictions implementing health in all policies (HiAP) to help inform their comprehensive health policy agenda, Healthy Baltimore 2015 (HB2015). This report attempts to answer the following research question: What should the BCHD keep in mind in order to get the success they want to have in implementing HB2015? This report provides the BCHD with findings on successful strategies to implementing HiAP using a case study approach. An examination of three jurisdictions' policies (New York City, Chicago, and California) will help inform the BCHD as it currently develops and plans the implementation of its comprehensive health policy agenda.

The findings are presented in the following five categories: capacity building, engaging stakeholders outside city agencies, engaging city agencies, communication strategy, and prioritizing issues. Much can be learned from other jurisdictions, however there are aspects of implementation that are community- specific, and thus, Baltimore should consider the findings in this report and adapt the recommendations to the context of the City and the community's needs.

Paper Presentation

2. Government's Role in Promoting Physical Activity: An Analysis of Case Studies for the Baltimore City Health Department. This report analyzes case studies from around the country for the Baltimore City Health Department (BCHD). The research question is as follows: what is government's role in promoting physical activity within government agencies and in the city as a whole? This report is a part of a larger effort to redesign communities to prevent obesity. The methodology consisted of an analysis of overall feasibility, implementation strategies, and challenges. The author proposes the following policies to help BCHD achieve this goal: (1) Governments should adopt policies and regulations that support active environments (2) Governments should model a "healthy fit" workplace. Safety issues and limited resources and infrastructure to support the built environment were cited as the greatest challenges to implementation.

Paper Presentation
Baltimore City Public Schools

3. Designing and Implementing a Fee-For-Service Model: Lessons from Baltimore City Public Schools' FY13 Funding Analysis. Baltimore City Public Schools' (City Schools) Central Office currently provides an array of services to schools that exceed current funding models and minimum legal requirements. In order to understand the value of these services and to restore the integrity of City Schools' funding models, Central Office administration commissioned City Schools' Office of New Initiatives (ONI) to conduct a funding analysis and design a fee-for-service schedule for eight district services across two central offices. This paper discusses the short-term and long-term goals of FY2013 Funding Analysis, the costing methodology used to create the fee-for-service model, data limitations and organizational roadblocks, and recommendations for successful implementation. Lessons from the FY2013 Funding Analysis will inform future cost analyses and policy decisions, as City Schools continues to reexamine the role of the Central Office in delivering services to schools.

Paper Presentation

4. Performance Measurements for Family Institute Baltimore City Public Schools. The Family Institute, an office within the Office of Engagement in Baltimore City Schools, is a program developed to bring the community and parents of its students together to get involved in the education of its students. The Family Institute offers workshops and online resources to parents in order to help them become knowledgeable, skillful participants in their child's educational performance. This report provides the evidence model this program was based, provides evidence for twenty-two of the program offerings, as well as develops performance measures for the Family Institute workshops. The performance measures are then converted into a workshop survey to be completed by participants. The survey provides information on the workshop quality and its effectiveness in presenting knowledge and changing the attitudes of parents. The survey tool provides a way for Family Institute to compare workshop quality in an objective manner.

Paper Presentation

5. Performance Measures for Baltimore City Public Schools' Site-Based Mentoring Program. This paper describes the development of suggested performance measures for Baltimore City Public Schools' Site Based Mentoring Program. Before presenting the performance measures, I present a rationale for using site based mentoring, including its potential to increase retention, improve teacher effectiveness and its cost effectiveness. From there, I describe City School's site based mentoring model in greater detail, presenting it as a logic model. Next, I present other school district's mentoring programs and analyze their performance measures, for use in guiding the creation of City School's measures. Finally, I present the suggested performance measures for City Schools, both output and outcome measures, and conclude with recommendations for how these performance measures should be used.

Paper Presentation

6. The Impact of Supplemental Educational Services in Baltimore City Schools. The No Child Left Behind Act of 2001 (NCLB) became the most influential piece of federal education legislation in generations. Ten years later, the legislation is being retooled at a state-by-state level through a waiver process introduced by the Obama Administration. One provision under considerable debate is NCLB's Supplemental Educational Services (tutoring) requirement for low performing Title I schools. This paper attempts to evaluate the effectiveness of the federal tutoring mandate in Baltimore City Schools, which spent nearly $13 million on the program in 2010-11 alone. Using a difference in differences methodology with controls for observed student characteristics, this paper isolates the effect of tutoring on the Maryland State Assessment proficiency rates of 1,384 participating students in grades 3-8 compared to 5,409 eligible students in grades 3-8 who did not participate.

The program overall did not appear to have a significant impact on mean MSA proficiency rates during the 2010-11 school year in reading or math. Findings show small positive effects of SES on mean MSA proficiency rates in fourth grade math, middle school reading, and for students retained from the previous grades, though significant negative effects are found in fifth grade reading MSA proficiency rates. Among tutoring providers serving at least 300 students, six providers had no effect on mean MSA proficiency rates and one provider, Innovative Educational Program, demonstrated a large and significant positive effect on reading performance.

Paper Presentation
County Government

Montgomery County Department of Health and Human Services

7. Performance Measures: Montgomery County Health and Human Services. The Montgomery County Health and Human Services department has taken on the task of reviewing existing performance measures as a result of multiple years of budget cuts, funders asking for measurements of effectiveness, and the desire to manage using data. This project is a review of the performance measures used within three service areas. The client has asked that the performance measures be reviewed for each program to identify measures of effectiveness and performance. For the purpose of this paper it is assumed that programs were operating as intended and improving the life of the target population.

Paper Presentation
Montgomery County CountyStat

8. Recommended Performance Evaluation Metrics and Analysis of Performance-Indicative Components for Ride-On, the Montgomery County Department of Transportation's Transit Division. Montgomery CountyStat develops performance metrics and works with county offices and agencies to evaluate and improve performance. The Department of Transportation's Transit Division (Ride On) was identified as having an incomplete portfolio of performance metrics currently in use, making it difficult to produce performance improvements. The current project seeks to correct this by identifying similar jurisdictions and surveying their performance programs. New performance metrics are then proposed, which CountyStat can add to existing metrics in order to improve performance tracking, with the ultimate goal of inducing data-based performance improvements. This report also examines two programs which are indicative of performance and of great interest to CountyStat: Mystery Riders, and Real-Time Arrivals. Finally, this report analyzes revenue streams and cash outflows among similar transit systems, and identifies strengths and weaknesses in Montgomery County based on this data.

Paper Presentation
Montgomery County Public Schools

9. Social Media and Education: Benefits, Concerns, and Application. This report, prepared for Montgomery County Public Schools, examines the benefits, concerns, and application of using social media in the classroom. The report examines and synthesizes the current body of literature. Additionally, two case studies of high schools provide further insights. Based on the literature review and case studies, key findings and recommendations are presented.

Paper Presentation
Montgomery County Office of Community Partnerships

10. Patterns of Vulnerability and Barriers to County Services: Needs Assessment of the African and Caribbean Populations of Montgomery County. This report examines the service needs and experiences of the African and Caribbean populations of Montgomery County, particularly examining the accessibility and utilization of County services. Using a community-based participatory needs assessment model, the author conducted four focus groups with members of the African and Caribbean communities. The focus group study revealed three overarching barriers to County services and resources. These include lack of knowledge of County services; inconsistency in accessing and navigating County services; and inability to access services due to language and cultural barriers.

The author proposes the following recommendations to strengthen service accessibility and utilization by the target population: (1) strengthen the target population's knowledge of County services through enhanced outreach; (2) reexamine points of access to County services by the target population; (3) address language and cultural barriers to County services; and (4) conduct additional research to further explore barriers and patterns of vulnerability.

Paper Presentation
Montgomery County Office of Legislative Oversight

11. Montgomery County's Deferred Retirement Option Plan: A Comparative Synthesis of Past Programmatic Research. A Deferred Retirement Option Plan (DROP) is an optional program that allows employees who are eligible for retirement to continue to work and receive their defined benefit pension. The pension benefit is credited to an account while the employee collects a salary and continues to work. The most common reason cited to create a DROP plan is the retention of skilled employees and institutional knowledge. Given certain design elements and other conditions, they can effectively do this at costs that are relatively low. However, DROPs often end up costing more than anticipated and several jurisdictions have scaled them back in recent years. This paper conducts an analysis of seven different DROPs which shows that DROPs generally cost more than alternative scenarios where employees either retire (costing 28% more on average) or continue working rather than entering DROP (costing 20% more on average). It also makes recommendations for how jurisdictions can structure their DROP plans to minimize costs.

Paper Presentation
Federal Government

U.S. Department of Education

12. Bullying in Schools: A Policy Analysis. Bullying in schools has negative short and long term social, physical and psychological effects on both bullies and victims. The problem of bullying in schools is not new but increased public attention due to school violence, cyberbullying, and student suicides has pressured government agencies to act. The Department of Education has begun to address the problem of bullying but sought an analysis of further options. This policy analysis draws on the research literature on causes and effects of bullying, current survey data on prevalence, news articles on current efforts, and Department documents and statements to provide an overview of the problems of bullying and the potential policy options for the Department. No single option will solve the problem of bullying in schools. However, by focusing resources on expanding and improving research on bullying in schools, and establishing definitions of bullying the Department can have a positive long term impact on the prevention of bullying.

Paper Presentation
U.S. Department of Health and Human Services

13. Analysis of the LGBT Population and its Take-up of ACF Services. Population-based survey data reveal that 3.8% of adults (roughly 9 million individuals) in the United States identify as lesbian, gay, bisexual or transgender (LGBT). As part of recent Department of Health and Human Services (DHHS) efforts to assess LGBT needs, the Administration for Children and Families (ACF)—the DHHS agency responsible for programs promoting the economic and social well-being of families, children, individuals, and communities—requested that a needs assessment of LGBT populations be conducted to summarize current data and locate data gaps on low-income LGBT individuals and families that ACF programs serve.

The study was conducted by a University of Maryland School of Public Policy student in conjunction with DHHS staff from October 2011 to May 2012. It addresses the research areas requested by ACF: summarizing available data on low-income and “at-risk” (with regard to participating in social safety net programs) LGBT individuals and families; describing the take-up of ACF services by LGBT populations; and identifying research gaps and opportunities for ACF to improve administrative data systems.

Paper Presentation

14. Applying Economic Analyses to Teenage Pregnancy Prevention Programs. The government has the opportunity to save billions of dollars by successfully replicating programs that reduce teenage pregnancyi. Although the Department of Health and Human Services identified 28 empirically effective teen pregnancy programs, no comprehensive economic analysis has been conducted. The central question this report answers is; "What type of economic analysis should be applied to the programs identified in the List of Evidence Based Teen Pregnancy Prevention Programs?" The benefits and limitations of the four different types of cost analyses are analyzed. Examples of cost-feasibility, cost-benefit and cost-effectiveness are applied to the three curriculums of: Teen Health Project, It's Your Game Keep it Real and Promoting Health Among Teens! Abstinence-Only.

The author recommends conducting a cost-effectiveness analysis to select the program most efficient in reducing the rate of teenage pregnancy. Currently, however, only five of the evaluations of effective programs report impact data on teenage pregnancies. Thus, additional impact data needs to be collected from the effective programs in order to conduct a thorough cost-effectiveness analysis. Additionally, the author recommends that future funding of pregnancy prevention programs require mandatory reporting of specific cost and pregnancy rate. If resource restraints makes conducting randomized experiments infeasible, then the author recommends designing pre-post evaluations or interrupted time series evaluations. Because data collected is dependent on the counterfactual, outcome data on the age of sexual initiation and the frequency of contraceptive use, and the number of sexual partners should be considered.

Paper Presentation

15. Estimating the Cost of Health Disparities: Summative Evaluation and Analysis. In spite of impressive medical and policy advances in total population health over the last century, the benefits of improved health have not been felt equally by all Americans. Health disparities are comparative measures that describe these differences in health status, access to health care, and quality of healthcare between subpopulation groups. Since 2003, the Agency for Healthcare Research and Quality has reported annually on over 240 measures of quality and access issues related to disparities in healthcare. The purpose of this report is to supplement current health disparity descriptions through the identification of methods used to estimate the cost of health disparities. Couched in Cost of Illness methodology, existing costing techniques make use of direct and indirect cost calculations and apply them to disparity measures. Elements of a Cost of Illness study include perspective, top-down or bottom-up cost approaches, time period, and cost units. The findings of this report suggest that application of bottom-up microcosting methods applied to absolute disparity measures are most applicable and useful for estimating the cost of health disparities.

Paper Presentation

16. Improving Integration Measures for Dual Eligible Special Needs Plans. Those who are dually eligible for Medicare and Medicaid face health status, social, and economic vulnerabilities at a much higher level than the rest of the population, and take up a disproportionate amount of both Medicare and Medicaid spending. Adding to the high levels of spending on dual eligibles, Dual Eligible Special Needs Plans (D-SNPs), a type of Medicare Advantage plan specifically for dual eligibles, had not been required to prove integration and coordination of services until recent healthcare reforms. This paper aims to determine the link between care integration and care coordination and quality for dually eligible beneficiaries by analyzing reporting requirements for D-SNPs. It recommends that the Model of Care quality reporting requirement be updated to include measures pertaining to the level of access beneficiaries have to both Medicare and Medicaid providers, the quality of transitions of care between Medicare and Medicaid providers, and the nature of payments provided by the D-SNP to Medicare and Medicaid providers.

Paper Presentation
U.S. Government Accountability Office

17. National Sick Leave Policy: Needs Assessment: How Would a National Paid Sick Leave Policy to Recover from Short-Term Illnesses Impact American Workers? Approximately 39 percent (44 million) of private-sector workers do not have access to paid sick days to recover from short-term illnesses, affecting their health and financial stability. This paper uses an extensive review of literature to address the current policies relating to sick leave, the populations affected by lack of access to sick leave, problems associated with lack of access to paid sick leave, and the potential impacts of a national paid sick leave policy. The paper discusses future research options to better understand the potential impacts of a national paid sick leave policy.

Paper Presentation

18. Research Synthesis on the Validity, Reliability, and Effectiveness of the Danielson Framework for Teaching, the Classroom Assessment Scoring System, the Education Value-Added Assessment System, and the New York City Value-Added Model. This research synthesis looks at four of the most prominent teacher evaluation models, the Danielson Framework for Teaching (FFT), the Classroom Assessment Scoring System (CLASS), the Education Value-Added Assessment System (EVASS), and the New York City Value-Added Model, to examine how valid, reliable, and effective these programs are in measuring teacher effectiveness. This research synthesis contributes to the discussion of appropriate teacher evaluation measures, especially in light of the dramatic changes in state teacher evaluation policies over the past few years and the proposed reauthorization of the Elementary and Secondary Education Act (ESEA). The proposed reauthorization would require student achievement to be a significant part of teachers' evaluations and would require districts and administrators to use teacher evaluations to make personnel decisions. The findings presented in this research synthesis include that the four teacher evaluations studied are not completely valid or reliable, which compromises their effectiveness in making high stakes personnel decisions. Further, the synthesis displays the need for more empirical research on the validity, reliability, and effectiveness of value-added models in particular.

Paper Presentation

19. Transition Assistance for Veterans: Further Research on the Transition Process Needed for VA to Better Meet the Needs of OEF/OIF Veterans. In order for VA to successfully meet its goal of easing the reentry of new veterans into civilian life, past research should be synthesized to determine what is known about how well veterans fare during the transition so that this knowledge can be applied to adjust existing efforts and/or create new programs.

The purpose of this report is to answer the following question: what is known about how well veterans fare during the first two years of transition to civilian life? In order to answer this broader question, four more specific research questions will be evaluated. The four research questions are: (1) What information, if any, does VA collect about transition outcomes, and how adequate is VA's research? (2) What is known, either by VA or others, about the extent to which transitioning veterans experience negative outcomes during the first two years after leaving service? (3) Are certain individuals at a higher risk for a difficult transition? and (4) Is there transition support that VA is not providing that some experts believe is critical to success?

Paper Presentation
U.S. National Endowment for the Arts

20. Portfolio Review Reporting for the Art Works Grant Programs at the National Endowment for the Arts. Portfolio Review Reporting for the Art Works Grant Programs at the National Endowment for the Arts. This report develops a template for grant portfolio review reporting for the National Endowment for the Arts. The report template will aid program managers in managing their grant portfolios in light of a new 2012-2016 agency Strategic Plan and new grantee forms. Development of the report was in furtherance of the Arts Endowment's goal, as stated in the 2012-2016 Strategic Plan, to be an "Efficient and Vigilant Steward of Public Funds."

The report and performance measures were developed through a review of the agency's 2012-2016 Strategic Plan, consultation with Arts Endowment staff, and research into grants management and comparison portfolio review programs at the National Science Foundation and National Institute of Food and Agriculture. The report itself is divided into three general categories: (1) Summary Statistics; (2) Long-term Trends; and (3) Special Topics. Within each broad category are measures which capture all three major phases of the grant life cycle: awards, active grants, and closed grants. Maps, column graphs, and pie charts are used to display many of the measures.

Paper Presentation

Folger Shakespeare Library

21. Implementation Evaluation: The Folger Library's Shakespeare Set Free Toolkit. This report uses an implementation evaluation to examine the Folger Shakespeare Library's Shakespeare Set Free Toolkit, a comprehensive and multifaceted resource for teachers to use in their classrooms as they tackle teaching Shakespeare to students at all grade levels. This project aims to answer the following questions: (1) Is the Shakespeare Set Free Toolkit useful to teachers? (2) Does the Toolkit's program design match the Toolkit's implementation in classrooms? and (3) How can the Toolkit be improved?

An implementation evaluation was conducted by surveying teachers who have the Toolkit in their classrooms, meeting with key stakeholders and reviewing literature on the challenges of teaching Shakespeare. The evaluation found that the Shakespeare Set Free Toolkit is useful to teachers and assists them in sparking student interest in Shakespeare and his plays. However, the evaluation also found that parts of the Toolkit can be improved, and reinforced during trainings, to enhance the experience of teachers as they use the Shakespeare Set Free Toolkit to teach Shakespeare in their classrooms.

Paper Presentation

Back to top