Applied Research Center¹
at the Andrew Young School of Policy Studies

The Applied Research Center is one of the foremost centers for evaluation, survey research, and policy analysis in academic settings in the U.S. In 1998, the Center won the American Evaluation Association Award for the Best Evaluation of the year and was commended by the Georgia General Assembly for providing them with independent and accurate assessments of public programs. The Center has two main divisions, one for evaluation and policy analysis, and the second for survey research. Gary Henry, the founding Director, is on leave to the Packard Foundation this academic year. Judith Ottoson is the Acting Director of the Applied Research Center.

The Center has extensive experience in measuring performance of programs in areas as diverse as education, transportation, health, environment, and information systems. The evaluation and policy analysis projects have included many studies of educational programs. The Center's work in analyzing the effects of Georgia's innovative HOPE Scholarship and in evaluating the nation's first universal pre-kindergarten program has brought national attention. Evaluations conducted by the Center staff range from large-scale statewide and national programs to school-based innovations.

The survey research division conducts many surveys each year including the quarterly Georgia State Poll. The center maintains state-of-the-art telephone and web-based survey technology and is staffed by some of the best-trained and most versatile interviewers and programmers in the Southeast. The Lab provides expertise not only in data collection, but also in data management, study design, and project management.

Although the Center's evaluators, survey researchers, and policy analysts often work together on projects that require surveys as part of a larger analysis, they also use other kinds of methodology, such as focus groups, observation, and document review to understand policy implementation and impact.

In This Page:

• Projects
• Reports

Projects

Studies of Pre-K Programs in Georgia. Gary Henry. There are two current longitudinal studies evaluating early childhood programs across the state of Georgia. The first longitudinal study, which began in 1996, is a five-year evaluation of Georgia's Pre-Kindergarten (Pre-K) Program. This study followed a probability sample of over 3,000 four-year-olds from 203 Pre-K classes selected from across the state. Student outcomes, including teacher assessments of academic and behavioral skills and promotion, have been collected and analyzed annually. Current efforts are focused on analyzing data and report writing from the fifth year data collection. Previous years' reports are among the most downloaded documents from the Applied Research Center's website.

The second longitudinal study, also known as the Georgia Early Childhood Study, began in July 2001. This study samples approximately 650 children participating in a variety of preschool settings, including Pre-K, Head Start, and other preschool facilities serving four-year-olds. This study will help determine the impact of the various types of preschool experiences on children's future school success, as well as investigate the reasons behind parents' choices for child care arrangements. Children will be followed into the first grade. Each year, data collection efforts will include: individualized standardized assessments with sampled children in key developmental areas, teachers' and parents' assessments of children's skills and behaviors, classroom observations rating the quality of the classroom environments, surveys with teachers and program directors regarding early childhood education, surveys with parents about preschool arrangements and family demographics.

The Office of School Readiness funds both of these projects, with the current year's funding set at $385,111. Total funding for the two projects is $1,038,388.

Office of School Readiness: Registry Project. Gary Henry and Judith Ottoson. The project offers accurate data on children's Pre-K experiences and provides a base for future research and evaluation. The project conducted for the Office of School Readiness consists of three major data components: class rosters, lead teacher credentials, and resource coordinator reports. The project is in its fourth and final year. ($776,252)

Assessing the Skills of Local Government Managers. Greg Streib. The Applied Research Center, in partnership with the Department of Public Administration and Urban Studies, developed a knowledge-based assessment instrument and a "multi-rater" assessment instrument in conjunction with the International City/County Management Association (ICMA). The instruments assist local government managers in their professional development by providing feedback on 17 practices determined by local government managers to be important for effective local government management. Now that the instruments have been developed and validated, the Applied Research Center oversees instrument scoring, report generation, database maintenance, and analysis of the results. ICMA. ($236,764)

The Impacts of High Stakes Accountability on Teachers' Professional Development and Student Achievement. Gary Henry. The Spencer Foundation has funded a two-year study of the impacts of high stakes accountability in the South. This study is being conducted in collaboration with researchers at the University of North Carolina. Through case studies and surveys, researchers will attempt to assess how differences in state accountability systems affect teachers' professional development, attitudes toward student learning, and, if possible, student achievement. ($210,000)

Probing Motorists' Perceptions of Highway Quality. Theodore Poister. This research is designed to learn more about Pennsylvania motorists' satisfaction or dissatisfaction with the highways they drive on and the basis upon which they rate these highways. Using a detailed telephone survey of motorists in eight counties, this research will provide the Department of Transportation with a better understanding of its customers' concerns regarding highway maintenance issues. Based on this information, the project will also pilot a revised mail-out survey instrument for soliciting customer feedback annually on a statewide basis. Pennsylvania State. ($199,030)

Customer Satisfaction Survey 2001-2002: Telephone Survey. Charlotte Steeh. In its second year these monthly surveys measure the satisfaction of employers and individual clients with Georgia's workforce development programs as implemented by the Georgia Department of Labor. The federal Workforce Investment Act of 1998 (WIA) requires each state to carry out this evaluation annually. A total of 10,500 telephone interviews will be completed for the 2001-2002 fiscal year with a sample of employers and clients. ($193,403)

Evaluation of the National School Health Leadership Institute. Judith Ottoson. This project evaluates the efforts to coordinate school health through the training of designated school health personnel. The three-year national Institute involves a series of training and booster sessions, along with technical assistance to support on-site implementation of Institute learning. This five year project has completed its second year. The evaluation will inform decision makers about school health policy. American Cancer Society. ($125,429)

Rural Health 2001: RDD Telephone Surveys. Charlotte Steeh. This county-by-county telephone survey of residents in Georgia's rural counties was conducted in cooperation with the Georgia Health Policy Center for the Networks for Rural Health from 1999 - 2001. The survey assesses respondents' opinions toward and respondents' utilization of a wide range of health care services including physician and hospital services, mental health services, nursing homes and substance abuse programs. Additionally some demographic and health status data were collected including the information concerning respondents health insurance. In all, surveys have been completed in eighteen rural counties. Seven of these were completed in 2001. ($110,327)

North Carolina Parent, Teacher and Principal Surveys. Charlotte Steeh and Dana Rickman. For a second year, North Carolina parents, teachers, and principals were surveyed on a variety of topics including teachers' professional development, principals' leadership, interactions with parents, and performance of schools within the state. Results are presented in Governor Easley's report on the progress of North Carolina to become among the top ten states in education performance in the nation. North Carolina's Education Research Center. ($100,000)

Preparing Tomorrow's Teachers to Use Technology (PT3). Judith Ottoson and Gary Henry. This project addresses the need for preparing new teachers for the best use of technology in the classroom and increasing their use of technology within the existing curriculum. In addition to increasing opportunities for mentorship and collaboration, the program strives to set student standards for technology proficiency and collaboration at all levels of education. As the PT3 Evaluator, ARC will assess implementation and impact on participating pre-service teachers and in-service teachers to determine the extent of preparation provided to PT3 teachers to use technology in the classroom, and their teaching efficacy beliefs regarding the use of technology in the classroom. College of Education/U.S. Department of Education. ($89,810).

High Museum of Art M.A.P. Evaluation. Gary Henry. A central goal of the Metro Atlanta Partnership for Visual Arts and Learning (MAP) is to equip educators in two metro Atlanta school systems to use the High Museum of Art's innovative reinstallation of its permanent collection as a tool for improving student literacy. Key objectives include a) providing substantive content; b) implementing a field-tested model to develop teacher mastery of new content and approaches; and c) using a classroom-based pedagogical focus to ensure the effective translation of new material. As the external evaluator, ARC conducts surveys, focus groups, interviews, and observations to identify strategies for program improvement, assess the impact of MAP on teacher and student success, and determine if there is a "scale-up" impact on nonparticipating teachers and students at these same schools. This led to a request by the High Museum of Art to develop an evaluation plan for the newly endowed Goizueta Foundation Teacher Institute for Visual Arts and Learning which will draw on MAP evaluation research findings and include an ongoing evaluation research component. Metro Atlanta Partnership for Visual Arts and Learning. MAP. ($78,503)

Performance Outcome Measures Project: Georgia Division of Aging 2001-2002. Charlotte Steeh. For the second year the Georgia Division of Aging Services is participating in a national evaluation coordinated by Westat to learn how in-home services have helped those who receive them and how satisfied clients are with their care. Approximately 1600 telephone interviews were conducted with a sample of clients who have received Personal Support Services during the first six months of 2000 and with an independent sample of 400 caregivers. ($71,364)

Roles and Responsibilities of Educational Organizations and Individuals. Gary Henry. This study involves the examination of the roles and responsibilities of educational organizations and individuals regarding improved student achievement. Case studies of the central offices of two school systems were conducted, as well as a series of focus groups and interviews. ($59,270)

State Educational Policy. Gary Henry and Darlene Opfer. This project engages knowledgeable policy scholars with faculty throughout the South at two conferences focused on innovations in state-level educational policy. This project is in response to the Spencer Foundation's Southern Initiative and proposed to begin a long-term research program on the influences on and impacts of state educational policymaking. ($50,000)

Monitoring and Evaluation of the Leadership in Fighting an Epidemic (LIFE) Initiative. Judith Ottoson. Through an IPA arrangement, the ARC was involved in developing and implementing an evaluation plan to assess the effectiveness of HIV/AIDS/STD/TB programming in the initial fifteen LIFE countries, 14 in Africa and India. The initiative is a joint venture between the CDC and USAID, in collaboration with in-country governmental agencies, NGOs, and multi-national organizations, such as UNAIDS. Countries decide which programs are best suited to meet needs. The M&E evaluation plan needs to be flexible, collaborative, and implementable to answer questions of effectiveness about diverse programs in diverse settings. Global AIDS Program, Centers for Disease Control and Prevention. ($39,475)

Blood Donation Decision. Charlotte Steeh. This national telephone survey of 1750 people who had not donated blood was a pioneering effort sponsored by the Winship Cancer Institute at Emory University and funded by the National Blood Foundation. Examining the attitudes of nondonors will help health officials increase the national blood supply. ($33,968)

Evaluation of Georgia's Title II Program. Gary Henry and Margaret Brackett. In the first year of this project, the evaluation team worked with various stakeholders to develop measures for each of the program objectives, and then they gathered data on these measures and presented a plan for obtaining data on the measures that were not currently being collected. Georgia Board of Regents. ($33,459)

Needs Assessments in Central Georgia for Nursing and Business Technology Degrees. Charlotte Steeh. The Division of Nursing and Health Sciences and the Division of Business and Social Science at Gordon College each solicited a needs assessment for the degree they are planning to offer at some time in the future. Four hundred telephone interviews were completed with 18 - 50 year old adults living in the twenty counties surrounding Gordon College to determine the demand for a business technology degree. For the nursing degree, former graduates of the two-year nursing program were contacted to see if they would be interested in returning to school. In addition, directors of nursing at hospital and clinics in the central Georgia area were interviewed. ($31,194)

Treatment Needs Assessment 2000-2001: Telephone Survey. Charlotte Steeh. This study, conducted for the Georgia Department of Human Resources, provided an inventory of current substance abuse treatment programs in the state of Georgia. After existing programs were identified, an administrator in each program was asked to complete a telephone interview regarding the services the program provides. The information will be the basis for identifying gaps between treatment need and service availability in Georgia. ($30,912)

State Employees Survey. Lloyd Nigro. The state employee survey content and sampling parameters were developed by Professors Nigro and J. Edward Kellough of UGA working with the staff of the State Merit System. The ARC Survey Lab contracted with the Merit System to design and carry out the survey and to build the resulting data file. These data were provided to the Merit System and used to produce several reports on state employees' perceptions of GeorgiaGain and Act 816-Civil Service Reform for the Commissioner of the Merit System. ($25,088)

Child Care Worker Compensation Study. Sally Wallace. This study assists the Urban Institute in the examination of the state of childcare worker compensation in Georgia. ($24,994)

Evaluation Framework for the National Foundation for the Centers of Disease Control. Judith Ottoson. This project involves the development of a framework to guide evaluation efforts of grantees and the decision-making process by foundation management. NFCDC. ($24,965)

Management Academy for Public Health. Judith Ottoson. Consultation is provided to the Centers for Disease Control Foundation on coordinating the internal and external evaluation of the Management Academy for Public Health at the Schools of Public Health and Business at the University of North Carolina. ($21,194)

InfoNet of Public Health Officials (INPHO). Judith Ottoson. A retrospective, summary assessment of the ten year INPHO project was conducted for the Centers for Disease Control Foundation. ($15,950)

Evaluation of the WISE Program. Gary Henry. Interviews and focus groups were conducted on three campuses that receive work-study funds for students from Georgia's WISE Program. Georgia's Student Finance Commission. ($14,220)

Breast Health Project for African American Women (BRIE). Charlotte Steeh. Follow-up interviews were conducted with African American women in southern Georgia to determine the effectiveness of information designed to reduce the risk of breast cancer and promote methods of early detection. The project was funded through the School of Health and Human Sciences at Georgia State University. ($13,044)

Follow-up Survey of Georgia Tech Alumni. Charlotte Steeh. The Office of Assessment at Georgia Tech University asked for Survey Research Laboratory (SRL) help in boosting the response rate of a survey of alumni that had been administered as a mail survey. The SRL completed telephone interviews, using a shorter instrument, to 289 alumni who had not responded at all to the original mail survey. ($8,996)

Tobacco Retailers 2001: Telephone Survey SRL. Charlotte Steeh. The survey research lab called 757 tobacco retailers in Georgia and administered a 3-minute survey to determine whether or not the business still exists at the same location as last year. The information is used to help the staff in the Georgia Department of Human Resources monitor tobacco prevention steps that retailers are required to implement. ($4,881)

Georgia State Poll 2001: 4 RDD Telephone Surveys. Charlotte Steeh. Four times a year the SRL conducts this survey of 800 Georgia residents. Various groups and organizations that purchase question time fund the Poll. Recent participating organizations include the Atlanta Regional Commission, Georgia Department of Transportation, Atlanta Regional Consortium for Higher Education, and the Georgia Office of Consumer Affairs.

Communities in Schools of Atlanta Evaluation Consulting (CISA). Mark Rivera and Craig Gordon. Semi-structured interviews, focus groups, and a document review are being conducted to develop recommendations for improvements to the CISA evaluation plan and reporting practices. The CISA program aims to help reduce absenteeism in schools and to provide caseload assistance to children and families with need. ($5,000)

Advancing Careers Through Education and Training (ACET) Collaborative Project. Laura Henderson. The Applied Research Center has entered into the first year of a multi-year project with the United Way of Metropolitan Atlanta, Inc., and the Georgia Early Learning Initiative (GELI) in a statewide collaborative effort involving multiple public and private entities, including other state universities. The ACET Collaborative Team's (ACT) Project Manager, Advancing Careers Through Education and Training (ACET), will oversee the project (hereafter referred to as the "ACT Project" or "Project") designed to provide quality technical assistance and training to child care workers in an effort to improve the state of child care in Georgia. The ARC will oversee the Evaluation component of the ACT Project, as well as undertake responsibility for the Classroom Observation (CO) component of the Project.

The CO component will focus on evaluating the quality of the learning environment in approximately 175 centers, and 700 classrooms providing services to infants, toddlers, three- and four-year-olds. Two-thirds of these centers will receive ACT technical assistance, while the remaining third will serve as a control cohort.

The Evaluation component of the ACT Project also has responsibility for summarizing the various efforts of the total Project, including a mid and end of year report on the coordinated efforts of the six overriding teams of the ACT Project. These teams are: Needs Assessment, Training,
Accreditation/Technical Assistance, Public Awareness/Marketing, Financial Assistance, and Evaluation.

The full ACT Project is funded at approximately 4.5 million, with the Applied Research Center's Evaluation component funded at $355,000.

Under review

Research on Survey and Statistical Methodology. Charlotte Steeh. National Science Foundation. ($176,296, under review)

Reports

Life after Welfare - Report of the Georgia Welfare Leavers Study. E. Michael Foster and Dana K. Rickman. February 2001. Atlanta, GA: Georgia Department of Human Resources, Division of Family and Children Services.

Pre-K Longitudinal Study: Findings from the 1999-2000 School Year. Gary T. Henry, Craig S. Gordon, Andrew Mashburn and Bentley D. Ponder. April 2001. Atlanta, GA: Office of School Readiness.

Evaluation Report-National School Health Coordinator Leadership Institute. Judith M. Ottoson, Greg Streib, John Thomas, Mark Rivera and Esra Alkan. August 2001. Atlanta, GA: American Cancer Society.

Muscogee County Even Start Family Literacy Program, Using Focus Groups to Provide Feedback on Program Effectiveness. L. W. Henderson. September 2001. Columbus, GA: Muscogee County Board of Education.

Fulton County Zone 4 Needs Assessment. Alvin Glymph, Melodie Horn and Dawn Smith. June 2001. Atlanta, GA: Fulton County District Attorney's Office.