2a. Assessment System
2a.1. Describe the process used to design an assessment system that will collect information on candidate proficiencies outlined in the unit’s conceptual framework, state standards, and professional standards. Who was involved in developing the assessment system and how?
The assessment system was designed by a leadership team consisting of the Division Chair of Education, the university’s Director of Institutional Assessment, the Provost/Executive Vice President of Academic Affairs, and the Coordinator of Assessment and Accreditation for the Education Division. This team drafted the overarching structure of the system to ensure alignment with the GaPSC Standards and the Thomas University Conceptual Framework. The structure was then manifested in admission and transition standards, assessment instruments, aligned learning outcomes, and organizational groups charged with specific activities for collection and analysis of evidence of learning, including aggregated and disaggregated data. All of the faculty of the Division of Education, including instrumental adjunct faculty, participated in the creation of the above through regular meetings, individual work assignments, and working retreats to refine instruments and align outcomes. Partner PK-12 teachers who serve as clinical supervisors, along with partner PK-12 school administrators, met with Education Division faculty to give feedback and participate in norming exercises to ensure interrater reliability.
2a.2. Unit Assessment System: Transition Point Assessments
Table 4 can be found at Prompt 2a.6 below.
2a.3. Describe plans for evaluating the unit’s assessment system. Who will be involved and how?
Structures are in place to ensure systematic evaluation and refinement of the assessment system by unit faculty, professional school partners, and university administrators. Once a year at the end of each spring semester, all program faculty meet with PK-12 stakeholders to review assessment data and the assessment system; a required outcome of these annual assessment retreats is the completion of the Annual Program Report which, among others, includes the following questions: What changes need to be made in your program’s candidate assessments and scoring guides or assessments of program operations in order to provide more meaningful and useful data? Overall, how can the assessment system be changed to provide more meaningful and useful evidence regarding candidate performance and program operations? The professional community stakeholders who participate in the assessment retreats are members of the unit’s advisory committee and include PK-12 teachers, administrators, and faculty in the Divisions of Math and Science and Humanities and Social Science. The Annual Program Reports are reviewed by the Division Chair of Education along with the Coordinator of Assessment and Accreditation, as well as the Director of Institutional Assessment. Under the guidance of the Division Chair of Education, the Faculty Council along with the Assessment Committee are responsible for evaluating the overall effectiveness of the unit assessment system.
2a.4. Describe how the unit plans to ensure that its assessment procedures will be fair, accurate, consistent, and free of bias.
The unit takes effective steps at both the unit and program levels to eliminate sources of bias in performance assessments and works at both levels to establish the fairness, accuracy, and consistency of its assessment procedures. Assessments are fair when they assess what has been taught. To that end, at the beginning of the assessment system development process, program faculty reviewed the curriculum to ensure that candidates were provided opportunities to learn, practice, and demonstrate the expected proficiencies identified in standards. In addition, faculty constructed state professional standards matrices as part of state program review. These matrices are charts showing where in the curriculum the candidate has opportunities to learn and practice what is specified in the standards. Fairness also means that candidates understand what is expected of them on the assessments. Instructions and timing of assessments are clearly stated and shared with candidates, as can be seen in Standard 1 of this document. In addition, candidates are given information on how the rubrics are used to score the assessments and how they count toward completion of their program.
Assessments are accurate when they measure what they purport to measure. To this end, the unit has aligned assessments with the standards and learning proficiencies that they are designed to measure; that is, the same or consistent categories of content appear in the assessments that are in the standards. Assessments are reviewed annually during the Assessment Retreat to determine if they are well-aligned with standards, appropriate for the standard being assessed; if revisions are indicated, they are so noted in the Annual Program Reports.
Closely related to accuracy is the elimination of bias. To ensure that the results of assessments adequately reflect what candidates know and can do, the unit and programs work to remove problems with the assessment instruments that introduce sources of bias and thus adversely influence candidate performance. Assessment instructions and scoring guides are reviewed by program faculty to identify and eliminate problems with assessments such as missing or vague instructions, poorly worded questions, and poorly reproduced copies that make reading difficult.
The unit aims to ensure that assessments are consistent and produce dependable results or results that remain constant on repeated trials. Several strategies are systematically employed to help ensure that unit and program assessments are trustworthy. At each annual faculty retreat key assessment reliability training is conducted. All clinical school faculty are trained during clinical teacher training sessions and school-based orientation meetings each semester with the Coordinator of Placements and Partnerships.
2a.5. Describe the assessments and evaluation measures that will be used to manage and improve unit operations and unit programs.
The unit’s system includes a comprehensive and integrated set of evaluation measures that are used to monitor candidate performance and manage and improve program and unit operations. The unit assessment system provides regular and comprehensive information on applicant qualifications, candidate proficiencies, competencies of graduates, unit operations, and program quality, and uses the data and its analysis for the purposes of improvement of both program and unit operations. The assessment system begins with an alignment to unit and state standards. All unit programs are aligned to the unit’s conceptual framework and to the Georgia Standards Commission Professional Standards. Assessment data are gathered at multiple points and multiple assessments are used including both internal and external data. Data are regularly compiled, summarized, analyzed, and used.
The Coordinator of Assessment and Accreditation ensures that all data are collected and housed in LiveText, the University CAMS system, and the EDUbase. Analysis of aggregated and disaggregated performance data enables program assessment and ongoing program improvement. Faculty committees meet on an ongoing basis to review program data, discuss needs for improvement, and make suggestions to improve unit operations and unit programs. Each May an assessment retreat is held that involves all program faculty together with program stakeholders in review of data, discussion of what the data mean, and consideration for what changes the data may indicate are needed.
The Unit’s assessment system includes a strong component focused on faculty quality that involves both university and unit level assessment. It begins with the hiring process where the unit is committed to recruiting high quality faculty who are exemplary and reflective teachers and scholars and also serve the community. All potential faculty members are screened thoroughly before hiring and oversight is provided by the VP for Academic Affairs. Thomas University systematically completes a comprehensive evaluation process of full-time and adjunct/part-time faculty with the goal of continuous improvement. The formal evaluation process requires faculty to annually submit a Reflective Self Profile demonstrating teaching effectiveness, scholarship/professional development, and service activities. During fall and spring semesters and summer term, student evaluations of all full and part-time faculty members are administered by the university’s Office of Institutional Assessment. Results of student evaluations are analyzed and course summaries are distributed to the faculty and Division Chairs as aggregated and disaggregated data after each semester ends. Peer evaluations are organized by the Division Chair and are conducted annually as a part of the annual review process. The performance of the faculty member is reviewed annually by the Division Chair (or in the case of the Division Chair by the Provost/VPAA) in relation to the submitted Profile, course evaluations, and the supervisor’s observation and review of evidence. As part of the annual evaluation process, the faculty member identifies one to three goals in each area for the following academic year. The Division Chair discusses the faculty member’s annual performance and goals and the Division Chair completes the Thomas University Faculty Evaluation Summary. The faculty member is given the opportunity to review the Chair’s evaluation. If a corrective action plan is required (one or more areas not meeting expectations), it is developed with the faculty member. Both the faculty member and the Division Chair sign the evaluation and submit it to the Provost/VPAA.
2a.6. (Optional Attachments) Tables, figures, and a list of links to key exhibits related to the unit's assessment system may be attached here.
Displayed in Table 4 are current Unit-level evaluation measures at five transition points (all programs use the same instruments).