The UPRM Teacher Preparation Assessment System Procedures 2016 (5.1.1) serves as the guide in which the TPP describes unit endeavors to assess its operational processes. The guide, created in 2009 and revised periodically, is how the UPRM TPP assures a systematic quality engagement in a continuous improvement process that is sustainable and evidence-based with the primary goal of increasing teacher candidate and completer effectiveness in the classroom as reflected in Standards 1 through 4. The guide describes in detail how the TPP collects data (5.1.1 page 8) and uses inquiry to establish program priorities, enhance program components, and test innovations in order to improve learning and development among P-12 students impacted by UPRM TPP prepared teachers.
UPRM institutional policy requires every department, program, and organization to develop a strategic plan for improvement which includes professional accreditation when applicable. The UPRM TPPs began unit and program assessment system in 2006 in preparation for NCATE precondition submission. Data were collected and analyzed prior to that point, just not as part of a systematic assessment process. Since its inception, the assessment system has undergone modifications, refinements, and customizations. The unit and program assessment system was envisioned as and is treated as a dynamic, formative process (5.1.1 pages 6-11). Annually the assessment committee (5.1.1 page 5) reviews the instruments, analyzes collected data and the system itself to improve the program, the data collection and analysis, and the assessment process using available technology to increase efficiency. The assessment system is based on the UPRM TPP Conceptual Framework (2.2.3).
The Conceptual Framework is consistent with UPRM Student Learning Outcomes, Puerto Rico Professional Standards for Teachers, CAEP standards, and Interstate Teacher Assessment and Support Consortium Model Core Teaching Standards and Learning Progressions for Teachers (2013) (2.2.3 pages 22-43). It sets forth ten core teacher proficiencies that represent expectations for all UPRM candidates. These proficiencies are based on current teaching/learning theory, neuroscience learning research, and accepted wisdom of practice to assure that candidates develop the knowledge, skills, and dispositions required to be effective teachers in Puerto Rico and mainland school systems (2.2.3 pages 10-21).
When candidates apply to any teacher preparation program at UPRM, they are required to go through a screening process that allows the program to follow their performance. Candidates that successfully meet screening criteria and are admitted subsequently pass through a series of transition points (5.1.1 pages 14-38) that provides the UPRM TPPs multiple assessments to evaluate, inform, and modify operational effectiveness:
1. Transition Point #1: Entrance to the Teacher Preparation Program – UPRM sets a minimum admission IGS* for each program according to its capacity (faculty and facilities). Students who enter the Agricultural Education, Physical Education Teaching, or Mathematical Education programs must meet the minimum admission IGS. Students admitted to other UPRM programs must approve 6 credits of Foundations of Education courses with a minimum 2.50 GPA and complete an individual or group dispositions interview satisfactorily to be admitted to a curricular sequence for teachers. Note: a curricular sequence for teachers is in effect a second major (30 credits) and each of the 13 available sequences is specialty specific.
2. Transition Point #2: Enrollment in Theory and Methodology Course – overall grade point average of 3.0 or better and 18 credits or more in major with 3.0 GPA in that major; all the Foundations of Education courses (15 credits) approved with a minimum 3.0 GPA; and a score of 80% or higher on Educational Philosophy Project Progression Interview rubric.
3. Transition Point #3: Admission to Teaching Practice – overall grade point average of 3.0 or better and 21 credits or more in major with 3.0 GPA in that major and the corresponding Methodology course approved with a B or better.
4. Transition Point #4: Program Completion- Approve the Teaching Practice course with a B or better; score 80% or higher on the final Classroom Observation Instrument for Teaching Practice evaluation; score 80% or higher on the Electronic Portfolio with Teacher Candidate Work Sample rubric; and attend exit group interview.
After program completion the UPRM TPPs conduct follow-up surveys of its program completers and their employers; and analyzes the annual PCMAS Survey and Teacher Certification Tests (PCMAS – College Board) results. The TPPs will conduct a Mixed Method Program Impact study annually beginning in the fall of 2016.
The principal places where the unit operation data is collected and used (5.1.1 page 38) are in the Agricultural Education Department (AgEd), the TPP office, the Division of Extension and Professional Studies (DECEP), the Office of Institutional Research and Planning (OIIP) and the Information Technologies Center (ITC). The AgEd and TPP Directors collect operational data to plan course offerings, assign teaching duties, and coordinate clinical practice. The directors and their academic advisors use candidate performance data to guide candidates through their teacher preparation program or sequence. The directors also handle faculty evaluations and follow up surveys. The Teacher Education Assessment Board reviews all of the available data relevant to the quality of the teacher preparation program. The Dean of Academic Affairs coordinates the discussion of the Teacher Education Assessment Board’s findings with the pertinent faculties, departments, and personnel including the Arts & Sciences faculty, the TPP Permanent Committee, and the Teacher Education Executive Committee.
Every year, diverse faculty committees and boards review the assessment artifacts and instruments in order to modify the same to respond to emerging needs and actual use in light of the data, the feedback from the field, and ongoing research. The guiding principle behind collecting, aggregating, and analyzing data is to make decisions that lead to better-prepared candidates, more effective programs, and improved unit performance.
The TPPs regularly and systematically uses data to evaluate program efficacy and to initiate changes (5.1.1 pages 39-41) that includes, among others; identify needed course offerings, plan course offerings, counsel students during enrollment, and encourage students to take initial education courses as free or recommended electives. The students’ course grades and specific course assessments are analyzed to monitor and improve their learning.
UPRM TPPs use a variety of assessments and evaluations to gather feedback and suggestions in order to improve operations, programs, courses, and services. Teacher candidates offer feedback each semester on instructor performance through the campus-wide Student Opinion Survey (Cuestionario de Opinión Estudiantil-COE). The COE covers general academia, teaching, service, research, and creative work that facilitates student learning. At URPM COE results are taken into account for teaching assignments, for tenure, and for promotion. The departments use aggregated COE data to plan faculty professional development.
Both the TPP and the AgEd directors receive and review COE results each semester. The directors review and discuss each faculty member results in a personal goal setting conference with that member. This conference receives special attention if the results denote a significant need for improvement. When a director spots areas for improvement repeated over time, he or she arranges targeted professional development activities with the UPRM Professional Development Center (CEP).
Each teacher candidate fills out an end-of-program evaluation after completing the teaching practice course. The form asks candidates about the quality of the program, the advising, the field experiences, and the connection with the conceptual framework. The form includes open-ended questions about the teacher preparation program and requests suggestions for improvements. End-of-program evaluation results are aggregated by program and discussed in faculty meetings to identify improvements needed in courses, in faculty performances, in administrative processes and/or in physical facilities. Candidate evaluations of cooperating teachers are a major factor in subsequent school placements. Alumni surveys, admission and retention data, and the Teacher Certification Test scores (PCMAS) as well as PCMAS survey results are used across the UPRM TPPs to assess program effectiveness and evaluate operations in order to improve both (5.1.1 pages 37-38).
In summary, the UPRM TPPs monitor candidate quality and progress through four transition points with cumulative GPAs, GPAs in education courses, GPAs in major, Educational Philosophy Project Progression Interview rubric, Classroom Observation Instrument for Teaching Practice evaluations, and e-Portfolios with TCWS. The End of Program survey, the annual PCMAS Survey, and the Puerto Rico Teacher Certification Tests provide qualitative and quantitative data that the TPPs analyze to measure the quality of their completers and their programs. The TPPs will conduct a Mixed Method Program Impact study annually beginning in the fall of 2016 to acquire more information about completer impact on P-12 students.
The UPRM TPPs’ quality assurance system embraces national, professional and state standards and expectations, and the legal requirements of the teaching profession as well as reflects its own vision and conceptual framework. UPRM TPPs work as a team to make their programs better and continue to prepare the most effective teachers on the island.
Quality Assessment measures: Relevant, Verifiable, Representative, Cumulative and Actionable Cumulative GPAs are the university norm for measuring student progress. The education course and major GPAs are relevant, cumulative, and as requisites for methodology and teaching practice, clearly actionable. The Classroom Observation Instrument for Teaching Practice evaluation and the e-Portfolio with TCWS were designed and are used to assure that candidates develop and demonstrate all the skills and knowledge needed to teach effectively. The construction and reliability of the Classroom Observation Instrument for Teaching Practice is explained in 5.2.1. The construction and reliability of the e-Portfolio with TCWS is covered on page 11 of 5.1.1. The College Board® develops the PCMAS survey and the Puerto Teacher Certification Exams in response to Puerto Rico Department of Education requirements and uses the standard statistical measures to assure their validity and reliability.
Continuous Improvement: Systematic and Purposeful
Cumulative education course and major GPAs are relevant, cumulative, and as requisites for methodology and teaching practice, clearly actionable. The Classroom Observation Instrument for Teaching Practice evaluation and the e-Portfolio with TCWS were designed and are used to assure that candidates develop and demonstrate all the skills and knowledge needed to teach effectively. The construction and reliability of the Classroom Observation Instrument for Teaching Practice is explained in 5.2.1. The construction and reliability of the e-Portfolio with TCWS is covered on page 11 of 5.1.1. The College Board® develops the PCMAS survey and the Puerto Teacher Certification Exams in response to Puerto Rico Department of Education requirements and uses the standard statistical measures to assure their validity and reliability. The TPP directors use Student Opinion Survey results to monitor and guide professor teaching effectiveness (5.1.1. page 49). The TPP aggregates the End-of-Program Survey responses to identify needed course, faculty performance, administrative process and/or physical facilities improvements. The Mixed Methods Research Plan will collect more information about the quality of completer performance in terms of national InTASC and state professional standards (PR-PST) (4.1.3 page 5). The Plan will use seven data collection instruments (4.1.3 pages 10-11). Current and proposed data collection is both systematic and purposeful.
The PRDE collected teacher performance evaluation data on a sample population last year for the first time which it has yet to share with UPRM. The UPRM Mixed Methods Research Plan was designed to collect information directly about the quality of its completers’ performance in terms of national InTASC and state professional standards (PR-PST) (4.1.3 page 5). The Plan will use seven data collection instruments (4.1.3 pages 10-11). The TPP will use collected data and PRDE teacher performance evaluation data, if made available, to make decisions about current and future courses and programs (standard 4, 5.1.1 page 37, and 4.1.3 page 13).
School partners, Teaching Practice Centers and the PRDE, are involved by the law and policy letter regulating Educational Practice Experience Centers (2.1.1, 2.1.2), monthly meetings to discuss matters related to clinical experience (2.1.3). Candidate placements and cooperating teacher selection is by agreement every semester (2.2.1, 2.2.3). The TPP offers the Cooperating Teacher Certification course (http://uprm.edu/p/eppcaep/cooperating_teacher_course), the syllabus of which was jointly prepared (http://uprm.edu/cms/index.php?a=file&fid=14042). Superintendents, practice center directors, and TPP and school clinical experience supervisors met to evaluate the revised Classroom Observation Rubric and the TPP Conceptual Framework (http://uprm.edu/p/eppcaep/uprm_tpp_conceptual_frame).
Teacher candidates evaluate TPP professors in the COI, clinical practice supervisors (http://uprm.edu/p/eppcaep/ppmes_evaluation), cooperating teachers (http://uprm.edu/p/eppcaep/co-op_teacher_evaluation), and the program (http://uprm.edu/p/eppcaep/end_of_program_survey).
The Mixed Methods Research Plan will use qualitative and quantitative methodologies to address a list of central research questions about the quality of UPRM TPP alumni performance in terms of national InTASC and state professional standards (PR-PST). Data will be collected from the key stakeholders in the UPRM alumni teaching environment: the UPRM alumnus, the alumnus’ P-12 students, and the alumnus’ immediate supervisor.
In summary, the UPRM TPPs’ quality assurance system embraces national, professional and state standards and expectations, and the legal requirements of the teaching profession as well as reflects its own vision and conceptual framework. The evidence shows that UPRM TPPs work with the stakeholders to make programs better and continue to prepare the most effective teachers on the island.