West Texas A&M University

Buff Transit Tracker
SSR Addendum Selected Improvement Plan

SELECTED IMPROVEMENT PLAN

1. Preliminary Response to the Selected Improvement Plan (SIP). Needs to be addressed

Use the Rubric for Evaluating the Capacity and Potential in the SIP to provide feedback on

(FFR, p. 27, paragraph 1)
a. The EPP goes on, however, to acknowledge that the capacity to implement and complete the development of a supplemental technology system for candidates and program-level data collection ‘will present challenges to the EPP in innovation and design’. While adding a statement of commitment to continuous improvement through technology, the EPP’s capacity for implementing and completing the SIP is not entirely clear.

Response:

Since the EPP’s submission of the Self-Study Report (SSR) in early April 2016, the EPP conducted a comprehensive needs assessment to determine the central needs of the department. Stakeholders, faculty, staff members, candidates, and directors of both traditional and alternative certification routes for initial certification were canvased. All stakeholders within the EPP were asked the question, from their perspectives, what did we need a supplemental technology system to accomplish? After gathering internal and external data from our comprehensive needs assessment and reviewing the various perspectives of the people canvased, the EPP generated a prioritized list of items to determine what components a supplemental technology system needed to have to improve and strengthen the EPP’s operational effectiveness and consistency of data-collection.

With the Dean’s recommendation, the Department Head and Director of Accreditation met with WTAMU Instructional Technology (IT) personnel and laid out our prioritized list of needed components for a proprietary supplemental technology system that would enhance gathering data for the existing WTAMU Learning Assessment Reporting System (LARS) and would provide program-specific tools that are relevant to the EPP’s identified needs. In our meeting with IT personnel, the EPP was assured that WTAMU certainly had the capacity and the ability through our programmers to create and develop the technology supplement that the EPP needs and could easily implement.

The EPP next met with WT’s premier programmer, Mr. Carl Welch. After discussing our prioritized needs for the EPP, Mr. Welch also assured us that developing a technology system to accomplish what we wanted would not be difficult and could be implemented easily within the EPP. He would write the code, develop and test the program, and have it ready in three to four weeks once the project was approved. Before the EPP project could be started; however, Mr. Welch informed us that we would have to follow university protocols in submitting a three-part Creative Request to IT. He would complete the technological part and the EPP would need to complete the other two parts of the request.

Working with Mr. Welch, the EPP completed and submitted the Creative Request to the WTAMU IT Department. The Creative Request, subsequent approval process and communications are provided in the Addendum Exhibits as evidence.

The EPP received final approval in August 2016, but was informed that our technology supplement EPP project would fall into the scheduled prioritized workflow of the University. With the selection and appointment by the A&M System of a new University President for WTAMU on September 1, 2016, many items have been placed on hold during the transition until the new President’s IT priority items are completed.

The EPP anticipates receiving the final EPP technology supplement from IT in Fall 2016. When this occurs, the EPP will announce a new projected implementation timeline of Spring 2017. Upon receipt of the completed project for implementation, all EPP stakeholders will be trained in the use of the technology supplement system prior to our full implementation.

Once implemented, the EPP will test the technology supplement system’s components viability and reliability in 2017. The EPP will collect data from the Office of Teacher Preparation and Advising, candidates, and stakeholders in the operation of the supplemental technology system. Based upon the evidence, adjustments, refinements, and/or additional components may be added to the supplemental technology system to continually improve the technology system.

Through the ongoing process of conducting a comprehensive needs assessment, collaboration with WTAMU IT staff in the development of an innovative design for a proprietary technology supplement for the EPP, the submission and approval of our Creative Request, the EPP demonstrates that the EPP has the capacity to implement and complete the development of a supplemental technology system for our candidates and program-level data collection.

The quality of the innovative design and unquestionable probability of EPP implementation of the supplemental technology system was reinforced by the University and IT Departmental approval of the project . Working together with our stakeholders, the EPP will undeniably meet the challenges of the innovative design of our supplemental technology system and demonstrate our capacity to implement and complete the Selected Improvement Plan (SIP) of the EPP.

[See Addendum Exhibit (AE87) The EPP’s Creative Request: Supplemental Technology System Project ].

(FFR, p. 27, paragraph 2)
b. The EPP indicates on page 31 of the SSR that the focal area for the SIP is Standard 5, Component 5.1 but in the Description of Selected Area for Improvement the EPP identifies that it ‘has selected CAEP Standard 5.3 and the use of technology for data collection as our foci’. The stated goals of improving data collection and ease of access to candidate and program data appear to be aligned with Component 5.3. While the EPP articulates a basic plan for developing a technology supplement, there is no direct and specific evidence stated has having [sic] the potential for positive impact on the EPP and its candidates.

Response and Correction:

The EPP has selected CAEP Standard 5.3 and the use of technology for data collection as our focus for Selected Improvement of the EPP. On page 31 of the SSR, Component 5.1 was an oversight. We apologize for any lack of clarity in the SSR.

As stated in the SSR and reiterated in the FFR, the EPP is totally committed to developing and initiating a technology supplement system to meet our goals for continuous improvement and increase operational effectiveness. Our primary goal of improving data collection through the use of the supplemental technology system designed to meet specific criteria as prioritized in the EPP’s comprehensive needs assessment will serve to accomplish our goal. Meeting this goal will allow the EPP to improve and provide efficiency of access to program and candidate-level data for our candidates, our partners, and the EPP. The supplemental technology system will allow the EPP to input, collect, and assess candidate and program data more easily and more consistently for improved operational effectiveness.

As evidence of our commitment and capacity to initiate the plan, the EPP has designated key personnel and WTAMU resources to develop, implement, and assess the technology supplement system. The EPP has worked closely with the Instructional Technology (IT) Department of WTAMU in the creation and development of the technology supplement system. Since the plan for the system has received WTAMU approval, the EPP is now awaiting final development of the technology supplement system that we anticipate to receive from IT later this Fall 2016.

Data entry personnel will include the Office Manager of the Office of Teacher Preparation and Advising and two advisors. The EPP is currently seeking to fill a third advisor’s position with the reorganization of the former two offices of Teacher Preparation and Advising and the Panhandle Alternative Certification for Educators (PACE). The Office Manager and three advisors will be trained to input candidate-level data including contact and demographic information, routes to initial certification, specialty licensure/ certification areas, GPA data, and candidate progression through the EPP. Other data for the EPP that our key personnel will input and upload is outlined on the following pages.

Faculty will be trained to input candidate-level data including Program Educational Outcomes and Ethical and Professional Dispositions of Candidates data. Any dispositional infractions that occur will also be reported in the technology supplement system. Infractions will generate emails to schedule a meeting with the candidate, Department Head, Director of the Office of Teacher Preparation and Advising, faculty members reporting the infraction, and other EPP personnel. A process for candidate dispositional infractions and improvement of those infractions will be included within the technology supplement system.

Additional key personnel of the EPP will complete oversight of the implementation of the technology supplement system. These key personnel will include the Department Head, the Director of the Office of Teacher Preparation and Advising, the Assistant Director of Teacher Preparation and Advising, and the Director of Accreditation.

Phase I of the implementation of the technology supplement system will include training of key personnel, the data-entry of specified information, and close monitoring of the validity of data and reliability of the system. The timeline for implementation of Phase I is anticipated to be Spring 2017. Online evaluations in Survey Monkey will be developed for our key personnel for each semester of 2017. Candidates will receive training in the use of the system and for access of their EPP information. During the first phase, our partners will submit to the EPP their PDAS/T-TESS evaluations of clinical teachers that will be entered by key personnel. Once the EPP has completed the testing of the system in 2017, we will then implement Phase II in Fall 2018.

As evidence of the EPP’s capacity to implement and complete the plan, Phase II includes the comprehensive training of our school-based partners and EPP-based evaluators (such as cooperating teachers and University Field Supervisors). Stakeholders will be trained each semester in the use of the EPP’s technology supplement system of how to upload candidate- level data including Field Observation Evaluations, Weekly Progress Reports, and Student/Clinical Teacher PDAS/T-TESS evaluations for Clinical Teaching.

Based upon evaluation data collected from Phase I, the EPP will implement refinements, changes, and improvements in the system, continue training EPP-based faculty and personnel, and monitor the successful implementation of the technology supplement system. Evaluations will continue to be collected that are completed by key personnel and stakeholders at the end of each semester. The EPP will review all evaluation data at the end of each academic year to determine any changes or improvements that are needed in the system. Those changes will be implemented into the system to be active for the following school year.

Resources used by the EPP for the successful implementation of the technology supplement system will include WTAMU IT Department staff, EPP-based key personnel, school-based personnel, and our candidates.

The EPP’s Table 1 Timetable for the implementation of the technology supplement system (TSS) and Table 2 Selected Improvement Plan Rubric are provided on the following pages:

Table 1. EPP’s Timetable for Technology Supplement System Implementation

Rubric for Evaluating Capacity and Potential in a Selected Improvement Plan

Spring 2016

Fall 2016

Spring 2017

Summer I, II
2017

Fall 2018

EPP Comprehensive Needs Assessment

Phase I: Receipt of final TSS from IT Dept.

Phase I: Continue training of key personnel

Phase I: Continue monitoring of the TSS

Phase II: Implement Phase II of the TSS

Meet with WTAMU IT
Department

Train EPP Key Personnel

Test innovations and effects of selection criteria on subsequent progress and completion of TSS

Train personnel as needed

Train all stakeholders

Meet with WTAMU
programmer

Implement EPP’s TSS:
Upload candidate and program level data

Survey Monkey survey for evaluation of TSS

Evaluation Surveys to evaluate success of the TSS implementation

Monitor implementation of Phase II

Collaborate, create, and develop Creative Request for the TSS

Survey Monkey survey to evaluate TSS

Use results of evaluation data to improve program elements and processes

Collect, analyze, and use results of evaluation data to improve program elements and processes

Monitor validity of data and reliability of TSS and track progress over time

Submit Creative Request to WTAMU

Monitor data- entry, ease of use, etc.

Work with IT EPP key personnel to improve elements and processes of the TSS

Work with IT and EPP key personnel to improve elements and processes of the TSS

Use results of evaluation data to continually improve program elements and processes

Received WTAMU
Approval August 2016

Collect baseline data and assess evaluation data against the EPP’s
performance goals and standards at the end of each semester

Make any changes and improvements as needed to the TSS

Make any recommended changes and improvements to the TSS as indicated by evaluation data

Determine success of implementation of the TSS by overall evaluation ratings

 

Direct and specific evidence of the potential for positive impact on the EPP and our candidates will be gained with the successful implementation of the supplemental technology system as based on our evaluation system data. Through the component tools that the system will provide, an ease of both data input and data access to the EPP database for candidate and program data includes the following:

For the EPP:

  • Candidate contact information;

  • Demographic information such as gender, race, ethnicity, etc.;

  • Route and Program data: Elementary, 4-8, Secondary, Special Education; Traditional or Alternative Certification; Specialty Licensure/Certification Areas; and Degree Plans;

  • GPA data: including GPAs at admission (2.75 or higher) and Grades of at least a “C” in all education courses;

  • TEA Policies and Requirements;

  • ASEP and LBB data: state certification exams data (TExES Content and TExES PPR Exams);

  • Online Candidate and Evaluator Trainings for T-TESS;

  • Program Educational Outcomes and Candidate Development: Ethical and Professional Dispositions of pre-, mid-, and post-assessments; Ethical and Professional Disposition Infractions; Notification processes to schedule meetings to address dispositional infractions by candidates;

  • Placement of Candidates: Methods 40 hours of Field Observations and 13 weeks of Clinical Teaching including the August Experience;

  • Candidate and Program Assessments;

  • Cooperating Teachers Weekly Progress Reports of Clinical Teachers;

  • Cooperating Teachers Evaluations of Clinical Teachers;

  • University Field Supervisors Evaluations of Clinical Teachers (PDAS/T-TESS data);

  • Student/Clinical Teacher Evaluations;

  • Certification data;

  • Candidate and School Partner Survey Data;

  • TEA data including TEA Completer Exit data and Principal Survey data;

  • Track candidates throughout the progression of the program;

  • Track completers after graduation.

For Our Candidates:

  • Candidate Access to Personal Information;

  • Degree Plan: Specialty Licensure/Certification Information;

  • EPP Handbook;

  • EPP Communications, Meetings Information, etc.;

  • Seminars and Online Trainings by Candidate Requests;

  • PEOs and Ethical and Professional Dispositions Development;

  • Candidate Coursework: Reflection Papers (Phases I, II, and III);

  • Candidate Placement: Methods Field Observations and Clinical Teaching;

  • Weekly Progress Reports for Clinical Teaching;

  • Survey Monkey;

  • Intervention and/or Remediation Services;

  • Assessments and/or State Certification Exams;

  • Certification and Graduation;

  • First Teaching Job Information;

  • Beginning Teacher Impact on Student Learning Information.

In the SSR Addendum, Tables 1 and 2, and the Addendum Exhibits, the EPP demonstrates our plan for developing a technology supplement that will potentially have direct and positive impact on the EPP and our candidates. By having ease of input and efficacious data retrieval with this new technology supplement system, the EPP will improve our operational effectiveness with efficiency and consistency with valid and reliable data and will continue to improve our service to our candidates.

[See Addendum Exhibit (AE87) The EPP’s Creative Request: Supplemental Technology System Project].

(FFR, p. 27, paragraph 3)
c. The EPP narrowly defines its proposed use of data and evidence by merely stating that the added technology supplement will ‘improve data collection and retrieval of candidate and program data’ and that the success of the SIP will be measured by ‘the successful implementation of the technology supplement’. The EPP does indicate that the goal is to improve and provide efficiency of access to candidate and program-level data by the EPP, its clients, and partners, which is a needed improvement to assess program elements and operations.

Response and Clarification:

In the SSR Addendum and Addendum Exhibits, the EPP has articulated our goal to improve and provide efficiency of access to candidate and program-level data by our candidates, our partners, and the EPP in order to assess our program elements, operations, and effectiveness.

For clarification, the EPP believes the success of the Selected Improvement Plan (SIP) will be measured by the successful implementation of the technology supplement system and that our candidates, our partners, and the EPP will measure that “success” through an ongoing evaluation system in Phase I and Phase II .

With the technology supplement system, the EPP will ensure successful implementation through an ongoing evaluation system for Phase I and Phase II used by our candidates and partners. With Survey Monkey and other online survey instruments, the EPP will gather baseline data from our stakeholders at the end of the first semester to determine the effectiveness and viability of our technology supplement system. For the timelines of the evaluation system of the technology supplement system , the EPP will administer and collect data from the evaluation surveys at the end of each semester of our candidates and key personnel in Phase I, our partners, candidates, and EPP-based personnel in Phase II. Candidates and school partners will evaluate the effectiveness of the technology system through their use and access to the system. We will test the technology supplement system after implementation of Phase I through each semester of 2017 and make any necessary changes and/or improvements to the technology system. Phase II will be implemented in Fall 2018.

For the final overall data-based evaluation of the EPP’s implementation of the technology supplement system and system elements , the EPP will examine the evaluations of our users of the system including data-entry staff, the Director of the Office of Teacher Preparation and Advising, advisors, administrators, and faculty members. Additionally, the EPP will collect the cumulative evaluative data of all stakeholders each semester to see how effective the technology supplement system and system elements have worked and to make any readjustments, changes, and improvements that are necessary to improve the system. The EPP will continue to work with WTAMU IT staff in the ongoing improvements of the technology supplement system.

The EPP will use the collected data and evidence each semester to determine the efficacy of the technology system and to evaluate the program elements and operational effectiveness of the EPP. The EPP will examine the established processes of working with our candidates, partners, and EPP-based personnel in trainings; testing; candidate progress and development; PEOs and dispositions; placements; and the evaluations of field observations and clinical teaching experiences of our candidates. The EPP will analyze these data to determine the overall effectiveness and successful implementation of the technology supplement system. A Likert scale of 1-5 will be used for the Survey Monkey evaluations of the technology supplement system. The criteria will echo the same criteria that are used on the state T-TESS appraisal document that are used to evaluate Texas educators.

The rating or evaluation scale for the EPP’s technology supplement system will include:

  • Ratings of Well Above Expectations, Above Expectations, At Expectations; Below Expectations; and Well Below Expectations:

  • Distinguished (5) Well Above Expectations;

  • Accomplished (4) Above Expectations;

  • Proficient (3) At Expectations;

  • Developing (2) Below Expectations; and

  • Improvement Needed (1) Well Below Expectations.

An average scale score or Mean of 3 or higher will indicate a successful implementation of our system.

(FFR, p. 27, paragraph 4)
d. While more immediate and efficacious access to data by the EPP and its partners points to needed improvement in the existing LARS Assurance System, the SIP is limited in its defined scope and level of specificity regarding the ways in which achievement of its goal of implementing the technology supplement and training users will demonstrate a higher level of excellence beyond what is required in the standards.

Response:

The EPP has defined a much broader scope and level of specificity of the ways that the achievement of our goal to implement the technology supplement system; training users such as our candidates (clients), partners, and EPP-based key personnel; and providing access to program and candidate-level data demonstrates a higher level of excellence that lies beyond what is required in the standards for the EPP.

CAEP Standard 5, Component 5.3 Continuous Improvement of the EPP states that for the EPP: The provider regularly and systematically assesses performance against its goals and relevant standards, tracks results over time, tests innovations and the effects of selection criteria on subsequent progress and completion, and uses results to improve program elements and processes.

The WTAMU Learning Assessment Reporting System (LARS) enables the EPP to meet CAEP Standard 5, Component 5.3 through the annual and systematic assessment of the performance of each program against EPP’s goals and aligned standards. Each program measures our mission in Program Educational Outcomes (PEOs), tests any innovations, tracks results over time, and examines the effects of program criteria on candidate progress and completion.

Each program within the EPP uses the results of these annual assessments to “close the loop” and improve program elements and processes.

Following and completing the required assessment protocols and annual assessment cycle established by West Texas A&M University, the EPP has met CAEP Standard 5, Component 5.3. The Vice President of the Office for Learning Assessment reports these completed program assessments to the Texas A&M University (TAMU) Board of Regents in College Station, Texas each year for final approval. The EPP has met university requirements for program assessment of all our programs.

However, in achieving the goal of implementing the technology supplement system, training users, and using data from the ongoing evaluation system to improve our program elements and processes, the EPP demonstrates a higher level of excellence beyond what is required in the standards. The EPP believes that the creation, design, development, implementation, and ongoing evaluation of a technology supplement system to supplement what is required by the University in LARS goes beyond what is required in the standards. Using the University assessment system annually successfully meets the standards, both for CAEP and for WTAMU. The EPP’s implementation of our own system designed with EPP-specific tools to supplement the existing LARS system will consistently improve our program elements and processes over time goes well beyond these standards.

The improved efficacy of service to our candidates, our partners, and the EPP will be provided with this implementation of the technology supplement system. The technology supplement system will be continually assessed each semester through data-based evidence collected from the evaluation system to determine the success of the implementation.

As previously discussed in the SSR Addendum on pages 213-214, the overall data-based evaluation of the EPP will be demonstrated by an average score or Mean of 3=“Proficient” or higher on the evaluation rating scale by our stakeholders. The EPP will use evaluation data results to improve all program elements and processes. For example, analyses of data will indicate the effectiveness of our placement of candidates in diverse settings and working with diverse P-12 student populations for both field observations and student/clinical teaching experiences. Input data from our school-based cooperating or mentor teachers, University Field Supervisors, and the candidates themselves will help the EPP to determine what elements are working best and what improvements are needed to enable the EPP and our processes to work even better and more efficiently.

The implementation of the technology supplement system will enable the EPP to not only meet standards, but also to go beyond in the demonstration of achieving an even higher level of excellence than what the standards require.

Thank you.

[See Addendum Exhibit (AE73) Learning Assessment Reporting System (LARS)].

[See Addendum Exhibit (AE87) The EPP’s Creative Request: Supplemental Technology System Project ].

(FFR, pages 28-29)

Rubric for Evaluating Capacity and Potential in a Selected Improvement Plan

Table 2. Selected Improvement Plan (SIP) Evaluation Rubric.

Rubric for Evaluating Capacity and Potential in a Selected Improvement Plan

Selected Improvement Plan of the EPP

Please see the EPP’s response in the SSR Addendum on pages 206-216.

Capacity to initiate the plan

For the EPP’s commitment, key personnel, and resources, please see the EPP’s response in the SSR Addendum on pages 206-207.

Capacity to implement and complete the plan

Please see the EPP’s response in the SSR Addendum on pages 207-209.

The potential of goals to have a positive impact on the EPP and its candidates

Please see the EPP’s response in the SSR Addendum on pages 210-212.

Identified baseline and yearly objectives that will lead to a successful CIP

Please see the EPP’s response in the SSR Addendum on pages 210-215.

Identified final data- based evidence that results in a successful CIP

Please see the EPP’s response in the SSR Addendum on pages 210-215.

Potential to demonstrate a higher level of excellence beyond what is required in the standards

Please see the EPP’s response in the SSR Addendum on pages 213-215.

Overall Evaluation of the SIP

Please see the EPP’s response in the SSR Addendum on pages 213-216. Thank you.