Standard 4
Standard 4


Washburn has put an emphasis on having a positive impact on student learning for many years. We have collected data from several different sources in order to demonstrate that our completers have a positive impact on student learning and that both completers and the schools are satisfied with the training provided.

4.1

The state of Kansas does not provide data to institutions regarding alumni having a positive impact on student learning. However, surveys have been conducted that focus on all IHE’s and Washburn graduates have been a part of this sample. These surveys have been organized and conducted the office of Educational Innovation and Evaluation at Kansas State University (KSU). Our EPP does not have a specific breakdown of how Washburn graduates compared to other IHE’s, but we do have the summary scores from the state that include Washburn alumni (see 2013 – 2015 KS Statewide Employer Survey Summary file)

In the spring of 2017 faculty in the department developed a pilot research study to specifically determine the impact on learning of our graduates and to follow up on these program completers. It was decided to first target alumni in Topeka USD 501 schools as we have many of our alumni teaching in Topeka. Approval for the research project was obtained from the WU IRB and USD 501 (see 4.1 Impact files).

The purpose of this study is 5-fold. Working with our P-12 professional partners in USD 501 Topeka Public Schools; 1). We will collect P-12 student impact data specific to the students they have taught (We will collect state, district, school and/or classroom data relevant to the P-12 student.); 2). Analyze the district personnel evaluations of our alumni to determine if they are teaching content reflective of their license; 3). Analyze the district personnel evaluations of our alumni to determine if their teaching is effective as judged by their administrators/employers; 4). Conduct follow-up interviews with the administrators who evaluate our alumni; and 5).
Interview the WU Alumni (program completers) to determine if they effectively apply the professional knowledge, skills and dispositions that the preparation experiences were designed to achieve; their perception of their preparation as relevant to the responsibilities they confront on their job; and that their preparation was effective. Data will be summarized and may be based on academic scores as well as non-academic factors gathered from the surveys and interviews. As of this writing data collection is progressing but is not yet available for analysis.


As a unit we regularly collect data regarding impact on learning and these data include:
1. Gain score templates completed by student teachers based on the units they teach during student teaching
2. KPTP focus area E
3. Non-academic P12 student impact data based on responses by students in the schools
4. Student Teaching Summary Evaluations completed by mentor teachers and University Supervisor’s (indicators 1.4, 3.5, 3.2, and 4.3)
5. Follow up study with building principals
6. Completer self-assessment of their impact


4.2

Items within the Content Tests, the KPTP, the Student Teaching Summary Evaluation, the P12 Survey on the Impact on Student Learning, and the Professional Conduct and Dispositions evaluation all focus on demonstrating an understanding of the skills, dispositions, and contextual factors that can influence learning, planning and conducting instruction (including the use of technology), assessment, and reflection and align well to the 10 InTasc standards. In addition, assessments within the professional development methods courses (i.e. such as the lesson plan templates used in methods blocks) are also consistent with standard practices. We solicit input from mentor teachers, university supervisors, building principals, P12 students and our candidates. Data on content tests, the KPTP, Student Teaching Summary Evaluations, and Professional Conduct are collected on all candidates and these data are provided in the evidence. The 4.2 Disposition Evaluation Level shows disposition scores from the Student Teaching Summary Evaluation for four consecutive semesters by the grade level taught.

Professional knowledge and skills are assessed through Praxis II content tests (standard 1), the KPTP (teacher work sample), our Student Teaching Summary Evaluations (completed by mentor teachers and university supervisors) and our Professional Conduct and Dispositions evaluation. Links to these evaluations can be found in the Washburn Initials Terms and Links file. We do have external benchmark data for KPTP scores and Praxis II content test scores (see standard 5 evidence).

We have used multiple measures to document that candidates have a positive impact on student learning and that candidates effectively apply the professional knowledge, skills, and dispositions expected of them. These measures include:

KPTP

This teacher work sample is completed during student teaching and submitted to the state for scoring. The KPTP is based off at least a two week unit plan and contains contextual information, objectives, lesson plans, assessment and reflection. Scores obtained from students are presented as whole class, focus groups and focus students. The data obtained in the KPTP is then submitted using the unit’s Gain Score Template which provides a consistent way for candidates to summarize the pretest-posttest scores for their students.

Academic Gain Score Template Data from Completing Student Teachers for KPTP’s:
Fall 2013: Average gain score 41%
Spring 2014: average gain score = 50.87%
Fall 2014: Average gain score = 41%
Spring 2015: Average gain score = 34%
Fall 2015 (gain score formula and template revised): = 37%
Spring 2016: Average gain score = 51%
Fall 2016: Average gain score = 44%
Spring 2017: Average gain score = 37.9%

As a part of the KPTP candidates are required to address the whole class and focus students. Data we have reviewed on the KPTP’s show that focus students are primarily those students on IEP’s, at risk, ESL, or have some other issues.

KPTP Focus student data (addressed impact on P12 learning for all students)
Screen Shot 2017-06-21 at 11.26.11 AM

Non-Academic P12 Impact Data

Completing student teachers are asked to survey their P12 students using a non-academic measure of impact on learning. This 12-item survey is based off of Ferguson’s (2011) work on the 7 C’s. We first piloted this survey in the Spring of 2015. Since that time we have had responses from more than 2000 students in the schools. Trend data from these surveys can be found in the 4.1 Impact on Student Learning Nonacademic file and the number of responses by candidates and students is listed below:

In the Fall 2015 the form was revised slightly and used with the Elementary/middle grades Student Teachers (n= 15 of 16). We had responses from 308 children in grades K-7

The form was revised again slightly and used in the Spring 2016 and used with Elementary Student Teachers (n=33). We had 661 responses from children in grades K-7.

In the Fall 2016 all of the Elementary Student teachers (n=16 of 16) completed this survey. We had 526 responses from children in grades K-7 with some responses from both placements

In the Spring 2017 semester we had 11 of 12 Elementary student teachers complete the survey. We had responses from 202 children in grades K-7.

We have done some pilot work to extend this survey to secondary-level candidates and plan to have students of those candidates also complete a similar survey.


Student Teaching Summary Evaluations
This evaluation is completed by university supervisors and mentor teachers during student teaching for all teacher candidates. A copy of this evaluation can be found in the Washburn Initials, Terms and Links file. Indicator 1.4 asks “The teacher candidate had a positive impact on student learning”.  The results over a three year period are very consistent especially considering the number of university supervisor and mentor teachers who would have completed these ratings. The
average score for 7 semesters is 3.67 with a range of 3.59 – 3.72. Data on the reliability of this EPP assessment is provided under standard 5.

Spring 2014: 3.70/4.0
Fall 2014: 3.70/4.0
Spring 2015: 3.72/40.
Fall 2015: 3.64/4.0
Spring 2016: 3.60/4.0
Fall 2016: 3.69/4.0
Spring 2017: 3.59/4.

Completer Self-Assessment

The completer self-assessment survey is given at the end of student teaching and is given to all program completers (see the Washburn Initials Terms and Links file). The one page survey addresses seven areas including content knowledge, assessment, diversity, classroom management and instruction. Completers are asked to submit the survey to indicate how well prepared they feel they are. One item addresses training to have a positive impact on student learning.

For 2014- 2015 the item regarding training to have a positive impact on student learning was: 3.61 on the 4-point scale. For 2015-2016 the item regarding training to have a positive impact on student learning was 3.63/4.0 scale, and for 2016-2017 the score was 3.70/4.0. The response rate for both sets of data was greater than 80% as these data are collected at a final required meeting of all student teachers.


Follow-up Survey of Building Principals

In 2014 we added an item to our follow-up survey of program completers that is completed by building principals. Based on data we obtain on the positions our completers secure, we follow-up with the building principals in those schools to find out how well they are performing. In the spring of 2014 building principals gave our completers a score of 3.59 on the 4-point scale. In 2016 the principals gave our completers a score of 3.41 on the 4-point scale. The full set of data on this survey can be found in the evidence, 4.3 Follow-Up Results file.

Validity and Consistency

Assessments used within the unit have content validity as they are consistent with InTasc and state standards and the standard practices of the profession. More information on the validity and reliability of EPP assessments can be found in standards 3 and 5. In the fall of 2016 we conducted an analysis of all our non-proprietary assessments and surveys to help ensure that they are sound, valid, and appropriate and work to address issues is under way.
Internal Consistency

Scores from 3 different sources showed consistent results (all on a 4 point scale):

Mentor teachers and university supervisors for fall 2016 = 3.69
Candidate Self-Assessment for fall 2016 = 3.70
Follow-up Building Principals 2016 = 3.41



4.3

We collect data on the employment of our completers every semester and do our best to keep these data up-to-date. This has proven to be a challenging task as completers can move out of the area or out of state, change names, or choose to not pursue a teaching position. Over several years we have determined that slightly more than 75% of our alumni are in teaching positions although some years and semesters have a higher percentage.

We have not made systematic efforts to collect data on employment milestones or promotions. We do, however, keep track of honors or awards our candidates or completers earn and these are stored in our department database, EDMS.

We have conducted a follow-up survey of building principals for many years as a way to gain more information on the quality of our alumni. The results of this survey can be found in the 4.3 Follow-Up Results file. We send building principals a survey based on our jobs data for completers. The items in the survey have changed some over the years but are consistent with our unit SLO’s and InTasc standards. We added a specific item regarding having a positive impact on learning in 2014. These data indicate that employers perceive completers’ preparation was sufficient for their job responsibilities.

4.4

We have conducted a self-assessment survey at the conclusion of student teaching for several years and these data provide evidence that the program completers feel their training was adequate and appropriate. The items asked have been revised slightly over the years, but are based on our Student Learning Outcomes, our conceptual framework, and InTasc standards. We believe we have established content validity as the survey items adequately and representatively sample the content area to be measured. The survey includes questions about preparation regarding content knowledge, the ability to plan and implement instruction, diversity, assessment, classroom management, impact on student learning and overall training. We survey program completers at the end of student teaching and get responses from more than 90% of them as this survey is conducted as a part of their final seminar meeting. The average score for elementary candidates for six semesters for the item on preparation on impact on student learning was 3.65 on the 4-point scale with an overall difference in scores of only .15. The average score for P12 and secondary candidates for six semesters was 3.59 with an overall difference in scores of only .35. Given the consistency of these scores we feel these data are reliable. The 4.4 Self-Assessment Results file shows the trend data for this EPP survey. Item 7 on this survey asks completers directly if they feel that the Education program at Washburn prepared them for a successful career in teaching. The average for six semesters (4 point scale) was 3.58 for elementary completers and 3.37 for P12 and Secondary completers which are scores well within the target range.

The impact study currently underway and discussed in 4.1 should also provide us with information regarding completer satisfaction with their preparation.



4 Evidence Summary
4.1 & 4.3 IRB Application Spring 2017
4.1 & 4.3 WU Impact Study
4.1 2013 - 2015 Employer Mean and SD - All Institutions FINAL
4.1 Academic Gain Scores Reported by Student Teachers
4.1 Impact on Student Learning NonAcademic
4.2 and 3.3 Dispositions Evaluation.level
4.2 and 5.4 External Benchmarks kptp
4.2 Student Teaching Summary Evaluation Items and Validity
4.3 Follow_up_Results of School Principals
4.4 Self Assessment Results



[ WU Home ] [ Directory ] [ A-Z Index ] [ Sitemap ] [ Contact WU ] [ Statements & Disclosures ] [ Accessibility ] [ Search ]
© 2015 Washburn University, 1700 SW College Ave, Topeka, Kansas 66621 785.670.1010
Contact webmaster@washburn.edu with questions or comments.