Speed Reading Evaluation

What is the rationale for doing this?

How does it fit with institutional strategy?

What are the main features of the provision?

What are the aims and expected outcomes?

The evaluation exercise aims at outlining the impact of the new speed reading course on doctoral researchers. This feeds into the University Graduate School (UGS) strategy for ongoing evaluation, improvement, and quality assurance. As this was the first year the UGS had run the course it was a good opportunity to embed more a thorough evaluation approach.  We were also able to include a quantative element by comparing the scores in speed reading tests taken on the day, and at the focus groups several months later.

Feedback from participants was sought at four different occasions:

·       Prior to attending the course participants were asked to fill out a short survey, stating why they wanted to attend the speed reading workshop and giving three outcomes they wanted to achieve. These were then reviewed by the trainer and used to shape the workshop.  (level 0 evaluation)

·       The normal post-course form was given out at the end of the workshop. This combined open-text questions with a Likert scale to capture the most and least effective elements of the workshop. These were combined into one spreadsheet to give cumulative data over the course of the year. (level 1, level 2 evaluation)

·       Speed Reading tests were taken throughout the workshop, and the results collated and saved.

·       Three months after attending, participants were sent an electronic survey (hosted by GoogleDocs). This asked participants how their working style had changed, and how they used the skills they had learned during the course (level 3, level 4)

·       A cross-section of participants was invited to attend a focus group after all the courses had been run. This captured the change in people’s working styles as well as suggestions for future improvements. We also gave focus group participants another speed reading test, and compared this result to the test taken at the workshop.  This allowed us to track whether improvements were still being made in reading speed. In all cases, reading speed showed an improvement between the workshop and the focus group, of between 21% and 29%, showing that the skill was being embedded into participants working practices.

We aim to be able to demonstrate the impact of the Speed Reading course at a range of levels on the Kirkpatrick model of evaluation, as well as implementing suggestions to make the course more effective.

Are there any pre-requisites for engagement, e.g. levels of skill, years of experience, essential pre-activities?

How many participate in each 'activity'?

There are no prerequisites other than having attended the speed reading course.

How do you monitor effectiveness?

Who do you seek feedback from?

Do you have benchmarks?

The practice helps us to make our provision more effective and to respond to the needs of doctoral researchers as outlined in the evaluation.  It also enables us to demonstrate the impact of our provision in the short and medium term. This is of especial importance when working on funding applications.

The main challenge was to get enough people to fill out the evaluation forms. We made the pre-course survey a prerequisite of attending the course, and trainers on the course checked that everyone had filled out an evaluation at the end of the day. For the 3 month survey getting responses was more challenging. We decided on offering people who completed the survey a chance to enter a prize draw to win Waterstones vouchers. After the draw was announced the number of completed surveys increased.

We will continue to evaluate the Speed Reading provision along the same lines.