Weeks 5 & 6 Evaluation Methods

For these two weeks we were asked to address the following:
Outline some information about the type of evaluation you believe will best suit your project with your reasoning
Summarise the article(s) you have found (300 words) - full reference, evaluation design (paradigm, model, questions, methods), findings and how it aligns with the evaluation project you would like to conduct.
==============================================================

I found the article by Professor Tom Reeves The Six Facets of Instructional Product Evaluation helped me considerably in understanding more about the methods of evaluation that there are available.

I have been using Blackboard to a considerable extent in recent years using blended delivery. During this time I have conducted only informal evaluations including written student surveys, peer evaluation of site content by other staff members etc. This paper gives me a perfect platform to now perform evaluation in a more formal setting when I begin my new eLearning task of developing everything (some old, but most new) in Moodle instead.

I need to gather feedback from students and perhaps other staff members during the development and implementation process of developing the new Tourism and Travel qualification on Moodle. As I said before Blackboard is being phased out and I am meant to have Moodle up and running by the beginning of Feb 2010.

Having thought about this for a while, I think that I should base my evaluation on Formative evaluation. I think I should carry out a questionnaire for example once I have completed the first module in Moodle and get students/staff to assess it - perhaps in terms of the Key Usability Principles identified by Jakob Neilsen which I think are solid guidelines to use:

• Structure - is the online module well organised?
• Simplicity - is it user friendly?
• Visibility - is everything readily/easily available for the student on the site to complete the module?
• Feedback - are students able to contact the lecturer easily and in a timely manner?
• Tolerance - will the students be able to cancel, go back or exit without problems?
• Reuse - will the students be able to go back and find information already examined/studied easily without too much trouble?

I think this ties in well with The Project of Evaluation Toolkit which mentions some 'High value' questions that are useful to ask because they will add real value to any evaulation, for example those which will most likely highlight areas that can readily be improved.

I think that if I was able to do an evaulation on one module (perhaps before it officially goes online?) I should be able to identify problems so that I could make improvements or adjustments as I continue to implement the full extent of transferring (or let's face it completely reworking) everything I currently have on BlackBoard to Moodle. I would imagine that I would need to do this after every module is completed.

I think that the evaluation would be best done through an online survey of users and then back that up with a focus group interview and perhaps a different staff questionnaire would be required so I am not relying on just one type of evaluation as recommended by Reeves and Hedberg (2003). This would mean I would be applying the Multiple Methods Evaluation Model so that the data collected can be triangulated.


References:
Reeves, Thomas, C. and John G. Hedberg (2003), Interactive Learning Systems Evaluation, Englewood Cliffs: Educational Technology Publications
Project of Evaluation Toolkit. Retrieved May 20th 2009, University of Tasmania http://www.utas.edu.au/pet/index.html