Evidence generated by complex and innovative processes such as OER release is often itself complex, context-specific and difficult to generalise. These kinds of initiative require significant organisational change and may include external partners and stakeholders with very different cultures and practices. Evaluation, in particular is challenging and ranges from evaluating specific OER for fitness of purpose, changes in staff attitudes, impact on learning and teaching and longer term impact on institutional practices and the wider community.

This page offers some resources to support evaluation activities and also points to what people have already learned through evaluation into this area.

UKOER Evaluation

UKOER Programme evaluation was supported by the UKOER Evaluation and Synthesis Team. The team, led by Glasgow Caledonian Academy, provided a range of support activities to support individual project evaluation across the three years and developed a Framework to support project evaluation and programme synthesis. Evaluation & synthesis was an iterative, two-way process and included liaison not only with project teams and their evaluators but also with the programme management team, the various support teams, and concurrent OER initiatives.

Engaging projects with the framework was challenging due to short (one year long) project time-frames. The team developed an Evaluation Toolkit which provided various visual ways to connect with the framework and offered routes through the programme themes. The toolkit also provided further resources to support evaluation of both projects and OER development and release.

The Team produced a yearly synthesis report highlighting key lessons and emerging themes:

• McGill, L., Beetham, H., Falconer, I. and Littlejohn, A. JISC/HE Academy OER Programme: Pilot Phase Synthesis and Evaluation Report. JISC, 2010 https://oersynth.pbworks.com/w/page/29688444/Pilot%20Phase%20Synthesis%20and%20Evaluation%20Report
• McGill, L., Falconer, I., Beetham, H. and Littlejohn, A. JISC/HE Academy OER Programme: Phase 2 Synthesis and Evaluation Report. JISC, 2011 https://oersynth.pbworks.com/w/page/46324015/UKOER%20Phase%202%20final%20report
• McGill, L., Falconer, I., Littlejohn, A. and Beetham, H. JISC/HE Academy OER Programme: Phase 3 Synthesis and Evaluation Report. JISC, 2011 https://oersynth.pbworks.com/w/page/59707964/ukoer3FinalSynthesisReport

These fed into a wider review of HEFCE funded OER initiatives (still reporting Jan 2013).

UKOER Project Evaluation documents

• The BERLiN experience - academic focus group feedback report: BERLiN Project (University of Nottingham
• The BERLiN staff survey report BERLiN Project (University of Nottingham
• Open Exeter Project Final evaluation report - University of Exeter 
• OpenSpires Project Evaluation Framework - University of Oxford
• OTTER Project Evaluation reports - University of Leicester
• Academic advisor evaluation report (Unicycle Project, Leeds Metropolitan University)
• Evaluation framework mapping (Unicycle Project, Leeds Metropolitan University)
• C-change Project Evaluation Report (C-change Project, Subject Centre for Geography, Earth & Environmental Sciences (GEES)
• Baseline and change survey results (Humbox Project)
• SPACE project evaluation materials
• Semi-structured interview guides to identify user needs for curriculum heads
• Semi-structured interview guides to identify user needs for learners
• Product development workshop evaluation form
• Evaluation framework
•  Interview questions for pedagogy steering group and Interview questions for user group (SPACE Project)
• Evaluation questionnaires
• sesame-project-baseline-tutor-survey
• sesame-project-pilot-tutor-survey
• sesame-project-pilot-student-survey
• sesame-project-final-tutor-survey (SESAME Project)