evaluation of a medical education intervention

How is a medical education intervention evaluated?

It is one tough taks to suggest an education reform for a medical school, to fork out money for structural change, and convince the reluctant faculty to change whatever seems to be working fine to make things better.

but there is another tough nut to crack - the evaluation of such interventions once it is done.

how could we evaluate the success of any medical educational intervention?

Educational evaluation is the systematic appraisal of the quality of teaching and learning- just as assessments do, Evaluation can have a formative role, identifying areas where teaching can be improved, or a summative role, judging the effectiveness of teaching.

Four general approaches to educational evaluation have been suggested by Wilkes and Bligh (1999) -

• student oriented - Predominantly uses measurements of student performance (usually test results) as the principal indicator.
• programme oriented - —Compares the performance of the course as a whole to its overall objectives and often involves descriptions of curriculum or teaching activities.
• institution oriented - Usually carried out by external organisations and aimed at grading the quality of teaching for comparative purposes.
• stakeholder oriented  - Takes into account the concerns and claims of those involved and affected by the course or programme of education including students, faculty, patients, and the governing body of medical education in the country. 

If, lets say, we had one year to evaluate the success of a project (intervention), what kind of data should we collect in order to approach evaluation from these above-mentioned four criteria? 

• student oriented - we could get the examination results (of whatever the medical school is currently using) at the start of the year, and at the end of the year where the intervention has taken place, as well as asking students for self-assessments about their own learning. 
• programme oriented - we could assess if the changed curriculum more appropriately achieve the outcomes that was meant to reach, and this assessment would be most relevant if there is a rubric for each of the outcomes set. (if there is no outcomes yet, perhaps it is an idea to make one at the start of the year)
• institution oriented - request external organizations to grade the quality of teaching. if the country that the school exist in does not have the External organization for quality control, there are international bodies that do so -one example is the WFME (World Federation for Medical Education) is involved in accreditation of medical schools, and they can assess medical schools if the said medical school is fit to produce medical graduates. 
• Stakeholder oriented - a survey of the perceived adequateness of medical education before and after the intervention could be undertaken, where the survey is directed to the stake-holders - students, faculty members, and patients (whom were possibly involved in the education process) and SPs (used in the OSCE intervention) 

references;
Wilkes M1, Bligh J; Evaluating educational interventions; BMJ. 1999 May 8;318(7193):1269-72.