Bibliography of Articles on Multi Source Feedback

Definition: Multi source feedback (MSF) or 360 degree feedback is the process by which multiple individuals, representing discrete informant groups, provide anonymous feedback to recipients. In medicine, the informant groups may include peer, referring physician, referral physicians, patients, co-workers (e.g., nurses, pharmacists, physical therapists) and self.

For the purpose of this bibliography*, we have included the literature in which there are two or more sources (excluding self).

Medical Students

• Tyler KM, Peer-level multiple source feedback for fitness to practice, Medical Educ 2006, 40:482-483

Canadian Residents

• Johnson D, Cujec B. Comparison of self, nurse, and physician assessment of residents rotating through an intensive care unit. Crit Care Med 1998; 26:1811-6.

International Residents

• Allerup P, Aspergren K, Ejlersen E, Jorgensen G, Malchow-Moller A, Moller MK, Pedersen KK, Rasmussen OB, Rohold A, Sorensen B, use of 360-degree assessment of residents in internal medicine in a Danish setting: a feasibility study, Medical Teacher 2007; 29:166-170

United Kingdom Residents

• Archer J, Norcini J, Southgate L, Heard S, Davies H, mini-PAT (Peer Assessment Tool): A valid component of a national assessment program in the UK? Adv in Health Sci Education 2006, DOI 10.1007/s10459-9033-3
• Archer J, Norcini J, Davies HA, Peer review of paediatricians in training using SPRAT, BMJ 2005; 330: 1251-1253.
• Carr S, The Foundation Programme assessment tools: an opportunity to enhance feedback to trainees? Postgrad Med J, 2006; 82(971):576-9
• Davies H A and Archer J C, Multi source feedback using Sheffield Peer Review Assessment Tool (SPRAT) - development and practical aspects. Clinical Teacher 2005; 2(2): 77-81.
• Hesketh EA, Anderson F, Bagnall GM, Driver CP, Johnston DA, Marshall D, et al. Using a 360 degrees diagnostic screening tool to provide an evidence trail of junior doctor performance throughout their first postgraduate year. Med Teach 2005;27(3):219-33.
• Whitehouse A, Hassell A, Wood L, Wall D, Walzman M, Campbell I. Development and reliability testing of TAB a form for 360 degrees assessment of Senior House Officers’ professional behaviour, as specified by the General Medical Council. Med Teach 2005;27(3):252-8
• Whitehouse A, Hassell A, Bullock A, Wood L, Wall D, 360 degree assessment (multisource feedback) of UK trainee doctors: Field testing of team assessment of behaviors, Med Teacher 2007; 29:171-176

United States Residents

• Davis JD, Comparisons of faculty, peer, self and nurse assessment of obstetrics and gynecology residents, Obstetrics gyn 2002; 99(4):647-651
• Higgins RS, Bridges J, Burke JM, O’Donnell MA, Cohen NM, Wilkes SB, Implementing the ACGME general competencies in a cardiothoracic surgery residency program using 360-degree feedback. Ann Thorac Surg 2004; 77(1):12-7
• Joshi R, Ling FW, Jaeger J, Assessment of a 360-Degree Instrument to Evaluate Residents’ Competency in Interpersonal and Communication Skills, Acad Med 2004 79: 458-463.
• Musick DW, McDowell SM, Clark N, Salcido R, Pilot study of a 360-degree assessment instrument for physical medicine and rehabilitation residency programs. Am J Phys Med Rehab 2003; 8295):394 – 402
• Reisdorff EJ, Hughes MJ, Castaneda C, Carlson DJ, Donohue WA, Fediuk TA, Hughes WP, Developing a valid evaluation for interpersonal and communication skills, Acad Emergency Medicine 2006; 13(10):1056-61
• Risucci DA, Tortolani AJ, Ward RJ, Ratings of surgical residents by self, supervisors and peers. Surg Gyn Obs 1989; 169(6):519-526
• Wood J, Collins J, Burnside E, Albanese MA, Propeck PA, Kelcz F, Spilide JM, Schmaltz, Patient, faculty, and self-assessment of radiology performance: A 360-degree method of measuring professionalism and interpersonal/communication skills, Acad Radiol 2004; 11:931-939.
• Wooliscroft JO, Howell JD, Patel BP. Resident-patient interactions: the humanistic qualities of internal medicine residents assessed by patients, attending physicians, program supervisors and nurses. Acad Med 1993; 68: 680-7.

Canadian Practicing Physicians

• Fidler H, Lockyer J, Toews J, and Violato C, Changing Physicians’ Practices: The Effect of Individual Feedback, Academic Medicine, 1999; 74: 702-714.
• Hall W, Violato C, Lewkonia, R, Lockyer J, Fidler H, Toews J, Jennett P, Donoff M, Moores D, Assessment of physician performance in Alberta: the Physician Achievement Review Project, Canadian Medical Association Journal, 1999; 161:52-57.
• Lockyer, J, Violato C, Fidler H, Likelihood of Change: A study assessing surgeon use of multi source feedback data. Teaching and Learning in Medicine, 2003; 15: 168-174.
• Lockyer J, Multi source feedback in the assessment of physician competencies. Journal of Continuing Education in the Health Professions, 2003; 23(1): 23: 4-12.
• Lockyer J, Violato C, A multisource feedback program for anesthesiologists, Can J of Anesthesia Can J of Anesthesia, 2006, 53:33-39
• Lockyer J, Blackmore D, Fidler H, Crutcher R, Salte B, Shaw K, Ward B, Wolfish N, A study of a multi source feedback system for international medical graduates holding defined licenses, Medical Education 2006; 40: 340-347.
• Lockyer JM, Violato C, Fidler H, The assessment of emergency physicians by a regulatory authority. Academic Emergency Medicine, 2006. 13(12):1296-303
• Lockyer JM, Violato C, Fidler H, What Multi Source Feedback Factors Influence Physician Self Assessments? A Five Year Longitudinal Study, Acad Med 2007, 10S (In Press).
• Sargeant JM, Mann KV, Ferrier SN, Langille DB, Muirhead PD, Sinclair DE. Responses of rural family physicians and their colleagues and coworker raters to a multi-source feedback process: a pilot study. Academic Medicine, 2003: 78 (S10): S42-S44.
• Sargeant J, Mann K, Ferrier S. Understanding family physicians reactions to MSF performance assessment: perceptions of credibility and usefulness. Med Educ, 2005. 39:497-504
• Sargeant J, Mann K, Sinclair D, Ferrier S, Muirhead P, van der Vleuten C, Metsemakers J Learning in practice: Experiences and perceptions of high scoring physicians, Acad Med 2006, 81(7):655-660
• Sargeant J, Mann K, Sinclair D, van der Vleuten C, Metsemakers J, Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use, Adv in Health Sci Educ 2006, 10.1007/s10459-006-9039-x
• Sargeant J, Mann K, Sinclair D, van der Vleuten C, Metsemakers J,Challenges in multi source feedback: Intended and unintended consequences, Medical Education 2007, 41(6):581-596
• Violato C, Marini A, Toews J, Lockyer J, and Fidler H, Feasibility and Psychometric Properties of Using Peers, Consulting Physicians, Co-workers, and Patients to Assess Physicians, Academic Medicine, 1997 (supplement); 72 S82-S84.
• Violato, C, Lockyer, JM, Fidler H, Multi source feedback: A method of assessing surgical practice, British Medical Journal, 2003; 326: 546-548.
• Violato C, Lockyer J, Fidler H, The assessment of pediatricians by a regulatory authority, Pediatrics 2006; 2006, 117: 796 - 802
• Violato C, Lockyer J, Self and peer assessment of pediatricians, psychiatrists and medicine specialists: Implications for self-directed learning, Adv in Health Sciences Educ 2006, 11:235-244

United States Practicing Physicians

• Lipner RS, Blank LL, Leas BF, Fortna GS. The value of patient and peer ratings in recertification. Acad Med 2002; 77 (Suppl 10): S64-6.
• Ramsey PG, Wenrich MD, Carline JD, Inui TS, Larson EB, LoGerfo JP. Use of peer ratings to evaluate physician performance. JAMA 1993; 269: 1655-60.
• Weinrich MD, Carline JD, Giles LM, Ramsey PG. Ratings of the performances of practicing internists by hospital-based registered nurses. Acad Med 1993; 68: 680-7.

United Kingdom Practicing Physicians

• Bennett H, Gatrell J, Packham R, Medical appraisal: collecting evidence of performance through 3600 feedback, Clinician in management 2004; 12(4):165-172
• Evans R, Elwyn G, Edwards A, Review of instruments for peer assessment of physicians, BMJ 2004; 328: 1240-1245
• Griffin E, Sanders G, Craven D, King J, A computerized 3600 feedback tool for personal and organizational development in general practice. Health Informatics Journal 2000; 6:71-80

Reference Books & Core References from Industrial Psychology

• Bracken DW, Timmreck CW, Church AH, The handbook of multi source feedback: The comprehensive resource for designing and implementing MSF processes, San Francisco: Jossey-Bass, 2001
• Smither JW, London M, Reilly RR: Does performance improve following multisource feedback? A theoretical model, meta-analysis and review of empirical findings. Personnel Psychology 58:33-66, 2005.
• Tornow WW, London M and CCL Associates, Maximizing the value of 360-degree feedback: A process for successful individual and organizational development, San Francisco: Jossey-Bass, 1998

Useful Web Sites

• Gray A, Lewis A, Fletcher C, Burke E, Mackay J, Kubelius E, Lindley P, 360 degree feedback: Best practice guidelines, 6/6/ 2003, http://www.psychtesting.org.uk/ [go to downloads and do a search for 360 degree feedback to retrieve the pdf]
• College of Physicians and Surgeons of Alberta.This website provides copies of the questionnaires being used in Alberta for family physicians, surgeons, anesthesiologists, medical/pediatric/psychiatry, episodic care (e.g., emergency room, locum physicians, hospitalists), and radiologists along with back ground information about the program and the feedback report.
• Foundations Program (United Kingdom).Copies of the questionnaires used are contained within the pdf for the Physician Learning Portfolio document.

Updated: June 16, 2008