Indian Journal of Clinical and Experimental Ophthalmology

Print ISSN: 2395-1443

Online ISSN: 2395-1451

CODEN : IJCEKF

Indian Journal of Clinical and Experimental Ophthalmology (IJCEO) is open access, a peer-reviewed medical journal, published quarterly, online, and in print, by the Innovative Education and Scientific Research Foundation (IESRF) since 2015. To fulfill our aim of rapid dissemination of knowledge, we publish articles ‘Ahead of Print’ on acceptance. In addition, the journal allows free access (Open Access) of its content, which is likely to attract more readers and citations of articles more...


  • Article highlights
  • Article tables
  • Article images

Article statistics

Viewed: 52

PDF Downloaded: 33


Maheshgauri, Bakare, Bhavsar, and Magdum: Triad of rubric assessment, constructive feedback and video recorded surgeries key assessment for competent ophthalmic surgeon


Introduction

Small-incision cataract surgery (SICS) is a widely accepted, appropriate, and affordable procedure that can deliver high-quality visual outcomes.1, 2, 3, 4

Despite this need, concerns remain in several regions over the safety, quality, and efficiency of surgical training for cataract surgery.5, 6

Ophthalmic surgery involves very precise surgical skills which are difficult to teach. The surgical technique of a resident is also difficult to assess. Currently, teaching ophthalmic surgical skill is not structured. Every surgeon have their own way of imparting surgical training, which creates confusion amongst the residents, thus hampering their learning skills.

Hence there is need for education system to come up with some teaching and assessment tool for evaluation of surgical performance of the post graduate students.

It is also desirable for any assessment method to be objective, thus removing bias and favoritism.

For assessing performances in clinical settings Miller developed a theoretical framework known as Millers Pyramid clinical assessment where bottom levels begins as ‘Knows’, then ‘Knows How’, ‘Shows how’ and at top ‘Does’. The top level ‘Does’ has been considered as gold standard of assessment in surgical training measured by direct faculty observation. There may be some barriers for assessment of surgical trainings of trainees e.g. fear of being labelled for strict and harsh teacher, it may be challenging for supervisor to assess progress of underperforming trainees because of poor documentation, lack of opportunities to observe performance, or fear of legal action, hence the Master of surgery degree is conferred without assessing actual operative skill of surgeon which may put health of society at risk. Hence there is need for education system to come up with some assessment tool for evaluation of surgical performance of post graduate student.

In practice, ICO has developed and implemented various assessment tool to evaluate the different level of millers pyramid in form of task specific checklist called Ophthalmic surgery clinical assessment Rubric(OSCAR).1, 7

The ICO OSCAR for SICS was developed by experts at the ICO using a modified Dreyfus scale (novice, beginner, advanced beginner, and competent).8

Significance of the study:

  1. Formative Assessment and constructive feedback in surgical training will improve the competency of new ophthalmic surgeons.

  2. Structured surgical training will be relatively easy to observe and perform, as trainee learns what is required to be competent.

  3. Ultimately improve patient care.

Aim & Objective

  1. To develop more standardize surgical training.

  2. To assess efficacy of new tool which can improve surgical skill of PG student.

  3. To know the effect of constructive feedback on surgical performance.

Materials and Methods

This study was prospective observational study, carried out in Department of Ophthalmology of Medical College and Research Centre between January 2019 to February 2020 after institutional ethical clearance.

Seven final year Post Graduate (PG) residents were chosen for the study with convenience sample technique Small incision cataract surgery training assessment was done by Rubric designed by tool called ICO-Ophthalmic Surgery Competency Assessment Rubric. The same tool was used to assess video recorded cataract surgery of residents by 2 different faculties and the response was noted. Assessor independently assessed the surgical technique without any prior influence and knowledge about previous surgical assessment, PG resident’s surgical video were numbered as PG 1 to PG 7 to maintain anonymity and prevent information bias. The assessor simply circled the observed performance description at each step of the procedure. The ICO-OSCAR score was completed. Observations of each resident were recorded.

At the end of each case assessor immediately discussed case with student to provide timely, structured, specific performance feedback. Surgical performances were recorded and their assessment were done by different faculties with same tool. Post-operative patient’s record were reviewed for outcome of surgery and complications if any. Rubric Score were analysed with inter rater agreement. Post assessment feedback was collected from faculties as well as resident about the assessment technique. Patients were assessed in post-operative period for visual recovery and any other complications. The recorded Data was entered in Master Chart on Word Excel document.

Result

Research analysis requirement: statistical analysis (descriptive and inferential), graphical representation Specific statistical test applied: kappa statistics Statistical packages used: SPPS and graph pad.

Table 1

Oscar score of PG by different assessor as well as self-minimum and maximum

Minimum

Maximum

Oscar score (1)

Oscar score (2)

Oscar score (Student)

Oscar score (1)

Oscar score (2)

Oscar score (Student)

PG 1

72

75

70

96

94

92

PG 2

65

62

58

90

88

86

PG 3

69

65

60

94

92

90

PG 4

72

75

70

96

94

92

PG 5

65

62

58

90

88

86

PG 6

55

49

50

80

82

80

PG 7

55

49

50

80

82

80

Table 2

Mean score of scorer 1 & 2 is almost same. There is not much standard of deviation of mean is noted

Mean

SD

Oscar score (1)

Oscar score (2)

Oscar score (Student)

Oscar score (1)

Oscar score (2)

Oscar score (Student)

PG 1

81.33

81.83

77.50

8.55

6.65

8.19

PG 2

77.57

74.29

71.00

8.83

9.34

9.85

PG 3

80.57

79.29

75.43

8.98

10.45

10.67

PG 4

81.33

81.33

77.50

8.55

6.65

8.19

PG 5

77.57

74.29

71.00

8.83

9.34

9.85

PG 6

70.5

71.5

66.67

9.35

12.45

10.8

PG 7

70.5

71.5

66.67

9.35

12.45

10.8

Figure 1

OSCAR Score of two assessor is almost same that shows there is no subjective variation in the assessment of students by assessor

https://typeset-prod-media-server.s3.amazonaws.com/article_uploads/f5c546cc-a2ad-4a4b-a705-a60f0f33ac0d/image/36685db9-484b-47bb-868d-5d8bebfe0c12-uimage.png

Final Master Table: Table 2 Interclass correlation coefficient on chronbachs alpha analysis shows is 0.961 in 95% confidence interval it shows there was a very good agreement in all 3 raters.

Table 3

ntraclass correlation coefficient

Intraclass Correlationa

95% Confidence Interval

Lower Bound

Upper Bound

Single Measures

.895b

.572

.961

Average Measures

.962

.800

.987

Intraclass Correlation Coefficient

F Test with True Value 0

Value

df1

df2

Sig

Single Measures

68.162

44

88

.000

Average Measures

68.162

44

88

.000

Two-way random effects model where both people effects and measures effects are random.

Inference

There was a very good inter rater agreement (0.96). Cohen’s Kappa: 0.65 again shows good interrater agreement. (Table 2, Table 3)

From these statistics, we found ICO-OSCAR offers a reliable way to objectively assess the surgical skills acquired during residency training. OSCAR TOOL has very good inter rater agreement i.e. (0.96).

Descriptive feedback analysis of student & observer regarding this assessment tool & entire procedure of this assessment revealed:

  1. Constructive feedback from assessor & recorded surgeries definitely helped PG students to improve surgically. Which ultimately resulted in best outcome for patient in terms of good visual acuity.

  2. Both assessor as well as examine found this tool very easy to use and non-subjective, structured and above all can be used as learning tool as well as it has described the expected performance. With possibilities of error which novice can make and if person has an idea beforehand it is least likely he will repeat the possible errors. This is very important in learning phase.

Our study results showed Oscar score gradually increased in all PG student with every successive cases. There was no much difference in the score between the assessors.

Discussion

Continuous professional development relies on the link between performance and an educational process aimed at improving knowledge and skill, one of the most broadly used frameworks for assessing skills is Miller’s Pyramid. This Pyramid has a series of levels of achievement beginning with knowledge (at the base) and ending with routine application in the clinical setting.9 Intraoperative evaluation is the gold standard. Intraoperative assessment achieves highest level evaluate ‘DOES’ in Miller’s pyramid. In a study of the Objective Structured Assessment of Technical Skills, Goff et al. (2002) had two faculty members complete checklists to rate the performance of OB/GYN trainees; ratings increased with amount of training.10

While standardized patient examinations have been successfully implemented in medical education, certification, and licensure, they cannot be used to assess all aspects of competence.7, 11

In our study we have seen it has very good inter rater agreement similar findings were noted by Golnic KC et al.8, 12

In our studies we have taken descriptive feedback on this assessment method by students as well as assessor and we found out it is reliable tool, adding educational value, it is feasible and acceptable by all.

As it is time tested, skills sharpen with multiple attempts or practice. Similarly if multiple assessment with constructive & time specific feedback is given it will lead to better surgical performance. Hence multiple assessment improves abilities of trainee in the surgical performance. This type of assessment tool serves three distinct purposes.13, 14 It minimizes subjectivity by precise instruction, the rubric states what is expected of the trainee to attain competence at each level. Self-Assessment by reflective practice is possible when the trainee’s surgical procedures are recorded. This ICO-Oscar tool was easy to use for assessor as well as learner. ICO-Oscar Tool in our study has been precise and valid for PG student’s surgical assessment. This tool can be used by Training institutes as bench mark for trainers to decide competency and necessary certification for Small Incision Cataract Surgery proficiency.

Conclusion

Good quality assessment not only satisfies the needs of accreditation but also contributes to student’s learning. Assessment methods should match the competencies being learnt and the teaching formats being used.

Source of Funding

None.

Conflict of Interest

The authors declare no conflict of interest.

References

1 

A Hennig J Kumar D Yorston A Foster Sutureless cataract surgery with nucleus extraction: outcome of a prospective study in NepalBr J Ophthalmol200387326670

2 

S Ruit G Tabin D Chang L Bajracharya DC Kline W Richheimer A prospective randomized clinical trial of phacoemulsification vs manual sutureless small-incision extracapsular cataract surgery in NepalAm J Ophthalmol20071431308

3 

Y Riaz SR Silva JR Evans Manual small incision cataract surgery (MSICS) with posterior chamber intraocular lens versus phacoemulsification with posterior chamber intraocular lens for age-related cataractCochrane Database Syst Rev20131010CD008813

4 

P Gogate JJ Optom S Deshpande K Naidoo Meta-analysis to compare the safety and efficacy of manual small incision cataract surgery and phacoemulsificationMiddle East Afr J Ophthalmol20152233629

5 

A Haripriya DF Chang M Reena M Shekhar Complication rates of phacoemulsification and manual small-incision cataract surgery at Aravind Eye HospitalJ Cataract Refract Surg201238813609

6 

P Gogate Comparison of various techniques for cataract surgery, their efficacy, safety, and costOman J Ophthalmol2010331056

8 

KC Golnik H Beaver V Gauba AG Lee E Mayorga G Palis Cataract surgery skill assessmentOphthalmology20111182427

9 

BW Williams PD Byrne D Welindt MV Williams Miller's Pyramid and Core Competency Assessment: A Study in Relationship Construct ValidityJ Contin Educ Health Prof20163642959

10 

BA Goff PE Nielsen GM Lentz GE Chow RW Chalmers D Fenner Surgical skills assessment: A blinded examination of obstetrics and gynecology residentsAm J Obstet Gynecol200218646137

11 

JJ Norcini DW McKinley Assessment methods in medical educationTeaching Teacher Educ20072323950

12 

K C Golnick H Beaver Gauba V Lee Ag E Mayoroga G Pallis SalehG, cataract cataract surgical skillassessment ophthalmology201111855

13 

SG Honawar How good a surgeon are you? - Standardized formative assessment of surgical competence for ophthalmology residents in trainingIndian J Ophthalmol20176597778 10.4103/ijo.IJO_849_17

14 

M Swaminathan S Ramasubramanian R Pilling J Li K Golnik ICO-OSCAR for pediatric cataract surgical skill assessmentJ AAPOS20162043645



jats-html.xsl

© This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.


Article type

Original Article


Article page

703-706


Authors Details

Rupali Maheshgauri, Prachi Nilraj Bakare*, Deepaswi Bhavsar, Renu Magdum


Article History

Received : 08-06-2021

Accepted : 22-06-2021

Available online : 03-01-2022


Article Metrics


View Article As

 


Downlaod Files

   








Open Abstract (Increase article citation) Wiki in hindi