Tag: Medical

Impact of online learning on sense of belonging among first year clinical health students during COVID-19: student and academic perspectives | BMC Medical Education

Online student cross-sectional survey

Demographic characteristics

A total of 179 out of the possible 663 students (27{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} completion) completed the online survey in June 2020. Median age of students was 19 years (IQR 18–28 years) and there were approximately three times as many females as males (Table 1), reflective of the undergraduate health sciences cohort (70{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} female). Student numbers were also reflective of the broader enrolment numbers in the programs (i.e., occupational therapy is the largest program). Just over half (53{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}; n = 94) of students had no prior experience in undertaking a Bachelor degree, and 76{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of students had not completed any online courses prior to enrolment.

Table 1 Demographic characteristics

Quantitative results to the sense of belonging questionnaire

In terms of students’ sense of belonging to the university, the majority felt ‘quite’ or ‘extremely’ happy with their choice of university (74{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) and felt ‘quite’ or ‘extremely’ welcomed by the university (68{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}). While most students felt respected by both staff (70{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) and students (60{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) at the university, students reported less connectiveness (23.5{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) to the university. Only 20{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of students reported they felt they were understood as an individual, and only 13{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} felt they ‘quite’ or ‘extremely’ mattered to others at the university (Table 2).

Table 2 Online learning and Sense of Belonging to the University [1]

Table 3 shows how the online learning experiences impacted on students’ perception of the course; 27{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of students felt ‘quite’ or ‘extremely’ connected to staff while 16{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of students felt ‘quite’ or ‘extremely’ connected to other students. While 49{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of students rated 4 and above for the level of respect that they received from other students and their contribution towards the subject, students who had prior higher education felt less respected than students who had no prior higher education (p = 0.03). When asked how the online subject had contributed to understanding, knowledge/skills in their chosen health profession, about half of the students rated the online subject highly (rating 4 and above). Students who had prior higher education indicated higher ratings of understanding and knowledge/skills compared to students without prior higher education (p = 0.07 and p = 0.03 respectively). There was also a significantly higher proportion of students with no prior higher education who identified the online learning experience as either ‘quite’ or ‘extremely’ likely to impact their intention to continue with their current course (p = 0.001).

Table 3 Impact of online profession-specific subject on perception of the course

Qualitative results

Qualitative findings provided insight into experiences of staff and students during the rapid, unplanned transition to online learning. Student questionnaire responses included two open-ended questions expanding on enablers and barriers to sense of belonging. These yielded 145 enablers and 254 barriers to students’ feeling a sense of belonging. Data were subjected to qualitative content analysis by two authors and categories are presented in Additional file 1.

Three focus groups were conducted: two student sessions, each with two students enrolled in Speech Pathology and Paramedicine, and one academic session with five participants. Four full time academics and one casual academic participated from a total population of nine eligible academics. Using the processes described in the methods, focus group analysis was compared with the survey content analysis and the authors identified synergies between them. Findings were then integrated under a global theme, underpinned by organising and basic themes. The following themes reflect triangulation between academic and student focus group data in addition to survey responses.

Global theme—navigating belonging during the COVID-19 crisis: a shared responsibility

“We are in this together…making the best of this”

This theme explores sense of belonging creation during this period as a shared process, where participants perceived they worked together to get through the crisis. Students and academics encountered many challenges as they transitioned to online learning but despite hard times, were able to engage positively. The global theme revealed students and academics were navigating belonging during the COVID-19 crisis, and this journey was a shared responsibility. Both groups were working to achieve positive student engagement that would in turn create a sense of belonging in first-year students. A strong commitment of working hard to make the best out of this was mutually acknowledged.

Students perceived academics had done “a really good job at making sure we belonged…in those first few weeks that we were on campus but even more so probably while we were in Zoom” (Student-Astrid-Focus Group). Academics perceived students were actively engaged in making online learning work and were collegial and collaborative.

The shared experiences about navigating belonging during the COVID-19 crisis, have been captured under four organising themes: dimensions of belonging, individual experiences and challenges, reconceptualising teaching and learning, and relationships are central to belonging. Within each organising theme, basic themes were identified that provide depth to the organising theme (Fig. 1). Additional files 1 and 2 present a summary of the quotes obtained from the open-ended surveys and focus groups respectively, that contribute to the themes in Fig. 1.

Fig. 1
figure 1

Pictorial representation of the global, organising, and basic themes

Organising theme: dimensions of belonging

This theme outlines that belonging is a multidimensional experience with several facets underpinning participants’ experiences. Students and academics identified several dimensions of belonging in relation to first year students’ experiences, as illustrated by two basic themes that sit under the organising theme: what it means to belong, and layers of belonging.

Basic theme: what it means to belong

This theme explores the idea that belonging at university is underpinned by feeling valued and connected. Academics and students agreed that having a sense of being valued by the university and a desire to have an active connection across all aspects of university life was important for students.

Belonging as a student was gained through a connection with the “vocation” (Student-Claire-Focus Group) or the course and career, and with people who will “be there” (Student-Claire-Focus Group) for them. Furthermore, support of academics was critical to gaining a sense of belonging. It was noted by academics and students, that when students feel they belong at university, they are actively engaged in their learning, and this sense of belonging in turn shapes their overall identity. Students can then “actually sort of relax and become themselves” (Staff-Brooke).

Belonging to their cohort, their course, their future profession, and their university was important for students. One academic noted that the “concept of acceptance” is part of the sense of belonging and goes “both ways” (Staff-Brooke).

Both academics and students agreed that the rapid change to online learning due to COVID-19, meant that developing a sense of belonging was challenged.

Basic theme: layers of belonging

This theme identified layers of belonging reflected in participants’ experiences. Peer, academic and professional layers each contributed to an overall sense of belonging and key examples are provided below.

Peers

Belonging to peers was described as “having that connection to someone that’s going through exactly the same thing as what you’re going through” (Student-Astrid-Focus Group). Students were concerned that when learning moved online that this sense of belonging would be jeopardised by less opportunities for in-person interaction.

Academics

Being connected to academics was perceived by students as directly impacting learning, with one student commenting: “…when they’re not connecting with the teacher, they’re not connecting with the content, they’re not connecting with the feedback. That’s when you develop this sense of feeling like you just don’t belong” (Student-Emily-Focus Group).

Academics perceived it was also important for students to develop a sense of belonging to the university community.

Profession

Belonging to a profession was identified as an important feature of belonging by academics and students. Studying a degree with a clear professional identity facilitated first year students to feel they belonged compared to those undertaking general health science degrees which may have multiple pathways and career options less directly aligned to first year studies.

One academic actively encouraged first year students to belong to their professional association as a way of fostering belonging in first years.

Organising theme—Individual experiences and challenges

This theme outlines that while there are similarities in participants’ experiences, individuals have unique contexts and factors shaping their experiences. Academics and students reflected upon personal impacts of the COVID-19 pandemic on their teaching or learning and how they responded as individuals to the ensuing challenges. Two basic themes emerged: Challenges of transition and recognising different learning preferences.

Basic
theme
—challenges of transition

This theme explored the significant challenges of transitioning to online teaching and learning. For some students, the transition to online learning offered potential benefits of flexibility and reduced travel time. Two of the four students in the focus groups opted for online learning opportunities available in other subjects of study prior to the pandemic to efficiently manage their study and external commitments. Nonetheless, the pandemic brought a raft of personal challenges that diminished these expected benefits. Covid-related changes to family employment, reduced access to childcare support and non-optional home schooling presented new concerns.

Clearly, students missed the opportunity to focus attention on their learning needs when balancing childcare demands and home-schooling during lockdowns.

Unlike a conventional online courses where students choose or plan to be online, the sudden, unexpected, and unplanned move to online study was prefaced by a short period (four weeks) of in-person class time. This initial in-person time was identified as being key to relationship building.

Academics identified positive experiences and challenges during the transition to online learning. The rapid change presented a problem to be solved and individuals could “embrace it and to work effectively…as a team” (Staff-Jane). Quickly strategizing and responding to the demands of online learning required team knowledge, experience, and support. Hence, enhanced team culture was a further positive for academics, being “present for each other” (Staff-Brooke).

Basic
theme
:
recognising different learning preferences

This theme identifies experiences of online learning influenced by personal attributes, individual expectations and learning preferences. Such key factors impacted students’ capacity to maintain focus on academic goals after the rapid change to online learning. Some students reflected that barriers were not solely a feature of online learning environments, reporting that competing priorities, including work commitments and limited contact time with staff as pre-existing challenges to belonging. However, some students directly attributed their limited engagement and reduced motivation to the online learning environment.

Students suggested that active engagement “comes down to personality” (Student-Astrid- Focus Group). If a student was not shy they were comfortable to come forward and participate online. Some students perceived clear links between personal discipline, engagement, commitment, and achievement in online learning environments.

Further, students perceived effective (and ineffective) online group functioning reflected personalities of individual members, with some groups/personalities seen as being able to organise whilst other groups lacked leadership and cohesion.

Students who perceived themselves as active engagers reported being drawn towards other students who demonstrated motivation to interact and learn. Other students perceived their personalities or learning preferences were misaligned with the expectations of belonging in online learning environments and focussed upon tasks rather than connection.

Academics recognised student diversity and a need to reflect and re-evaluate expectations of students in online environments. They accepted that some students may be quietly engaging and learning to belong, but this was harder to observe in online compared to in-person learning environments.

Organising theme—relationships are central to belonging

This theme identified the relationship between all parties as a fundamental aspect of creating a sense of belonging. Two basic themes were influential in shaping perceptions of how relationships and connections contribute to belonging: collaboration with peers is fundamental, and effective and regular communication with staff is necessary.

Basic
theme
—collaboration with peers is fundamental

This theme revealed collaboration with student peers was a key element of creating a sense of belonging. The degree of social interaction with student peers and opportunities to create friendships contributed to feelings of belonging. Accordingly, students found it problematic when peers neglected to turn cameras on during classes, making interaction very difficult. Visualisation of peers and use of cameras in online classes impacted students’ opportunities to get to know each other.

Challenges posed by online learning were further highlighted in the student survey through a focus on non-academic aspects of university and campus life. Typically, university campuses offer interactional opportunities through clubs, sport, and shared spaces to learn and socialise. Campus life, students suggested, may facilitate learning and personal development. Absence of this type of interaction was linked to barriers in developing friendships and consequently a lesser sense of belonging as reflected in Additional file 1.

Basic theme—
communication
with academics is necessary

This theme outlined that communicating with academics was a key component of creating a sense of belonging. With less opportunities for peer support, there was stronger reliance on the academic-student connection, although students reported positive and negative interactions with academics during online learning.

Positive interactions and individualised communication with academics enhanced student sense of satisfaction and belonging. Furthermore, students in the focus groups reported a feeling of trust and a bond created by a shared challenge. Survey responses echoed this sentiment, noting that academics were “non-judgmental and supportive” (Student Survey 18) and created a sense of camaraderie. However, when students perceived impersonal communication from academics, they felt less connected or believed that teaching had become a “transaction” (Student-Astrid- Focus Group). Perceived levels of enthusiasm and engagement from academics influenced student’s perceptions of connection and belonging.

Students identified the online environment as a barrier to communication with academics. While systematic and university level communication was perceived as a useful source of information, students prioritised individualised communication from academic staff as key to belonging.

Academics concurred that effective communication was challenged in online environments, missing non-verbal cues and responsivity that characterises a classroom environment. Although the online learning environment provides an opportunity for academics to connect professionally with students, there were students who left their cameras off, with one academic noting they didn’t push this issue because there are many reasons for students choosing this option.

Organising theme: reconceptualising teaching and learning

This theme reveals how academics and students reconceptualised their expectations and modes of teaching and learning, to manage the crisis. It was not easy for academics or students, and many strategies were employed to make it work, with two basic themes emerging: challenges to online teaching and learning, and strategies to engage and connect.

Basic theme:
challenges
of online teaching and learning: “how do I make this work?”

This theme outlined many challenges faced by both academics and students during a rapid change to online mode. With the rapid change to online learning, academics asked themselves, ‘How do I make this work?’.

Managing workload

Academics reported their workload increased significantly, and they “found it a juggling act” (Staff-Louise) to meet their teaching requirements. Administrative loads consequently increased when reduced in-person contact with students led to more electronic communication. Academics needed to up-skill in online teaching in a short time frame and perceived this responsibility as all encompassing.

The rapid switch to online learning attracted significant academic workload, implementing and adapting content to see how material “might play out in a Zoom environment…[where]…everything takes longer” (Staff-Natalie).

Some students noticed a temptation to disengage from online learning, which meant balancing their workload and study demands became a challenge as they also faced significant workload and stressors in their personal lives due to COVID-19.

Class dynamics

Academics and students spoke about the change to classroom dynamics. The online environment was noted as being one in which it was difficult to read the room to see how students were progressing with their work. Others tried to use humour to enliven a class, only to have the Zoom frame freeze, killing the mood they were trying to create. Hence, staff felt teaching online was less conversational, flexible and responsive compared to face-to-face. Moreover, academics missed hands-on practical elements; a big shift for some programs.

Technological challenges

Academics learnt new skills quickly, but often these skills would be challenged when technology failed. Some academics reported a sense of vulnerability due to technological ineptitude but acknowledged that making mistakes in front of students could humanise the experience. Academics also acknowledged that some students did not have adequate technological resources to meet changes in their learning requirements when classes were placed online.

Basic theme: strategies to engage and connect

This theme reflected the strategies academics and students employed to remain engaged and connected. Academics worked hard to enhance online learning and hoped to connect with students and engage them in activities. Students too were active and appreciated academics’ efforts to facilitate engagement and connection. Underlying many of the strategies adopted by academics was a deep concern for student welfare during this time. Therefore, many academics aimed to ensure students were engaged and connected with each other and with the academic team. Academics built in small group opportunities during online teaching so students could connect, learn, and socialise.

Staff also spoke about informing students they could contact staff for support. One staff member described crossing the divide and actively discouraging a ‘them and us’ dynamic between students and staff.

A variety of teaching tools were identified by staff to build connection and promote engagement. Such tools included interactive quizzes, ice breakers activities, integrating reflective practices into activities and ‘drop in’ sessions. Staff also encouraged students to establish social media groups or other group experiences outside the classroom. Some staff members arrived early to zoom classes and left late to enable students to connect informally.

Students appreciated staff attempts to provide these activities. Students found these initiatives helpful, recognising staff placed effort into knowing students personally and focussing on student wellbeing and achievement. Students cited examples of provision of extra resources, mini-lectures, additional question and answer sessions, and fast response times to student queries. Students also initiated their own engagement strategies, including using group and personal messaging over platforms such as Facebook messenger.

Are Online Medical Billing and Coding Courses Worth It?

Healthcare professionals discussing how online medical billing and coding courses are worth it

Health care billing and coding is the intersection of technological know-how and health care. In this industry, you are going to use numerous technological know-how methods to assist with the billing and reimbursement strategies that are utilized all over the health care industry. You may well also manage billing and electronic overall health data for your employer.

The professional medical billing and coding field is critical in health care simply because it provides client-facing experts the information and facts they want to present high-quality treatment and aid maintain business in an business that requires it to perform properly.

Catina Flagg with the text Catina FlaggHealth care billing and coding programs can give the marketplace awareness you will need to crack into the industry or upskill for your occupation. The techniques you’ll master can unlock new vocation alternatives and offer vocation advancement likely, said Catina Flagg, a medical college member at Southern New Hampshire University (SNHU).

“Medical coding supplies you with selections. You can conveniently get the job done whole- or portion-time, and you may perhaps even be in a position to work out of the workplace,” Flagg stated. “Some healthcare providers are outsourcing their medical coding and coding work, which indicates you may possibly be capable to function from property — at the time you’ve acquired some encounter and credentials.”

What Should You Count on from Medical Billing and Coding Courses?

In medical billing and coding classes, you are going to discover what professional medical billing and coding is, targeted on the skills and information required for a career in the discipline. The essential areas of examine consist of:

  • Coding classification units
  • Existing Procedural Terminology (CPT) and evaluation administration
  • Degree two of the Health care Frequent Course of action Coding Method (HCPCS)
  • Modifiers and terminologies
  • The International Classification of Conditions, Tenth Revision, Medical Modification (ICD-10-CM)

These spots of study will prepare you by delivering the “necessary health-related terminology, the cycle of health care claims and the health-related coding system,” Flagg explained.

The time it usually takes to comprehensive health-related billing and coding programs is dependent on a pair of variables, such as the place and how you choose the classes. At SNHU, for illustration, you can get programs on the web, which are only 8 weeks very long. That indicates you can acquire health care billing and coding techniques in as tiny as eight weeks.

Opportunity advantages of having health care billing and coding classes include things like:

  • Freelance and agreement career selections
  • Work possibilities that offer versatile get the job done environments
  • Doing work remotely

If a flexible career is something you motivation, consider what a career in the medical billing and coding area may offer you.

What Types of Careers Are Readily available in the Medical Billing and Coding Area?

The careers available may perhaps vary dependent on your instruction amount and practical experience. But the job needs in this subject assistance a variety of instruction amounts. So, whether you are just obtaining begun or currently have a degree, there is a vocation route for you.

Some roles to take into account:

  • Overall health data technologists
  • Clinical and well being company professionals
  • Medical information specialists
  • Healthcare registrar staff

The role of medical records expert is 1 option if you are just obtaining started out in the area. To be a health-related records expert, you can expect to commonly will need at the very least a postsecondary nondegree award, such as a certification, in accordance to the U.S. Bureau of Labor Figures (BLS). The median fork out for the posture in 2021 was $46,660, and jobs are envisioned to mature 7{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} in this discipline by means of 2031.

There are several web-sites, this kind of as the American Academy of Expert Coders (AAPC), that present exams accessible to those people searching to get accredited in health-related coding, this sort of as:

  • Accredited Inpatient Coder (CIC)
  • Qualified Outpatient Coding (COC)
  • Qualified Professional Coder (CPC)
  • Certified Chance Adjustment Coder (CRC)

Other roles in the industry may perhaps call for a degree, this kind of as an affiliate diploma. For case in point, the roles of wellbeing details technologists and health care registrars had a median income of $55,560 in 2021. And they ordinarily have an associate diploma schooling, according to BLS. In addition, the area is expected to expand 17{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} via 2031 — a lot quicker than regular.

There are also job chances, this sort of as management positions, for those with a bachelor’s degree. For example, healthcare and overall health services professionals normally keep a bachelor’s diploma, and their median fork out in 2021 was $101,340, in accordance to BLS. In addition, this discipline is envisioned to increase 28{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} by 2031 — substantially faster than average.

No matter your education and learning level, there are occupation selections obtainable in this developing industry that healthcare billing and coding classes can aid you achieve.

Should really You Consider Healthcare Billing and Coding Courses?

Having healthcare billing and coding courses makes it possible for you to check out the place of examine before committing to a total diploma system. Although, if the healthcare billing and coding area interests you, you may well ponder what diploma choices are readily available. At SNHU, the healthcare billing and coding course is a section of the overall health information administration (HIM) bachelor’s diploma program.

What is HIM? HIM focuses on mastering about health and fitness details and rules, medical devices management, facts assortment and storage, and evaluating information and facts program purposes, amid quite a few other topics.

If you’re considering a HIM degree, then professional medical billing and coding courses are very likely involved inside of the program. You may well also be able to transfer any credits you earn into a various program of your deciding on as nicely.

Irrespective of whether you acquire professional medical billing and coding programs separately or as aspect of a diploma plan, you can have the gain of understanding “(the) valuable abilities and expertise needed in the several health care small business and administration roles,” Flagg reported.

Find out far more about SNHU’s on the web professional medical billing and coding course: Obtain out what skills you may find out and how to ask for details about the program.

Nicholas Patterson ’22 is a author at Southern New Hampshire University. Connect with him on LinkedIn.

Identifying the challenges of online education from the perspective of University of Medical Sciences Students in the COVID-19 pandemic: a Q-methodology-based study | BMC Medical Education

This cross-sectional study was carried out making use of the Q methodology through the following six ways employing Barry and Proops system [19].

Phase 1 and 2: defining the concourse

At this phase, a concourse space was shaped with the identification of the topic or thought of the research. The introduced sights on the issue elevated for the concourse can be formed from a review of texts and industry experts in this field [19].

In this analyze, the subject and notion for the concourse ended up the difficulties of online instruction all through the COVID-19 pandemic. The concourse included a selection of diverse components similar to the investigate subject that was discussed amid students. The college students (P-set) who also had contributed earlier to the enhancement of the initial set of statements. 30-one particular students participated in semi-structured interviews, and we tried to establish their subjectivity about the investigation topic using the Q system [20].

In this review, the concourse (sample of individuals) provided pupils of the University of Professional medical Sciences (paramedical learners) who had sufficient info about online instruction in the course of the COVID-19 pandemic.

Move 3: screening and assortment of statements (Q-sample)

All through the semi-structured interviews with 31 pupils, 70 statements were being extracted about the perceived challenges of online training. The Q objects were picked very thoroughly so that goods did not overlap, and at the same time, no point of view ought to be missing. Thus, the collection procedure will take the most time and hard work of all the steps of the Q methodology. For that reason, research staff removed comparable unrelated, and ambiguous statements from the Q set. Eventually, 50 statements ended up chosen.

Phase 4: chosen P-set

College students who participated in the concourse (interviews) were selected as a sample of individuals to take part in sorting in the Q analyze (P-set). In the existing review, pupils were being picked by purposive sampling to include learners who experienced an academic, qualified, experimental romantic relationship or preceding understanding about the matter of examine. This choice of samples designed the members with extra assorted mentalities enter the analyze. It is advised that in Q scientific tests, the range of contributors to sort statements should be less than the selection of statements close to the study subject matter [21]. In the existing study, the quantity of members who ranked the difficulties of online training applications was 31 (Table 1).

Desk 1 The Q-established statements and component arrays in the study of difficulties online instruction among pupils

Stage 5: Q-form

At this stage, the typical distribution table in the variety of a Likert scale from − 5 to + 5 was created offline. Strategies on distributing the expressions on the standard distribution table have been supplied. In the 1st phase, the goal of the study is the number of statements chosen by way of the job interview. In the 2nd stage, location the statements in 3 columns: “I agree”, “I have no opinion,” and “I disagree. In the 3rd phase, the statements (mandatory) are distributed in the standard Likert distribution diagram (− 5 to 5+), describing the rationale for deciding on the two ends of the Likert scale from their issue of perspective and ultimately entering the demographic info. Hence, in Q, the sorting process is subjective [19]. In other text, sorting goods in the ordinary distribution enable just about every participant to current their inner viewpoint through sorting.

Phase 6: investigation and interpretation of things

Students’ details obtained from Q sorting have been entered into PQ-Approach software package model 2.35. The process of evaluation and interpretation was carried out in 3 levels: (a) identification of aspects, (b) conversion of components into factor arrays (c) interpretation of aspects working with variable arrays.

  1. A)

    Aspect Identification

The extraction of factors in PQ-Approach computer software was carried out through the subsequent sequential methods: (a) principal element assessment, (b) identification of latent variables, (c) varimax rotation and evaluation of loading aspects for unique values higher than 1.00, d) estimation of the percentage of variance described by the discovered things and (e) differentiation of interpretable things with at least two correlated Q kinds [22].

  1. B)

    Convert variable to component arrays

The correlation in between each individual Q form and one particular discovered element signifies the degree of conversation among the Q types and the determined components [19, 23]. The manual flagging in PQ-Strategy application was utilized for this research. The correlation coefficients of at least .364 have been viewed as as the minimize-off point (the absolute price of the aspect load is bigger than ((frac2.58sqrtN)). That factor load was 99{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} considerable, respectively, and the price of N was equal to the number of Q statements (N = 50). Sorted for recognized factors [24]. Requirements specified on a variable are utilised to produce a element array for that aspect. The aspect array represents the sorting of that factor (point of look at) making use of z-scores. The element array for each individual variable decided the degree to which each and every expression was in the spectrum, so a additional exact interpretation of each and every issue (subjectivity) was obtained in accordance to the place of just about every expression. (P-value< 0.05 vs. 0.01) is also determined from the Z score to distinguish expressions [25].

  1. III)

    Factor interpretation using factor arrays

Distinct Q expressions were identified, and factors were interpreted textually. The defining expressions for a factor were those that had a rating value of “+ 5”, “+ 4”, “4-,” 5- “in factor arrays that had different scores (P < 0.05) in a given factor Compared to their scores on other factors, the post-P-set interview was conducted at the end of Q sorting to confirm the diagnosis and interpretation of item subgroups among the identified factors.

Postgraduate Online Medical Education during the COVID-19

Introduction

Online learning (eLearning) was gradually incorporated into medical education over the past 20 years, which has paralleled the increased use of eLearning across all workforce sectors.1 A review published in 2006 concluded that eLearning would be “one of the most important developments in the delivery of postgraduate medical education.”1 The authors of that review article, and many others who shared similar views in the early internet era, could not have known how that prediction would be tested. However, as the world came to be immersed in the SARS-CoV-2 (COVID-19) pandemic in 2020, eLearning surely became important in the delivery of postgraduate medical education.

The COVID-19 pandemic caused a massive change worldwide; affecting all areas of workforce including education. This state of emergency has led to many modifications within the healthcare system, such as cancelling elective surgical procedures, reducing the volume of acute-care surgeries, closing all outpatient clinics, limiting the presence of trainees on service and calling-off departmental educational activities.2–6 This resulted in significant interruptions of clinical rotations. In addition, traditional in-person academic activities such as face-to-face teaching and simulation labs were halted; examinations, courses and conferences were postponed on an international level.7,8 Almost overnight, online learning transitioned from its status as a developing option to becoming mandatory if education were to continue.

The rapid transition from traditional face-to-face to eLearning has transformed the way medical education was delivered and posed many challenges to trainers and trainees involved.4,9 Synchronous and Asynchronous eLearning modalities have been utilized by several institutions during this period. Numerous platforms were utilized for delivering academic content; the most frequently used were ZOOM and Microsoft-Teams.10,11 Innovative teaching modalities took place, including the “flipped-classroom” method, where learners were provided with didactic materials and pre-recorded videos prior to the educational session.11–13 Other strategies were implemented to accommodate for teaching clinical skills and ensuring the continuity of clinical education, achieved through video-recorded surgical procedures as well as providing telehealth patient consultations.11 With regard to evaluation, the most commonly reported assessment method during this period was in the form of multiple-choice questions; other studies reported the conversion of the standard Objective Structured Clinical Examinations (OSCE) to an online version.11,14

Numerous obstacles and challenges have been reported as a result of this massive transition. Trainers, teachers, and educators were required to rapidly adapt to digital technologies; trainees encountered difficulties with poor bandwidth connectivity; accessibility and time management issues were evident, as well as communication challenges due to the lack of non-verbal language.11,15

The resulting global experience with online medical education is being shared primarily as information gathered from user surveys. Although quantitative data are essential, detailed qualitative data are as necessary today as they were in the early studies, to allow comprehensive and reliable investigations of this complex intervention comprising “multiple human components (teachers, learners, etc.) interacting in a nonlinear fashion to produce outcomes which are highly context dependent.”16

Accordingly, we distributed an online survey to postgraduate medical learners and teachers in Riyadh, Saudi Arabia. These data are expected to supplement the expanding total of literature, adding to the reported experiences and possibly contributing to the development of strategies that can resolve specific issues, gaps, and deficiencies in online postgraduate medical education. The aim of this study is to provide qualitative and quantitative assessments of postgraduate online medical education during the COVID-19 pandemic amongst trainers and trainees in Saudi Arabia.

Research Objectives

The research had three main objectives. The first was to describe the experiences, coping, perceptions, satisfaction and preferences for online learning by medical trainers and trainees. The second was to determine how the experiences correlated with perceptions, satisfaction, and preferences. The third and foremost objective was to test the null hypothesis: no difference between trainers and trainees on various aspects of online learning.

Materials and Methods

In this cross-sectional study, a questionnaire was developed through a review of recent publications on online learning during the Covid-19 pandemic in addition to the experiences of the researchers involved. At first, the questionnaires used in previous similar studies were carefully reviewed by the authors. Then, more questions were added based on the experiences of the researchers involved— whom were either active trainers, learning managers or decision makers during the pandemic. The researchers discussed the items of the questionnaire for relevance and finalized only 43 of them.

The questionnaire was pre-tested on a pilot number of potential respondents, with care being taken to exclude them from the main survey. The questionnaire comprised several sections addressing the sociodemographics of participants and measures of experiences, perception, satisfaction, and preferences. Table 1 shows the list of variables used in the research and their corresponding questions. Nominal variables were recorded as multiple-choice questions, while ordinal variables were scored based on a 4-point or 5-point Likert Scale.

Table 1 Variables of the Study

Institutional Review Board (IRB) approval was granted on 1 July 2020 from the local IRB at King Fahad Medical City—under category “Exempt” based on Good Clinical Practice (GCP) Guidelines. The questionnaire included a cover-letter describing the purpose of the study along with a statement of informed consent for research participation—which was developed in accordance with the local IRB guidelines. No participant identifiers were collected as part of our survey and the responses were anonymous. The involved researchers maintained adherence to GCP guidelines throughout the duration of the study.

The questionnaire was distributed by e-mail on August 21, 2020, to 1200 trainers and trainees of academic medical centers within the Riyadh 2nd Health Cluster, which included King Fahad Medical City, Prince Muhammad Ibn Abdulaziz Hospital, Al Yamamah Hospital, King Salman Dialysis Center and three specialized dental centers. The sample size was computed using the Raosoft online formula as 205; based on a total population of trainees and trainers of 1200, alpha error of 0.05, 95{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} confidence level, and a 2:8 distribution based on the estimated trainee-to-trainer ratio. A follow-up was made 1 week after the initial e-mail via a reminder e-mail and/or phone call. Data received were checked and edited for consistency and accuracy. Open-ended items were coded, except for 2 questions (Q18, Q38) which were analyzed qualitatively. The variable “position” was defined as either “trainee” or “trainer.”— “Trainee” was coded to include residents R1-R4, interns, fellows and pre-scholars; “Trainer” was coded to include consultants, assistant consultants, and program directors.

Statistical analysis was carried out using SPSS v.26 to cross-tabulate frequencies of the variables and test for association using the chi square statistic, with significance being set at 0.05 using 2-sided asymptotic p values. Both the Spearman and the Kendall tau correlation coefficients were computed with a critical value of significance of 0.05 and 2-sided p values. The main objective of the analysis was to determine and explain the differences between trainers and trainees. Where indicated, multivariate logistic regression models were used to test for association while controlling for confounding to determine independent associations.

Results

Study Sample Socio-Demographics (S1–5)

A total of 1200 emails were sent out, of which 207 were returned giving a response proportion of 17{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}. Table 2 shows the sample characteristics. There were significant differences between trainees and trainers in age, gender, household size and specialty, but only age showed independent association after running a logistic regression model containing all the socio-demographic variables.

Table 2 Sample Characteristics

Transition from Traditional to Online Learning and Changes in Institutional Policies, Procedures, and Support (Q1, 2, 9–14)

Table 3 shows reported transitions and changes due to the pandemic. A high proportion, 82.1{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, rated their pre-pandemic computer and internet experience as high or medium and there was no significant difference between trainees and trainers. One-quarter, 25.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, had no online learning experience before the pandemic, with a higher proportion among trainees. There was no significant correlation between their pre-pandemic computer and internet competency with pre-pandemic online learning experience. Age was not correlated with the pre-pandemic computer and internet competency but was significantly negatively correlated with pre-pandemic online learning experience (rS= −0.257, P < 0.000).

Table 3 Transition from Traditional to Online Learning and Changes in Institutional Policies, Procedures, and Support

There was a positive correlation between the reports of ‘redesigning teaching courses’ and “change of procedures to accommodate online learning” (rS= 0.360, P < 0.0001). Major or drastic changes were reported in both by 44.9{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} and 39.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of respondents, respectively. The trainees differed from trainers in reporting provision of guidelines before the start of online learning activities (P < 0.024). About 27.3{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of trainees reported inadequacy, while a higher proportion of trainers, 58.5{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, reported absence of guidelines. About 70.1{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of respondents reported a high level of institutional support for online learning with no significant difference between trainees and trainers.

A high proportion of respondents, 80.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, reported coping well or very well with online learning, with no significant differences between trainers and trainees. Coping was negatively correlated with age (rS= −0.151, P < 0.030), positively correlated with the pre-pandemic computer and internet competency (rS= 0.202, P < 0.004) but not correlated to pre-pandemic online learning experience (rS= −0.094, P < 0.177).

The transition from traditional to online learning was associated with stress. About 24.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of trainers reported extreme or major stress compared to 20.7{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} among trainees, and there was overall significant difference in stress between trainers and trainees. Stress was correlated with the pre-pandemic computer and internet competency (rS=−0.162, P < 0.020), but not with prior experience of online learning or with age; as (rs= 0.035, P = 0.562) and (rs= 0.045, P = 0.453), respectively.

Online Learning During the Pandemic: Experiences, Perceptions and Satisfaction (Q3–8, 15–17, 19)

Zoom was the preferred software by 94.2{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of respondents. All modern equipment were used equally by trainees and trainers; laptops, smartphones and tablets, with desktop computers being the least (9.2{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}). More than half of the respondents, 53.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, reported spending 4 hours or more per day on online learning activities, but there was no significant difference between both groups (P < 0.224). The majority of online learning activities, 92.3{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, took place at home during quarantine period, with no significant differences between trainees and trainers.

The 2 most popular learning activities were lectures and seminars/webinars accounting for a total 87.9{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of all activities, but trainees reported more lectures while trainers reported more seminars/webinars. Case presentations/discussions was the most popular form of assessment at 49.3{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, followed by short oral examinations 15.5{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} and online OSCE 12.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}; the rest of the assessment methods were rarely used. It is noteworthy that there were significant differences: trainees reported more case presentations/discussions, 55.8{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, while trainers reported more short oral examinations 26.4{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} and online OSCE 20.8{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}.

Overall perception of online learning was very positive or positive, totaling 73{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, with no significant differences between trainees and trainers. Overall perception was correlated with age (rS= −0.0213, P < 0.002), with stress (rS= −0.359, P < 0.00), with coping (rS= 0.672, P < 0.00) and with satisfaction (rS= −0.835, P < 0.000).

On a Likert scale of 1–5, 71.1{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of respondents were either satisfied or highly satisfied and 3.9{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} were very unsatisfied, with no differences between trainees and trainers. Satisfaction was correlated with age (rs = −0.136, P < 0.020), Pre-pandemic computer and internet competency (rs = 0.146, P < 0.016), stress (rS= −0.363, P<0.0001), and coping (rS= −665, P < 0.0001) but was negatively correlated to overall perception (rS= −0.835, P < 0.000).

While the response to the item on difficulties of online learning was 39.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, 15.0{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} reported technical problems as the most common problem, with trainers reporting 22.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} and trainees reporting 12.3{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}. A question on challenges and opportunities presented by online learning revealed that 39.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} agreed that online learning presented challenges and opportunities while 15{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} thought it did not; however, this item had a non-response of 42.5{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}.

Comparison of Online to Traditional Face to Face Learning (Q20–32)

Respondents were asked to compare online to traditional learning on 12 items using a 5 point the Likert scale. These items were highly correlated. In total, the proportion, of “strongly agree” varied between 19.8–47.8{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} among trainees and 8.2–35.3{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} among trainers. Significant differences between trainees and trainers were observed in 6 out of 12 variables. Learners were more satisfied with learner to teacher communication 35{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} vs 30.9{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}; teacher to learner communication 39{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} vs 17{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}; reduction in academic stress 72.1{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} vs 62.3{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}; overall satisfaction 57.1{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} vs 39.7{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, and academic stimulation 66.9{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} vs 75.5{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}. Trainers agreed that online learning had less stress 62.3{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} vs 72.1{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} and that it gave more time to teachers 75.5{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} vs 66.9{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}.

A wide diversity of subjective responses was given to the open-ended item asking for what was missed from traditional learning. The most common factors being interaction and associated items, like body language and engagement. Other subjective responses related to classroom dynamics such as good teaching, attention, understanding, explanation and concentration. Others related to the need for human contact such as commitment, passion and activity. The rest of the responses were more objective and measurable such as clinical practice, supervision, time, communication, verbal feedback, and workshops. Some responders said nothing was missing.

Preferences Regarding Online Learning (Q33–38)

The reports on preferred time for online learning activities showed variability with no significant differences between trainees and trainers. The highest was evening hours after work 30.4{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} closely followed by afternoon working hours 29.5{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, morning working hours 22.2{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, and night after work 15.0{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}. Weekends were the least popular 2.9{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}. Most respondents, 86{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}, preferred the duration of online learning activities of not more than 2 hours with no significant difference between trainees and trainers. Lectures and case discussions were preferred by 95{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of both trainees and trainers as the most effective online learning activities. Respondents considered multiple choice questions 54.6{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} and problem-solving questions 28.5{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} as the most effective assessment methods. With regards future preferences, 64.3{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} preferred combining traditional and online methods but a respectable proportion of 25.5{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} preferred continuing eLearning as the sole method of education. A negligible proportion of 9.7{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} wanted discontinuing online learning and returning to traditional methods.

Many respondents did not answer the item about what aspects of online education should continue post pandemic. Lectures and case presentations/discussion were the most mentioned, but these did not come with online learning. Five preferred returning to the traditional methods. One wanted “everything in online learning to continue post-pandemic”.

Discussion

Differences Between Trainers and Trainees

Online medical education is not new; however, the rapid switch to exclusive online learning worldwide required institutions to take a deep dive into what was previously considered a complementary educational tool. Although most publications are related to medical school education, several considerations and issues are common among all educational levels.11,17 Our research adds to the increasing number of reports documenting issues and perceptions in response to the transition to online learning during the COVID-19 pandemic.

The major null hypothesis of this study states that there was no difference between trainees and trainers on all variables relating to online learning within our study. Knowing the differences is important for tailoring future online activities to suit the abilities and expectations of trainees and trainers alike. Gender differences were not significant in our study; however, a generation gap was obvious from the data. The trainers being older differed in being more married, having larger households and specialties. Besides socio-demographic variables, trainees and trainers had significant differences on 11 variables that can be grouped as transition from traditional to online learning, online assessment activities, and comparison to traditional learning.

Transition to eLearning, Stress, Coping, Perception and Satisfaction

In the transition to online learning, trainees had more years of prior experience with online learning, which reflects the generation gap. The trainees, being younger, are well versed in computer technology and internet use compared to older trainers.

Several variables were correlated with age as the underlying determining factor. Previous experience of online learning by trainees (younger in age) has made their transition to pandemic online learning easier. Younger age explains better coping with online learning through its positive correlation with pre-pandemic computer and internet competency. The younger trainees had lower overall perception, which may be related to having less experience in learning methods and outcomes in general.

A recent literature review summarized barriers and solutions to developing and implementing online learning programs for medical students and postgraduate trainees in global settings; however, they were not complicated by the challenges of a worldwide pandemic.17 Time and infrastructure issues were 2 barriers during such comparatively relaxed environment in studies reported from 2006 through 2015. A pre-pandemic United States national survey of 214 internal medicine residency program directors reported that synchronous online learning was used by 40{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} and asynchronous learning by 72{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of programs.18 The asynchronous programs were considered to be more accommodating of resident schedules and duty hour restrictions. However, even in those non-urgent settings, faculty development was considered to be less than adequate (30{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) or non-existent (56{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) by 86{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of respondents. In our survey, 74{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of trainers reported being provided with few (18{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) or no (56{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) guidance prior to implementing online learning. Trainees in our study were more prepared; however, barely one-third (33{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) had received adequate guidelines before embarking on their online learning activities. Conversely, a survey study at the College of Pharmacy (COP) at King Saud bin Abdulaziz University for Health Sciences in Saudi Arabia reported that almost two-thirds of the students believed that the COP was well prepared for the complete transition to online learning during the COVID-19 pandemic.19 In our study, the trainees were more aware of guidelines before the start of online learning—they must have sought sources other than the trainers. Most likely, they looked for or demanded the guidelines.

As with any transition from the familiar to the new, the introduction of online learning was associated with stress. The trainers experienced more stress with eLearning, which is explained by their shorter experience with online learning before the start of the pandemic. A focus group study of 60 undergraduate medical students’ perceptions of online learning was carried out in Qassim region of Saudi Arabia during April and May 2020. The study reported that, similar to observations in most studies of online learning, technical issues were common barriers.15 The authors emphasized that providing technology training courses to teachers is essential. Deficiencies in these skills could contribute to stress during the transition period. In our study, almost one-fourth (24{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) of teachers experienced major or excessive stress while transitioning to an online learning setting; while only 4{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} had a negative coping experience once learning activities were underway.

Our results indicated that perception of online learning was positively correlated with coping, which is logical, and was negatively related to stress, which is understandable. Its negative correlation with overall satisfaction is explainable by the clash between the ideal of a high perception and the actual satisfaction from the reality of the experience. Indeed, as online teaching methods are being regarded as an efficient tool for learning, the quality of eLearning was expected to be comparable to traditional methods; and learning outcomes were not expected to be compromised.19 Most reports of experiences after the transition to online learning in medical education are also related to medical school education.20 For example, a survey study involving pre-clinical students was carried out at the University of California at San Diego during March and April 2020. They reported that, in general, students believed the quality of instruction and their ability to participate were negatively affected by remote learning.21 However, the short interval that has transpired since the start of the pandemic did not make-way for examining online learning outcomes achieved by students, with either positive or negative attitude towards it. Furthermore, in November 2020, 30 residents in a Mexican general surgery residency program (PG1-PG5) participated in a study surveying their experience after transitioning to online learning since April 2020.22 Although the academic and organizational level was considered higher than that provided by traditional learning, by 47{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} and 67{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of participants, most (57{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) were neutral about whether there was a concomitant increase in their academic performance; and whether the changes had been more useful for their training (53{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) compared with their previous training. A systematic review of 29 qualifying articles examined the impact of COVID-19 on all aspects of surgical training, including the transition to online learning.23 Acknowledging decreased hands-on surgical experience; patient exposure was ubiquitous, which was in some cases accommodated partially by simulations and telemedicine. Although two studies reported 65{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} and 82{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of trainees had favorable opinions of their online learning; in one study, 65{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of trainees believed that even their theoretical training had been negatively affected.

Online Learning and Assessment Activities

Trainees differed from trainers in preferences of learning activities. Trainees had a higher proportion of lectures, while trainers had a higher proportion of seminars/webinars. There were notable differences between trainees and trainers regarding assessment activities; trainees mentioned highest participation in case presentations/discussions, while trainers mentioned more short oral examinations and online OSCE. The differences in describing the activities that took place are difficult to explain, since they experienced the same learning activities. There is a possibility that the question item was not understood by one or both groups; they may have indicated their preferences instead of their observations or the preferences biased the observations. We have no data on the distribution of respondents by hospital. It is possible that trainers who responded to the questionnaire were from hospitals that practiced online learning differently from other hospitals to which most trainees belonged.

OSCE assessment was not commonly reported by our study participants. The fact that 15{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of teachers and 9{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of students reported it as an assessment method and did not recommend it for the future, suggesting that they may have had a negative experience. Although OSCE has been in use for many years, its effectiveness and role continues to be examined. Some pre-pandemic studies on OSCE assessments for medical students reported higher stress and difficulty levels compared to traditional assessments.24 However, positive experiences of OSCE use in medical school have also been reported. A small survey study in a teaching hospital in Dammam, Saudi Arabia, reported that 63{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of students and 80{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of faculty believed that OSCE provided a fair assessment of clinical skills; and approximately two-thirds agreed that it was an enjoyable experience.25 Postgraduate students have also reported positive experiences with OSCE. In 2012, a survey of 66 internal medicine residents’ perceptions of OSCE was examined in Saudi Arabia, after implementation of OSCE as part of the final clerkship exam in 2008.26 On a 5-point Likert scale with 5 indicating strong agreement, the mean score was 4.5 for items asking if the exam was well administered, well structured and if staff guidance was helpful. The mean agreement score for the items asking whether OSCE was stressful was 3.5 and was 2.3 for intimidating. The authors concluded that the overall perception of the residents towards OSCE was favorable and encouraging.

Prior to the COVID-19 pandemic, a small number of postgraduate training programs reported their experiences with virtual OSCEs.27 Subsequently, when the UCL medical school in London canceled face-to-face assessments in response to COVID-19, an online 18-station timed OSCE was convened.27 Assessments were similar to those used in traditional OSCE, including clinical communication skills, written communication, practical skills, examination skills, and professionalism. The authors shared 12 practical tips compiled from their experience and from the literature that can help in the design and delivery of online OSCE. The Harvard School of Dental Medicine developed an online OSCE during the COVID-19 pandemic using the Zoom eLearning platform, because it featured breakout rooms where private mini-sessions could be created by the host.14 Students signed-in and were allocated to their breakout rooms; then progressed through the rooms when the allotted time had passed. Most students thought the online OSCE was as successful as traditional OSCEs, and all students believed they were able to completely demonstrate their knowledge. Examiners also had positive impressions of the online OSCE and emphasized the importance of calibrations and run-throughs prior to launch. Technical issues were the only difficulties encountered. The authors believed there was value in moving the in-person assessments online in the post-pandemic era.

Comparison to Traditional Learning

Trainees differed from trainers on 6 out of 11 items comparing online to traditional learning. Trainees were more apt to agree that online learning was better for teacher-learner communication, reducing academic stress, overall satisfaction and academic stimulation, whereas the trainers disagreed. These disagreements are explainable by the phenomenon of the generation gap considering the experience with computers and the internet, which is higher in the younger trainees. Trainers agreed more than trainees that online learning gave more time to teachers. This is understandable because online learning saves transient time between events such as movement to and within the hospital.

A recent meta-analysis of studies reported from 2000 to 2017, which compared online with offline undergraduate medical education, reported that knowledge and skills were significantly improved with online learning.28 Although not all studies in the review signify that online learning was more effective, none concluded that online learning was less effective than traditional learning. A single-center US study after the COVID-19 pandemic began included 81 emergency medicine and internal medicine residents. They reported that participants preferred in class interactions with peers (85{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) and lecturers (80{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}); with 62{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} reporting decreased engagement with lecturers during online conferences.10 Residents were significantly more engaged in other tasks during online conferences compared with in-class attendance. In our study, 35{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} of participants disagreed that online learning was academically more stimulating compared with traditional learning settings.

Changes in communication opportunities may contribute to decreased acceptance or effectiveness of online learning. An Egyptian survey (N = 78) and focus group (N = 25) study examined faculty perceptions of medical school responses to the COVID-19 pandemic. They reported that communication issues between faculty and students led to student detachment.29 Almost two-thirds (63{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) of our teachers disagreed that online learning enhanced teacher-learner communication, and 57{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} did not agree that it improved learner-teacher communication. Although our students expressed more positivity, only 39{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} believed learner-teacher communications were improved in the online setting, and 35{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} believed teacher-learner communication was improved.

A survey of 538 clinical years (fourth through sixth year) medical students was performed in all medical schools across Jordan; less than 2 months after a state of emergency was proclaimed in response to the COVID-19 pandemic in 2020.30 Over half of the participants (62{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) reported poor interaction with instructors as a drawback, with only 14{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} reporting better interaction in an online learning setting. More students would prefer a hybrid approach in the future whether they were satisfied (22{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}), neutral (24{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}), or dissatisfied (29{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) with the online learning; and a return to traditional learning was preferred by more students dissatisfied with online learning (15{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) compared with neutral (4{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) and satisfied (1{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) students. The majority of our teachers and students advocated for hybrid (65{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) or online only (26{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) education in the post-pandemic period; accordingly, efforts to maximize the benefits provided by online education are warranted.

A cross-sectional study in India included 55 postgraduate surgery residents who were without previous exposure to online teaching; the study was performed 1 month after transitioning to online didactic training during the COVID-19 lockdown.31 The transition included an orientation program for all teachers and residents. Individual items on the quality of online teaching did not indicate perceived superiority of either online or traditional learning; however, the authors considered the overall quality perception of online teaching to be favorable. The participants in our study were almost exclusively involved in theoretical learning and teaching. Transitioning to online education in this capacity does not require the capabilities, infrastructure, and inputs that are required to provide online clinical training. Overall, one-half of our study participants disagreed that online learning was better for teaching skills, with a higher percentage of teachers sharing this view (63{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) compared with students (45{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}). Similarly, a survey study of medical schools was conducted in Libya in mid 2020; stating that over half (55{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db}) of the participants disagreed or strongly disagreed that online learning can be used for teaching clinical aspects of medical sciences.32

Recommendations

A review of 14 studies on adaptive processes to the COVID pandemic in undergraduate and residency programs concluded that re-modulating the educational approach provided positive opportunities for personal and professional growth.11 However, the authors acknowledged that these qualitative narrative studies did not systematically analyze the characteristics and results of the changes that were introduced. They also believed that many of the enhancements that were described would require economic inputs that are not achievable in many parts of the world. Another study also suggested that the shift toward online education may have a lasting positive impact.33

The historical use of online videos for teaching procedural skills to postgraduate medical learners was exemplified when a systematic review was able to identify 20 qualifying articles published between 2009 and 2019.34 In the pre-pandemic period, the use of online videos was considered complementary to the more traditional teaching models. Virtual lectures could allow expert educators to disseminate beneficial knowledge to programs that do not have that level of expertise. Within an institution, cross-disciplinary education may be facilitated. It is considered that on demand virtual asynchronous lessons can be an inexpensive way to improve both access and content quality.35 These authors also believed that cross-institutional virtual collaborations can be part of a low-cost time conservative strategy; providing specialized training that otherwise may not be available in the learners’ institutions.

Several recommendations have emerged as medical training had to be adjusted for the limitations of COVID-19 on a global scale. Telehealth initiatives have been launched and/or expanded in several settings. When permitted by their institutions, residents can benefit from participating in virtual patient visits, reviewing charts and engaging in patient counseling under the supervision of the attending physician.36 Studies for examining the benefit of incorporating telemedicine into resident curricula are warranted. A few programs have responded to the pandemic by creating virtual video-based clinical training.37 A US undergraduate surgical education curriculum developed in response to the pandemic, emphasized the importance of using an interactive live-streaming platform for surgical experience, as well as patient-facing telehealth visits.38 The authors concluded that their virtual surgical education could be expanded for use in the post-pandemic era. Accordingly, institutions could benefit from cooperating on the development of valid strategies to incorporate clinical training into their postgraduate educational programs. Cleveland Clinic in Abu Dhabi constructed a 3-level pandemic response approach for developing and facilitating interventions determined to be necessary to maintain residency training.39 Online didactic education was supplemented by converting rounds to virtual platforms. However, their detailed framework description awaits assessment of its effectiveness and resident perceptions and satisfaction.

It is evident that currently published studies must be examined for their contributions to online learning development and implementation strategies in the future. Surveys such as our study can provide the basis for undertaking additional studies; to identify and adopt creative methods for effective online learning delivery and assessment. Tracking and identifying student skill gaps has become even more essential in a setting with reduced clinical learning opportunities. Innovation opportunities triggered by the pandemic should be taken advantage of; virtual and augmented reality technologies may be particularly important for teaching practical skills such as emergency interventions and surgical techniques.

In summary, our study adds to the baseline provided by numerous other studies of online learning experiences during the first few months of the COVID-19 pandemic. These early publications should be followed with additional studies, where details of program design and modifications in response to these initial observations are shared and realistic assessments of program effectiveness are performed. The availability of adequate qualitative details can contribute to collaborative participation in developing standardized strategies that can overcome the challenges of the pandemic and increase the quality of medical education in the future.

Limitations of the Study

The actual study sample of 207 was adequate according to calculations, but the researchers had been ambitious sending out 1200 questionnaires to capture as much diversity as possible. The relatively low response proportion of 17{af0afab2a7197b4b77fcd3bf971aba285b2cb7aa14e17a071e3a1bf5ccadd6db} is explained by the general digital fatigue in the society, because people receive many research questionnaires through the social media. The sampling could have been more representative if it was stratified to account for differences between training hospitals and academic centers. The study could have been enriched by additional qualitative research on some variables to obtain more in-depth understanding of online learning, which is a new phenomenon.

Conclusion

The main finding of the study is the difference between trainees and trainers in their experiences with online learning. This is explained by the generation gap in the acquisition and use of modern technology between the older trainers and the younger trainees. Overall, there was high proportions of coping, perceptions and satisfaction with online learning. The majority of the respondents also preferred continuing online learning combined with traditional methods in the post-pandemic period.

Acknowledgments

Dr. Valerie Zimmerman for her great work in reviewing the manuscript.

Disclosure

The authors report no conflicts of interest in this work.

References

1. Harden RM. Trends and the future of postgraduate medical education. Emerg Med J. 2006;23(10):798–802. doi:10.1136/emj.2005.033738

2. An TW, Henry JK, Igboechi O, et al. How are orthopaedic surgery residencies responding to the COVID-19 pandemic? An assessment of resident experiences in cities of major virus outbreak. J Am Acad Orthop Surg. 2020. doi:10.5435/JAAOS-D-20-00397

3. Bansal P, Bingemann TA, Greenhawt M, et al. Clinician wellness during the COVID-19 pandemic: extraordinary times and unusual challenges for the allergist/immunologist. J Allergy Clin Immunol Pract. 2020;8:1781–1790.e3. doi:10.1016/j.jaip.2020.04.001

4. Ferrel MN, Ryan JJ. The Impact of COVID-19 on medical education. Cureus. 2020;12:3. doi:10.7759/cureus.7492

5. Ahmed H, Allaf M, Elghazaly H. COVID-19 and medical education. Lancet Infect Dis. 2020;20:777–778. doi:10.1016/s1473-3099(20)30226-7

6. Cipollaro L, Giordano L, Padulo J, Oliva F, Maffulli N. Musculoskeletal symptoms in SARS-CoV-2 (COVID-19) patients. J Orthop Surg Res. 2020;15:1. doi:10.1186/s13018-020-01702-w

7. The Royal College of Surgeons of Edinburgh. Statement on COVID-19 and its effect on surgical training RCSEd. Available from: https://www.rcsed.ac.uk/news-public-affairs/news/2020/march/statement-on-covid-19-and-its-effect-on-surgical-training. Accessed March 26, 2022.

8. AAMC. COVID-19 MCAT FAQs. Available from: https://students-residents.aamc.org/applying-medical-school/article/coronavirus-covid-19-and-mcat-exam/. Accessed March 26, 2022.

9. Daniel SJ. Education and the COVID-19 Pandemic. PROSPECTS. 2020;49(1):91–96. doi:10.1007/s11125-020-09464-3

10. Weber W, Ahn J. COVID-19 conferences: resident perceptions of online synchronous learning environments. West J Emerg Med. 2021;22:1. doi:10.5811/westjem.2020.11.49125

11. Giordano L, Cipollaro L, Migliorini F, Maffulli N. Impact of Covid-19 on undergraduate and residency training. Surgeon. 2020;19:e199–e206. doi:10.1016/j.surge.2020.09.014

12. Williams DE. The future of medical education: flipping the classroom and education technology. Ochsner J. 2016;16(1):14–15.

13. Schwartzstein RM, Roberts DH. Saying goodbye to lectures in medical school — paradigm shift or passing fad? N Engl J Med. 2017;377(7):605–607. doi:10.1056/nejmp1706474

14. Kakadia R, Chen E, Ohyama H. Implementing an online OSCE during the COVID‐19 pandemic. J Dent Educ. 2020;85(S1):1006–1008. doi:10.1002/jdd.12323

15. Khalil R, Mansour AE, Fadda WA, et al. The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia: a qualitative study exploring medical students’ perspectives. BMC Med Educ. 2020;20:1. doi:10.1186/s12909-020-02208-z

16. Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ. 2010;10:1. doi:10.1186/1472-6920-10-12

17. O’Doherty D, Dromey M, Lougheed J, Hannigan A, Last J, McGrath D. Barriers and solutions to online learning in medical education – an integrative review. BMC Med Educ. 2018;18:1. doi:10.1186/s12909-018-1240-0

18. Wittich CM, Agrawal A, Cook DA, et al. E-learning in graduate medical education: survey of residency program directors. BMC Med Educ. 2017;17:1. doi:10.1186/s12909-017-0953-9

19. Shawaqfeh MS, Al Bekairy AM, Al-Azayzih A, et al. Pharmacy students perceptions of their distance online learning experience during the COVID-19 pandemic: a cross-sectional survey study. J Med Educ Curric Dev. 2020;7:238212052096303. doi:10.1177/2382120520963039

20. Dost S, Hossain A, Shehab M, Abdelwahed A, Al-Nusair L. Perceptions of medical students towards online teaching during the COVID-19 pandemic: a national cross-sectional survey of 2721 UK medical students. BMJ Open. 2020;10(11):e042378. doi:10.1136/bmjopen-2020-042378

21. Shahrvini B, Baxter SL, Coffey CS, MacDonald BV, Lander L. Pre-clinical remote undergraduate medical education during the COVID-19 pandemic: a survey study. BMC Med Educ. 2021;21:1. doi:10.1186/s12909-020-02445-2

22. Gonzalez-Urquijo M, Gonzalez-Hinojosa DE, Rojas-Mendez J, Rodarte-Shade M. Transferring face-to-face sessions to virtual sessions in surgical education: a survey-based assessment of a single academic general surgery program. Eur Surg. 2021;53(2):55–59. doi:10.1007/s10353-021-00691-2

23. Hope C, Reilly JJ, Griffiths G, Lund J, Humes D. The impact of COVID-19 on surgical training: a systematic review. Tech Coloproctol. 2021;25(5):505–520. doi:10.1007/s10151-020-02404-5

24. Azim Majumder A, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019;10:387–397. doi:10.2147/amep.s197275

25. Alsaid A, Al-Sheikh M. Student and faculty perception of objective structured clinical examination: a teaching hospital experience. Saudi J Med Med Sci. 2017;5(1):49. doi:10.4103/1658-631x.194250

26. Alaidarous S, Mohamed TA, Masuadi E, Wali S, AlMalki A. Saudi internal medicine residents׳ perceptions of the objective structured clinical examination as a formative assessment tool. Health Prof Educ. 2016;2(2):121–129. doi:10.1016/j.hpe.2016.04.001

27. Hopwood J, Myers G, Sturrock A. Twelve tips for conducting a virtual OSCE. Med Teach. 2020;1–4. doi:10.1080/0142159x.2020.1830961

28. Pei L, Wu H. Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Med Educ Online. 2019;24(1):1666538. doi:10.1080/10872981.2019.1666538

29. Shehata MH, Abouzeid E, Wasfy NF, Abdelaziz A, Wells RL, Ahmed SA. Medical education adaptations post COVID-19: an Egyptian reflection. J Med Educ Curric Dev. 2020;7:238212052095181. doi:10.1177/2382120520951819

30. Al-Balas M, Al-Balas HI, Jaber HM, et al. Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: current situation, challenges, and perspectives. BMC Med Educ. 2020;20:1. doi:10.1186/s12909-020-02257-4

31. Srivastava V, Pandey V, Tiwari P, Patel S, Ansari MA, Shukla VK. Utility of real-time online teaching during COVID era among surgery postgraduates. Indian J Surg. 2020;82(5):762–768. doi:10.1007/s12262-020-02592-2

32. Alsoufi A, Alsuyihili A, Msherghi A, et al. Impact of the COVID-19 pandemic on medical education: medical students’ knowledge, attitudes, and practices regarding electronic learning. PLoS One. 2020;15(11):e0242905. doi:10.1371/journal.pone.0242905

33. Westerman ME, Tabakin AL, Sexton WJ, Chapin BF, Singer EA. Impact of CoVID-19 on resident and fellow education: current guidance and future opportunities for urologic oncology training programs. Urol Onco. 2021;39(6):357–364. doi:10.1016/j.urolonc.2020.09.028

34. Srinivasa K, Chen Y, Henning MA. The role of online videos in teaching procedural skills to post-graduate medical learners: a systematic narrative review. Med Teach. 2020;42(6):689–697. doi:10.1080/0142159x.2020.1733507

35. Smigelski M, Movassaghi M, Small A. Urology virtual education programs during the COVID-19 Pandemic. Curr Urol Rep. 2020;21:12. doi:10.1007/s11934-020-01004-y

36. Kwon YS, Tabakin AL, Patel HV, et al. Adapting Urology residency training in the COVID-19 Era. Urology. 2020;141:15–19. doi:10.1016/j.urology.2020.04.065

37. Pettitt-Schieber B, Kuo M, Steehler A, et al. Implementation and evaluation of eight virtual surgical electives for medical students during the COVID-19 pandemic. Am J Surg. 2021;222(2):248–253. doi:10.1016/j.amjsurg.2021.01.032

38. Chao TN, Frost AS, Brody RM, et al. Creation of an interactive virtual surgical rotation for undergraduate medical education during the COVID-19 Pandemic. J Surg Educ. 2020;78:346–350. doi:10.1016/j.jsurg.2020.06.039

39. Abdel-Razig S, Ahmad W, Shkoukani MA, et al. Residency training in the time of COVID-19: a framework for academic medical centers dealing with the pandemic. Perspect Med Educ. 2020:1–6. doi:10.1007/s40037-020-00622-z