The Applicability and Effects of Flipped Learning on 'Public Health Nursing' Courses
ⓒ 2021 Korean Society of Muscle and Joint Health
Abstract
Flipped learning is a novel teaching strategy for encouraging students to engage in the learning process. This study aimed to redesign the public health nursing course and examine the implications of flipped learning on learning outcomes, self-efficacy, and self-leadership in undergraduate programs.
A one-group, pretest-posttest design was used. A total of 80 students participated in this study. The flipped learning method was employed between April 14 and June 5, 2016. The data were analyzed using descriptive statistics and an independent t-test.
The self-efficacy of the lower 25% group based on academic performance was significantly increased; however, self-leadership did not show any change after utilizing flipped learning. Overall, 65.4% of the students were satisfied with their class.
Flipped learning was an effective strategy for students with low achievement. Despite these advantages, it was considered to reduce the burden of studying.
Keywords:
Flipped learning, Self-efficacy, Population health, Self-leadershipINTRODUCTION
Healthcare is currently one of the most rapidly and dynamically changing fields. As advances in information and medical technology increase, newly graduated nurses experience a gap between nursing education and practice. As a result, they struggle to update their skills and knowledge. Today, the explosive expansion of knowledge has led to "content saturation" of the curriculum. Nurse educators have responded to this by moving away from a content-laden curriculum to improve critical thinking (Forbes & Hickey, 2009). The traditional lecture-based teaching model is not an effective way for students to acquire new knowledge (Hattie, 2009). Therefore, nursing educators have emphasized the importance of active learning strategies that allow students to engage in the learning process and interact with the curriculum's contents via diverse activities that promote critical thinking (Forbes & Hickey, 2009).
Flipped learning is a relatively new instructional method that initially took place in a classroom setting but has recently moved to the online environment (Bergmann & Sams, 2012). This method provides students with interactive learning opportunities and activities such as problem-based learning, debates, presentations, concept maps, etc., within the classroom setting under a teacher's guidance (Herreid & Schiller, 2013). It alters classroom dynamics from passive to active and self-directed learning. Furthermore, indirectly delivering educational content, the lecturers act as facilitators who engage students in individual or group-learning activities by assimilating pre-class content during class time. Flipped learning has reported controversial and mixed results between traditional and flipped class settings (Geist, Larimore, Rawiszer, & Al Sager, 2015). Flipped learning helps students improve their educational outcomes, including academic achievements (Chao, Chen, & Chung, 2015), critical thinking (Burke, 2017), self-leadership (Lee & Park, 2018), and communication skills (Lee & Eun, 2016). However, some studies have indicated the prevalence of less satisfaction with the flipped class method because of the time required for preparation (Missildine, Fountain, Summers, & Gosselin, 2013).
Many flipped learning studies have been reported in nursing on clinical, practice-oriented subjects such as anatomy, physiology, and pharmacology (Geist et al., 2015; Missildine et al., 2013). There have been no studies published on flipped learning in public health nursing for undergraduate students. In the previous semester, this course was completed in lecture-based teaching and exams to evaluate goal achievement. Many students consistently complained that they had difficulty learning the role of public health nurses other than knowledge. Flipped learning in public health nursing is competence-based; it prepares students to acquire the ability to think about population health issues as a primary care nurse critically. Flipped learning results in more engaged and interactive learners.
Most previous studies have focused on flipped learning's overall effects as learning outcomes, such as academic achievements (Hu et al., 2017). The students with a low-or mid-level grade tended to increase their academic achievement after the flipped class, compared to high-level students (Park & Park, 2018). There has been limited research on student experiences from an educational, psychological perspective (van Alten, Phielix, Jassen, & Kester, 2019). Several studies have recently evaluated flipped learning effects, looking at various outcomes, such as learning motivation (Lee & Hwang, 2016), self-efficacy, critical thinking, communication skills (Lee & Eun, 2016), and satisfaction.
AlJaser (2017) proposed that self-efficacy may have a strong positive correlation (r=.697) with academic achievement; therefore, it can be considered a reasonable variable for evaluating the effects of flipped learning. Self-leadership, which is the motivation aspect of achieving one's objectives (Bryant & Kazan, 2012), leads students to move toward their self-designed goals while taking responsibility for their actions. Kan, Shin, and Kwon (2016) reported improvements in self-leadership after the application of flipped learning. However, some studies have shown that student satisfaction is even less due to increasing spending time, such as watching a pre-recorded class and prior book reading (Missildine et al., 2013; van Alten et al., 2019). Therefore, it is necessary to explore studies on the implementation process in flipped learning (Betihavas, Bridgman, Kornhaber, & Cross, 2016).
The purposes of this study were to 1) revise the current public health nursing course using flipped learning for undergraduate students and 2) explore the characteristics and effects of flipped learning, including self-efficacy and self-leadership, and student satisfaction.
METHODS
1. Study Design
A one-group, pretest-posttest design was used to examine the effects of implementing the developed flipped learning course.
2. Participants and Settings
Eligible students were third-year undergraduate nursing students enrolled in the "Public Health Nursing" course. Before the class began, the students were provided with information concerning the change to flipped learning. Further, any questions or concerns expressed by the students were addressed. Participants who gave free and voluntary informed written consent to participate in the study were included. If they did not want to participate in this study but wanted to experience the flipped class activities, they could join the class. However, those who did not complete the questionnaires were excluded. Among the 92 third-year undergraduate students, a baseline survey was obtained from 89 students. Six students did not complete both pre-post surveys, and two students were on leave of absence, and one declined the survey. Finally, 80 students were included in the analysis.
This study was conducted at D University in Korea between April 14 and June 5, 2016. According to the University's practices with which the author is affiliated, this course is in the form of a block lecture, and each session lasted two hours. The classes were divided into two, each comprising 40 students. The survey was conducted using questionnaires administered to 80 students.
3. Preparation and Support for Flipped Learning from the University
The author's University encourages its faculty to redesign courses to implement flipped learning. It provides technical support and guidelines based on those set forth by the Center for Teaching and Learning (CTL) at D University. The Learning Management System (LMS) was reconstructed to allow access to pre-class content at any time via computer and mobile devices. Before the course began, the faculty was provided with a CTL training program that included information on "How to make a pre-class video" and "Strategies for designing and managing participation activities for students."
4. Need Assessments
A need assessment was conducted to identify students' needs and explore learning experiences in a public health nursing course. Students who had previously taken this course (lecture-based course) were eligible to participate. In recruiting students, the author provided the study's aim through emails, the social network service, and selected those willing to participate in the interview. Six students were selected from the upper, middle, and lower academic achievement groups. The author developed open-ended interview questions (What is your general learning experience in the current public health nursing class? What difficulty did you face in the public health nursing class? What are your expectations or needs from the professor teaching public health nursing? What do you think about a new teaching method, including flipped learning? What kinds of support are needed to study public health nursing?). The author conducted the interview only once in a seminar room at the University, and it lasted for two hours. During the interviews, the interview took field notes.
5. Course Redesign Process for Flipped Learning
The course redesign process was conducted in accordance with the ADDIE model (Figure 1). ADDIE is a model of instructional system design (Branch, 2009). It comprises the following instructional design phases: analysis, design, development, implementation, and evaluation.
Change in the average attendance rates for the pre-recorded lectures according to the academic achievement.
In the analysis phase, literature reviews with flipped learning based on nursing were conducted to examine studies using flipped classes and implementing relevant strategies in higher education. In the design phase, class plans were developed for weekly classes over 15 weeks. Weekly learning goals were established based on the learning objectives of the middle and small classifications by subject area, as proposed by the Korean Academy of Community Health Nursing Society. During the development stage, the course was redesigned to encourage students to participate in class while using problem-solving and teamwork abilities actively. Based on the weekly lesson plans, detailed teaching plans for each class were developed, including pre-class, in-class, and post-class plans, following the flipped learning class design guidelines presented by the CTL. According to the week's learning objectives for the pre-class plans, the author developed new materials and related reading assignments each week. In the implementation phase, each class was carried out by the lesson plan. The students attended the 2-hours class and completed small group activities led by the professor.
Regarding pre-class online attendance, attendance was counted if the students attended the lecture within the professor's timeframe within the LMS program. Planned activities were suggested for developing in-depth learning through the sharing of knowledge with other students. These were in the form of problem-solving activities that were to be completed in small groups of 5~6 persons. When the students seemed to be disengaged, the faculty encouraged them to engage in classroom activities through pre-planned questions actively. Students' achievements were evaluated using a small quiz in the evaluation stage, and a guide for the last post-class was provided.
6. Description of the Module and Learning Strategies for Flipped Learning
The pre-class modules helped students actively participate in class activities because they could study the pre-recorded lectures and content beforehand. These modules included learning objectives, guidelines for required textbook reading, a pre-recorded introductory video about 20-minutes-long, and supplementary reading material. The module was developed by the author using the Office Mix program. For in-class learning, the students were instructed to watch a relevant lecture online before attending the class, and they took an exam on the pre-class content each time.
During class, the students were asked to watch the relevant lectures online before arriving at the class. These lectures averaged 25 minutes in length and consisted of narration (by the author) over a video file developed using the Office Mix program. The in-class module was designed based on two types of strategies (individual and group activities). For the individual activity, 3~4 quizzes were provided to each class via the smart communication tool to verify students' understanding of the pre-recorded lectures and increase interest and feedback. Moreover, the think-pair-share strategy with rapid and post response methods was used for group activities. The rapid response method requires faculty to present discussion questions to the students and facilitate student discussion regarding the question; then, they show the results to the other students during class. The students have 24~36 hours to answer the discussion question in the post response method, and then they submit their results online. After class, they were provided with a summary note and feedback regarding the post-response activity. In facilitating student participation in class, a small quiz was given in each session, and the results reflected each student's academic achievement.
7. Measurement
A questionnaire survey was administered to measure the effectiveness of flipped learning. It comprised questions on academic self-efficacy (28 items), self-leadership (18 items), learning satisfaction (7 items), and the general characteristics of the participants. The length of the pre-class, attendance, and the number of plays were reported to assess the flipped learning course results.
8. Self-efficacy
Self-efficacy was measured using an academic self-efficacy scale (Kim & Park, 2001). Context-specific self-efficacy can effectively predict the academic performance of a student in an overall academic situation. This scale consists of three subscales, each rated on a 5-point Likert scale: assignment difficulty preference (10 items), self-regulated efficacy (10 items), and self-confidence (8 items). Higher scores indicate higher self-efficacy. This scale has strong internal reliability (Kim & Park, 2001): Cronbach's ⍺=.84, .76, and .74 for the difficulty of assignment, self-regulated efficacy, and confidence, respectively. In this study, assignment difficulty preference was .84, self-regulated efficacy was .75, and self-confidence was .82. The overall reliability of the scale was .83.
9. Self-leadership
Self-leadership was measured using a questionnaire developed by Kim (2002) based on Manz's conceptual framework and initial questions (1983). It consisted of 18 items, including self-observation (3 items), rehearsal cueing strategies (3 items), self-goal setting (3 items), self-reward (3 items), self-punishment (3 items), and constructive thought (3 items). Higher scores, measured on a 5-point Likert scale, are associated with higher levels of self-leadership. Reliability at the time of development was .87, and the reliability in this study was .80.
10. Learning Satisfaction
Learning satisfaction was measured at the end of the class without a baseline check. The measure consisted of seven questions using a seven-point scale and one open-ended question provided by the CTL at D University. Questions about satisfaction, class operation, degree of understanding, improvement in interest, learning time, class participation, intention to retake, and willingness to recommend the class to others were asked. These are commonly used measures relating to the subjects for which the University carries out flipped learning. One open-ended question is, "What are the most important difficult things you have learned so far in this class?"
11. Academic Achievement
Academic achievement was measured using the final course score and grade point average (GPA). After implementing flipped learning, the students were divided into four groups according to their academic score: below 25%, 25~50%, 50~75%, and above 75%.
12. Ethical Consideration
The Institutional Review Board of D University approved this study (IRB -***15-03-007), the data were collected at the end of the class by a trained teaching assistant who was not otherwise involved in this study to ensure voluntary participation so that the students felt free to respond to the survey.
13. Data Analysis
Descriptive analysis was performed for participants' demographics and flipped learning-related variables, such as length of the pre-class, attendance rate, and several plays using SPSS 23.0 (SPSS Inc. Chicago, IL). A paired t-test was conducted on self-efficacy and self-leadership to compare the changes between the pre-course and postcourse surveys, according to the academic performance quartile (upper 25%, upper 25~50%, lower 25~50%, lower 25%). Finally, student satisfaction was measured at the end of the class using the LMS.
RESULTS
1. General Characteristics of the Survey Participants
In terms of survey participants, 80 students completed the 15-week course. The general characteristics of the participants are shown in Table 1. They were in their third year of college, and 76 (95.0 %) were female. Since a single school year was targeted, 67 (83.8%) of the students had a mean age of 22.5 years (SD 1.2); however, there were also transfers and returning students. Regarding the students' interest in public health nursing, 35 (43.8%) had a moderate interest, and 52 (65.0%) said they were satisfied with their nursing major. All students surveyed had never previously been involved in flipped learning classes.
General Characteristics of Participants(N=80)
2. The Findings through Focus Group
The needs collated through the focus group interview were as follows:
The students said, "The class was not bad because it did not need my efforts to join the course. However, sometimes the course culminated regardless of students' understanding, with a weekly plan prepared. In particular, those who had a low GPA had difficulty zeroing in their weaknesses, and those who knew theirs did not know how to make up for them. Moreover, it was found that a conventional lecture made the students passive and bored. The students needed to receive appropriate feedback from the professor at the end of the class and catch up.
3. Operational Characteristics of the Developed Flipped Learning Course
It was assessed using the length of the pre-class, attendance, and number of plays.
The pre-class length: The length of the pre-recorded lectures was between 20.5 and 35.0 minutes, and the average time was 23.42 minutes (SD 5.21). The students' average attendance rate for the ten online pre-classes was 7.3 (77.3%). Figure 1 shows the change in the average attendance rates for the pre-recorded lectures according to academic achievement. It was the lowest at the beginning of the first lecture and the highest for the fifth lecture. After that, it dropped, and all groups had their lowest rates right after the midterm exam, after which it tended to rise again. The first session had the lowest (54, 67.5%), while the fifth session had the highest (71, 88.8%).
Attendance: The students were divided into quartiles based on their final academic scores to identify a difference in pre-recorded lectures' attendance rates according to the students' academic scores. As a result, the attendance rates for the upper 25.0 %, upper 25.0~50.0%, lower 25.0~50.0%, and lower 20.0% were 87.5%, 82.0%, 74.6%, and 65.0%, respectively. The upper 25.0% showed the highest attendance rate. Overall, the pre-recorded lectures' attendance rates were significantly correlated with self-efficacy (r=.33, p<.05).
The number of plays: The average number of times the pre-recorded lecture was played was 2.3. There were differences in the academic scores. The average number of plays for the upper 25%, upper 25~50%, lower 25~50%, and lower 25% were 1.6, 1.3, 3.0, and 3.0 times, respectively. Regarding the reasons for repeated watching, insufficient understanding of the lecture (41; 50.3%) and pre-test review (28; 35.0%) were the most common.
Regarding the device enrollment of the pre-recorded lectures, 26 (32.5%) of the students attended the lessons using the computer only, 36 (45.0%) took the course via mobile devices only, and 17 (21.3%) took the course using both. The most common pre-class lecture time preference was "20 minutes or less" (66; 82.5%), while 7 (8.8%) answered "over 30 minutes."
4. The Effects of Self-efficacy and Self-leadership
Table 2 shows the effects of flipped learning on self-efficacy and self-leadership. Self-efficacy was 101.91±13.47 before and 105.04±10.74 after taking the class. Thus, it increased by 3.3, showing a statistically significant difference (t=-3.46. p<.001). The self-leadership scores did not show any difference. When comparing the differences in self-efficacy according to academic performance quantities, it was observed that the lower 25% group showed a significant increase.
Changes in Variables According to Academic Performance Before and After the Flipped Class(N=80)
5. Students' Satisfaction with the Flipped Learning Class
Overall, 49 out of 80 (64.5%) responded positively to class operation (Item 1, Agree: 43; Strongly agree: 6). Additionally, 45 (59.2%) of 80 responded that the flipped learning content was understandable (Item 2, Agree: 41; Strongly agree: 4). In particular, in-class activities were the most exciting and helpful (54.4%) (Item 3, Agree: 31; Strongly agree: 5). However, 73 (96.1%) students answered that they spent more time on this subject than any other (Item 4). The number of those who responded that they wanted to take flipped learning again was not very high (Item 6, 35.6%).
DISCUSSION
This study examined the effects of incorporating a flipped learning method into a public health nursing course that previously used a traditional lecture format for undergraduate students. This study's main finding is that students' self-efficacy, especially those with lower academic scores, significantly improved when the flipped learning method was used. This result is consistent with recently published meta-analyses, such as Hu et al. (2018) and Tan, Yue, & Fu (2017). They investigated the effects of flipped learning and its effectiveness for increasing academic achievement among lower or average performing students than higher-performing students (Lee, 2015). However, it is not easy to find more studies that detail the effects of flipped learning and explain how they differ based on the students' academic performance. Therefore, there is a need for further research to investigate students' characteristics that affect flipped learning.
In the present study, the mean score of self-efficacy was significantly increased from 101.91 to 105.04 (t=-3.46, p<.001), and the group with a lower 25.0% improvement compared to the other three groups. Self-efficacy, one of the predictors of academic achievement (Andrew, 1998) was reported to account for 8.0% of the variance in students' academic achievement (Chacko & Huba, 1991). This result supports that flipped learning is indirectly effective in low-or mid-level grade students, although this study was conducted without comparison.
Self-leadership showed no significant change after they had experienced the flipped class. This finding is not consistent with some previous studies (Kan et al., 2016; Lee & Park, 2018). Self-leadership is considered an important variable in learning outcomes because flipped learning requires students to prepare for their learning materials before their pre-class by using online instructional videos and text readings (Betihavas et al., 2016; Burke, 2017; Choi et al., 2015). Self-leadership facilitates the pre-class learning performance of students. However, this has not yet been studied. Furthermore, it is necessary to examine whether self-leadership is an appropriate outcome variable of flipped learning and other factors, such as moderating or mediating, influence flipped learning.
In this study, most of the nursing students were satisfied with the flipped learning method. However, according to a literature review, students were not always satisfied in a flipped class and achieved mixed results with the flipped learning method (Moracos, Islam, Yu, Banow, & Schindelka, 2015). Among the students in this study, 21 (27.7%) preferred not to take another flipped learning class again. After class, they reported that the time burdens, such as watching a pre-recorded lecture and reading the textbook before class and the need to complete a short quiz at every level, were significant factors. Thus, they did not want to take another flipped learning course. Some studies have reported that students and instructors were more reluctant about this pedagogy than the traditional teaching class (McLaughlin et al., 2014; Tan et al., 2017). Therefore, it is necessary to ensure appropriate engagement time for the pre-class class. In this study, the average time spent in these pre-recorded lectures was about 23.4 minutes, and many offered the opinion that a class is time-consuming if it is longer than 20 minutes. While taking the pre-class lecture, the students repeatedly stopped the video to take notes before watching. Thus, the time they spent learning was about one hour. According to a previous study, Gilboy, Heinerichs, & Pazzaglia (2015) recommended that advanced learning lessons last for only 10~15 minutes because online lectures can bore students. McLaughlin et al.(2014) provided pre-recorded lectures with an average length of 34.6 minutes. Choi et al.(2015) suggested 25~30 minutes and reported that it is difficult for students to learn for more than 40 minutes. In this study, students reported that less than 20 minutes was a suitable length for a pre-class lecture. Nonetheless, further research is needed to investigate a more convenient and effective pre-recorded lecture duration.
Building an online support system that enhances accessibility to pre-class video and in-class activities is necessary to increase the flipped learning effects. The University provides students with pre-recorded lectures under their preferences using either a computer or mobile device. Among them, 45.7% participated in the pre-recorded lecture utilizing a cellphone or tablet during their commute time. These students had a strong willingness to move from the computer to mobile learning upon resolving system errors. Additionally, it is necessary to supplement and maintain the University's information system to minimize mistakes in the use of mobile lectures. Some of the pre-recorded lectures were made PowerPoint presentations with an instructor talking head and edited the video by themselves. There are some problems in watching the pre-recorded video, including noise and sound problems. Some students complained about these kinds of problems in class. These problems may affect student learning. Besides, in the course redesign development stage, it was not easy to design in-class activities. There is insufficient information about the actual time and activities about in-class activities through a literature review. In this study, the author conducted small group activities, consisting of 5~6 persons in each group. One class consisted of eight groups. Each module lasted 2 hours, including small group activities (about 30 minutes) and group presentation (20 minutes), pre-learning test (10 minutes), and quiz feedback (10 minutes), summary (20 minutes). However, this is not enough time for group activities.
There were some significant limitations to this study. First, this study used a one-group, pre-post design without a control group because of ethical issues. The author did not assign the students to a traditional lecture group to increase research validity. To compare flipped learning effects, two comparison groups, namely one for the traditional class and the other for flipped learning, are desirable. However, it may have caused an ethical issue if students were assigned to a particular group. Therefore, only one group was assigned to both pre-study and post-study.
Second, the flipped class course in this study was conducted in eight weeks in a block format. Its progress was faster than that of a general class. Further studies are needed to investigate the retention effect of flipped learning, including long-term follow-up. Although these limitations exist, this study could serve as a good indicator of the need to expand new nursing education strategies.
CONCLUSION
This study presented revising process currently used public health nursing course for flipped learning and evaluated the learning model's effects. As known in the previous studies, students were satisfied with the teaching method overall. This study showed that those with lower academic scores significantly improved students' self-efficacy when the flipped learning method was used.
The author presents the following recommendations for designing flipped learning. Time burdens were a point considering for students. Students report that it takes much time to complete the flipped course; therefore, instructors need to develop appropriate engagement strategies to decrease time-consuming. Because the attendance rate in an online class is an indicator to monitor the students' participation, especially for students with low academic achievement, instructors constantly should have to check it. Also, the effects included in this study were based on a small number, one-group design, and short-period within eight weeks. Therefore, it is essential to share concrete strategies that can further enhance flipped learning courses' performances in the future and strategies to enhance students' participation in nursing education.
Acknowledgments
This study was supported by the research fund of Daegu University in 2018 (No. 20180112).
CONFLICTS OF INTEREST
The author declared no conflicts of interest.
References
- AlJaser, A. M. (2017). Effectiveness of using flipped classroom strategy in academic achievement and self-efficacy among education students of princess Nourah bint Abdulrahman University. English Language Teaching, 19(4), 67-77.
- Andrew, S. (1998). Self-efficacy as a predictor of academic performance in science. Journal of Advanced Nursing, 27(3), 596-603.
- Bergmann, J., & Sams, A. (2012). Flip Your Classroom: Reach Every Student in Every Class Every Day (1st ed.). Eugene, Oregon: International Society for Technology in Education.
- Betihavas, V., Bridgman, H., Kornhaber, R., & Cross, M. (2016). The evidence for 'flipping out': A systematic review of the flipped classroom in nursing education. Nurse Education Today, 38, 15-21.
- Branch, R. M. (2009). Instructional Design: The ADDIE Model (2010th ed.). Newyork: Springer.
- Bryant, A., & Kazan, A. L. (2012). Self-leadership: how to become a more successful, efficient, and effective leader from the inside out. New York: McGraw-Hill.
- Burke, S. M. (2017). Cultivating critical thinking using virtual interactive case studies. Journal of Pediatric Nursing, 33, 94-96.
- Chacko, S. B., & Huba, M. E. (1991). Academic achievement among undergraduate nursing students: the development and test of a causal model. Journal of Nursing Education, 30(6), 267-273.
- Chao, C. Y., Chen, Y. T., & Chung, K. Y. (2015). Exploring students' learning attitude and achievement in flipped learning supported computer-aided design curriculum: A study in high school engineering education. Computer Applications in Engineering Education, 23(4), 514-526.
- Choi, H., Kim, J., Bang, K. S., Park, Y. H., Lee, N. J., & Kim, C. (2015). Applying the flipped learning model to an English-Medium nursing course. Journal of Korean Academy of Nursing, 45(6), 939-948.
- Forbes, M. O., & Hickey, M. T. (2009). Curriculum reform in baccalaureate nursing education: Review of literature. International Journal of Nursing Education Scholarship, 6(1), 1-16.
- Geist, M. J., Larimore, D., Rawiszer, H., & Al Sager, A. W. (2015). Flipped versus traditional instruction and achievement in a baccalaureate nursing pharmacology course. Nursing Education Perspectives, 36(2), 114-115.
- Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior, 47(1), 109-114.
- Hattie, J. (2009). Visible learning: a synthesis of over 800 meta-analyses relating to achievement. New York: Routledge.
- Herreid, C., & Schiller, N. (2013). Case studies and the flipped classroom. Journal of College Science Teaching, 42, 62-66.
- Hu, R., Gao, H., Ye, Y., Ni, Z., Jiang, N., & Jiang, X. (2017). Effectiveness of flipped classrooms in Chinese baccalaureate nursing education: A meta-analysis of randomized controlled trials. International Journal of Nursing Studies, 79, 94-103.
- Kan, J. S., Shin, M. S., & Kwon, M. S. (2016). The effects of project-based flipped learning model on self-directed learning ability, self-leadership and learning competency. Journal of Fisheries and Marine Sciences Education, 28(5), 1478-1491. (in Korean).
- Kim, A. Y., & Park, I. Y. (2001). Construction and validation of academic self-efficacy scale. The Korean education Review, 39(1), 95-123.
- Kim, H. S. (2002). The relationship between self-leadership and job satisfaction of middle school teacher. Unpublished master's thesis, Soongsil University, Seoul.
- Lee, B., & Hwang, S. Y. (2016). Class experience of the students on 『Pregnancy, Delivery and Puerperium』 nursing course through flipped learning: Mixed method research. Korean Journal of Women Health Nursing, 22(4), 221-232.
- Lee, M. K., & Park, B. K. (2018). Effects of flipped learning using online materials in a surgical nursing practicum: A pilot stratified group-randomized trial. Health Informatics Research, 24(1), 69-78.
- Lee, S. (2015). The current status of Korean grammar education using flipped learning. The Language and Culture, 11, 221-262(in Korean).
- Lee, Y. S., & Eun, Y. (2016). The effect of the flipped learning on self-efficacy, critical thinking disposition, and communication competence of nursing students. Journal of Korean Academy Society of Nursing Education, 22(4), 567-576.
- Manz, C. C. (1983). The art of self-leadership: Strategies for personal effectiveness in your life and work. Englewood Cliffs, New Jersey: Prentice-Hall.
- McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N., Davidson, C. A., Griffin, L. M., et al. (2014). The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Academic Medicine, 89(2), 236-243.
- Missildine, K., Fountain, R., Summers, L., & Gosselin, K. (2013). Flipping the classroom to improve student performance and satisfaction. Journal of Nursing Education, 52, 597-599.
- Moraros, J., Islam, A., Yu, S., Banow, R., & Schindelka, B. (2015). Flipping for success: Evaluating the effectiveness of a novel teaching approach in a graduate level setting. BMC Medical Education, 15, 27.
- Park, E.O., & Park, J. H. (2018). Quasi-experimental study on the effectiveness of a flipped classroom for teaching adult health nursing. Japan Journal of Nursing Science, 15(2), 125-134.
- Tan, C., Yue, W., & Fu, Y. (2017). Effectiveness of flipped classrooms in nursing education: Systematic review and meta-analysis. Chinese Nursing Research, 4, 192-200.
- van Alten, D. C. D., Phielix, C., Jassen, J., & Kester, L. (2019). Effects of flipping the classroom on learning outcomes and satisfaction: A meta-analysis. Educational Research Review, 8, 100281.