Flipped Classroom Educational Model ( 2010-2019 ) : A Bibliometric Study

Nowadays, teaching and learning activity employing the flipped classroom model has an important position in the process of providing education. This research aimed at identifying and analyzing articles examining the flipped classroom model that has been published in several reputable international journals issued in the 2010-2019 timeframe, which was conducted using bibliometric studies. The research was conducted using a 4-stages systematic mapping method: (1) searching for articles using the Publish or Perish application in the Scopus database, (2) classifying the articles for the bibliometric analysis, (3) checking and completing the metadata of those articles, and (4) conducting bibliometric analysis using VOSviewer application. The bibliometric analysis produced seven findings, as follows: (1) the trend of flipped classroom publications continued to increase from 2013-2019; (2) the ten most contributive journals has published 88 articles by 2019; (3) the ten most cited articles has produced 1,155 citations; (4) the three highest order of author keywords most widely used in flipped classroom articles were flipped classroom, active learning, and blended learning; (5) author collaboration with strong links only occurred in 21 authors through one document; (6) institutional collaboration with strong links formed through 28 collaborating institutions; and (7) state statistics were formed into three clusters and spread across various countries through contributions from authors who were in charge of 456 institutions. The flipped classroom model can be concluded as an educational model that is currently popular among researchers.


Introduction
This article discusses the trend of using flipped classroom model in education from the perspective of bibliometric studies. With the development of technology in educational practices, it has encouraged the development of learning models that involve technology in the process of transmitting knowledge. Flipped classroom as an educational product that involves a touch of technology has become an integrated part of educational practice in various fields, such as in the language (Hsieh et al., 2017), mathematics (Ogden, 2015), medical education (Tang et al., 2017), music (Doi, 2016), etc. Flipped classroom is an educational concept where in the conventional teaching and assignments components of a subject are altered (Davis, 2016). Specifically, flipped learning is an educational method in which the direct instruction transfers from one group to another, where the resulting group is converted into a complex immersive learning experience as the instructor directs students through their implementation of principle and innovative research on the subject (Love et al., 2015). Students view educational material outside the classroom and address it with fellow students or adapt what has been taught in the classroom to new content in a flipped classroom (Shih et al., 2019a).
Recent research on Flipped Classroom has been carried out by several researchers including the following. Zheng et al. (2020) conduct pre-experimental research by selecting a seventh-grade Information Technology Course lesson as the learning content in a flipped classroom. Their research intends to demonstrate that participatory in-class learning in the flipped classroom can be conceptualized not only as a collaborative process of discrete actions but also as a structured activity. The whole research focuses on how to create an effective flipped learning comprehensively, and offers a new method for the classroom interaction analysis.
Recent research that specifically examining flipped classrooms from literature analysis (meta-analysis) has been conducted by Shi et al. (2020). Their research focuses on the importance of the instructions in the flipped classroom on the educational objectives for university students. Their research aimed at identifying high-quality observational scientific journals that investigate the educational objectives of university students, and using meta-analyzes to evaluate the optimal feasibility of flipped classroom guidance. Bibliography is obtained through searches in the Web of Science (WoS) database, the Education Resources Information Center (ERIC), and Elsevier ScienceDirect. Meanwhile, the analysis was carried out using Review Manager 5.3 software. Their research indicated that flipped instructions in the classroom could have a significant impact on the cognitive academic performance of university students as contrasted to conventional lessons. The analysis of the moderator variable indicates that the teaching method was the only important component analyzed that affects the development of flipped instruction in class. Such findings show that the flipped learning method in the classroom helped university students develop their cognitive performance through a wide analysis of the interdisciplinary research papers available at the moment. It was also found that the flipped classroom was more successful when instructors integrated personalized, productive and participatory instructional strategies. Al-Shabibi et al. (2019) have also conducted a bibliometric study of Flipped Classroom with a focus also on the issue of the effectiveness of flipped classroom. Their research is aimed at discovering the efficacy in educational objectives as per the flipped classroom approach. Their research evaluates the different factors that certain researches attempted to examine on the plan, its implication on those factors and the rationale for their beneficial influence. Results of the study show that the flipped classroom offered students with a more informative manner that will lead to greater educational performance and greater experience for learning and work environments in the 21st century.
Apart from the well-established and abundant literature, there has not yet been found a bibliometric study of flipped classrooms that is truly comprehensive in the last 10 years, especially for bibliometric studies that use Scopus as a single database. The researchers believe that bibliometric study is significant to map bibliography information of some certain field (Batanero et al., 2019a;Otchie et al., 2020). Therefore, this article aims at complementing other important aspects in the flipped classroom research map to obtain a more comprehensive mapping. In contrast to previous research, this research focuses on a bibliometric analysis of the flipped classroom education model by referring to seven aspects, namely publication trends, the most contributive journals, citation patterns, author keywords, author collaboration, collaborative institutions, and country statistics from authors who publish flipped classroom articles. Therefore, the research is conducted based on the following problem formulation.

Literature Review
Flipped Classroom is an incorporated teaching strategy which intelligently utilizes latest methods to provide education fulfilling the needs of the learner in the 21st century (Al-Shabibi et al., 2019;Hodges et al., 2015;Mortensen et al., 2015). The basic principle of a flipped classroom is to substitute explicit instruction with video content and to allow learners inside the classroom to concentrate on critical learning experiences with their teachers (Chen et al., 2014;Koponen, 2019). In this approach, learners are researchers by themselves by utilizing technologies successfully through studying beyond the school boundary, encouraging analytical thought, self-development, cognitive skills and collective work among students make a difference in the educational outcomes of the learners (Baytiyeh et al., 2017;Chen et al., 2019;Koponen, 2019;Shih et al., 2019b;Ye et al., 2019).
The flipped classroom is deemed one type of mixed learning, represented as a combination of any two teaching innovations (Caner, 2012;Eryilmaz, 2015;Radia, 2019;Said et al., 2013). Usually, teachers develop or implement virtual learning videos or notes for learners to learn outside of class, so that classroom instruction is made available to the students via immersive and higher-order exercises such as team conferences, problem-solving activities, and presentations (Hava et al., 2018;Jovanovic et al., 2019;Tütüncü et al., 2018;Yilmaz et al., 2017). The flipped classroom is also called the reversed classroom (Shih et al., 2019a). Flipping the classroom implies the activities that have typically taken place inside the school is reversed, and vice versa (Davies et al., 2013;Enfield, 2013;Kim et al., 2014;Lage et al., 2000;Strayer, 2012). Several researchers have claimed more explicitly that the following two elements must be used in a flipped classroom; computer-based teaching for students learning outside the classroom and immersive social learning experiences for inside classroom learning (Bishop et al., 2013;DeLozier et al., 2017;Hwang et al., 2015;Moffett, 2015).
Researchers have concluded that based on empirical evidence, there were some important benefits of the flipped classroom. First, learning is managed and paced individually as learners have access to non-class learning processes (Kim et al., 2014;Lai et al., 2016;O'Flaherty et al., 2015). Increased flexibility and adequacy indicate that learners have more ability to change their own educational process and take active participation in the learning process than they could in a conventional classroom (Bruff et al., 2013;Evans et al., 2016;Israel, 2015;Yousef et al., 2015). Second, relationships in the classroom are improved as the connections between teachers and students are always more involved and engaging (Adnan, 2017;Amiryousefi, 2019;Bergmann et al., 2012;Gilboy et al., 2015;Hantla, 2014;McLaughlin et al., 2014;Zou et al., 2018). Advocates say that their potential is to increase the chance for learners to directly participate in learning activities by reducing the amount of time when learners listen passively (Jovanovic et al., 2019;Qiang, 2018;Sletten, 2017). Third, the efficacy of flipped learning on academic performance has also been reported due to its development of adequate classroom time for student thinking and team sharing (Kong, 2014;Thai et al., 2017;Zainuddin et al., 2016).
By implementing the flipped educational method in the classroom, more resources will be expended on interactive learning events, such as community meetings, problem-solving exercises, hands-on tasks, experiential learning and other forms of social activity, as well as individual instruction rather than just teacher-led lessons (de Araujo et al., 2017;Nouri, 2016;Uzunboylu et al., 2015). Nevertheless, several problems were also mentioned for introducing a flipped classroom. Most of them being that teachers sometimes need time to plan more for a flipped classroom (Abeysekera et al., 2015;Arnold-Garza, 2014;Berrett, 2012;Blair et al., 2016;Davies et al., 2013;Enfield, 2013;Zainuddin et al., 2016). Another major challenge stems from the preference of some students towards a conventional and passive method of learning where information can be extracted to them (Berrett, 2012;Forsey et al., 2013;Touchton, 2015).

Methodology
This research employs a bibliographic design of studies using systematic and explicit mapping methods (Batanero et al., 2019b;Garza-Reyes, 2015;Hudha et al., 2020). Meanwhile, for the stages of bibliographic studies using four stages as

Search Procedure
This research used Publish or Perish (PoP) software as an application used for searching the database of bibliographic. Meanwhile, Scopus is the database source for searching bibliographies using the PoP application and is one of the largest databases providing peer-reviewed literature (Ballew, 2009), therefore, Scopus is chosen as the sole database in this research. Scopus contains a far wider variety of products than other repositories (Salisbury, 2009;Shareefa et al., 2020), and it accounts for about 70% more publications compared to WoS (Lopez-Illescas et al., 2008;Shareefa et al., 2020). Several criteria have been established for all bibliographies included in the analysis, which included the following three aspects: (1) the type of bibliography was only journal; (2) the title of the article has to contain "Flipped Classroom"; and (3) the search year was limited to the period of 2010-2019 (last 10 years). Since the searches on the Scopus database via PoP are limited to a maximum of 200 articles in one search, hence, the bibliographic searches were conducted annually in March 2020. Figure 1 depicts the process of searching bibliographies in the PoP application.

Figure 1. Bibliographic Search of PoP Applications
The results of bibliographic search were saved in the EndNote X9 application and saved into a CSV file, which were opened in the Excel application. The saved files were checked and equipped with metadata.

Bibliographic Filters
The bibliography selection and sorting were conducted using several criteria, namely: (1) containing the context about Flipped Classroom; (2) using English; and (3) published by established or reputable bibliographic database publishers. Each bibliography to be included or excluded from the bibliometric analysis process was examined by tracing it into the Scopus database extracted from the PoP application. The chosen bibliography type was only journal type. Some of the bibliographies appearing in the search process in the PoP application were not selected because they were in the form of Conference Articles, Erratum, Notes, Editorials, Reviews, Cloning, or articles that are not equipped with abstracts.
The initial search results using the PoP application resulted in 406 bibliographies, which were sorted into 346 selected bibliographies. There were 60 bibliographies that were not selected because they did not meet the established criteria. Table 1 presents the total number of bibliographies from each year resulting from searches through the PoP application.

Bibliography Completeness
To conduct the filtered bibliography analysis, the metadata was examined and completed. The examination covered aspects to be analyzed, including the title of the article, names of authors, agencies and countries, abstracts, author keywords, article links, publishers, and publication year. After the metadata was complete, the bibliometric analysis was conducted.

Bibliometric Analysis
The analysis of bibliometric was conducted based on the seven following aspects: (1) publication trends, (2) journals that published most articles about flipped classrooms, (3) articles that were most cited, (4) author keywords most widely used in flipped classroom articles, (5) author collaboration, (6) institutional collaboration, and (7)  . VOSviewer is operated to efficiently work with massive amounts of data and to offer a wide range of different visuals, analysis, and observations (Rafols et al., 2012;Van Eck et al., 2010;Van Eck et al., 2014). In addition, VOSviewer can create maps of publications, author, or journals based on platforms of co-citation or maps of keywords centered on distributed channels (Hudha et al., 2020). The file type inputted into the VOSviewer application for analysis is the EndNote bibliography file.

Publication Trend-based Analysis
The      Table 2 above shows that some journals listed in number 5-7 published the same numbers of papers (7 papers). Then, each journal listed in number 9-10 published 6 papers.       Table 3 shows that 10 papers with most significant impact were published in 2013-2016, in which in 2013 (three papers), 2014 (three papers), 2015 (two papers), and 2016 (two papers).

Author Keyword-based Analysis
The author keyword analysis was done by using the VOSviewer application. Figure 4 depicts 880 author keywords analyzed with a minimum size of three occurrences, 68 author keywords generated were strongly connected. The 68 author keywords were divided into 12 clusters which are presented by different colors. Cluster 1 (red) consists of 12 items, cluster 2 (green) consists of 10 items, cluster 3 (blue) consists of 9 items, cluster 4 (yellow) consists of 9 items, cluster 5 (purple) consists of 8 items, cluster 6 (aqua) consists of 4 items, cluster 7 (orange) consists of 4 items, cluster 8 (brown) consists of 4 items, cluster 9 (pink) consists of 3 items, cluster 10 (salmon) consists of 3 items, cluster 11 (lime) consists of 1 item, and cluster 12 (gray) consists of 1 item. Figure 4 shows that keyword using flipped classroom term was resulted in the most results. The keywords with results occurred in cluster 11 (teaching mode) and 12 (nursing education) indicated that research on flipped classrooms are still limited in that fields.

Author Collaboration-based Analysis
The results of author collaboration analysis using VOSViewer with the number of occurrences of at least one document, from 964 authors produced 21 authors who were strongly connected. Each author had 20 links and one document. Author collaboration was only formed into one cluster (red) consisting of 21 items. The author collaboration pattern was depicted in Figure 5.

Figure 5. Network Visualization of Author Collaboration
Figure 5 above shows that the 21 authors collaborated through one same document, hence resulted in 20 strong links. It shows less collaboration in flipped classroom studies, since the total numbers of authors analyzed are 964 authors.

Institutional Collaboration-based Analysis
The results of the analysis for institutional collaboration were depicted in Figure 6. Of the 445 institutions, with an analysis of the minimum occurrence of one document, 28 institutions were strongly connected. The institutional collaboration was divided into five clusters. The cluster 1 (red) consists of 7 items, cluster 2 (green) consists of 6 items, cluster 3 (blue) consists of 6 items, cluster 4 (yellow) consists of 5 items, and cluster 5 (purple) consists of 4 items.

Analysis Based on State Statistics-based Analysis
Country statistical analysis was performed using the GPSVisualizer application (gpsvisualizer.com) to get the coordinates of each place or country from the authors. The red pin shows the location of the coordinates generated through the Maps application (maps.co). Figure Figure 8 shows that the numbers of contributors of papers on flipped classroom consisted of three clusters. Specifically, a small cluster with 10 and less contributors, an average cluster with 11-100 contributors, and a big cluster with more than 100 contributors. The highest cluster was in the United States.

Discussion
Over the past ten years, the publication of established research results in Scopus indexed journals on the topic of the new flipped classroom began in 2013. Starting in 2013, research in the flipped classroom area had continued to show significant improvements until 2019, never even experiencing a decrease again in the number of publications. The total number of publications reached 346 articles. This indicated that flipped classrooms have become very popular in educational practice over the years. Twice the number of publications also showed a fairly high upward trend that occurred from 2014 to 2015 with an increase of 24 articles, and from 2017 to 2018 with an increase of 38 articles. This pattern of publication trends continues to increase while simultaneously demonstrating quite high research productivity. This rise may be induced by various causes, such as the size of the organization, organizational requirements, structures of incentive and reputation, and human psychological mechanisms such as a preference for intrinsic rewards (Kwiek, 2015(Kwiek, , 2016Leisyte et al., 2012;Ramsden, 1994;Stephan et al., 1992;Teodorescu, 2000).
The high number of publications is inseparable from the contributions of various journals that consistently map out flipped classroom research areas. The ten journals presented in Table 2 were the most contributive Scopus indexed journals in publishing articles about flipped classrooms in the last ten years. The journals selected by the researchers for publication also indicated the credibility and reputation of the journal including the credibility of the publisher. Although, there were only journal managers who were out of control in maintaining the quality and quantity of publications as happened in the Journal of Advanced Oxidation Technologies so that the discontents are indexed by Scopus (last indexed in 2018). This case was also a note for researchers who were dedicated to being able to choose a place of publication that has a reputation for research that is valuable and has the potential to have an impact on another research. Basically, the perception of the publication wherein research is published has an impact on how research is evaluated and has become an important factor of research performance (Campbell et al., 1999;Macdonald et al., 2007).
Of all the articles published, it can be identified which articles have the most impact on other studies. Citation is one indicator to measure the prowess of a scientist. As implied in the research from Qi et al. (2017) that extraordinary scientists were measured by the number of their quotes. Figure 3 presents that the articles about the flipped classroom with the highest impact came from articles published in 2015 with a total number of citations reaching 770 out of 31 articles. Specifically, the article with the highest impact was written by Gilboy, M.B., et al. A paper has described how to incorporate the flipped classroom and illustrating the misconceptions of this strategy across 2 collegiate nutrition classes to the students. Their results indicated that the majority of the 142 participants who passed the assessment favored the flipped method compared to conventional learning environments (Gilboy et al., 2015). This article also illustrates that research on the topic of flipped classrooms was quite developed in the field of nutrition studies. Judging from the title of the journal from Table 3, the research of flipped classrooms is also quite developed in the fields of Pharmaceutical, Physiology, Chemical, Medicine, and Information Technology.
In the publication of flipped classroom research results, some keywords were most often used by authors. Keywords reflect the meaning of the whole sentence and contain the essence of the sentence (Hao et al., 2014). In a research article, some of the keywords used by the researcher (author keyword) mean that they reflect the essence of the whole topic being studied. As summarized in Figure 4, the most frequently used author keyword in flipped classroom research was the 'flipped classroom' term of cluster 1 with 56 links and 201 occurrences. The second-largest sequence was the term 'active learning' of cluster 6 with 30 links and 35 occurrences. The third highest sequence was the term 'blended learning' from cluster 4 with a total of 19 links and 20 occurrences. Through these three author keyword terms, it can be indicated that the model or approach of the flipped classroom in its research was closely related to educational practices contains the principles of active learning and touches with blended learning mode.
Flipped classroom articles also showed the author collaboration practices. The evidence suggested that cooperation was driven by purposes such as the requirement for materials (financing and materials), enhancement of one's field skills and expertise, advancement of technologies and enhanced research accessibility (Eduan et al., 2019;Sooryamoorthy, 2014Sooryamoorthy, , 2016. However, based on an analysis result using the VOSviewer application, the practice of collaboration in the research of flipped classrooms with strong links was not high, only occurring in 21 authors out of 964 authors (2.18%). The collaboration pattern that occurred in the 21 authors is shown in Figure 5. Specifically, the collaboration occurred in the article titled "Results of a Flipped Classroom Teaching Approach in Anesthesiology Residents" published by "Journal of Graduate Medical Education." This data provided an opportunity for researchers to carry out collaborative research on the topic of flipped classrooms to enrich the element of collaboration between researchers in developing educational practices with flipped classrooms. Collaboration can enhance the careers of young researchers (Li et al., 2019;Qi et al., 2017).
In contrast to author collaboration, aspects of institutional collaboration showed a higher number of strong links. VOSviewer's analysis presented in Figure 6 showed that out of 445 institutions, 28 institutions form a strong collaboration map divided into five clusters. From each cluster, several universities were quite prominent in collaborating. For example, the University of California had the most collaboration articles from cluster 1 with 4 articles and 5 links, followed by the Harvard School of Dental Medicine with 3 articles and 1 link. Stanford University (9 links) and the University of Wisconsin (10 links) became the university that had the most collaboration articles from cluster 2 with 3 articles each. The University of North Carolina had the most collaboration articles from cluster 3 with 4 articles and the number of links 7. Stanford University School of Medicine had the most collaboration articles from cluster 4 with 2 articles and the number of links 8. The University of Northern Colorado became the university that had the most collaboration articles from cluster 5 with 4 articles and with the number of links 3. For certain circles, international collaboration could be done to pursue Top University because this aspect has become one of the assessment indicators of QS World University Ranking (OCallaghan, 2020). However, on a count basis, 28 collaborating institutions were still categorized as minimal from 445 listed institutions. This could be caused by financing problems and the benefits that can be obtained. As explained by Katz et al. (1997) that there was no mechanism to consistently quantify both the costs and advantages of partnership, and therefore no method to decide if the advantages truly exceed the costs.
The location of the authors traced through the coordinates of the institution is shown in Figure 7. The distribution of these authors has indicated that those interested in the flipped classroom research were quite widespread in various parts of the world. Through this author mapping, it can also be seen which countries were contributors to flipped classroom articles, otherwise, it can also be identified countries that were not involved in the flipped classroom research. The number of locations for each contributor based on region clustering is shown in Figure 8. The locations that were the largest contributors were included in the red area cluster with a total of 110 contributors. The second most cluster with 98 contributors is shown in yellow. The total number of yellow clusters was 311 contributors. Meanwhile, the third largest cluster with a total of nine contributors is shown in blue. The total number of clusters in blue was 35 contributors. The five countries producing the most flipped classroom articles were occupied by the United States of America, China, Taiwan, Australia, and Indonesia. This showed that research productivity in these countries was quite high with various influences such as the type of leadership in the institution and also the intrinsic or extrinsic motivation of academics (Ramsden, 1994). In Indonesia, for example, an increase in the number of publications indexed by Scopus in the last ten years showed a significant number, which in 2010 had 3,336 Scopus documents and in 2019 it reached 41,871 Scopus documents (Sinta, 2020). One of the factors influencing the increase was because researchers who successfully published articles in Scopus-indexed journals were given additional funding or incentives in the form of money from universities (Pratikno et al., 2018).

Conclusion
To conclude, there were seven problems asked at the outset, which can be answered as follows. First, publications about flipped classroom in Scopus indexed journals have continued to increase in number over the past ten years.
Second, 88 of the ten journals that produced the most articles have been published. The top-ranking journal published 16 articles, and the tenth-ranked journal published six articles. Third, most citations occurred in articles published in 2015 with a total of 770 citations. The most cited article was written by Gilboy, M.B., et al. in the field of nutrition studies with 238 citations. Fourth, the most widely used author keywords in the top three were flipped classroom, active learning, and blended learning. Fifth, author collaboration with strong links only occurred in 21 authors through one document. Sixth, institutional collaboration with strong links only occurred in 28 institutions. Seventh, the authors came from 456 institutions or locations spread out from 52 countries.

Suggestions
Research in this topic can still be expanded by using other applications and various other bibliographic databases to enrich bibliometric mapping in the realm of flipped classroom research. Bibliometric studies like this can be one solution for researchers when the field research has to be delayed due to Covid-19. Flipped classroom can be a solution for teaching and learning process focusing on student as center of learning. The opportunities that can be benefited by inter-institution collaboration on flipped classroom research.  Hudha, M. N., Hamidah, I., Permanasari, A., Abdullah, A. G., Rachman, I., & Matsumoto, T. (2020). Low carbon education: