A worldwide consensus has emerged regarding the benefits that can be brought to education through the appropriate use of evolving digitalization, including information and communication technologies. The range of possible benefits covers practically all areas of activity in which knowledge and communication play a critical role, from improving teaching and learning processes to better learning outcomes.1,2 Clearly, the way digitalization is used will depend on the subject being taught, the learning objectives, and the nature of the students.

The use of new technologies in education implies new teacher roles, new pedagogies, and new approaches to teacher education. The successful integration of digital technologies in the classroom depends on the teacher's ability to structure the learning environment to merge new technology with pedagogy and develop socially active classrooms, encouraging cooperative interaction, collaborative learning, and group work.3 These goals require a different set of classroom management skills, including the ability to develop innovative ways of using technology to enhance the learning environment and encourage technology literacy, knowledge deepening, and knowledge creation.

Digital competence has been defined as one of the eight key competencies for lifelong learning. The Council of the European Union4 recommends member states develop digital competence by promoting and taking advantage of appropriate digital learning strategies and environments in education. Digitalization rebuilds education, learning, and research and requires development of new skills from teachers. For example, recent studies among healthcare faculty have shown that the use of digital technology in teaching to ensure ongoing information exchange and to maintain contact between faculty and students poses a challenge.5 The need for changes in pedagogical practices, such as the use of digital technology, is highlighted. Digipedagogical competence requires pedagogical competence, the knowledge to utilize digital technology in teaching and learning, use of digital technology, and consideration of student-centered learning during digitalization without forgetting the ethical issues.

With digital technology, it is possible to improve efficiency in education and offer students better possibilities for distance and lifelong learning.6 The use of technologies, such as e-learning, computer-assisted learning, Web-based applications, virtual reality, and augmented reality, has increased learner satisfaction7,8 and motivation8,9 and developed students' problem-solving10 and critical-thinking skills7 better than conventional education methods11,12 even as it has been said slow and laborious work for teachers.9,13 Quite often, faculty feel a lack of confidence using new technologies in their teaching.14

The European Commission has published the Digital Education Action Plan,15 which covers schools, vocational education, and higher education. It defines how education can make better use of digital technology and how digital competencies can be supported. The report suggests supporting digital readiness by strengthening schools' digital capacities.15 Many studies have been conducted showing how to use various technologies in teaching and learning, but the consensus and summary of those studies are missing. Moreover, we lack summarized research knowledge about the technology-related aspects or conclusions of digital interventions, which we address in this review.

AIM

The aim of this review is to synthesize and describe published research related to digital learning interventions in higher education, referring to their technological outcomes. The focus is on technology-related aspects or conclusions that the study participants (students) and their teachers have considered beyond the learning outcomes. The questions that guided the review process were: (1) What are the reported positive and negative technological outcomes of digital learning interventions in higher education? and (2) What suggestions and recommendations did the authors of the reviewed studies present regarding the future development of digital teaching and learning, as well as related interventions?

METHODS

The scoping literature review was chosen because it is used to identify the nature and extent of research evidence16 and the gaps in the existing research literature.16,17 The methodology for this scoping review was based on the framework outlined by Arksey and O'Malley,17 which comprises five key phases: (1) identifying the research question; (2) identifying relevant studies; (3) selecting studies; (4) charting the data; and (5) collating, summarizing, and reporting the results. A descriptive summary and inductive content analysis were used to summarize and disseminate the findings. This review is intended to draw attention to the comprehensive coverage of the available literature and therefore does not exclude studies of lower scientific quality, as it is intended to provide an account of available research.17,18

Search

Articles were identified by searching five electronic databases: CINAHL (EBSCO); MEDLINE/PubMed; Eric (ProQuest); Scopus; and a Finnish database, Medic. A university librarian was consulted for searches. The searches were conducted on May 2018, including all articles from the beginning of 2015. The following similar search terms were used across databases: digital, mobile, virtual, games, social media, Web-based, Internet, information technolog*, ICT, communication technolog*, technology uses in education, educational technology, E-learning, electronic learning, online courses, teach*, learn*, education, higher education, universities, postsecondary education, interve*, teaching method, and teaching technique. Two searches were conducted in four international databases, the first including the search term "intervention" and the second including the search term "teaching method." In the Medic database, the search terms were digital, mobile, virtual games, social media, Web-based, Internet, information technology, ICT, communication technology, technology uses in education, educational technology, e-learning, electronic learning, eoppiminen, digitaal*, virtuaal*, mobiil*, opetus, teach*, learn*, education, and koulutu*. No additional manual search was executed because the number of existing articles was large. Criteria for inclusion and exclusion were established (Table 1).

Table 1 Search Criteria

Search Outcome

The initial search yielded 1252 titles, which were imported to the bibliography management tool RefWorks (Ex Libris, Jerusalem, Israel), and duplicate entries were eliminated, leaving 1159 titles. Based on the inclusion and exclusion criteria, two authors (M.S. and A.H.) scrutinized the titles, abstracts, and content to reduce the number of articles to 132. After the full-text articles were screened, 91 were accepted, of which five additional articles were excluded with consensus discussions between four authors (M.S., A.H., L.S., T.S.). Eighty-six articles were eventually included in the review (Figure 1).19

FIGURE 1. A flowchart of the process of study selection (PRISMA [Preferred Reporting Items for Systematic Reviews and Meta-analyses]).

Data Analysis

To obtain an overview of the selected articles, they were summarized in a data extraction matrix according to the first author, year, country, study purpose, academic discipline(s), main digital means of intervention, methodology or design, and data collection method(s). For the first research question about the reported technological outcomes of digital learning interventions, the outcomes were first divided into two categories: positive and negative. Then an inductive content analysis was conducted in the two categories. For the second research question, the authors' suggestions and recommendations for future interventions from a technological viewpoint were searched, mainly in the discussion, limitations, and conclusion sections, and sentences including the words and phrases "should," "need," "recommend," "to meet the needs," "can be used," and "in the future" were searched for and categorized inductively.

RESULTS

Characteristics of the Included Studies

Several characteristics of the studies (n = 86) included in this review were categorized and presented (see Supplemental Digital Content 1, http://links.lww.com/CIN/A69), including the study purpose, academic disciplines, main digital means of intervention, methodology and design, and data collection method(s). The studies were conducted in 30 countries on six continents (Asia n = 32, North America n = 24, Europe n = 18, Australia n = 4, Africa n = 2, and South America n = 3). Three studies were conducted in two countries and continents (Sweden and Hong Kong, United States and Lebanon, UK and Somaliland). Quantitative (n = 65), qualitative (n = 3), and mixed-methods (n = 18) studies were included, with various designs.

The studies represented six academic disciplines: health sciences (n = 44), social sciences (n = 13), humanities (n = 12), engineering and technical sciences (n = 12), physical sciences (n = 2), and life sciences (n = 2). One article included social sciences and health sciences (n = 1). The participants were students from these disciplines. Intervention duration varied from several hours to one academic year, although it was not reported in all articles, and in some articles, it was unclearly described. The categorization of technological outcomes, including authors' suggestions, is presented in Figure 2.

FIGURE 2. The categorization of results.

Positive Technological Outcomes

Positive outcomes were sorted into seven categories: (1) effective and participatory form of learning, (2) individualized and self-paced learning, (3) motivation to learn, (4) resource saving, (5) satisfaction with the technological method, (6) automatized technical solutions enabling learning and teaching, and (7) enabling a wide learning environment. As a most often mentioned outcome, digital formats of learning were seen as an effective and participatory form of learning.20-52 Although learning in a virtual environment can be time consuming, the engagement and effectiveness were reported as high.28,33,40 Improved confidence,27,41,46 reduced anxiety,41,46 enhanced active participation21,24 and problem-solving capacity,42 augmented analogical reasoning,20 and stimulated information retrieval skills23 were mentioned as benefits of digitally derived education. Digital devices allowing for synchronous and asynchronous interaction21,22,35 and multiple opportunities to use text, audio files, and videos22,26 or to have discussion environments integrated with contents,35 as well as the game's repeatability,50 were seen as advantages. Having the possibility to contribute to co-construction of knowledge with a teacher or with peers was expressed positively.21,22 as was the possibility of sharing material with others.31 Furthermore, learning clinical topics and complex treatment protocols29 and the ability to use learned skills in various future situations23,25,51 in a clinical setting during the studies40 were emphasized. A digital method combined with traditional learning was suggested as useful in reinforcing learning.38,45,47 The digital method's effectiveness in professional training of students26 and its ability to encourage the students to take risks (such as in practicing a foreign language) while learning the issue27 were highlighted. A platform supporting interactivity also was reported to help the students share their thoughts and exchange ideas, even for those not used to doing so publicly,39 and digital technologies were seen as diminishing differences between the students, thereby increasing equality in learning.21

The students mentioned individualized and self-paced learning often.32,34,36,39,41,52-62 It gave them access to the relevant information outside the allocated teaching period41,52,53,55,57-62 and unrelated to the place,32,41,55,56,58,61,62 thus increasing their autonomy.34,58 In case of individual learning-based problems, the mechanism of the tool in identifying problems through statistics was considered helpful.36 Students also valued meeting the various needs and preferences that the digital forms of learning made possible.32,54,59

The use of digital resources enabled a wide learning environment.23,36,59,63-65 This finding was manifested, for example, in the educational module's strength in providing authentic clinical scenarios65 and presenting real-world problems23 or using videos, which gave learners access to additional material.59 Moreover, a digital tool complementing theory material was considered beneficial.36 The fact that a small mobile device can contain persuasive technology for educational interventions was considered innovative, flexible, and promising.63 Some interventions contained a variety of online classroom strategies, providing a large set of methods.64 Based on the students' feedback and teachers' observations, the digital method used provided students a new way of learning and increased their motivation to learn.23,28,30,32,33,36,38,41,42,46,49,60,63,65-68 They said learning was more exciting, paid more attention to instruction, and were more interested in the class.30 The students were even reported to show extra involvement in their learning due to their high interest in game-based learning.28

Automatized technical solutions enabling learning and teaching and instruction being integrated into the content54 had clear advantages,31,33,36,41,53-55,62,65 as the students, for example, mentioned the benefit of immediate or provided timely feedback,31,36,53,55,65 and teachers emphasized the advantages of the automatic-correction technique.36 Standardized content was also considered beneficial, minimizing the variation in teaching.62 In general, students indicated their satisfaction with the technological method of use.29,35-37,44,51,52,58,63,69-72 The satisfaction was expressed regarding usefulness in learning,36,44,51,70,71 general user friendliness, and ease in using the tool.37,44

Digital learning was also seen as a way to save resources,24,32,33,36,45,48,50,52,55,70,73-75 as some learning events could be arranged in a seminar room instead of simulation facilities,50 or using free24 or relatively low-cost48 digital applications for learning. In many cases, digital learning allowed students living far from campus to study from home32,45 or from their home countries rather than traveling abroad.74 Also, for example, students listened to audio files during their free time or when traveling.52

From the teachers' perspective, some digital learning platforms planned short and standardized, reducing the running time and lecture preparation.75 Teachers' reduced time in monitoring and correcting students' problems was considered important because the digital tool monitored the students' progress continuously.36 From the educational organization's perspective, running costs and contributions to savings of revenue were identified,32,73 as was the relatively small number of faculty members teaching.70 Also, one digital device's smaller cost in comparison to another's was noticed.33

Negative Technological Outcomes

Outcomes that represented negative aspects were sorted into five categories: (1) technical problems, (2) need for resources, (3) degree of difficulty, (4) insufficient instructions and feedback, and (5) undesirable structure and design. Technical problems were mentioned most often as a disturbing factor in learning interventions.51,67,73,74,76-83 The quality of the sound,73,76,82 small screen size,78 or other quality problems in visual presentation73,82 were reported, as were poor Internet connections67,74,77,79 or lack of Internet access outside the university campus.79 Managing multiple devices was reported as difficult,81 as was using all the digital environments' components.83 Moreover, a need for resources was reported.28,38,51,52,83-85 In addition to the labor and financial resources that are needed to develop a digital application or e-learning material,38,85 virtual teaching requires time and effort.52 Time is also needed for learning,28 as well as time and support to enhance knowledge and skills necessary to use the digital platform before its implementation.83 Some digital applications require ongoing Internet connectivity or a fast or at least sufficient Internet connection.51,84

The learning module's degree of difficulty42,51,80,86-88 was manifested in various ways. Regarding games, the students can become uninterested due to low demand for or inconveniences in playing the game87 or experience insufficient and too simple feedback from the online tool.80 On the other hand, the system can also be too complex, requiring lot of learning before one can use the tool.51 Moreover, the game can turn out to be insufficiently realistic to create a sense of immersion,42 and as the authors indicated, the games created by specialists can restrict the students' creativity, or the students are able to memorize the game's content.86 In general, the students may feel that the module is too overwhelming and not adequately targeted to them.88Insufficient instructions and feedback30,58,88 from instructors was reported; learners hoped for more support from teachers58 and appropriate feedback,58,88 as well as more guidance in the learning platform.30 Eventually, the students were discouraged from participating by the learning platform's undesirable structure and design30,65,88; some students also indicated their own preferences in using mobiles, such as viewing landscape versus portrait views or using another user interface.30

Suggestions and Recommendations From Authors of the Reviewed Studies

Based on tested interventions, the reviewed articles' authors gave their suggestions or presented recommendations for future digital teaching and learning and related interventions. They were divided into five categories: (1) learning environments; (2) learning/teaching techniques, applications, methods, and devices; (3) teacher activities; (4) learner resources and characteristics; and (5) duration and timing of intervention. Suggestions related to learning environments36,45,52,83,89 varied. The importance of an authentic learning context (eg, a museum) was highlighted,83 but a fully equipped laboratory for teaching language was considered unnecessary.52 Authors also pointed out that using only the game to teach demanding tasks is difficult,36 indicating the need for more versatile learning environments. Furthermore, authors recommended the use of technological platforms that students already use,89 along with paying attention to collaborative aspects of interaction in mobile learning environments.83 Last, the virtual learning environment was recommended to train students based at remotely located institutions.45

Concerning learning/teaching techniques, applications, methods, and devices,23,25,29,34,41,42,47,48,52,56,79,83,84,86-94 the importance of selecting an appropriate tool or method for a certain group of students was mentioned.52 Furthermore, assessing the impact of mobile learning tests on student performance89 and adding more features in assessment90 were proposed. Future developments for the tested application or platform included adding transitional stages between components86; increasing attractiveness, using an access monitoring system, and assessing individual performance87; including listening,34 practice exercises,91 or cases29 in the platform; adding repetitive features and feedback,42,91 interactivity,91 or more intervention characters94; developing additional video recordings47 or free access to them41; and adding voice recognition48 or multimedia.29 Moreover, a combination of several applications or content-learning methods was suggested,42,94 as was using multiple types of applications for the students to learn before they entered the workforce.89 The authors also emphasized the importance of the developmental phase of application; for example, it is important to pay attention to instructional design and testing, as well as how the learner is guided effectively to the content and given meaningful feedback.88 Improving the ICT infrastructure79 and high-quality Internet connections84 were recommended. Some comments were related to gathering more learner-related analytics data.23,83,92 Some suggestions were contradictory. For example, some suggested that faculty should be cautious when selecting an application to make sure it can be used to meet the students' learning objectives,56 and others stated that concern over learning outcomes alone should not dissuade educators from employing games,25 implying that digital methods can also help enhance skills besides the stated learning outcomes. According to Millis et al,93 it is essential to know how the game-like elements contribute or detract from learning, because student engagement in the learning activities is important.

Teacher activities32,37,40,52,53,59,62,72,79,80,82,83,88,95-99 contained several suggestions, such as capacity building for lecturers to adopt a blended learning approach79 or devoting time to learning and gaining experience in running webinar sessions.82 The authors emphasized that the benefits do not depend only on the technology but on how well it is used to promote reflection in the learners,37 and they strongly recommended teachers redesign their courses and teaching methods when moving from traditional to digital presentation methods to achieve satisfactory learning effects.97 A teaching model for using certain applications was recommended.52 Many suggestions concerned the guidance or feedback,32,40,53,72 and advice for setting up virtual office hours where students could ask for teacher assistance was given.59 Besides feedback, technical support and content expertise were also considered important.80,88 A virtual interaction to enhance reflection with the expert teacher was suggested during the student field trips.83 Authors presented some advice for the selection of contents for educators96 and made remarks related to a lack of quality criteria for the application (app).95 Authors also suggested that educators meet students' needs by choosing multiple learning strategies or methods to teach them (medical) content,62,98 allowing customization of learning, thereby increasing students' motivation,53 or having Web-based learning available as on-demand.99

Learner resources were also emphasized.73,91,94,100-102 Fleming et al100 mentioned that it should be kept in mind that not all economically disadvantaged students have digital resources off campus. Similarly, Carlson et al73 brought up financial issues and family considerations but favored ICT solutions to help with internationalization at home. Adjusting teaching methods to serve students' needs and profiles was considered important in some articles,91,94,101 and paying attention to students' possibly inadequate computing skills when they enter universities was brought out.102

Park et al103 brought up the need for sufficient time designated for intervention, stating that the extension for the education period and repeated application would be needed in the future. Similarly, Lee et al41 pointed out the duration of intervention to improve the students' skills and knowledge competencies and suggested more measurement during the intervention, such as finding out how many times or how frequently students watch the video and the frequency with which it affects the outcomes. Time between the performance test and end of intervention was considered important,42 as was the experiment's duration.80 Hence, duration and timing of intervention and its activities41,42,80,103,104 were considered important factors in the future. Last, Thalluri and Penman105 presented a checklist for the use of Facebook for learning and teaching, examining the application from a variety of aforementioned perspectives.

DISCUSSION

The findings in this study contribute to the body of knowledge about digital interventions that are conducted among a variety of disciplines in higher education. The number of included studies (n = 86) was large, and their geographical area broad, implying that many interventions are being conducted globally and the need to find alternative or complementary technology-based methods or digital devices for already established teaching is evident. The digital learning interventions contained a variety of content, resulting in a mixture of digital methods, which were implemented in a various innovative ways. Based on the results, the positive outcomes outnumbered the negative ones, and the authors presented plenty of suggestions and recommendations based on their experiences with digital learning interventions.

Several articles reported the perceived effects of digitally enriched teaching. Digital learning methods encouraged students to take risks and enabled them to repeat the learning content to increase their skills and knowledge. Practicing communication or clinical skills with virtual patients or rehearsing challenging situations using a serious game can complement theory-based learning. However, we might ask whether combining multiple digital methods to obtain learning objectives is effective from the viewpoint of resources from teachers and students, as we can learn from these articles.

One of the most visible advantages of using digital platforms was the possibility to have control of one's own learning. The independence of the time and place of studying is essentially important to students who live far from their campuses but was also valued by other students for giving them the freedom to study when it best suited them. Using mobile devices in learning while traveling or during free time shows goal orientation and can eventually even influence learning outcomes. One interesting feature from the viewpoint of students and teachers was the automatized technical solutions that were built into applications or platforms. They allowed immediate feedback for students and provided teachers real-time information on students' performance. Although these articles do not go deeper into learning analytics, it is useful to know that applications are able to produce information about students' processes and are an evolving area of practice.106 The question is, are teachers skilled and motivated to use this information and how can they use it to benefit learners? Therefore, educational organizations are encouraged to offer digipedagogical support to teachers to gain full benefit from learning analytics to further develop digital learning and teaching processes.

Negative outcomes, although far fewer than positive ones, are still important for educators to reflect on. Technical difficulties, which were the most criticized in included articles, are very common and stem from variety of reasons. Irrespective of their origin, they caused students to have decreased learning experiences or, in worst cases, no learning experience at all if, for example, the Internet connection did not work. Such problems can decrease the motivation to study or cause feelings of inequality. Not only the students are harmed by inoperative devices or connections; every teacher knows the feeling of being unable to help students in need, and, in many cases, they have to give students a replacement task (and read it) afterward. When the synchronic learning situation requires student interaction, as described in many of these articles, or even live international collaboration of students, that experience cannot be replaced easily. Another important reminder for educators is the students' need for sufficient feedback and support in digital learning environments. These interventions tested digital means of learning, and it is assumed that the teachers were active in promoting learning. Hence, the students' complaints of insufficient instructions and feedback were fairly low.

The authors of these reviewed studies highlighted multiple, important issues that are good to keep in mind when the teaching is digitally modified. These issues concerned a wide range of factors from environmental opportunities to learner resources. Even though most suggestions were related to the methods or related technologies, the teacher's role and responsibility were highly emphasized. Thus, a competent teacher, who has technological and pedagogical skills,14 is always key in facilitating student learning processes, implying that up-to-date knowledge and skills must be a requirement for educators at all levels.107

Almost 10 years ago, UNESCO3 emphasized the use of new technologies in education. According to the results of this review, it can be said that this aim was fulfilled. A vast number of interventions have been conducted presenting varying integrations of digital technologies into teaching. Almerich et al14 disclosed that teachers feel a lack of confidence when using new technologies in their teaching. According to this review, however, we can say that teachers are willing to test new technologies.

LIMITATIONS

As the number of the included review articles was substantial, a quality appraisal was not conducted. As Arksey and O'Malley17 point out, it is not a "must" in scoping reviews. A positive outcome of not setting strict criteria can increase the variety of methodologies and range of study designs, which allows for a more comprehensive review of the digital intervention literature. The gray literature was not searched due to the large volume of selected studies.

Second, although this review did not specifically favor health-related interventions, more than half of the studies (n = 44) were from the area of health sciences. Thus, the outcome of the review was dependent on the databases selected for this review, of which three were related to health (CINAHL, PubMed, Medic). Searching additional or more varied databases might have identified additional relevant studies and balanced the variety of disciplines. However, as the aim of the review did not include deeper analysis of content, examining digital learning interventions from the point of presenting an overview of the existing literature was deemed feasible.

Third, this review communicates the self-reported technological outcomes of students and teachers described in the digital learning intervention articles. The exploration of the learning outcomes was left out to be reported at later phase. Therefore, a comprehensive synthesis of the literature reviewed was not a goal. The balance between breadth and depth of analysis was decided among the authors.

Fourth, the technological outcomes and authors' suggestions and implications for future digital learning interventions from a technological viewpoint may overlap slightly, as they sometimes represented fairly similar contents. In addition, authors of reviewed articles may have proposed further insights or recommendations for readers, but all of them were not interpreted as suggestions related to future digital learning interventions or from a technological viewpoint. Furthermore, very broad and general suggestions-or very detailed ones-examining specific technological solutions were not exported from the articles. If the suggestions were more related to learning outcomes, they were omitted, as were suggestions related to study design or research methodologies.

Last, although this review included 30 countries in six continents, it is worth noting that educational organizations throughout the world have significant variations in their technological capacities and infrastructure. Therefore, these findings are relevant in educational environments where appropriate digital resources for teaching and learning are available.

CONCLUSION

Higher education faculty are in a key position in educating future professionals who are expected to master digital environments as skillfully as other competencies in today's demanding and constantly changing workforce. The results of this scoping review reflect the importance of systematic digital learning intervention planning and provide information to educators and other professionals who are looking for effective and student-friendly ways to modify their teaching based on evidence. Overall, the main aim for educators to modify their teaching methods or carry out new technical solutions is to help students gain deeper understanding, improve skills, increase knowledge, remain motivated to learn, gain awareness, and learn to think critically.

Acknowledgment

The authors thank the Ministry of Education and Culture, Finland, for funding the study.

References