2024 VOL. 11, No. 2
Abstract: The 2018 International Computer and Information Literacy Study (ICILS) provides a comprehensive dataset with contextual information on German school environments. The data comes from responses by eighth-grade teachers and principals to a questionnaire about Information and Communication Technology (ICT) for teaching. We performed an exploratory re-analysis examining what influences teachers' adoption and use of ICT by employing six factors — Social Influence, Effort Expectancy, Performance Expectancy, Facilitating Conditions, Behavioural Intention, and Use Behaviour — from the Unified Theory of Acceptance and Use of Technology (UTAUT). We measured relationships between these constructs with a Structural Equation Model (SEM) consisting of 15 indicators provided by the ICILS study. The goodness of fit measures indicated confirmation of the model (RMSEA = .05, SRMR = .05, CFI = .93, TLI = .91). Our findings show that teachers' attitudes toward ICT and the school administration's support affected how ICT was applied in the classroom.
Keywords: ICT skills, structural equation model, teachers, ICILS dataset
The International Computer and Information Literacy Study (ICILS) is a large-scale, international assessment of eighth-grade students' CIL (Computer and Information Literacy) and CT (Computational Thinking) competencies conducted by the IEA (International Association for the Evaluation of Educational Achievement). In 2018, more than 46,000 students from over 2,200 schools in 13 countries were evaluated. The dataset is available for download in IEA's data repository for analysis and reproduction of their results. It is augmented by a complete set of responses to a questionnaire by the schools' students, teachers, principals, and ICT coordinators to contextualise their background and school environment. Germany was one of the participating countries, with 2,328 teachers from 182 schools surveyed (Fraillon et al., 2020a).
The Organization for Economic Cooperation and Development (OECD) recommended that Germany take measures to broaden the implementation of ICT equipment in schools, enhance teachers' digital skills, and strengthen the digital resources available in schools (OECD, 2020). However, the success of any ICT-based educational intervention hinges on teachers' attitude and self-efficacy to incorporate these tools into their teaching practices (Van Acker et al., 2013). Their self-efficacy is mainly influenced by their ICT skills (Aesaert et al., 2017). In addition to teachers' personal beliefs and perspectives, especially on the significance of technology for student learning, Ertmer et al., (2012) found that external support from the administration and colleagues is key to implementing ICT into teaching practice.
To gain a deeper understanding of the factors that impact teachers' acceptance and use of ICT in the classroom and their interplay, we re-analysed the German 2018 ICILS dataset with a focus on the relationship between factors derived from the school administration and the teacher survey data.
We use the Unified Theory of Acceptance and Use of Technology (UTAUT) by Venkatesh et al. (2003) to understand technology adoption in educational settings. The model aims to explain and predict factors influencing user acceptance and technology usage behaviour. UTAUT integrates several earlier theories, such as the Technology Acceptance Model (TAM) (Davis et al., 1989; for a systematic literature review with a focus on educational contexts, see Granić & Marangunić, 2019), Theory of Reasoned Action (TRA) (Hartwick & Barki, 1994), Theory of Planned Behaviour (TPB) (Ajzen, 1991), and others. UTAUT posits four key constructs that determine an individual's intention to use technology and the subsequent actual usage behaviour (Davis et al., 1989):
Performance Expectancy: This refers to the degree to which an individual believes that using the technology will help them achieve gains in job performance. Users who expect technology to improve their performance are more likely to accept and use it.
Effort Expectancy: This is the degree to which an individual believes using the technology will be free of effort. In other words, it is the ease of use associated with the technology. If technology is perceived as easy to use, users are likelier to adopt it.
Social Influence: This construct refers to how an individual perceives that “important others” (such as peers, superiors, or colleagues) believe they should use the technology. The more the user perceives social pressure to adopt the technology, the more likely they will use it.
Facilitating Conditions: These are the factors an individual believes are in place to support the use of the technology, such as the availability of resources, infrastructure, and technical support. If users believe that there are adequate facilitating conditions, they are more likely to adopt the technology.
When focusing on teachers implementing ICT in their classrooms, we describe the four key UTAUT constructs as follows (see Figure 1):
Performance Expectancy: The extent to which teachers believe that adopting ICT tools will improve their teaching effectiveness and enhance student learning outcomes. A teacher who is confident that ICT helps them deliver engaging lessons and assess student performance is said to have a high "performance expectancy."
Effort Expectancy: The perception of individual ease and collaborative effectiveness in using ICT tools for teaching. Teachers who perceive ICT as simple to use and can seamlessly integrate it into their teaching methods have high "effort expectancy" scores.
Social Influence: The impact of school principals on teachers' decisions to adopt ICT in their classrooms. Teachers might be more inclined to use ICT if they perceive that the administration holds positive views or expects it.
Facilitating Conditions: Teachers with access to appropriate hardware and software, reliable internet connectivity, ongoing technical support, and PD (professional development) opportunities that equip them with the necessary skills to integrate ICT into their teaching practices enjoy facilitating conditions.
Building on the conceptual groundwork provided by the UTAUT model, the following research question is presented: Can a structural equation model be derived from the German 2018 ICILS dataset that describes influences on ICT adoption in German schools?
The ICILS 2018 dataset for Germany is publicly accessible through IEA's data repository. It provides continuous scaled indices collated from individual teacher and principal responses to a Likert-scale questionnaire (Meyer & Mikheeva, 2019). To be confident that we operate within the methodological boundaries, we use these variables in our analysis without further modifications and follow their technical report guidelines. From the set of available variables, we selected those that conceptually most match the UTAUT constructs we outlined in the Framework section.
For instance, the variable "Negative views on using ICT in teaching and learning" measures the extent to which a teacher indicated agreement with individual questions regarding adverse outcomes of using ICT. Bad outcomes include poorer written expression, a distraction from learning, and an impediment to concept formation by their students. Furthermore, teachers judged whether ICT use leads to plagiarism, limited personal communication, and worse calculation and estimation skills among students.
On the other hand, the variable "Positive views on using ICT in teaching and learning" captures how much teachers agree that it yields positive outcomes. The questionnaire asked teachers whether such practices develop student interest in learning, collaboration, problem-solving, and planning abilities. Further questions inquired about its helpfulness in letting students access better information sources, work at an appropriate level for their learning needs, and improve academic performance.
With our assumption that positive views correspond to a high "Performance Expectancy" and that negative beliefs affect it negatively, we treat this variable as an indicator of a larger latent construct. Hence, as shown in Figure 2, we modeled "Performance Expectancy" with two variables capturing teachers' positive and negative views toward ICT in teaching and learning. Similarly, we conceptually assign further variables to the remaining UTAUT constructs.
We map three variables from the ICILS study onto the "Social Influence" construct: "View on using ICT for educational outcomes," "ICT use expected of teachers," and "Expectations for teacher collaboration using ICT." These variables measure the importance principals place on students obtaining basic computer skills and how much they expect teachers to incorporate ICT in their practices or when collaborating with others.
To "Facilitating Conditions," we assigned "Availability of computer resources at school," "Teacher participation in reciprocal learning PD related to ICT," and "Teacher participation in structured learning PD related to ICT." The first variable measures how sufficient the physical and digital equipment is and whether teachers have enough preparation time and support to incorporate ICT in their lessons. The latter two assess how often teachers can observe their peers or receive formal training in such activities.
We instantiated "Effort Expectancy" with "Teachers' ICT self-efficacy," the teachers' belief in their ability to perform ICT tasks effectively. "Collaboration between teachers in using ICT" was also included, gauging how practical it is to work with colleagues to develop and improve ICT-based lessons. Higher values represent lower levels of effort.
Our model's "Behavioural Intention" construct signifies teachers' aspirations in engaging their students' digital literacy and problem-solving. "Emphasis on ICT capabilities in class" is their focus on cultivating CIL skills among students; "Teacher emphasis of teaching coding tasks in class" measures the emphasis on teaching CT skills within the classroom.
In our SEM, the "Use Behaviour" represents how teachers' intentions to use ICT in their teaching practices are enacted. We judged the variables "Use of ICT for classroom activities," "Use of general utility software," and "Use of digital learning tools" to suitably convey this goal. These delve into the frequency with which teachers perform ICT-based student-centered classroom activities and use general or specialised digital learning tools.
We had to account for ICILS's complex survey design to perform our analysis. The first step was preparing the data by merging the teacher-level variables selected in the previous section with those at the school administration level. We positioned findings from the teachers' perspective, with principals' attributes becoming those of teachers. For example, the variable "ICT use expected of teachers" from the principals' dataset was reinterpreted as "The expectation of a teacher's principal regarding their use of ICT."
The dataset merging process was done with version 5.0.16 of the IEA's International Database Analyzer to ensure that a teacher weighting factor was automatically selected. Its application as a sampling weight variable to the unified teacher and school data was required (Meyer & Mikheeva, 2019). In addition, 75 jackknife replicates (Gonzalez & Foy, 2000; Wolter, 1985) were incorporated into the merged dataset, following the recommendation for using the jackknife repeated replication method to accurately estimate standard errors in ICILS 2018 data (Meyer & Mikheeva, 2019). The programme generates SPSS 28 code that, once executed, produces the merged dataset for the SEM.
We modeled our SEM using the StataBE 18.0 software package (StataCorp., 2023). We loaded the prepared dataset and organised the model as discussed in our methodology. A visual representation can be found in Figure 2, where rectangular boxes correspond to variables in the dataset that we assigned to higher-level UTAUT constructs.
These are then linked together with the postulated relationships: according to UTAUT, "Effort Expectancy" determines "Behavioural intention," which, together with "Facilitating Conditions," influences "Use Behaviour." Our SEM includes these same relationships and, as UTAUT, treats "Social Influence" and "Performance Expectancy" as exogenous. Where we differ is hypothesising that the principals' "Social Influence" affects the teachers' perception of "Facilitating Conditions" in the school and that it, together with "Performance Expectancy," impacts teachers' "Effort Expectancy." The data preparation and Stata program code documentation are available on GitHub for transparency and reproducibility of our results.
We can then estimate the SEM to compute how strong the relationships are. Stata provides several estimators that output goodness-of-fit measures to validate our model. In total, our merged dataset has N = 2,328 observations but some of the rows have missing values. If we include them alongside sampling weights and the survey design, the MLMV (Maximum Likelihood with Missing Values; Maydeu-Olivares, 2017) estimator returns a coefficient of determination (CD) of .924 as a measure of model fit. Alternatively, a Maximum Likelihood (ML) estimator that removes observations with missing values (N = 1,902, CD = .930) produces a standardised root mean squared residual (SRMR) value of .053.
To derive the Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and Root Mean Squared Error of Approximation (RMSEA) as fit indices, it was necessary to forego the application of weights and employ default standard errors instead. Utilising MLMV, a CFI of .936, a TLI of .919, and an RMSEA of .047 was achieved. With the ML estimator, the resultant values were RMSEA = .050, CFI = .926, TLI = .908, and SRMR = .044.
ICILS' technical report (Fraillon et al., 2020b) categorises an RMSEA value between .08 and .05 as "satisfactory" and any value below this range as a "close fit." The calculated SRMR values fall under the cutoff value of 0.08 (Hu & Bentler, 1999). The TLI and CFI values, which reside between 0.9 and 0.95, suggest a decent model fit (Hu & Bentler, 1999). Notably, across all estimators, all SEM relationships were statistically significant (p < .05). These fit indices, taken together, point towards a good (Hu & Bentler, 1999) model fit.
The estimated model encompasses six latent constructs and 15 observed variables. Standardised factor loadings signify the strength and direction of the relationships, i.e., the amount of change in the observed variable corresponding to a one standard deviation change in the latent variable. The measurement model results for the weighted MLMV-estimated SEM are highlighted in Table 1. Except for computer resource availability reported by teachers, all factor loadings are above the 0.4 cutoff (Pituch & Stevens, 2015).
Table 2 describes the structural model by reporting the Standardised Path Coefficient and the Jackknife Standard Error for every relationship. The path coefficients measure the direct effect of one latent variable on another when all other factors in the model are held constant. Using the jackknife standard error in both tables provides a less biased statistic.
Table 1: Measurement Model Using the jackknife standard error in both tables provides a less biased statistic.
* From: Meyer & Mikheeva, 2019. ICILS User Guide. Section 3.3.4; StataCorp. 2023. Stata 18 Reference Manual. Variance Estimation, Jackknife Estimation.
Table 2: Structural Model
Figure 2 presents the estimated Structural Equation Model. It displays the factor loadings for each indicator, as presented in Table 1, and the path coefficients from Table 2. Furthermore, the figure shows the measurement errors for the indicators of the latent variables and their corresponding error variances.
This paper re-examines the German ICILS 2018 dataset to understand factors at both administrative and teacher levels that impact the integration of ICT in the classroom. The analysis is anchored in the UTAUT model to evaluate the significance of these influencing elements. The identified aspects that ultimately lead to the adoption of ICT in teaching and learning are presented next.
Teachers' Perceptions of Facilitating Conditions in the School Positively Influence Teachers' Effort Expectancy
The influence of Facilitating Conditions on Effort Expectancy in our model indicates that readily available computer- and support resources aid in establishing a conducive environment for integrating ICT into teaching, as it fosters teachers' self-efficacy and collaboration in using ICT. Investing in facilitating conditions through professional development and administrative support can impact teachers' willingness, confidence, and ability to integrate technology into their teaching practices. This aligns with the findings of Ikhlas and Dela Rosa (2023) and Kundu et al. (2020), who describe a positive relationship between teacher training and access to ICT equipment with teachers' ICT self-efficacy.
Teacher's Perceptions of Facilitating Conditions are a Significant Enabler of their Use Behaviour
In our model, the Facilitating Conditions directly affect the Use Behaviour. Teachers who employ utility software and digital learning tools in their teaching, or whose students perform tasks with ICT in the classroom, have adequate resources and administrative support to do so. This finding suggests that ICT is adopted into teaching practices when the necessary assistance and infrastructure are provided for teachers. This finding aligns with Mirzajani et al. (2016), who state that adequate resources are important factors that help bring ICT into the classroom. Educational stakeholders should consider investing in these conditions strategically as a lever to promote widespread and effective ICT integration. Nikolic et al. (2019) write that poor school equipment is not a crucial factor that prevents the introduction of ICT in the classroom.¬¬
Effort Expectancy is a Catalyst for Behavioural Intention in ICT Usage
Our model describes a substantial relationship between Effort Expectancy and Behavioural Intention. Easing technology adoption in education through the collegial exchange of knowledge and teachers' self-efficacy in handling ICT tools drives the intent to integrate technology into pedagogical practices. This interplay is affirmed by the existing literature (Hatlevik & Hatlevik, 2018; Vandeyar et al., 2013) on the co-constructivist concept of reciprocal ICT learning, where teachers collectively share and gain insights on using ICT in their teaching practice. Therefore, educational stakeholders must focus on initiatives that foster teachers' ICT skills and a collaborative culture. These can include targeted professional development programmes or establishing ICT mentorship systems within schools.
Behavioural Intention is a Vital Precursor to Use Behaviour
The strongest relationship described by our model is the link between Behavioural Intention and Use Behaviour. A commitment from teachers to incorporate digital literacy and computational thinking exercises into their lessons does materialise into ICT-rich lessons and activities. Their classes are assisted by general-purpose software and specialised digital learning tools; their students are frequently assigned ICT-based tasks in classroom activities. This finding is in line with Hatlevik & Hatlevik (2018) and Player-Koro et al. (2012), who found that teachers' confidence in their ability to use ICT and their positive views on its benefits for teaching encourage its adoption in classrooms. To support this, schools can offer focused professional development opportunities, providing teachers with the necessary knowledge and resources to integrate these technologies into their curriculum.
Teachers' Performance Expectancy Positively Affects Effort Expectancy for ICT Integration
The pathway from our analysis between Performance- and Effort Expectancy revealed that when teachers recognise the benefits and effectiveness of ICT for teaching, they see it as less challenging to implement in their classrooms. Educational stakeholders should actively encourage a positive view of ICT among teachers and practitioners. Strategies to achieve this include targeted professional development workshops that showcase successful ICT integrations in classrooms, provide access to easy-to-use technology resources, and create a supportive community where teachers can share experiences and strategies for using ICT effectively.
The Principal's Attitude towards ICT Influences Teacher Perceptions of Facilitating Conditions at the School
Given the relationship between the Social Influence and Facilitating Conditions, we found that teachers whose superiors advocated for ICT in education had more access to resources, training, and overall conditions that aided ICT integration. A principal's stance on technology sets the tone for its usage within a school. The positive outlook on the educational utility of ICT communicates a social norm that ICT should be an integral part of teaching practices, influencing teachers' classroom practices. This fits within the bigger picture of the direct positive correlation between principals' attitudes toward the use of ICT and the degree of its use in the classroom drawn by Ghamrawi et al. (2013), which we did not find.
The Successful Integration of ICT in Education Relies Heavily on Teachers' Willingness to Adopt these Tools in their Teaching Practices
Our findings show that the successful integration of ICT in education relies heavily on teachers' willingness to adopt these tools in their teaching practices. This observation is consistent with analogous research on pre-service teachers; for instance, Tondeur et al. (2018) demonstrated a positive correlation between their attitudes toward ICT, perceived ease of use, and competency in using ICT for educational practices.
By understanding the factors and relations presented in this analysis, educational stakeholders could develop targeted strategies to promote ICT use in the classroom, both for those currently in the profession and those preparing to enter it. These include investing in teacher training and collaboration, providing necessary resources and infrastructure, and fostering a supportive school environment. Ultimately, the goal is to empower teachers to harness the full potential of ICT, enhancing teaching and learning experiences for the betterment of all students.
Aesaert, K., Voogt, J., Kuiper, E., & van Braak, J. (2017). Accuracy and bias of ICT self-efficacy: An empirical study into students’ over- and underestimation of their ICT competences. Computers in Human Behaviour, 75, 92-102. https://doi.org/10.1016/j.chb.2017.05.010
Ajzen, I. (1991). The theory of planned behaviour. Organizational behaviour and human decision processes, 50(2), 179-211.
Davis, F.D., Bagozzi, R.P., & Warshaw, P.R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1002.
Ertmer, P., Ottenbreit-Leftwich, A., & Sendurur, E. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59. 423-435. 10.1016/j.compedu.2012.02.001.
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020a). Preparing for life in a digital world: IEA International Computer and Information Literacy Study 2018 International Report. Springer Nature.
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020b). IEA International Computer and Information Literacy Study 2018. Technical Report. International Association for the Evaluation of Educational Achievement.
Ghamrawi, N. (2013). The relationship between the leadership styles of Lebanese public school principals and their attitudes towards ICT versus the level of ICT use by their teachers, 02, 11-20. https://doi.org/10.4236/OJL.2013.21002.
Gonzalez, E.J., & Foy, P. (2000). Estimation of sampling variance. In M.O. Martin, K.D. Gregory, & S.E. Semler (Eds.), TIMSS 1999: Technical Report. Boston College.
Granić, A., & Marangunić, N. (2019). Technology acceptance model in educational context: A systematic literature review. British Journal of Educational Technology, 50: 2572-2593. https://doi.org/10.1111/bjet.12864
Hartwick, J., & Barki, H. (1994). Explaining the role of user participation in information system use. Management Science, 40(4), 440-465.
Hatlevik, I., & Hatlevik, O. (2018). Examining the relationship between teachers’ ICT self-efficacy for educational purposes, collegial collaboration, lack of facilitation and the use of ICT in teaching practice. Frontiers in Psychology, 9. https://doi.org/10.3389/fpsyg.2018.00935
Hooper, D., Coughlan, J., & Mullen, M. (2007). Structural equation modeling: Guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6.
Hu, L.-T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 6, 1-55.
IEA Data Repository. (2023). https://doi.org/10.58150/ICILS_2018_data.
IEA IDB. (2023). Analyzer. https://www.iea.nl/data-tools/tools.
Ikhlas, M., Dela Rosa, E.D. (2023). Profile and predictors of high school teachers' attitude and self-efficacy in utilizing ICT: An investigation from Indonesia. SN Social Sciences 3, 71. https://doi.org/10.1007/s43545-023-00654-2
Kundu A., Bej T., Dey, K.N. (2020). An empirical study on the correlation between teacher efficacy and ICT infrastructure. International Journal of Information and Learning Technology. https://doi.org/10.1108/IJILT-04-2020-0050
Maydeu-Olivares, A. (2017). Maximum likelihood estimation of structural equation models for continuous data: Standard errors and goodness of fit. Structural Equation Modeling: A Multidisciplinary Journal, 24, 1-12. https://doi.org/10.1080/10705511.2016.1269606
Meyer, S., Mikheeva, E. (2019). ICILS 2018 User Guide for the International Database. https://www.iea.nl/publications/user-guides/icils-2018-user-guide-international-database
Mirzajani, H., Mahmud, R., Ayub, A., & Wong, S. (2016). Teachers’ acceptance of ICT and its integration in the classroom. Quality Assurance in Education, 24, 26-40. https://doi.org/10.1108/QAE-06-2014-0025.
Nikolic, V., Petković, D., Denić, N., Milovancevic, M., & Gavrilovic, S. (2019). Appraisal and review of e-learning and ICT systems in teaching process. Physica A: Statistical Mechanics and its Applications.
OECD. (2020). Education policy outlook in Germany. OECD Education Policy Perspectives, No. 16, OECD Publishing. https://doi.org/10.1787/47b795b1-en
Pituch, K.A., & Stevens, J.P. (2015). Applied multivariate statistics for the social sciences: Analyses with SAS and IBM’s SPSS. Routledge. https://doi.org/10.4324/9781315814919
Player-Koro, C. (2012). Factors influencing teachers’ use of ICT in education. Education Inquiry, 3, 108-93. https://doi.org/10.3402/edui.v3i1.22015.
Stata Corp (2023). Stata Statistical Software: Release 18. StataCorp LLC.
StataCorp. (2023). Stata 18 Structural Equation Modeling Reference Manual. Stata Press.
Tondeur, J. (2018). Enhancing future teachers' competencies for technology integration in education: Turning theory into practice. Seminar.net, 14. https://doi.org/10.7577/seminar.2981
Van Acker, F., Van Buuren, H., Kreijns, K., & Vermeulen, M. (2013). Why teachers use digital learning materials: The role of self-efficacy, subjective norm and attitude. Education and Information Technologies, 18(3), 495-514. https://doi.org/10.1007/s10639-011-9181-9
Vandeyar, T. (2013). Practice as policy in ICT for education: Catalysing communities of practice in education in South Africa. Technology in Society, 35(4), 248-257. https://doi.org/https://doi.org/10.1016/j.techsoc.2013.10.002
Venkatesh, V., Morris, M.G., Davis, G.B., & Davis, F.D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 425-478.
Wolter, K.M. (1985). Introduction to variance estimation. Springer.
Author Notes
https://orcid.org/0009-0000-1541-022X
https://orcid.org/0000-0002-9725-268X
https://orcid.org/0000-0003-1170-8875
Daniel Petri Rocha is a computer science student at the Technical University of Munich pursuing his master's degree. He specialises in software engineering, focusing on developing effective open-source online learning environments. Email: daniel.petri@tum.de
Dr. Arne Bewersdorff's research focuses on fostering ICT and AI literacy, as well as interest in AI among students. His second focus is the effective integration of AI systems in STEM education to improve teaching and learning. Email: arne.bewersdorff@tum.de
Prof. Dr. Claudia Nerdel's research spans interdisciplinary science didactics, primarily in biology. She delves into the integration of digital media in education, the digital shift in biology and related STEM fields, including the application of artificial intelligence in schools and universities. Email: claudia.nerdel@tum.de
Cite as: Rocha, D.P., Bewersdorff, A., & Nerdel, C. (2024). Exploring influences on ICT adoption in German schools: A UTAUT-based Structural Equation Model. Journal of Learning for Development, 11(2), 311-322.