An Evaluation of the Usefulness and Ease of Use of the Aptus within the Samoan Education Context

Ioana Chan Mow, Edna Temese, Mose N. Mose, Tara Patu, Elisapeta Mauai, Ioana Sinclair, Fiafaitupe Lafaele, Joseph Namulauulu, Misioka Tanielu, Agnes Wong Soon, and Vensall Chan

VOL. 4, No. 3

Abstract: This paper describes the findings of the first trial of the Aptus device within the National University of Samoa. The Aptus is a device that enables wireless access to valuable educational resources in the absence of electricity and the Internet. The goals of this research were to explore the acceptance of using the Aptus to access e-resources within the context of education in Samoa, with user acceptance measured by evaluating the ease of use and usefulness of the Aptus. The findings of the trial indicated very positive perceptions of students and teachers at the university in terms of ease of use and usefulness of the Aptus within educational settings. From the results of Phase 1, a strong recommendation is made to adopt the Aptus as a technology for providing access to quality educational resources within the National University of Samoa. The study also recommends the need for training of teachers on the use of the Aptus and its applications, such as Moodle.

Keywords: Aptus, OwnCloud, OER.

Introduction

Information Communication Technologies (ICTs) are seen as potentially powerful enabling tools for expanding access to education and improving the quality of education (Kanwar, Aguti & Balasubramanian, 2014).  Innovations in ICT have revolutionised and enhanced the learning and teaching process, opening up new learning spaces, opportunities and better access to educational resources. Integration of ICT into schools, colleges, universities and other educational institutions has provided novel ways to access resources, implement information gathering and analysis as well as providing the means to improve the administrative and management capacity of educational institutions.

In Samoa, the integration of computer and communications technology into the school system is still in its initial stages and is implemented through a variety of projects, such as Schoolnet, the One Laptop per child (OLPC) and the CSL E-rate project (Vaa, 2015). However, one of the key issues hindering the full realization of the potential benefits of ICT is the issue of access and affordability. Access to quality teaching and learning resources through the use of technology is limited. The Samoa SchoolNet project is an ADB funded project aimed at introducing computers into schools, piloted in 2007 and expanded in 2013. While the SchoolNet project has been touted a success and provided access to computers and resources in 42 secondary schools, actual access to quality resources is still quite limited, even within SchoolNet schools where technology resources are woefully underutilized. SchoolNet schools have substantial teaching resources in electronic form but can be accessed only within the SchoolNet labs. Access to resources using technology is even more limited in primary schools with very few schools having access to computer technology. Of particular challenge is access to the Internet, which potentially provides a wealth of quality resources to facilitate and improve the teaching and learning process (Chan Mow, 2010; PRIF, 2015; Vaa, 2015).

At the National University of Samoa, there is an extensive network with about 300 PCs in seven computer laboratories and about 20 PCs in the university library. However access to these computer laboratories is limited, with priority given to computer classes. This leaves students with only access to the 20 PCs in the library, as there are no PCs in the classrooms. Hence, the issue of access to quality electronic teaching and learning resources is quite critical. The problem of access has been exacerbated by increasing student numbers. At NUS, from 2003 to 2011 student enrolments doubled from 1,423 to 2,823. Since then student numbers remained over 2,500 and rose to 3,357 in 2015 (Education Sector Plan 2013-2018; NUS Annual Report 2015-2016).

One innovative solution for such challenges is a recent innovation launched by the Commonwealth of Learning in 2014 (www.col.org/Aptus). The Aptus, also referred to as “classroom without walls”, is a device set that allows access to digital or electronic resources in the absence of electricity or the Internet (Ghosh, 2013). Developed through the Commonwealth of Learning (COL) the Aptus enables “reaching the unreached” through the use of appropriate and affordable technologies. The COL’s Aptus device consists of two components: a mini-PC that acts as a server with the capability of storing large quantities of e-resources, in combination with a wireless router, which allows multiple learners to access the e-resources stored on the server. Learners can access the e-resources using laptops, tablets or mobile phones. With the explosive growth of mobile phone usage in recent times (see Tables 1 and 2) and the ready availability of low-cost tablets, access to quality e-resources through the use of the Aptus provides a low-cost solution to accessing the benefits of ICT. The Aptus, in combination with the increasing availability of open educational resources (OERs), makes access to valuable educational resources achievable. As such, the Aptus device is suitable for remote and underserved areas in underdeveloped and developing countries, and transforms the learning and teaching experience using low-cost innovative technology.

The current study trialed the use of the Aptus and explored the feasibility of using the Aptus within the context of NUS and schools in Samoa to provide and improve access to e-resources. The initiative was a collaborative effort by NUS, Commonwealth of Learning (COL) and the Ministry of Education Sports and Culture (MESC) to provide innovative low-cost solutions to chronic teacher shortages, limited access to computers, particularly in non-Schoolnet schools, as well as provisioning of offline access to e-resources. COL contributed 15 sets of the Aptus and MESC provided transport and access to schools for the trial. The study was implemented in two phases. The first phase trialed the Aptus within the National University of Samoa. In Phase 2, the Aptus was trialed in selected primary and secondary schools within the urban area. At the end of each phase, the use of the Aptus was evaluated, using user perceptions on the ease of use and usefulness of the Aptus device in the provisioning of access to electronic resources. This paper is based on Phase 1 of the trial conducted at the National University of Samoa.

The research questions for this study were:

Hence, as mentioned earlier, the goals of this research were to explore the acceptance of using the Aptus to access e-resources within the context of education in Samoa, with user acceptance measured by evaluating the ease of use and usefulness of the Aptus based on user perceptions.

About Aptus

The Aptus device can store and distribute hundreds of textbooks for multiple grades, hold and stream hundreds of videos from theKhan Academy, host Moodle 2.5, a Learning Management System, as well as WordPress and/or Drupal, a Content Management System. To supplement all this, local content can be generated and added quite easily. Extant literature indicates that within educational settings the Aptus can be used in various ways. The Aptus can be used to access learning resources, software applications, and content from the Internet even when connectivity does not exist. It can be used as a mobile library that downloads and stores learning materials to be accessed offline. The Aptus router can be charged easily and also can be used as a solar charger for other devices.

The Aptus device has already been trialed in other countries. From April to June 2014, COL’s Aptus system was made available for performance testing to volunteers at 20 sites in 15 countries.      

Testers assessed such aspects as the start-up time, radius of connectivity, number and type of devices (laptops or smartphones /tablets), and maximum operation under one battery charge. Testers were also asked to assess the usefulness of OwnCloud, an advanced “cloud services” software application residing on the Aptus, which facilitates the local exchange of files… Results received from 10 locations in nine countries have shown that Aptus is a robust device, able to withstand long transit times. It starts up easily (less than two minutes to boot up), is able to connect to many devices (maximum of 20 in one location in Nigeria) and can last for close to three hours. The OwnCloud application was widely appreciated. Almost every tester said that content upload to Aptus at a local level is possible but should be made easier for users with limited knowledge of website management. (Commonwealth of Learning, 2014)

Theoretical Framework

This research is grounded within the theoretical framework of the technology acceptance model (TAM) designed by Davis (1989) on the factors that influence users’ adoption of technology in general. According to the Technology Acceptance Model, users’ acceptance of a given technology is affected by their perceptions on the usefulness and ease-of-use of that technology. Perceived usefulness was defined by Davis (1989) as “the degree to which a person believes that using a particular technology would enhance his or her job performance” (Davis, 1989) and perceived ease-of-use of a system was also defined by Davis (1989) as “the degree to which a person believes that using a particular system would be free of effort” (Davis, 1989).

Within the context of this research, users — both teachers and students — will use the Aptus if they perceive it to be useful and easy to use, and the effectiveness of the Aptus will be evaluated by measuring user perceptions of the ease of use and usefulness of the Aptus device. According to the Technology Acceptance Model, the more positive the responses to the above factors of perceived usefulness and perceived ease of use, then the more positive the attitudes of teachers and students will be to the use of ICT and the more likely they will be to use ICT in their teaching and learning.

Methodology

The research is quantitative and exploratory in nature and utilized the Technology Acceptance Model to conduct a multi-setting, case study approach. Purposive sampling was used to select the participants for this trial. Due to the number of users being limited to less than 20, classes selected were small classes, such as streams of Foundation Computer Studies (HCS081). The selected sample consisted of 223 students in the 16 to 18-year age range in 12 tutorial classes of HCS081, an Introductory Computer Studies class at NUS. HCS081 is a semester long introductory Computer studies course at the university foundation level (equivalent of a pre-college year) in which students learn basic introduction to Windows and Microsoft applications, as well as introductory level Java programming. Students in the HCS081 class are from the various disciplines, such as Arts, Commerce, Education and the Sciences.

Procedures

An Aptus device was setup for each of the selected trial settings. Students were asked to bring in their own devices, such as smart phones, tablets and laptops. Those without devices were provided with tablets. Distribution was to ensure that each student/teacher had access to either a tablet or mobile phone. The Aptus device was installed for use within each of the HCS081 tutorial classes. Before the trial, training was conducted to familarise tutors with the use of the Aptus. Training involved the following activities. For teachers activities included: i) setting up of the Aptus device ii) recharging the router, and iii) Moodle activities, such as the setting up of classes, enrolling students, uploading course notes and creating assessments. For teachers and students, activities trialed were: i) how to navigate the Aptus Home page, ii) downloading notes from the Aptus using OwnCloud a file sharing application, iii) uploading documents to the Aptus using OwnCloud and using the Moodle virtual classroom to access course notes, upload student assignments and conduct online quizzes. During training, teachers also discussed with the research team what activities were planned for the four weeks of the trial based on the four activities mentioned above. These activities formed their teaching plan for the four weeks of the trial. Upon completion of training, staff and students trialed the Aptus for four weeks using activities introduced during their training.

Upon completion of the trial, staff and students completed a survey evaluating user perceptions on ease of use and usefulness of the Aptus device. The questionnaire, which had both English and Samoan versions, was administered by the research team with the assistance of the teachers in the selected settings. The questionnaire consisted of mostly Likert-type questions. In the Likert-type questions responses were typically: i) strongly disagree ii) disagree iii) neutral iv) agree and v) strongly agree.

The survey questionnaire consisted of the following sections:

These items have been adapted from various questionnaires of studies based on the Technology Acceptance Model (Cassim & Obono, 2011; Teo, Su Luan, & Sing, 2008).

Analysis

The survey evaluated the effectiveness of the Aptus within an educational context based on two variables: i) the ease of use and ii) usefulness. The overall reliability of the instrument using alpha Cronbach index yielded 0.94 In addition to scores on individual items two aggregate scores were generated to measure positive attitudes for ease of use and usefulness. Ease of use aggregate score was calculated by summing responses for each of the questions on ease of use — Questions B1 to B9. This means that Ease of Use aggregate could take values between nine and 45. The nine items were entered into a factor analysis using a Principal Components procedure, which indicated that a single factor resolution was possible. That is, all nine items loaded strongly upon one factor.  SPSS Reliability analysis was then used to check on scale properties, when all nine items were summed to one scale. The internal reliability coefficient alpha was found to be 0.87, with all items contributing strongly. Ease of Use aggregate was created with a mean of 35.2, SD of 6.3, kurtosis of 0.6 and skewness of -0.79.

Similarly a Usefulness aggregate score was calculated by summing responses for each of the questions on usefulness with the aggregate score taking values between 11 and 55.The eleven items were entered into a factor analysis using a Principal Components procedure, which indicated that a single factor resolution was possible. That is, all eleven items loaded strongly upon one factor.  SPSS Reliability analysis was then used to check on scale properties, when all eleven items were summed to one scale. The internal reliability coefficient alpha was found to be 0.93, with all items contributing strongly. Usefulness aggregate was created with a mean of 45.6, SD of 7.1, kurtosis of 2.8 and skewness of -1.1.

Results

Ease of Use

Possible range of values for responses on individual items was from one to five with a natural midpoint of three representing neutrality. Actual results indicated very positive and all above average responses ranging from 3.4 to 4.3 (Table 1).

The possible range for the Ease of Use aggregate was from nine to 45, with 27 as the natural midpoint, representing neutrality. However, the actual mean was 35.4 (SD of 6.2), which indicated, on the overall, a high level of positive attitudes towards ease of use. The scores for Ease of Use ranged from 15 to 45, with 88 % of the scores lying above the natural midpoint of 27. In short, students indicated very high-level, positive perceptions in terms of ease of use.

Usefulness

As for ease of use, possible range of values for responses on individual items was from one to five with a natural midpoint of three, representing neutrality. Actual results on usefulness indicated very positive and all above average responses ranging from 3.8 to 4.3 (Table 2).

The possible range for the usefulness aggregate was from 11 to 55, with 33 as the natural midpoint, representing neutrality. However, the actual mean was 45.6 (SD of 7.1), which indicated, on the overall, a high level of positive attitudes towards ease of use. The scores for usefulnesss aggregate ranged from 13 to 55, with 95% of the scores lying above the natural midpoint of 33. In short, students indicated very high-level positive perceptions, in terms of usefulness of the Aptus.

Table 1: Individual Item Responses for Ease of Use

Feedback from Tutors

Responses from the four tutors who taught the 12 tutorial classes indicated that their perceptions of ease of use and usefulness were even more positive than students with all responses in the agree or strongly agree category. Tutors found Aptus easy to install, did not overheat easy to upload and download notes. Tutors were also very enthusiastic with the use of Moodle for managing their classrooms in particular communicating with students, conducting activities such as online quizzes. Tutors also reported students commenting that they were accessing better quality resources on the Aptus than in the university library. The only concern tutors had was with the complexity of technical details of managing Moodle such as student registration. Within the context of the current study, this pointed to the need for in-depth training for teachers on Moodle before the trial for primary and secondary schools. Further, with the long-term view of mainstreaming of the Aptus and Moodle within the school system, this points to the need for dedicated training for staff and students on the use of Moodle.

Table 2: Individual Item Responses for Usefulness

Gender and Age Effects

One way analysis of variance (ANOVA) procedures of individual items and aggregate variables on ease of use and usefulness showed no significant differences (p = .05)in responses based on gender. There were also no significant differences based on age, as this was a fairly homogenous sample with students in the 16 to 18-year age range.

Summary and Discussion

The results of the survey indicated very positive perceptions of staff and students to the usefulness and ease of use of the Aptus. Interviews with tutors indicated “great enthusiasm of students to learn using the Aptus, with tutors seriously asking for the opportunity to purchase their own personal Aptus to be used in everyday teaching.” Such indications point to the viability of the Aptus as a technology to be introduced into the teaching and learning environment to improve access to quality educational resources. As with the assertion made by Dennis Pack of Kiribati in Ghosh (2013), the Aptus could very well be the main technology used in our schools.

Another advantage of using the Aptus, according to interviews of tutors was the provision of the Moodle virtual classroom, a Learner Management System for managing classroom content. With the use of Moodle tutors were able to manage their class notes, exercises, and interactive testing for drill and practice on course content. This provision of Moodle for managing classroom content has been a priority at NUS for some time now, as staff had been requesting its availability for managing their classrooms virtually, making course notes available and hosting online discussions. The problem, as pointed out earlier, is the lack of access to the NUS computer network and labs by most students in order to take advantage of Moodle on the university network. Hence, the availability of Moodle on the Aptus platform provides much needed access to Moodle without having to access the NUS network or having to access the computer laboratories.

From the results of Phase 1, a strong recommendation is made to adopt the Aptus as a technology for providing access to quality educational resources within the National University of Samoa. The Aptus can be used by lecturers within classrooms so students can access course notes and other quality teaching resources made available on the Aptus, without the need to be inside a computer lab or to have the Internet or electricity. This would transform the learning environment in addition to providing much needed access to resources.

Additionally, the trials have also opened up possibility of other potential uses of the Aptus within the university. One such use is for staff meetings, where meeting documents can be loaded on the Aptus and participants can access the documents from the Aptus, thereby reducing paper float. Another innovative use is for the administration of university wide evaluation surveys. Several Aptus devices could be loaded with the survey and then mounted in various locations around campus facilitating ease of access for students. The research team aims to trial these two applications of the Aptus within the next academic year.

In Samoa, the growth of usage of mobile technology has been explosive. Since 2005, there has been a seven-fold increase in the number of mobile subscribers, from 22,000 in 2006 to 167,400 by 2010 (Commonwealth of Australia, 2008), and the price of phone calls has dropped by more than half (Va’a et al., 2012). Hence, in countries like Samoa with tremendous growth in usage of mobile phones, accessing e-resources through mobile phone by students is potentially a monumental breakthrough in achieving access to technology and the wealth of electronic resources.

To conclude, the following recommendations are made as a result of the findings of the current study:

References

Commonwealth of Australia. (2008). Pacific Economic Survey. Retrieved from http://pacificpolicy.org/wp-content/uploads/2013/07/pacific_economic_survey08.pdf

Cassim, K. M., & Obono, S.D.E. (2011). On the factors affecting the adoption of ICT for the teaching of world problems. Proceedings of the World Congress on Engineering and Computer Science WCECS, October 19-21, 2011, San Francisco, USA.

Chan Mow, I.T. (2010). Technology enabled education in Samoa: Issues, challenges, strategies and recommendations. In S. Mukerji & P. Tripathi (Eds.), Cases on interactive technology environments and transnational collaboration: Concerns and perspectives, (pp. 193-210). New York:  IGI Global.

Commonwealth of Learning. (2014). APTUS testers find it "Relevant" and "Robust", Connections/EdTech News, 19(2), 16. Retrieved from http://oasis.col.org/handle/11599/779

Education Sector Advisory Committee. (2013). Education Sector Plan (July 2013 – June 2018).  Malifa: Government of Samoa. Retrieved from this URL.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-339.

Ghosh, M. (2013). MOOC M4D: An overview and learners viewpoint on Autumn 2013 course. International Journal of Interactive Mobile Technologies, 8(1), 46-50.

Kanwar, A., Aguti, J., & Balasubramanian, K. (2014). How ICT can support teacher training. Retrieved from http://hdl.handle.net/11599/757

Pacific Region Infrastructure Facility (PRIF). (2015). Economic and social impact of ICT in the Pacific. Retrieved from https://theprif.org/index.php/resources/document-library/121-prif-ict-study-report-2015

National University of Samoa. (2015). National University of Samoa Annual Report: 2015-2016. National University of Samoa.

Teo, T., Su Luan, W., & Sing, C. C. (2008). A cross-cultural examination of the intention to use technology between Singaporean and Malaysian pre-service teachers: An application of the Technology Acceptance Model (TAM). Educational Technology & Society, 11(4), 265-280.

Vaa, R. (2015). A baseline study on technology-enabled learning in the Commonwealth Pacific Island Countries: Report. Burnaby, British Columbia: Commonwealth of Learning.

Va’a, F., Unasa, T. V., Fonoti, F. L., Chan Mow, I., & Amosa, D. (2012). Aspects of Economic Development. In Meleisea, M., Meleisea, P.S., & Meleisea, E. (Eds.), Samoa’s journey 1962-2102: Aspects of history. Wellington: Victoria University Press.

Authors:

Dr Ioana Tuugalei Vaai-Chan Mow is the Professor of Computing and Computer Education at the National University of Samoa (NUS). She is the COL Focal point for Samoa, the Team Leader for the National University of Samoa Aptus research team, and played a key role in the PACENETPlus, COLTEL and Faculty of Science Open and Distance Learning Project. Email: i.chanmow@nus.edu.ws

Dr Edna Temese-Ualesi is a Senior Lecturer and CISCO instructor, in the Computing Department at NUS. She is a member of the NUS Aptus Research team, the NUS COLTEL research group as well as the Faculty of Science Open and Distance Learning Project Team. Email: e.temese@nus.edu.ws

Mose N. Mose is a Lecturer and CISCO instructor, in the Computing department at NUS. He is a member of the NUS Aptus research Team, the NUS COLTEL research group, the VEX Robotics Team, as well as the Faculty of Science Open and Distance Learning Project Team. Mose is also the founder of the largest online shopping portal in Samoa - Makeki Online. Email: m.mose@nus.edu.ws

Tara Patu-Fritz is a Lecturer and CISCO instructor, in the Computing department at NUS. She is a member of the NUS Aptus research Team, the NUS COLTEL research group, as well as the Faculty of Science Open and Distance Learning Project Team. Email: tpatu@nus.edu.ws

Elisapeta Mauai is a Senior Lecturer in the Computing Department at NUS. She is the Coordinator of the NUS CISCO academy, a member of the NUS Aptus Research team, the NUS COLTEL research group, as well as the Faculty of Science Open and Distance Learning Project Team. Email: e.mauai@nus.edu.ws

Ioana Sinclair is a Lecturer and CISCO instructor, in the Computing department at NUS. She is a member of the NUS Aptus research Team, the NUS COLTEL research group, as well as the Faculty of Science Open and Distance Learning Project Team. Email: i.sinclair@nus.edu.ws

Joseph Namulauulu is a Lecturer and CISCO instructor, in the Computing department at NUS. He is a member of the NUS Aptus research Team, the NUS COLTEL research group, the VEX Robotics team, as well as the Faculty of Science Open and Distance Learning Project Team. Email: j.namulauulu@nus.edu.ws

Fiafaitupe Lafaele is a Lecturer and CISCO instructor, in the Computing department at NUS. She is a member of the NUS Aptus research Team, the NUS COLTEL research group, as well as the Faculty of Science Open and Distance Learning Project Team. Email: f.lafaele@nus.edu.ws

Misioka Tanielu is a Lecturer and CISCO instructor, in the Computing department at NUS. He is a member of the NUS Aptus research Team, the NUS COLTEL research group, the VEX Robotics team, as well as the Faculty of Science Open and Distance Learning Project Team. Email: m.tanielu@nus.edu.ws

Agnes Wong Soon is a Lecturer and CISCO instructor, in the Computing department at NUS. She is a member of the NUS Aptus research Team and the NUS COLTEL research group. Email: a.wongsoon@nus.edu.ws

Vensall Chan is a Lecturer and CISCO instructor, in the Computing department at NUS. He is a member of the NUS Aptus research Team, the NUS COLTEL research group, the VEX Robotics team, as well as the Faculty of Science Open and Distance Learning Project Team. Email: v.chan@nus.edu.ws