Effects of Internet Access During Examinations

Sugata Mitra and Ritu Dangwal

2022 VOL. 9, No. 1

Abstract: The scores obtained by students in examinations where internet access was allowed during the examination were compared with the scores obtained in traditional examinations where no assistance was allowed. These scores were then compared with those obtained in a standardised school examination on the same topic or subject, taken by the same students a year before. We observed that scores dropped by over 70% within a year of taking a traditional examination but could be significantly improved if internet access is allowed in the later examination. We further observed that scores in examinations where internet access was allowed were consistently higher than where internet access was not allowed. Finally, we report an analysis by rank and observe that student rankings change both over time and whether internet access was allowed or not. This leads us to suggest that use of the internet during examinations measures abilities that are different and more meaningful to our times than those that are measured by traditional examinations based on memorisation and unassisted recall.

Keywords: assessment, examination, cheating, internet access, memory.

Introduction

Use of the internet is not allowed during traditional examinations (TE) in most educational institutions in the world. Also not allowed during TE are books or notes in any form, talking or looking at the work of other examinees. All such methods that may be of assistance to an examinee during a TE are generally classified as “cheating” (Colnerud & Rosander, 2009) and are forbidden. These restrictions are put on an examinee because the purpose of an educational examination is to measure the knowledge an examinee has in the subject in which he or she is being examined. The words “knowledge” and “knowing” are circularly defined in most dictionaries. For example, in the Merriam Webster Dictionary, “knowledge” is defined as “The fact or condition of knowing something….” , while “knowing” is defined as “Having or reflecting knowledge….”. Similar circular definitions are present in the Cambridge and other dictionaries. Since these words are hard to define, it would be more precise to assume that a traditional examination measures an examinee’s ability to memorise, recall without assistance, and apply recalled memories to answer questions (which is considered good, see, for example, Persky & Fuller, 2021). This would explain why all assistive methods are forbidden during traditional examinations.

Outside of educational examinations, extensive use of the internet and all other assistive methods including asking others and looking at what others are doing, are not only allowed but actively encouraged in the hope that a correct answer or solution will be found to a question or problem (Apuke & Iyendo, 2018). The ability to answer a question or solve a problem quickly and accurately using any available resource is considered a desirable skill. This is opposite to what educational examinations (TE) attempt to measure. These diametrically opposite points of view are possibly a result of the origins of the TE from a time when it was important to prepare students to solve problems without any assistance whatsoever. A time when books, libraries, teachers, and even helpful friends, could not be made instantly available at the time of need. The existence of the internet and tiny devices that can access the internet at high speeds have changed the need for many skills that were vital in the 20th and prior centuries (van Laar et al., 2020). The changes have been too rapid for most educational systems to adapt.

In order to make changes in how or what educational examinations should measure, we first need to know the effects that assistive technology would have on TE, if they were allowed. In this paper we report the results of an experiment to measure the effects of allowing internet access during traditional examinations. For the sake of brevity, we will name examinations where internet access is allowed as Internet Assisted Examination (IAE).

Research Questions

If internet access is allowed during an examination:

1. Would the scores obtained by examinees be different than if internet access was not allowed?
2. Would the difference in scores, with and without internet access, be the same for each examinee?
3. Would the examinees learn while using the internet during examinations?

Experimental Design

We have used the standard experimental research model where single variables are manipulated while others act as controls.

1. Round 1: Take a group of students who have taken an end-of-school examination, such as the GCSE (UK), about a year previously and are now enrolled in a university. Divide this group into two random groups of approximately equal numbers. Name the groups Alpha and Beta.
2. Choose a question paper, QP1, that is equivalent to an examination already taken by the students in their end-of-school examination.
3. Administer QP1 to the Alpha group allowing the stipulated time to answer, i.e., the time normally allocated for that examination. They must work alone and not use any assistance of any sort.
4. Administer QP1 to the Beta group allowing the stipulated time to answer. They are allowed to use the internet for answering the examination, using any device (IAE). No supervision is required for this group.
5. Grade the answer sheets according to the scoring norms. This is the end of Round 1. At the end of this round, the same examination will have been answered by the two groups, one using TE, the other using IAE.
6. Round 2: Choose another question paper, QP2, equivalent to QP1, and repeat the steps above, this time group Alpha can use the internet, while group Beta must answer alone and without assistance.
7. Grade the answer sheets according to the scoring norms. This is the end of Round 2. At the end of this round, an examination equivalent to that in Round 1 will have been answered by the two groups in reverse order, i.e., the group that used TE would use IAE and vice versa. This ensures that the scores in Rounds 1 and 2 act as controls to each other.
8. Round 3: Group Alpha answers QP2 without the internet, group Beta answers QP2 with the internet.
9. Grade the answer sheets according to the scoring norms. This is the end of Round 3.
10. Round 4: Group Alpha answers QP1 with the internet, group Beta answers QP1 without the internet.
11. Grade the answer sheets according to the scoring norms. This is the end of Round 4. Rounds 3 and 4 provide further control information, as both groups would have taken both examinations in the TE and IAE modes.
12. Each round must be separated from the previous round by at least one week and preferably longer in order to reduce the effect of involuntary retention of information.

Method and Scores Obtained

1. Forty-eight students in the first year of their undergraduate studies in Computer Science at a university in western India volunteered for the study. All students had taken the ICSE (Indian Certificate of Secondary Education) prior to their admission to the university. They were divided into two random groups, Alpha and Beta, of 24 students each. Profiles of the groups are shown in Table 1.

Table 1: Profiles of Alpha and Beta sample groups

2. Two examination question papers were created for Geography and History. The papers were created using question papers from past ICSE examinations. This was straightforward because each question in the ICSE question papers has a score associated with it and a norm for scoring the question. Since each ICSE question paper is allocated a fixed time for answering, it is possible to estimate the time that should be allocated to any question by proportionately dividing the allocated time for the examination by the score in a particular question. Such selected questions from various years can then be assembled into an equivalent question paper. We chose Geography and History because these subjects showed little variability in their curricula over the years. We will name these equivalent examinations G, for Geography and H for History.
3. We then followed the steps as directed by the design section above. At the end of the four rounds, each group had answered G and H in the TE and IAE modes over a period of 77 days on days 0, 28, 66 and 77. The intervals were necessitated by the students’ normal schedule of studies at the university and were not under our control. The sequence of examinations and the scores obtained are shown in Table 2.

Table 2: Examination schedules and average percentage scores for Geography and History examinations with standard deviations in brackets

4. Finally, we collected the actual scores received by the participants in the school examination on Geography and History (it was a combined question paper) about one year before the commencement of this study. The average scores showed no statistically significant difference ( P(T < = t) two-tail = 0.58) between the two groups. The scores are shown in Table 3. Based on this, we assumed that the two groups were equivalent for our purpose.

Table 3: Scores obtained by the participants in their school final examination in the Geography-History paper

Analysis and Discussion

While we carried out a number of analyses on the data obtained, for the purpose of this paper, we decided to describe only those that showed clear results requiring little statistical interpretation:

1. We observed that students of computer science showed little recollection of what they had learned in Geography and History in their schools. The scores they obtained in these subjects during their school-leaving examinations dropped by more than 70% when an equivalent examination was administered after a year. However, when internet access was allowed in the same examination, the scores were only about 20% lower than what they had scored a year ago. Figure 1 shows the mean scores for equivalent TE and IAE examinations administered after one year, along with the scores they had obtained in the actual TE at the end of their schooling. The thee mean scores were found to be significantly different from each other.

Figure 1: Mean scores with standard deviations in the actual school-leaving examination
compared to scores in equivalent TE and IAE after one year

2. IAE scores were found to be consistently higher than TE scores. There were no exceptions to this in our data. Figure 2 shows the TE and IAE scores (combined for the Geography and History examinations) for all the participants. Figure 3 shows the scores separated for the two subjects.

Figure 2: Average of the scores in Geography and History for all the participants

Figure 3: Mean scores in Geography and History with and without internet access

3. We observed that there was a difference in scores depending on whether a TE was administered before or after an IAE for the same subject, namely, geography and history. TE scores in examinations administered after the same examination was administered using IAE a few weeks earlier were found to be significantly higher than TE scores in examinations administered before the IAE. The experience and knowledge gained during IAE seems to be acting as “preparation” for the TE. This is shown in Figure 4.

Figure 4: TE scores when administered before and after IAE

On the other hand, when IAE are administered after TE, we did not notice any significant difference. This is shown in Figure 5.

Figure 5: IAE scores when administered before and after TE

Curiously, when the IAE was administered after the TE, we observed a small drop in the IAE scores. It is possible that the earlier memorised knowledge from TE prevented effective use of the internet during IAE, however, this is speculative and needs more experimentation to establish.

4. We observed that relative ranking of students was poorly correlated. The ranks obtained by students in IAE show a correlation coefficient of 0.53 with the ranks they obtained in the actual school leaving examination. Such low correlation is to be expected since the score obtained by an examinee in IAE is determined by the ability to use the internet accurately and quickly, in addition to memorised knowledge, if any. Hence, students who scored highly in a TE could be overtaken by students who used the internet well in an IAE, and this will change student rankings. We noticed also that the rankings for equivalent TE administered after a year of the actual TE in school are not correlated (0.37). This could be explained by how much each student has forgotten. A high-scoring student who has forgotten more of the material than a lower scoring student may fall behind, but we have not tested for this.

Summary

The sample size in this study was not large and the findings cannot, therefore, be generalised.  A study with a larger sample is suggested. However, we do get a fascinating glimpse into the effects of internet access on examinations. The ability to compute (search), comprehend and communicate may lie at the heart of educational assessment for the generation growing up in the 21st century.

Conflict of Interest: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Author Contributions: The first author provided the design and analysis for this study. The second author developed the methods, conducted the experiments and collected the data.

Funding: This study was not funded by any research grant.

Acknowledgement: The authors are grateful to the participants who found the study of great interest, volunteered their time and cooperated at every stage of the experiments.

References

Apuke, O. D., & Iyendo, T. O. (2018). University students' usage of the internet resources for research and learning: Forms of access and perceptions of utility. Heliyon, 4(12), e01052. https://doi.org/10.1016/j.heliyon.2018.e01052

Colnerud, G., & Rosander, M. (2009). Academic dishonesty, ethical norms and learning. Assessment and Evaluation in Higher Education, 34(5), 505-517.

Persky, A. M., & Fuller, K. A. (2021, July). Students’ collective memory to recall an examination. American Journal of Pharmaceutical Education, 8638. doi:https://doi.org/10.5688/ajpe8638

van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M. & de Haan, J. (2020). Determinants of 21st-century skills and 21st-century digital skills for workers: A systematic literature review. Sage Open, 10(1). https://doi.org/10.1177/2158244019900176

Authors:

Sugata Mitra retired in 2019 as Professor of Educational Technology at Newcastle University in England, after 13 years there, including a year in 2012 as Visiting Professor at MIT MediaLab in Cambridge, Massachusetts, USA. He is Professor Emeritus at NIIT University, Rajasthan, India.

His work on children’s education include the ‘hole in the wall’ experiment where children access the internet in unsupervised groups, the idea of Self Organised Learning Environments (SOLEs) in schools, the role of experienced educators over the internet in a ‘Granny Cloud’ and the School in the Cloud where children take charge of their learning – anywhere. Email: sugata.mitra@niituniversity.in

Dr Ritu Dangwal is an Assistant Professor at NIIT University in India. She has a PhD in Organisational Psychology and an active research record spanning over 20 years. Her interests include children's education, self organised learning, open and distance education, psychology and assessment. Email: Ritu.dangwal@niituniversity.in

Cite this paper as: Mitra, S., & Dangwal, R. (2022). Effects of internet access during examinations. Journal of Learning for Development, 9(1), 129-136.