VOL. 3, No. 2
This paper explores and documents fundamental concepts and concerns that characterize or perhaps jeopardise the relationships between the ‘old’ research communities and ‘new’ policy maker communities working to improve the nature and scope of learning in the developing contexts of the global South using personal mobile digital technologies. As becomes apparent, these concepts and concerns are relevant and interesting across a broader range of domains, touching perhaps under-privilege and access to education and technology in both the global North and the global South, the uses of technology to extend, enhance and transform learning and the various pressures and determinants of policy-making and of the public funding of research.
This piece grows out of personal experience and reflection and is consequently partial and subjective; it is however an area that is difficult for researchers to explore objectively, analytically and comprehensively, for two main reasons. The first is that the area is necessarily subjective for researchers because researchers as a community themselves are part of it not separate from it, they are players and stakeholders in the relations and transactions being explored and discussed. The second is that the methodological tools do not really exist to explore and analyse these relations naturalistically; these relations and the transactions that comprise them are verbal, fluid, informal, privileged and political. Policy-makers are improbable research subjects.
This piece is an attempt to explain and explore what seems to be going on as these two communities and those alongside them try to build better and more productive relationships. It is partly exposition and explanation, partly exhortation and partly critique.
As some context for the development of this piece, I should mention briefly the process that catalysed it. Annually UNESCO in Paris holds a Mobile Learning Week. Leading up to the Week in 2015, UNESCO proposed a symposium that explored the interface between research and policy, assuming that both parties understood the other only poorly. This was a worthwhile exercise in itself but also a source of research data that might lead to findings that were useful to the two communities.
This document draws on two sets of qualitative data. In both cases, these are drawn from small self-selected samples, namely people who attended the symposium, more than 80 of them, and who chose to speak, and people who subsequently offered responses to an online survey. These people were a cross-section of experienced researchers, programme managers and policy makers, from both the global North and the global South. There was a sequence and development in that responses from the symposium informed the questions subsequently posed in the survey, seeking to probe for examples, counter examples and opinions. The aim was not to produce outcomes or findings that were objective or representative but to illustrate the breadth of experience, ideas and practices. This aim underpinned how the data as gathered and how it was analysed, and how subsequently presented. No formal research method or theory was used or assumed.
The moderators for each segment of the symposium were provided with scripts, in effect a focus group protocol, and used it to even-handedly move through their respective panels, seeking responses in turn and then throwing the topic open to the other participants. The vast majority of all the segments was audio recorded and transcribed, and notes taken independently verbatim.
Participants were emailed to participate in an online survey that consisted of a series of open-ended questions directly related to one or other of the themes from the symposium. There were twenty-eight respondents and most made substantial free-text answers to most of the twenty questions. We must recognise that the vast majority of these respondents identified themselves as researchers. Their responses must be seen in this context but are valuable nevertheless and in most instances draw on experience in the field and on experience of engaging with policy makers. These represent the deeply flawed attempts to use recognised methods to capture qualitative data objectively. They were however also informed by comparable contacts and collaborations with DFID, USAID, World Bank and World Economic Forum.
We must start by introducing the fundamental concepts.
The ideas and practices of what has come to be called ‘mobile learning’ date back to the turn of the century, to a small number of universities and organisations and to a handful of researchers, developers, activists and practitioners. Several years later the increasing power, functionality, coverage and ownership of mobile phones made them incredibly attractive to policy makers, programme managers, international agencies and humanitarian foundations working in the global South, wanting to provide educational opportunities and resources to people, communities and regions.
There is a list of free and accessible resources in the end section, including those specifically relevant to the global South.
The growth of mobile learning research, the growth of researchers working on mobile learning and the apparently attractive and benign possibilities of mobile learning suggest the need for much research to underpin any subsequent investment, and to underpin policy, capacity building and awareness raising. The current paper is intended to help the funders of research understand and optimize the research that they fund and the research findings that they use.
This paper starts by giving policy makers and other people funding research a practical analysis of different types of mobile learning research in order to articulate more realistically the nature of the insights and findings that they should expect from the mobile learning research community. It also looks briefly at the changes and trends in mobile learning research in order to give a better sense of what to expect in future.
The word ‘research’ can be used to refer to finding information, for example from Google or Wikipedia, or to refer to market research, from people in the street. We, however, are talking about the activities of experienced professional researchers, typically based in a university or institute environment, who gather data in the field or in the laboratory. The purpose of their work is typically to test, endorse, refine or refute some theory or hypothesis. At the highest level of research, for example, for an academic journal or a research conference, we are looking at work that should be:
These claims or attributes must each be substantiated by a comprehensive review of the existing academic literature, and at this highest level, research findings should be guaranteed by various kinds of peer review, iterative revision, impact ratings and community scrutiny.
These do perhaps represent a gold standard, most explicitly adopted in the universities of Western Europe. Standards differ and in many parts of the world, for instance the Middle East and the post-Soviet Republics, PhD research can largely focus on un-critical accounts of existing literature, and in East Asia respect for tradition or authority can hamper original thought.
With an emergent discipline such as mobile learning, there will always be differences in expectations and interpretations from the diverse constituent disciplines, from education (or pedagogy or didactics), mobile technology (or computer engineering), development studies and others. In this respect, however, mobile learning research is maturing and the past decade has seen an international professional body emerge, the International Association for Mobile Learning, along with a recognized scholarly journal, the International Journal of Mobile and Blended Learning, and annual international research conferences, such as mLearn. There are also regional conferences, such as IADIS Mobile Learning in Europe or mLearnCon in US. There are fewer in the global South, perhaps only conferences where the commercial aspect competes with the scientific aspect, such as eLearning Africa. This in itself imposes a pervasive bias onto which research is presented, who presents it and who hears about it.
Processes and standards are in place to ensure and enhance the rigour and thus the trustworthiness of research findings. These research findings constitute the traffic from researchers to policy makers, their funders and any other stakeholders, including the wider public. They are however detailed, structured and highly contextualized, and by their nature, difficult for non-specialists to read with the necessary scrutiny, scepticism and criticality.
This discussion of research may seem to have little significance to programme managers and policy makers. It does however articulate the standards, practices, drivers and culture that researchers inhabit, and helps us understand some of the causes of misunderstanding and miscommunication. Furthermore, research may have a number of roles or functions for policy-makers, donors and officials, and it is important to distinguish these and nurture appropriate expectations.
There is a history of deploying new mobile technologies on a small scale for a fixed term to test technological or pedagogic concepts or to act as exemplars. The technologies are however often ahead of what the market, especially the disadvantaged and underprivileged parts of the market, can afford. These small-scale fixed-term projects may thus have only limited significance for policy makers and may present unacknowledged challenges when policy makers seek to transfer the procedures, technologies and formats from their original context. With these new technologies, there is a history of researchers undertaking research projects that exploit each new hardware, software or network technology as it comes to market, sometimes losing sight of the longer-term impact or conceptual significance of such projects. These projects can represent a distraction for policy makers as, not only might there be questions of sustainability, reliability and affordability, but they are also ahead of the curve in terms of wider acceptance and understanding. There can also be a problem of churn. This is the phenomenon of continually going back to the start and can distract from research that focuses on affordable sustainable innovation around existing robust and ubiquitous technologies; policy makers have a role and have a responsibility in directing researchers towards activities with the most socially useful outcomes. Research projects promoted as exemplars are problematic since they may exemplify ideas that are unaffordable or unsustainable. Research projects that seem to succeed but on the basis of devices supplied – lent or given – to learners are also likely to be promoting ideas that are unaffordable or unsustainable. The same is true for projects that depend on one-off donations, for example, message bundles from mobile network operators. These problems may not arise where the underlying assumption is of specific devices becoming cheaper or network coverage improving.
There are occasionally research projects that propose to take a concept to scale. They are often but not always, preceded by smaller pilots. These larger projects should be designed to answer questions about wider community, organisational and institutional issues, not about pedagogy or technology, and should only proceed with robust, resilient and proven technologies and systems, adequate training and testing, and a clear understanding of the objectives and methods for gathering evaluation data. These are perhaps the most interesting and important research projects for policy makers, ministry officials and programme managers since they align most closely to their own work, interests and responsibilities.
Large-scale deployments and evaluations do not however only measure reactions and test responses, they also have the capacity to create or destroy the expectations, attitudes and goodwill around specific technologies and programmes and so must be planned and presented carefully. They should not be used to test untried technologies; these can be tested by simulation or in labs or with researchers, students or officials rather than the general public or vulnerable learners.
These are only a few of the possible types of research. We could enumerate others, for example, blue-skies, developing theory, testing new pedagogy, implementing new technology, importing approaches or theories tried elsewhere and so on. With each of these types of research comes a particular kind of outcome, a particular kind of finding, and our point here for both researchers and policymakers is to be clear, sceptical and explicit about the expectations, relevance and nature of any particular research idea and approach.
Having looked at research in general, it makes sense to move onto the specifics of mobile learning research and how it is evolving, especially in relation to the global South.
One undeniable observation is that the relatively upbeat global economic climate that supported the early mobile learning research community at the turn of the century is now downbeat, considerably reducing both the public sector and private sector support for research anywhere in the world. Over the same time, interest and understanding of mobile learning to support educational and humanitarian missions in the global South has increased, opening up the possibilities of research funding from international agencies, donors and foundations, and the mobile network operators have seen education as part of their business with a vast market in the global South. At same time we have seen mobile technology change from expensive, difficult, scarce and fragile to cheap, universal, robust, pervasive and ubiquitous, having the effect that whilst previously mobile learning needed research and researchers, now mobile learning has become easy, obvious, common-sense, apparently needing no further research.
Technological research will always be the domain of technology corporations; they have the money and expertise to get ideas to market. What are constantly changing and always varied are the attitudes, expectations, abilities and experiences of learners as they engage with mobile technology, as it transforms the learning they need and the world around them. Whilst technology research, or rather technology development, is universal and not negotiable or culturally specific, research with learners is highly specific to their context and their circumstances and local to the responsibilities of regional and national policy makers and ministry officials.
What is still not well understood is the larger context for technology and pedagogic change, specifically issues of scale and sustainability. Research is still needed but this research by its nature implies a greater commitment of resources than earlier short-term, small-scale research and requires continued scrutiny of context in the search for transferable, durable or generalised findings. This is partly because the issue of culture in mobile learning is so poorly researched and because most research has taken place against the un-remarked backdrop of English or Western culture, pedagogy and assumptions using global technologies and systems and so thus the issue of other cultural contexts seems to have seldom arisen.
Sadly, the professional and financial environment in which researchers work can mean that failure or the appearance of failure is not an option and from the start of the research process, the bidding, to its conclusion, the reports and publications, failure seldom makes an appearance. Our experiences showed that the community was aware of the need to learn from mistakes and failures. Researchers do have the responsibility to publish their findings in order to extend our knowledge of mobile learning. However, this can conflict with perceptions that journal editors prefer positive results and so undocumented mistakes are perpetuated.
We said earlier that a characteristic of research, or research that would be viewed as acceptable by universities, professional bodies, research journals, the press and the wider world, was that it should be ethical, that it should do no harm.
For researchers from the universities of the global North, research on communities in the global South and mobile technologies can pose several sorts of ethical challenge, ones that address whether harm might be done. These include the adequacy of any institutional ethical processes, the novelty, complexity and abstractness of mobile technologies and the difference and distance of any target or host communities, especially poor, marginal or rural ones, from their own experiences, expectations and values.
This connects with the Golden Rule vs. the Platinum Rule, the former telling us to treat others as we wish to be treated, superseded by the latter as people realized that how one might wish to be treated might not be the same as others would want to be treated. This difference can be connected to culture, religion, and societal values and norms—this was understood by many in the research community.
The research community at the symposium revealed a number of ethical practices that should be considered while working with community groups or marginalized people, these include:
These are only some operational hints. There are undoubtedly many more that are relevant.
Some organisations have explicit ethical clearance procedures (IRBs in USA); these may or may not be adequate for research with distant and different cultures using mobile technologies. Other organisations, sometimes ministries, corporations, NGOs, consortia, funders and consultants, do not have such procedures. Policy makers and others funding research should bear in mind that recipient organisations may have no ethics procedures or practices in place and consider their own role in creating appropriate expectations of the project lifecycle, from building in requirements for an ethical evaluation in proposals through to expecting to see ethics documented in publications and reports.
The majority of the participants in this study believed that ethical procedures should be different depending on the country; that target populations should be able to provide insights about their own ethics and that ethical procedures should have some flexibility and not merely represent the powerful.
These procedures, even within the participants’ own country, may be perceived as bureaucratic and paternalistic, producing only documentation and tokenism, and can be dominated by specific disciplines, for example, psychology or clinical medicine, and are thus unsuited to emerging areas of research of an interdisciplinary nature, ones where the nature of harm may be subtle and complex. It is possible to work within these procedures whilst also pressing for their improvement.
Perhaps regrettably, organisational ethics procedures often prioritise a public relations dimension to their work and are keen to make decisions that protect and promote the image of their organisation. The well-documented media panics about education and mobiles, especially with younger children, may be in their minds. They can also be methodologically conservative or biased, and not open to research methods that are aligned to online research with mobiles or culturally sensitive research with indigenous or remote communities.
There has been a variety of initiatives and technologies that use mobiles in the conduct of research in order to capture data, and these could be used to explore learners and learning with mobiles. This would be an authentic alternative to more traditional face-to-face or paper-based methods.
One research participant at the symposium described visionary educators as the catalyst for many mobile learning initiatives in schools. A concern raised at the symposium was how national guidelines were often developed from very little evidence.
There are also projects to get greater insights into how people including learners use their mobiles. These are intended to give educators a greater understanding of how learning content and learning interactions can be designed for people on the move in a wide variety of situations. This research is highly contextualized and highly specific to individual groups and communities.
The primary role of mobile learning researchers was a topic that provided the greatest variety of responses from the symposium participants. These included:
Again, this is only an indicative list but it illustrates the diversity of positions and possibilities.
Any society, community or culture is uniquely characterised by a specific balance between the formal, the established and the institutional on the one hand and the informal, the indigenous, the local, the vernacular on the other hand, and perhaps the counter-cultural, the subversive and the disruptive, too, amongst peer groups, communities, families, kinship groups and elders. This balance can correlate or explain differences in ethics across different communities, ethics in the shared sense of what is approved, acceptable, appropriate, allowable or permissible in terms of interaction, relationship, manners, exchange, humour, posture, language, discourse, fashion and behaviour in any community, in what constitutes harmful and what constitutes harmless. Individuals of course often belong to more than one community and probably aspire to be accepted by several others. Looking at the basis of the ethics in any community exposes the challenge confronting the procedures, practices and documents that attempt to regulate research interventions from a distant institutional culture and community. Countries and regions are also characterised by a multiplicity of religions existing side-by-side. These religions form the foundation of their communities' ideas about ethics and morality but differentiate each from the other. Furthermore, the rapidity of technological, specifically ICT, adoption, change and appropriation leads to generational differences, not always based on chronological age, overlaying other cultural or communal differences.
Any of these norms and characteristics is very different from those of research institutions in the global North and may be unfamiliar to local officials and policy makers. From the perspective of outsiders, of researchers coming from outside, there is much potentially that might be not worth mentioning or be taken for granted and many areas where harm might be inadvertently perpetrated. This is with varying granularity and relevance to communities from nations and religions down to villages and families. Also researchers coming from outside may have an inappropriate perspective on ends and means. An example might help; a pilot in East Africa involved in issuing mobile phones to children in rural areas finished after six months. The researchers retrieved the mobile phones and the children were distraught, having become attached to the phones. The researchers’ response was that they ‘couldn’t make omelettes without breaking eggs’, in effect that the scientific ends justified the scientific means.
Institutions with traditions and culture derived from the North and the West, because of their academic pre-eminence in the global South have been in a position to define research ethics globally and thus define harm on the basis of their own heritage and perspective. We should be aware of Western bias – confusing the responsibility to educate others with the right to educate them and we should see mobile learning research in this more balanced way. Perhaps when, in bygone days, delivering and supporting education was challenging and difficult then the moral imperative to educate was a useful spur. Now however pervasive and intrusive mobile technologies make it much easier to deliver and support education and the moral imperative to reflect first should come into play. Also, researchers from the global North can often assume that other cultures subscribe to the notion that informed consent need only come from the individual not from their community, or from elected representatives. This is a largely Western and individualistic notion. Community understanding of sophisticated and abstract mobile technology systems may grow slowly as research interventions progress. These are arguments for continued community involvement in ethics procedures, for continual communal consent rather than one-off individual consent. This is however further complicated by talk amongst the development community of unexpected consequences – how is the potential harm of interventions to be judged when outcomes seem to be so unpredictable?
Commercial companies and corporations in the mobile learning space, for example handset manufacturers, mobile network operators, broadcasters and content publishers, have sponsored research, either directly, through their corporate social responsibility divisions or through their foundations. They may do this in cash or in kind, and the latter is likely to be their own products or services. They may insist on timescales that researchers find too short for meaningful results, and they may have no formal ethics procedures.
Our research suggested that researchers are conscious of several issues. A concern was that the research conducted on mobile learning was connected with specific corporations providing that funding.
Concerns were echoed in the symposium,
Of course, in all the academic papers the record for research is behind the pay wall, it’s difficult to reach out the evidence.
A symposium participant from the business sector described how many vendors are looking for “proof of concept”, in that they need evidence that will justify the investments they were making in the education sector. The symposium participant described the need to have new products for the market tested with pilot studies.
So in conclusion, most research formats have some relevance to policy makers, ministry officials and programme managers but the diversity of research formats that they might either fund or might review means that considerable judgment and discretion are needed when looking for the policy implications.
This section looks at data, specifically at the different types and sources of data that the mobile learning research community can produce, in order to assist policy makers in the judgements they must make about the impact and relevance of data to policy formulation and implementation.
Research by its nature produces data. Data is however usually less trustworthy and relevant than policy makers and the public might like to think. There is a trade-off between relevance and trustworthiness. This is because trustworthiness will naturally increase if researchers minimize extraneous, confusing or confounding influences and factors. This however produces an increasingly artificial or simplistic research environment, like a laboratory, unlike most real situations. A specific example is the tendency in mobile learning research to issue learners with the same mobile device for the duration of the project. This was logistically straightforward, technically attractive and methodologically more rigorous since it reduced the confusion of getting outcomes from diverse devices. Unfortunately, the likelihood of sustainable mobile learning based on learners being given devices is remote and instead we see the BYOD, bring-your-own-device, policy as the way forward. Unfortunately, all our earlier evidence based on consistent supplied devices is not relevant to such situations.
Furthermore, constraints on time, effort and resources will always mean that there is less data than necessary. This means that researchers must always exercise judgment and skill in order to maximize interest, relevance and impact within these constraints. Sometimes this may mean trade-offs between the duration, spread, intensity and methods being deployed since research budgets are always finite. Various factors compete to consume resources so, for example, longer research must mean shallower research because intense research uses resources more quickly; likewise more sites cost more than fewer sites. So, whatever research formulation is chosen, it imposes limitations on the trustworthiness, relevance and generality of the findings. In particular, qualitative data will cost more to gather and more to analyse than quantitative data – quantitative data will tell you what people do but qualitative data will tell you why they do it.
We should perhaps say something about qualitative data and quantitative data since these are fundamental to research methods and research findings. Put simply, quantitative is usually structured, numerical and objective whilst qualitative is textual and subjective. This means that quantitative data can reveal much about learners’ behaviour but qualitative data is needed to explain it. Typical methods for gathering quantitative data include statistical methods, checklists, structured interviews, system logs and Likert scales whilst qualitative methods include semi-structured interviews, free-text surveys and focus groups but we should point out that these are conventionally face-to-face and not clearly aligned to mobile learners with mobile technologies. This means there are differences in how the data is obtained and thus in the cost of obtaining it.
As expected, within the research community, there was no individual research method that was preferred for researching mobile learning. Good researchers let the research questions determine the methodology used.
The research participants also describe a shift to include qualitative methods that can capture other less quantifiable evidence. A mixed methods approach was suggested by many of the participants – mixed methods describes research that collects both quantitative and qualitative data:
I understand the taxpayer’s wish or pressure again to see how money is spent. There should be a balance to be showed somewhere between, okay, results that can be quantified, also we have all these beautiful dimensions about, dimensions such as engagement, motivation, but also change, the cultural change in teaching practice, all the small elements that constitute what you say, the learning or teaching culture, the mind-set. This is something we cannot fully identify in quantitative thesis.
The research suggests that quantitative methods may be the choice for political gain but do not form the research agenda that can serve the policymakers and practitioners. Policy makers might need research on:
Many of these are policy-related rather curiosity-driven.
In assessing the relative costs and outcomes of different proposed ways of gathering research data (or evaluation data) in the field, policy makers and programme managers must recognize various trade-offs and reservations.
We have ethical and methodological reservations about power differentials between the researchers and their subjects. It does not feel right and the data is not worth having. We have experienced evaluation exercises in East Africa where senior representatives of the Ministry of Education and of the overseas Department of International Development, in a convoy of very expensive vehicles, went into poor rural schools to ask primary school teachers what they thought of the programme in question. Clearly the differentials of power and authority made any responses and reactions worthless.
In looking at the options for gathering research data in the field, there is a tendency to round up the usual suspects – that is, interviews, focus groups, questionnaires – and to avoid options that might be more imaginative and possibly more appropriate to mobiles, to people on the move, to people on the move learning informally and learning together, and to people who are not the usual categories of research subjects or correspondents.
It is always sensible for funders to ask researchers not only about the choice of individual research techniques but to also ask about their sequence and their relationships. Was one technique used to refine or corroborate another one? Did system logs check users’ recollections? Did a questionnaire or survey that gave objective data about behaviour lead to semi-structured interviews that provided data about reasons and motives? Did a broad survey identify areas for deep investigation? Did interviews identify topics for subsequent surveys? And so on. Policy makers reading researcher outputs should be asking questions about individual research techniques. Was it piloted? Is the documentation available? Did it use a recognized method?
This section also looks at data, specifically how the process of selecting, siting and sampling locations for mobile learning pilots and trials can constrain its value and impact in policy formulation, particularly at a regional and national level.
Where trials, evaluations and pilots or samples and test sites get located usually heavily influences the findings, outcomes and conclusions, and certainly does not usually give the kinds of national or regional validity that policy makers and ministry officials might like it to, especially when resources, personnel and time are under pressure. It is tempting, for example, to think that locating a trial or collecting data from every province or district gives findings that are nationally relevant. Locating a trial or collecting data from each tribal or ethnic community or from each type of agricultural community, for example, pastoral, nomadic, subsistence arable and peri-urban market gardening, also seems to provide an equally convincing evidence base so we must ask what is going on when we site trials or sample different groups. What is going on is that we build in the assumption that the variable, for example, tribal group or administrative region, is important and must be accounted for, must be neutralized, by spreading the sampling across its range. This may be true but ignores other variables, perhaps age, income or gender as having a possibly larger impact on our findings. The message to policy makers is that findings that seem to have a national significance may be less trustworthy than they seem at first and that extrapolation, moving from the data gathered in the field to some more general conclusions, may be unreliable however desirable and attractive it may be.
How to choose a sample (group of participants) and a setting (the location) for data collection is an issue that all researchers have to face. The majority of the participants who chose to answer this question said that it was dependent on the research methods that have been chosen. For example, if qualitative methods are selected there may only be a few people in the study, perhaps two or three and this research may take place in a small village. If quantitative methods are selected, the numbers may be in thousands or more, however, this could take place in a city or with multiple villages depending on the criteria being used. The site chosen should be as similar as possible to the proposed generalized area; therefore, if the research is to better understand how high school students use mobile devices to learn about science, the study participants should be high school students.
This section explores the various interpretations and definitions of the concept of 'mobile learning' in order to articulate the practical implications for mobile learning policy, and indeed mobile learning funding and implementation. Experience over the last decade has shown that this seemingly academic issue has considerable practical implications especially for the financing and regulation of mobile learning. UNESCO articulated a definition in its Policy Guidelines that could be used as a starting point. This section will place this definition in a wider context.
There have been a variety of interpretations of what mobile learning might mean but here we attempt to draw the attention of policy makers, programme managers and officials to the implications of this variety.
The earliest definitions focused on mobile technology as the defining characteristic, for example saying that mobile learning "...was any learning where mobile or portable or handheld technology was the main or sole delivery mechanism." These definitions were problematic because technology, its ownership and its use always moved on, leaving earlier definitions looking out-of-date and underpowered. There can also be problems with policy-making that rests on techno-centric definitions, e.g., as the basis for funding priorities or regulatory frameworks. One example is the UK MoLeNET programme for the TVET sector, amounting to some 14m GBP over three years and devoted to hardware and infrastructure. This defined mobile learning in terms of mobile technologies and then encountered problems adjudicating on exactly which maximum screen size constituted a mobile device, and also fighting the then-fashion for the UMPC, the so-called ultra-mobile PC of 2006.
Subsequent perceptions and definitions have shifted on to focus on the mobility of the learning and the learners, especially when they move across contexts, settings and situations. The focus has shifted to initiatives within education systems and their institutions to enrich and extend the existing ideas and practices of learning. The former has often sought to deploy high-end and emergent technologies within small research contexts using technology supplied to the learners. This has produced interesting and rigorous results but ones that are only trustworthy in situations where learners can be supplied with the necessary technology. The latter has used mobile technology to reach people and communities for whom other educational interventions would be too costly, difficult or dangerous. This obviously includes rural areas, sparsely populated areas, areas of poor infrastructure and challenging environment, and areas with no secure buildings and human resources to support any other technology or delivery. It also includes reaching marginal, sometimes stigmatized, communities, people from communities without the money, confidence or familiarity to exploit any other technology or delivery of education, and people unable to access other interventions because of cognitive or physiological disadvantages or disabilities. This perspective has one set of policy implications, and continues to be the dominant paradigm. It works with the existing institutions, professions, curriculum and ministries. It is however problematic in the context of fixed, stable, static institutions and professions in mobile, moving societies.
There is however also a cluster of definitions that emphasis the centrality of mobility and connectedness as a defining characteristic of modern societies and cultures anywhere; these definitions place learning with mobiles outside formal education; they see learning with mobiles as only a small part of people’s mobile lives and their lives with mobiles; they focus on the capacity of people and their mobiles to generate, share, transform, discuss ideas, information and images, that is, the capacity of people to shape their own mobile learning.
These different perceptions and definitions have very different implications for research methods, and for policy priorities; and for relationships between policy-makers, researchers and types of learners, their motivations, goals, attitudes and styles.
Many ministries and institutions are addressing the sustainability of mobile learning by considering BYOD (bring-your-own-devices) policies for learners, thereby displacing the expense of handsets and connectivity onto learners themselves but introducing learner agency, control and expectations into the equation.
Given that most schools in most countries have out-right prohibitions of mobile devices still in place, the current policy environment is something of a muddle. Research into attitudes, expectations and pedagogies should all be on the policy makers’ agenda if they want effective policy and sustainable deployment and implementation of learning with mobiles.
Some schools, usually acting on their own initiative rather than responding to national or regional prompting, have engaged in discussions with their learners designed to devise a new consensus about bringing mobile phones to and using them in schools, recognising that older children may be the head of a family deprived of parents and perhaps recognising the futility of on-going prohibition. Increasingly, there is software in use capable of enforcing a more nuanced policy for mobiles on school premises.
There is also vast amount of what we might call user-generated learning, that is, content, conversations and communities produced by learners or users themselves rather than by institutions, agencies and ministries. This represents a vast resource and opportunity for policy makers to exploit once they have used research to scope and map it, and to develop processes to monitor, endorse and publicise it. It consists, for example, of podcasts, Facebook groups, smartphone apps, web sites and social networks but is constantly changing and sometimes of dubious quality and doubtful provenance.
This section looks at the challenges and pitfalls inherent in different kinds of mobile learning research and how these different kinds are evolving, in order to calibrate expectations for policy makers. The nature of the ethical dimension of mobile learning and mobile learning research are explored in order to help policy makers develop more robust, consensual, appropriate and transparent mechanisms designed to reduce the risk of harm, especially in programmes and interventions with the disadvantaged and the marginal.
The following sections address briefly a succession of questions,
Educational research is difficult. This is because education is complex, situated and life long, making it difficult to attribute educational (or personal, social, economic or political) success in later life to specific educational interventions. These successes are in fact always multi-causal and they are often, to use a phrase popular in international development, ‘unexpected consequences’. Caution amongst educational policy-makers has perhaps led to preference for overly ’scientific’ or empirical methods, ones that treated education as a laboratory experiment, and for a predominance of short-term pre-test/post-test perspectives. Policy makers need to be aware of the complexity of the phenomena that they commission researchers to explore.
There is furthermore a global outbreak of over-testing and this sadly might seem to offer masses of free data for researchers to explore on behalf of policy makers.
In general, there is always a risk of measuring what is easy, cheap or quick to measure and avoiding measuring what is difficult, expensive or sustained. This can mean that simple, numerical and quantitative data and surveys are preferred to complex, qualitative and textual data. Technological systems and online methods, for example mobile phone network infrastructure and users’ handsets, can produce masses of ‘cheap’ and apparently independent data whilst semi-structured interviews are skilled and expensive. This can lead to a focus on objective outcomes and behaviours at the expense any understanding of the motivations and values that lay beneath them, and can lead to attributing effects to the causes that were measured rather than the ones that were not.
It is easier to measure formal learning than informal learning since the inputs, objectives, outcomes and environment can be documented and controlled. It is easier to measure the impact of training than of education, easier to measure skills acquired and knowledge acquired than it is to measure confidence gained and attitudes changed. It is easier to measure lower order skills than higher order ones. These remarks take on increased significance when we think of the potential of mobiles for informal learning, adult learning and community learning, and when we think of the potential in accessing the wealth of online resources, communities and ideas through the portal of the mobile. Employers globally talk of the economic importance of increasing entrepreneurship and building ‘soft’ skills, both areas where research and measurement are more difficult than similar work on traditional ‘hard’ skills, and experts globally talk of the need for lifelong learning. Mobiles are a universal, sustainable and effective technology for supporting, enhancing and delivering them all but policy makers need specific research to identify the best interventions, priorities and resources to deploy.
Mobility research, meaning research on the social and economic impact of mobile technologies and the mere fact of moving, is difficult. Mobility is a commonplace part of everyday life, not unusual, separate and isolated like using computers. Outcomes, changes and impacts are yet again multi-causal, long term and diffuse; they are social and affective, not just cognitive and objective; whilst researchers talk of unexpected consequences, people do not talk about undocumented causes. Credible, authentic and naturalistic research is difficult – researchers are still struggling to observe and measure social interactions and social practices without inhibiting and distorting what they are trying to observe. More powerful and more compact and portable technologies make this increasingly easier but never as easy as research on stationary people in static situations. In general, researchers know very little about contemporary mobile digital habits, especially in marginal or remote communities. The rapidity of technical change, the rapidity with which different cultures adapt and adopt the technology and the diversity and fluidity of these different cultures means that educators and policy makers have little idea about the ways in which mobiles fit into the lives of people, especially the lives of people least like themselves.
One participant at the symposium echoed this sentiment, saying,
I think it is also important to note that the research agenda for mobile learning is going to be much different, the devices are different, the scale is much larger, but also the contextual use is different. So whereas I may have approached E learning as part of a formal course, there can be social and informal learning which is one of the hallmarks of mobile learning.
Fortunately, there is a growing body of research, under the banner of m4d, mobiles-for-development, looking at the social and economic impact of mobiles on people in the developing regions of the world, the impact on fisher-folk, subsistence farmers, small traders and the sick. There is for mobile learning researchers and policy makers the need and the challenge to reach across to these other research communities; working with farmers, fishers or the sick represents an opportunity to develop and deliver mobile learning that is meaningful and integrated within the needs and aspirations of specific communities. Policy makers need to be aware of the breadth of inter-related disciplines from which they could commission research.
There is also a wealth of technical data available from networks and from devices that might for example give all of the SMS traffic through a dedicated support system or details of context, that is, time and location, of selected user activity. This kind of research may be cost-effective in terms of coverage and scale but is highly technical, in terms of accessing the data and storing it and analysing it, and ethically probably problematic, because of perceptions of surveillance and risks to confidentiality and anonymity but nevertheless has lots of potential.
Mobile technology is pervasive, ubiquitous, nearly universal and potentially intrusive. As the vehicle for educational interventions, it has infinitely greater capacity to reach into people's lives than computers or televisions and to reach beyond the traditional systems, institutions and buildings of education and into families and communities; consequently the ethical dimension of mobile learning research and mobile learning policy must be an integral component of research, policy and practice.
This section gives policy makers an overview of the context and background in which the mobile learning research community operates in order to understand how these contextual factors shape the capacity of the research community to contribute to policy. We look briefly at how research is managed, funded, reviewed, staffed and resourced, how researchers’ careers, opportunities and incentives are structured. These issues have a bearing on what research gets done or not done, and how it gets done, but also on how policy makers should view their relationships and interactions with researchers.
Most researchers work in universities or institutes, though whether these are in the global North or the global South makes a vast difference. Researchers are always under pressure to bring in extra funds and will bid for external projects and programmes. To build a research career they must be successful in bidding and in the global North their institute or university will have a sophisticated infrastructure to help them win bids. Compared to researchers in the global South, however, those in the North are much more highly paid and have travel and subsistence costs that are higher than their competitors in the universities and institutes of the global South. We see here the outlines of a cycle whereby successful researchers, most likely in the global North, become more successful, because of their track records, and other researchers, more likely in the global South, have trouble breaking into this cycle. The Strengthening Information Society Research Capacity Alliance (SIRCA) programme of IDRC, the International Development Research Centre, is one example trying to build capacity in the global South to break into this cycle.
For a research career in the global North, publications in respected journals are recognized steps in professional development and advancement and are part of developing a professional portfolio. Publication in research journals is sometimes rewarded in developing countries but usually not, especially not for novel and emergent disciplines. There must also be an open question about whether peer review (usually of the researchers in the global North by the researchers in the global North) reinforces certain perspectives and positions and ignores others. Many developing countries have capacity problems in relation to academic writing, research supervision, proposal writing and in managing projects and consortia so that the development of any research domain and research culture is skewed in favour of the voices, the players and the opinions of the global North and consequently the impact on policy making is skewed too. Conference attendance and conference presentation are often seen as stepping-stones to publication, as well as networking opportunities for subsequent writing, consortia and bidding. Lack of expertise and experience in hosting and managing research conferences means that relatively few take place in developing countries. Furthermore, these countries have limited budgets for academic travel so attendance at overseas research conferences is limited. These are part of the wider concerns for research in the global South.
The academic career structure in many developing countries does not incentivize, promote or reward research (as opposed to teaching or management). This means that local research capacity and expertise develop very slowly, perhaps producing a reliance or dependency on external and foreign researchers (or less qualified consultants acting as researchers). This might be acceptable for technology projects but perhaps not for social science projects, those that involve engaging with local users, officials, managers and other stakeholders. Many foreign or external researchers need to use or choose to use local research assistants or local institutional partners. This is one way in which local capacity can be built. It does however raise questions about whether foreign or external researchers should be required as a condition of their funding to work with local people and local partners, and additional questions about whether they should earmark part of their time and budget to training and mentoring activities. Other ways in which local capacity can be built are research exchanges, bursaries and scholarships at universities and institutes in the global North. These are valuable but only if there are mechanisms to embed the experience and expertise afterwards and only if there is a critical and on-going exploration about which knowledge, values, perspectives and techniques from the global North are transferable and relevant and which are not.
Many experts believe that bringing in local researchers is important. The research community had plenty to describe the positives gained from having local researchers involved in a study, saying:
It is absolutely important to employ/involve local researchers in addition to external researchers because local researchers will be able to bring local socio-cultural contexts in research data collection and analysis process which will be valuable to produce credible research evidence on what works and what doesn’t work. Moreover, local researchers can improve their skills and build their confidence in conducting similar research in the future, they can play a critical role in influencing policies by reaching out to decision makers and sharing research findings.
Other benefits include the local researcher having the ability to:
These are issues that policy-makers can help with throughout the research project life-cycle.
For policy makers working in developing countries, these very general remarks have a range of implications. The first is merely the need for heightened awareness of the factors at work when they deal with researchers and research, the second may be a call to review how research is funded and how research calls are constructed, selected, monitored and integrated into longer-term strategic planning and policy making. These implications do not probably have any extra relevance to learning with mobiles in developing countries except insofar as this is relatively suddenly a popular topic with perhaps less cumulative critical experience and expertise, and with considerable popular, political and media interest.
This has been a largely personal and informal attempt to make sense of the relations and interactions between researchers and policy-makers, both of them working with mobile technologies in the global South to support and deliver learning. The two communities both seek in their different ways and with differing emphasis to increase what we know and to use what we know for the benefit of the disadvantaged and the marginal so any attempt to improve this work must be worthwhile and a valuable foundation to more detailed follow up. There is clearly work that can be done at various levels. At the highest level, this work should build on the symposium described here, bringing together more researchers and more policy-makers and exploring the observations and impressions that we document here, perhaps as a community of practice, under the aegis of the Commonwealth of Learning or UNESCO, or national agencies (and funders) such as DFID, USAID or IDRC. This would trickle down to lower levels and specialist communities.
Acknowledgements
The author would like to thank UNESCO, specifically Mark West, for supporting the symposium and making notes and recordings available, and to Dr Helen Crompton of Old Dominion University for providing transcription and feedback.
Author:
John Traxler is Professor of Digital Learning at the Institute of Education, University of Wolverhampton, UK. He is a Founding Director of the International Association for Mobile Learning, Associate Editor of the International Journal of Mobile and Blended Learning and of Interactive Learning Environments. He is on the Research Board of the Association of Learning Technology, the Editorial Board of Research in Learning Technology and IT in International Development. John has co-written a guide to mobile learning in developing countries and is co-editor of the definitive book, Mobile learning: A handbook for educators and trainers, with Professor Agnes Kukulska-Hulme. E-mail: John.Traxler@wlv.ac.uk