Each week David Souter comments on an important issue for APC members and others concerned about the Information Society. This week's blog post looks at different ways to measure the Internet.
What do we mean by measuring the Internet? There are many different ways to answer:
-
Should we measure inputs, for example – the quantity and quality of infrastructure, the number of devices or domain names?
-
Should we measure outputs – how much businesses and individuals use the Internet or social media?
-
Or should we measure impacts – the difference that Internet access and use make to the ways in which economies and societies evolve?
The best answer to those questions is, of course, all three. For a full picture of the Internet – one which will design policies that maximize its value and minimise its problems – we need to understand all three. And that is difficult, for two main reasons.
The data challenge
First, ironically (given what Internet businesses now do with data), we don’t yet have the data. ISPs and companies like Google, Facebook, Amazon, mine our data for competitive advantage, but policymakers in most countries don’t have accurate numbers about Internet use among their citizens. The data that are published are often estimates.
Second, while it should be possible to quantify inputs and outputs, access and usage, it’s much harder to quantify impacts. The changes that result from the Internet’s growing influence on society, economy and culture, aren’t always immediately obvious. Impacts grow and change over time and as the Internet itself evolves from email to social media, from search to electronic commerce, from small to larger user groups. Vested interests also want to play up impacts that make them look positive and play down those that don’t.
What about the Internet environment?
Many impacts, in short, are hard to quantify. If we want to know what’s really going on we have to use qualitative evidence as well as numbers.
And the same is true of Internet environments. Assessing whether these ‘enable innovation and investment’, ‘foster empowerment’ or ‘facilitate surveillance’ isn’t something we can do with data only. It requires analysis, judgement and some degree of subjectivity. It’s still worth trying, though.
UNESCO’s Internet Universality concept
Which brings me to a new project managed by APC, for which I’m acting as research coordinator. The project’s to develop indicators for the ‘Internet Universality’ concept which was adopted by UNESCO in 2015.
The concept’s not concerned with the Internet as a whole, but with aspects of it which fall within UNESCO’s mandate to build Knowledge Societies. It identifies four principles which UNESCO considers fundamental to fulfilling the Internet’s potential for sustainable development:
-
that it should be built round human RIGHTS
-
that it should be OPEN
-
that is should enable ACCESS
-
and that it should be governed through MULTISTAKEHOLDER cooperation.
Finding indicator
From their initial letters UNESCO calls these the ROAM principles, and it’s seeking indicators for them. Not in order to make league tables that imply one country’s better than another, but to help governments and other stakeholders in any country to explore their Internet environments and see how they can move them towards an Internet that’s based round those four principles of Rights, Openness, Access and Multistakeholderism.
There’s a model for this in UNESCO’s Media Development Indicators, which were agreed in 2008. They’re concerned with traditional media, and they’ve been used to assess media environments in some thirty countries.
As well as the ROAM principles, there’ll be a special focus on gender and the Internet’s impact on children and young people. And as well as the principles themselves, there’ll be a section on cross-cutting issues, synergies and challenges.
Your opportunity to participate
APC and its partners in the indicators project – ict Development Associates (my consultancy) and three regional research centres (Research ICT Africa, LIRNEasia and DIRSI) – have been asked to identify existing indicators and develop new ones that will meet the need.
We’re approaching this through desk research into existing indicators and through online consultation. We’re asking anyone that’s interested to tell us what they think. There’ll be two phases of this consultation:
-
The first was launched at the WSIS Forum last week in Geneva. It’s concerned with general issues. What are your priorities for indicators? What is your experience with specific indicators? What do you think are the biggest gaps in indicators? How would you use these indicators once they are in place? You can contribute to this consultation – in any of the six United Nations languages – on the UNESCO website here.
-
We’ll use your contributions to set out options for indicators in an interim report that will be published in November. There’ll be a second consultation on those options, before a final report’s produced in April next year.
UNESCO will also arrange consultation events at Internet meetings during the next twelve months.
The challenges
It’s obvious, of course, that indicators for these principles can’t just be quantitative. Qualitative indicators, too, are going to be needed to assess rights, openness and multistakeholder participation, even access (for example, whether there are appropriate legal frameworks in areas like universal access or data protection). The final indicator framework will be a mix of different approaches.
That framework must also be viable in the majority of countries, including those with limited capacity for monitoring and measuring. Quantitative indicators need to be collectable in such environments; and qualitative indicators must be capable of being assessed consistently.
The indicator framework will have to be selective. The Media Development framework includes 47 indicators, most with more than one sub-indicator. That’s a lot to gather and assess. Even so, it isn’t comprehensive – nor can it be if it is to be feasible for small groups of analysts, researchers or policy designers.
We’re at the start here of a year-long process. The outcome of desk research and consultation should be a framework that can be used by anyone – governments or international organisations, researchers or human rights defenders, journalists or academics – to build a picture of the state of play in individual countries and suggest new policy directions.
The bigger picture
UNESCO’s Internet Universality indicators will add a new dimension to assessments of national Internet environments.
That will be concerned, though with UNESCO’s priorities to build Knowledge Societies and maintain human rights. These are important aspects of the Internet, but they’re not comprehensive. Measuring the Internet – its inputs, outputs and especially its impacts – is a constantly expanding task. Each year – each day, even – more people are doing more things on the Internet, using more bandwidth and more services.
The impact this has on societies, economies and cultures is increasingly profound and hard to measure. Even existing data sets fall quickly out of date amid such rapid changes in technology and markets. And that’s before we add the interaction between the Internet and other aspects of today’s emerging Information Societies, like the Internet of Things, big data and AI.
It’s important that we analyse these changes, and that we do so not just from the perspective of the Internet but from that of society, economy and culture as a whole.
And it’s important that, in doing so, we look to the future rather than the past. We need ways of assessing what the Internet is doing that are future-proof: that will be as relevant in 2027 as they are today. We need, in short, frameworks that can cope not just with known knowns but with known unknowns and, if possible, unknown unknowns. That’s very challenging.
In the meantime, why not help make a start by turning to UNESCO’s consultation page and tell UNESCO what you think’s the right way forward for measuring Internet Universality?
Next week I’ll look at the relationship between politics and social media.
Image: The ROAM Principles. UNESCO's project reports