posts_gdocs: 1KKAs2S3PBqQLeDtaC-QrSGT0IzewyNF7DyGToEIIb1M
Data license: CC-BY
This data as json
| id | slug | type | content | published | createdAt | publishedAt | updatedAt | publicationContext | revisionId | breadcrumbs | markdown | title |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1KKAs2S3PBqQLeDtaC-QrSGT0IzewyNF7DyGToEIIb1M | choosing-our-topics-and-metrics | article | {
"toc": [
{
"slug": "undefined-acknowledgements",
"text": "Acknowledgements",
"title": "Acknowledgements",
"isSubheading": false
}
],
"body": [
{
"type": "text",
"value": [
{
"text": "On ",
"spanType": "span-simple-text"
},
{
"children": [
{
"text": "Our World in Data",
"spanType": "span-simple-text"
}
],
"spanType": "span-italic"
},
{
"text": ", we present thousands of metrics on hundreds of topics.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"text": "But there are many more topics that we could work on, and metrics we could present. How do we choose them?",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"text": [
{
"text": "How do we choose which topics to cover?",
"spanType": "span-simple-text"
}
],
"type": "heading",
"level": 1,
"parseErrors": []
},
{
"type": "text",
"value": [
{
"text": "We cover a topic if we believe it helps our readers understand one or several of the world\u2019s largest problems. More specifically, this means that a topic will fit many of the following criteria:",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "list",
"items": [
{
"type": "text",
"value": [
{
"children": [
{
"text": "It affects many countries and people.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This can mean that it concerns every person, such as ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/health-meta",
"children": [
{
"text": "health",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ". It can mean that it affects many people in all countries, such as ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/poverty",
"children": [
{
"text": "poverty",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ". Or it can mean that it affects many people in fewer countries, such as ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/malaria",
"children": [
{
"text": "malaria",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ".",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It comes with great costs or benefits. ",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": "The costs or benefits can be direct, and shorten people\u2019s lives or mean they lead happier lives. An example is the ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/coronavirus",
"children": [
{
"text": "COVID-19 pandemic",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ", which has immediately affected people\u2019s well-being. But the costs or benefits can also be indirect, and worsen or alleviate other problems. An example is ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/agricultural-production",
"children": [
{
"text": "agricultural production",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ", which affects many people\u2019s access to nutrition.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It poses significant risks.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This means that it may not impose great costs at the moment, but may do so in the future. An example is ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/nuclear-weapons",
"children": [
{
"text": "nuclear weapons",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ", which have not been used in decades but whose use would be devastating.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It will remain important, or become more important in the future. ",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"url": "https://ourworldindata.org/poverty",
"children": [
{
"text": "Poverty",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": " is an example of a topic that will remain important, as many people remain impoverished even if fewer people live in extreme poverty than in the past. ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/artificial-intelligence",
"children": [
{
"text": "Artificial intelligence",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": " is an example of a topic that will become more important, as technological advances continuously expand its effects on people\u2019s lives.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is helpful to understand other topics.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " Many of the topics we focus on are problems in themselves. But we also provide data and research on major changes that help us understand and address these problems. An example is ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/world-population-growth",
"children": [
{
"text": "population changes",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ", which are crucial to better understand energy and education needs.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is poorly understood.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This means the public knows little about a problem or frequently misunderstands it, such as when related data is not described well. An example is ",
"spanType": "span-simple-text"
},
{
"url": "http://ourworldindata.org/plastic-pollution",
"children": [
{
"text": "plastic pollution",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ", where data and research were often missing from the public conversation.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is neglected elsewhere.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This can mean that other organizations do not cover it, or do so in a limited fashion. An example is ",
"spanType": "span-simple-text"
},
{
"url": "http://ourworldindata.org/biodiversity",
"children": [
{
"text": "biodiversity",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ", where data on global changes are hard to find elsewhere. This also means that if others cover a topic well, we are less likely to cover it ourselves.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It has data on it.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This means that other organizations or individuals are collecting and publishing data on the topic. We do not collect data ourselves, and instead make the data and research of others easier to access and understand.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "We have expertise on it.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " If we have someone on our team with deep knowledge of the area, we are more likely to cover the topic. Ideally, we would have both a researcher and a data scientist with this expertise. An example is ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/democracy",
"children": [
{
"text": "democracy",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ", where we expanded our work as our team grew.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "We have funding available for it. ",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": "While most of our funding comes from unrestricted resources, including ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/donate",
"children": [
{
"text": "reader donations",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ", we partially fund our work through grants that cover work on specific areas. Importantly, we only apply for these grants if we have editorial independence: that they are on topics we want to cover in depth anyway, and there are no requirements on ",
"spanType": "span-simple-text"
},
{
"children": [
{
"text": "how",
"spanType": "span-simple-text"
}
],
"spanType": "span-italic"
},
{
"text": " to cover the topic.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"text": "We evaluate ourselves how a topic fits these criteria. But we rely heavily on related research, especially research that is peer-reviewed.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"text": [
{
"text": "How do we choose which metrics to provide?",
"spanType": "span-simple-text"
}
],
"type": "heading",
"level": 1,
"parseErrors": []
},
{
"type": "text",
"value": [
{
"text": "For each topic, we work to provide the best metrics to understand it. What metrics are 'best' will often depend on our specific questions. Overall, a metric we provide will fit many of the following criteria:",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "list",
"items": [
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is quantitative",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": ". This can mean numerical measures, such as the number of people living in extreme poverty; or categorical measures, such as classifications of countries as either democratic or autocratic. We do not systematically provide information that is highly specific, such as personal experiences of living in poverty or under dictatorship.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It captures what we are trying to measure",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": ". This means that it neither leaves out anything essential, nor includes anything irrelevant. For example, an inadequate measure for whether a country is a democracy is the share of the population that voted. Looking only at voter turnout ignores whether citizens had more than one choice at the ballot box. And at the same time, it inadvertently considers citizens that were coerced to vote.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It covers large parts of the world. ",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": "True to our name, we seek metrics which cover as much of the world as possible. Only then can they help us understand global differences and changes.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It covers a lot of time. ",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": "This means both that the measure goes as far back in time as possible, and that it is as recent as possible. It then can help us understand both historical and very recent developments.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is comparable across time and space",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": ". This means that we prefer metrics that can be compared across years and countries. This allows us to evaluate whether countries are making progress or falling behind, and how countries are doing relative to another.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is reliable.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This means that the metric is consistent, i.e. it captures the phenomenon similarly when measured repeatedly, and therefore is precise, and captures the phenomenon with little error. A consistent and precise metric makes us more confident in what it tells us about the world.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "Its construction is transparent.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This means that we prefer metrics that come with a detailed description of how it was constructed, why it was constructed in this way, and with the underlying code and raw data. We, and you as our reader, then can evaluate its strengths and weaknesses in detail.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is easy to understand",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": ". This means that the metric captures something that people are broadly familiar with, and they can broadly make sense of its construction. It then can provide answers that people beyond experts can learn from.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is difficult to misuse.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This means that the metrics cannot easily be taken out of context and be misrepresented. This does not mean that the data cannot have flaws \u2014 we still provide flawed data if we think that we can learn something from it.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is maintained well.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This means that the data source updates the metric frequently, and provides reasonably up-to-date data. We often favor data from international institutions (such as the World Bank and the UN) and research institutions (such as the Global Carbon Project and the Varieties of Democracy project) over data from individual academic publications, because the former have the mandate and resources to keep this data up-to-date.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "Its values differ a lot from the same measure by another trusted source.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This means a metric captures disagreement across sources. It then helps us to be appropriately uncertain of our answers in light of disagreeing sources.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "It is accessible.",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": " This means that the data is published in a publicly accessible document and is licensed to be reused by us and preferably others. Only then can it help people answer their questions, on and beyond our site.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"children": [
{
"text": "We have the tools to visualize it. ",
"spanType": "span-simple-text"
}
],
"spanType": "span-bold"
},
{
"text": "This means a metric is structured such that our in-house visualization tool \u2014 the ",
"spanType": "span-simple-text"
},
{
"url": "https://ourworldindata.org/owid-grapher",
"children": [
{
"text": "Our World in Data Grapher",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": " \u2014 can display its information well. For example, our maps are set up to visualize national data, and currently cannot display metrics at the sub-national level or gridded data.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
}
],
"parseErrors": []
},
{
"type": "text",
"value": [
{
"text": "The topics and metrics we present are not set in stone, and we keep thinking about which ones to add. So if you think a topic or metric fits the criteria outlined here, please reach out to us at ",
"spanType": "span-simple-text"
},
{
"url": "mailto:[email protected]",
"children": [
{
"text": "[email protected]",
"spanType": "span-simple-text"
}
],
"spanType": "span-link"
},
{
"text": ".",
"spanType": "span-simple-text"
}
],
"parseErrors": []
},
{
"text": [
{
"text": "Acknowledgements",
"spanType": "span-simple-text"
}
],
"type": "heading",
"level": 2,
"parseErrors": []
},
{
"type": "text",
"value": [
{
"text": "I thank Edouard Mathieu, Esteban Ortiz-Ospina, Hannah Ritchie, and Max Roser for their very helpful comments and ideas about how to improve this article.",
"spanType": "span-simple-text"
}
],
"parseErrors": []
}
],
"refs": {
"errors": [],
"definitions": {}
},
"type": "article",
"title": "How we choose which topics to work on, and which metrics to provide",
"authors": [
"Bastian Herre"
],
"excerpt": "On Our World in Data, we present thousands of metrics on hundreds of topics. How do we choose them?",
"dateline": "February 27, 2023",
"subtitle": "On Our World in Data, we present thousands of metrics on hundreds of topics. How do we choose them?",
"featured-image": "default-featured-image.png"
} |
1 | 2023-07-20 13:19:05 | 2023-02-27 08:30:01 | 2024-03-18 15:41:59 | listed | ALBJ4LvVc6xZb7wKlK2Z3MSQO5qCNEVSFT71DURoN65rc2DrUTBfTSexkrzUkDKpfsh_fmkEG436uxVnObsbZw | On _Our World in Data_, we present thousands of metrics on hundreds of topics. But there are many more topics that we could work on, and metrics we could present. How do we choose them? # How do we choose which topics to cover? We cover a topic if we believe it helps our readers understand one or several of the world’s largest problems. More specifically, this means that a topic will fit many of the following criteria: * **It affects many countries and people.** This can mean that it concerns every person, such as [health](https://ourworldindata.org/health-meta). It can mean that it affects many people in all countries, such as [poverty](https://ourworldindata.org/poverty). Or it can mean that it affects many people in fewer countries, such as [malaria](https://ourworldindata.org/malaria). * **It comes with great costs or benefits. **The costs or benefits can be direct, and shorten people’s lives or mean they lead happier lives. An example is the [COVID-19 pandemic](https://ourworldindata.org/coronavirus), which has immediately affected people’s well-being. But the costs or benefits can also be indirect, and worsen or alleviate other problems. An example is [agricultural production](https://ourworldindata.org/agricultural-production), which affects many people’s access to nutrition. * **It poses significant risks.** This means that it may not impose great costs at the moment, but may do so in the future. An example is [nuclear weapons](https://ourworldindata.org/nuclear-weapons), which have not been used in decades but whose use would be devastating. * **It will remain important, or become more important in the future. **[Poverty](https://ourworldindata.org/poverty) is an example of a topic that will remain important, as many people remain impoverished even if fewer people live in extreme poverty than in the past. [Artificial intelligence](https://ourworldindata.org/artificial-intelligence) is an example of a topic that will become more important, as technological advances continuously expand its effects on people’s lives. * **It is helpful to understand other topics.** Many of the topics we focus on are problems in themselves. But we also provide data and research on major changes that help us understand and address these problems. An example is [population changes](https://ourworldindata.org/world-population-growth), which are crucial to better understand energy and education needs. * **It is poorly understood.** This means the public knows little about a problem or frequently misunderstands it, such as when related data is not described well. An example is [plastic pollution](http://ourworldindata.org/plastic-pollution), where data and research were often missing from the public conversation. * **It is neglected elsewhere.** This can mean that other organizations do not cover it, or do so in a limited fashion. An example is [biodiversity](http://ourworldindata.org/biodiversity), where data on global changes are hard to find elsewhere. This also means that if others cover a topic well, we are less likely to cover it ourselves. * **It has data on it.** This means that other organizations or individuals are collecting and publishing data on the topic. We do not collect data ourselves, and instead make the data and research of others easier to access and understand. * **We have expertise on it.** If we have someone on our team with deep knowledge of the area, we are more likely to cover the topic. Ideally, we would have both a researcher and a data scientist with this expertise. An example is [democracy](https://ourworldindata.org/democracy), where we expanded our work as our team grew. * **We have funding available for it. **While most of our funding comes from unrestricted resources, including [reader donations](https://ourworldindata.org/donate), we partially fund our work through grants that cover work on specific areas. Importantly, we only apply for these grants if we have editorial independence: that they are on topics we want to cover in depth anyway, and there are no requirements on _how_ to cover the topic. We evaluate ourselves how a topic fits these criteria. But we rely heavily on related research, especially research that is peer-reviewed. # How do we choose which metrics to provide? For each topic, we work to provide the best metrics to understand it. What metrics are 'best' will often depend on our specific questions. Overall, a metric we provide will fit many of the following criteria: * **It is quantitative**. This can mean numerical measures, such as the number of people living in extreme poverty; or categorical measures, such as classifications of countries as either democratic or autocratic. We do not systematically provide information that is highly specific, such as personal experiences of living in poverty or under dictatorship. * **It captures what we are trying to measure**. This means that it neither leaves out anything essential, nor includes anything irrelevant. For example, an inadequate measure for whether a country is a democracy is the share of the population that voted. Looking only at voter turnout ignores whether citizens had more than one choice at the ballot box. And at the same time, it inadvertently considers citizens that were coerced to vote. * **It covers large parts of the world. **True to our name, we seek metrics which cover as much of the world as possible. Only then can they help us understand global differences and changes. * **It covers a lot of time. **This means both that the measure goes as far back in time as possible, and that it is as recent as possible. It then can help us understand both historical and very recent developments. * **It is comparable across time and space**. This means that we prefer metrics that can be compared across years and countries. This allows us to evaluate whether countries are making progress or falling behind, and how countries are doing relative to another. * **It is reliable.** This means that the metric is consistent, i.e. it captures the phenomenon similarly when measured repeatedly, and therefore is precise, and captures the phenomenon with little error. A consistent and precise metric makes us more confident in what it tells us about the world. * **Its construction is transparent.** This means that we prefer metrics that come with a detailed description of how it was constructed, why it was constructed in this way, and with the underlying code and raw data. We, and you as our reader, then can evaluate its strengths and weaknesses in detail. * **It is easy to understand**. This means that the metric captures something that people are broadly familiar with, and they can broadly make sense of its construction. It then can provide answers that people beyond experts can learn from. * **It is difficult to misuse.** This means that the metrics cannot easily be taken out of context and be misrepresented. This does not mean that the data cannot have flaws — we still provide flawed data if we think that we can learn something from it. * **It is maintained well.** This means that the data source updates the metric frequently, and provides reasonably up-to-date data. We often favor data from international institutions (such as the World Bank and the UN) and research institutions (such as the Global Carbon Project and the Varieties of Democracy project) over data from individual academic publications, because the former have the mandate and resources to keep this data up-to-date. * **Its values differ a lot from the same measure by another trusted source.** This means a metric captures disagreement across sources. It then helps us to be appropriately uncertain of our answers in light of disagreeing sources. * **It is accessible.** This means that the data is published in a publicly accessible document and is licensed to be reused by us and preferably others. Only then can it help people answer their questions, on and beyond our site. * **We have the tools to visualize it. **This means a metric is structured such that our in-house visualization tool — the [Our World in Data Grapher](https://ourworldindata.org/owid-grapher) — can display its information well. For example, our maps are set up to visualize national data, and currently cannot display metrics at the sub-national level or gridded data. The topics and metrics we present are not set in stone, and we keep thinking about which ones to add. So if you think a topic or metric fits the criteria outlined here, please reach out to us at [[email protected]](mailto:[email protected]). ## Acknowledgements I thank Edouard Mathieu, Esteban Ortiz-Ospina, Hannah Ritchie, and Max Roser for their very helpful comments and ideas about how to improve this article. | How we choose which topics to work on, and which metrics to provide |