Overton has updated the Sustainable Development Goals filter on the platform to make it more precise and help users better understand the impact they’re having in priority areas.
Since the development of the Sustainable Development Goals (SDGs), governments, IGOs and researchers have been compelled to develop work that helps to address these pressing shared concerns and achieve the overall goal of a fairer and more sustainable world.
To track the progress towards achieving these goals, it’s helpful to be able to see what policy action is being taken on each front. To this end, the Overton app includes a filter which allows users to refine their results by SDG category, so they can see policy documents related to each specific goal.
“Though other bibliometric databases also track SDGs, Overton is unique in that it links the SDG to policy documents rather than research papers. Our view is that the SDGs are a policy initiative rather than a research one – a scholarly paper’s SDG classification isn’t really relevant until something is being done with it“
Euan Adie – Overton Director
Though the Overton app has included a SDG filter for some time, the recent update has improved its accuracy and ensured that more documents are being picked up.
The best way to classify SDGs is using machine learning. This is accomplished by first creating a training set of documents that the system can learn from. By showing it a large number of text snippets relating to each SDG, over time it can learn to identify new documents with a high level of accuracy. However previous attempts to use this method failed due to the difficulty of creating a large enough training set to ensure accurate categorisation.
However, the recent release of a free, open-source training set containing thousands of text examples for all 17 of the SDGs changed things. This training set was developed over the course of two years by a group called OSDG.
The OSDG project is a large-scale collaboration by hundreds of citizen scientists. A huge group of volunteers from UN agencies and universities across the world came together to analyse publicly available texts like policy documents and research papers in order to assess their relevance to SDGs.
The scale of this exercise – coupled with their systematic approach, in which several people analysed each text excerpt before agreeing its classification – makes it particularly trustworthy.
Overton took the OSDG training sets and built a new SDG classifier for policy documents and our system now has a much higher level of both precision and recall – the app picks up as many indications as possible while also reducing the likelihood of misclassification.
However, the classifier remains in development. It currently doesn’t ever assign SDGs 9 (“Industry, Innovation and Infrastructure”), 12 (“Responsible Consumption and Production”) and 17 (“Partnerships for the Goals”) as it wasn’t possible to ensure a high enough level of accuracy.
“Our current approach is to err on the side of precision – we would rather miss things than tell you something incorrect. The missing SDGs were very broad topics so we were seeing a lot of false positives, and we decided that the accuracy of the classifier on them just wasn’t high enough”
Euan Adie, Director of OVerton
Overton continue to work on the classifier to improve precision and accuracy.
