Metascience Novelty Indicators Challenge

A global challenge to transform how we measure novelty in research

The Metascience Novelty Indicators Challenge is a global initiative to develop and validate scalable indicators that can identify novelty in research. We are challenging academics, data scientists, researchers, and innovators worldwide to create tools that could reshape how we evaluate novelty and thereby support metascience progress.

The Metascience Novelty Indicators Challenge invites individuals or teams who can generate novelty scores for journal publications across a wide range of disciplines. 

Why do we need the Metascience Novelty Indicators Challenge?

If the global science system had responsibly used better and more timely indicators of research excellence, this could have a profound impact on the incentives of researchers, our understanding of the factors which make excellence more likely, and in turn, the pace of research progress. Better metascience indicators can help funders, governments, academic institutions, and individuals get more high-quality research out of limited resources.

Citation and journal-based indicators are widely used, albeit controversial, indicators. In some ways, these measures do track our intuitions on what ‘good research’ is, but it is an open question whether these indicators track a key virtue of research: novelty.

Metascience academics over the past decade have devised measures of novelty, with numerous examples available in literature. These measures are gaining interest in the niche world of academic metascience but have not yet translated into mainstream use or acceptability in the wider research ecosystem. We think the main reason is that these measures are not yet validated at scale and across different fields of research. We don’t know that they cohere with our human judgements of what novel research is, and so we cannot trust them. This sort of validation is expensive and technically challenging, but the Metascience Novelty Indicators project is taking it on because we believe in the potential explanatory power that such an indicator could offer.

Our mission is to advance the field by a) finding the best and most reliable indicators, and b) robustly validating these indicators for general use across a range of fields.

For more information about the rationale for this challenge, see our piece in Nature.

Why join the Metascience Novelty Indicators Challenge

• Contribute to advancing the field of metascience, including research assessment and the measurement of novelty 
• Connect with leading researchers in metascience and bibliometrics

The winner will also: 

• Receive a £300,000 grant to develop your indicator further
• Get the option to work closely with the UK Metascience Unit and Elsevier on developing and operationalising the novelty indicator at scale across the scientific corpus, for use in research evaluation
• Get the option to contribute to a peer-reviewed publication on the post-prize learnings on indicators.

Are you…

Anyone with a smart idea and the technical capability to deliver the metascience novelty indicators, especially:

• A university team studying metascience and novelty in research
• Post graduates, PhD candidates and students with relevant experience
• Data scientists and researchers working in bibliometrics, scientometrics, science of science, research on research, metascience, or related fields
• Industry practitioners, non-profit research experts, or commercial entities with expertise in research analytics

We want to hear from you.

Learn how to take part