SDG Mapping Results for 1,000 Publications from Large Language Models: GPT-4o, Mixtral, Llama 2, Llama 3, Gemma 2, Qwen 2 and GPT-4o-mini

Randomly selected 1,000 publications from the Swinburne University of Technology research bank were used for SDG mapping tasks with the large language model GPT-4o and the open-source models Mixtral, Llama 2, Llama 3, Gemma 2, Qwen 2 and GPT-4o-mini.

The input to each model consisted of the publication’s title and abstract.

The designed prompt is as follows: 

PROMPT = '''Analyze the publication and determine the SDGs it aligns with. Evaluate against all the 17 SDGs provide the reason for alignment. In the end summarise the confidence levels(%) for each assigned SDG in JSON format. For example: {example}. Title: {title} Description: {description}'''

example = '''{ 'Goal 6': 0.67, 'Goal 11': 0.50, 'Goal 3': 0.25}'''