In 2010, ISS ESG’s head of Climate Solutions Max Horster started one of the first companies to measure the climate impact on investments. From investment carbon footprinting to climate scenario analysis, from climate-linked proxy voting to climate neutral investments via offsets: Over the years, the team pioneered a wide range of today’s leading methodologies and approaches across all asset classes. In 2017, Max and his team joined ISS ESG to form the first climate specialist unit of a global ESG service provider. Today, they cover over 25,000 issuers on up to 600 individual climate-linked data points and have screened over $4 trillion of AUM on their climate risks and impact. On the occasion of its 10th anniversary, the ISS ESG Climate Team shares 10 lessons from 10 years of helping investors to tackle climate change.
Lesson 6: Human versus Machine: Rightsizing the expectations for Artificial Intelligence in ESG
“Fire your ESG analysts – who needs them? The future of ESG ratings just requires data scientists.” When the first Artificial Intelligence ESG service providers came to the table around 2012, the value promises ranged from bold to aggressive. Now, almost a decade later, the positioning is much humbler and more realistic.
There is no doubt that ESG ratings will benefit from innovations in machine learning and Artificial Intelligence over time. As with any other industry, we can’t sustain our data collection and rating practices in the same way we did 30 years ago, when the ISS rating methodology was first developed. And we don’t: a lot has changed over the past 30 years and we are employing AI specialists to constantly evaluate how new technologies can help us improve our current practices as well as processing large alternative data sets, such as from satellites, news reports or other third party sources. After 30 years of rating companies and almost 10 years of testing AI abilities, the results are promising, but by no means the revolution that one might have expected at the outset of AI entering the ESG space.
Let’s look at the current process of ESG and see where the AI solutions stand.
Step 1: Data identification. Every rating starts with identifying data. This is currently the greatest opportunity for AI, machine learning and specifically Natural Language Processing (NLP): crawlers can go through text contained in Corporate Sustainability Reports and find, for example, where greenhouse gases, CO2 references or related topics can be found. We can train an algorithm to recognize Codes of Conduct for 500 companies, so that the machine will be able to identify the Codes of Conduct for company 501 to 10,000 – a great way to scale up coverage. What is constantly underestimated, however, is the training element. This requires “domain experts,” i.e., professionals who understand in what form and shape a Code of Conduct can appear in company reports. And this is just a simple example – think of identifying supply chain issues, human rights violations, climate management strategies, etc. Data scientists might be able to program the algorithm, but it takes ESG sector specialists and thousands of labeled training data points for each category to properly educate the algorithm.
Step 2: Data collection and extraction. After localizing the data, the right data points need to be extracted and harmonized to be comparable – both from companies and from third party sources. Based on step 1, machine learning can help streamline workflows, but not replace the human analyst. By providing URLs and report page numbers, this saves data collectors’ and analysts’ time as they can immediately jump to the relevant section and process the information.
What machines can do here is, for example, determine if GHG emissions and climate are mentioned at all. So they can, with high confidence and within seconds for thousands of companies, make the claim that a company does not address climate change at all or that a company says something about climate change. This, however, doesn’t mean that a company really addresses climate change – the word climate could just pop up in a different context and result in a positive hit.
What machines cannot do yet is extract quality and error-proof data automatically. Machines cannot (yet) exactly determine, for example, if the reported greenhouse gas emissions are complete (or just cover headquarters or certain markets), what timeframe they refer to (fiscal year, calendar year, a few months) or what underlying greenhouse gases have been converted into CO2 equivalents.
AI and machines are great for quantity – ploughing through huge amounts of data. They are, however, still far short of expectations when it comes to quality data. In other words, machines can make data processes more efficient, but they can’t produce a result that is good enough to base climate reporting on, let alone investment decisions.
Step 3: Rating companies. The AI-vory tower idea is that the machine takes indicators and weights, and creates an automatic rating that is either good to go or just requires final analyst approval. In reality, ESG ratings are much more nuanced than that. Especially if the rating must be transparent and explainable rather than providing surprising, black-box fueled results.
There is more to an ESG rating than simply defining thresholds for sectors. Take a utility company. It requires a specialist with deep company knowledge to differentiate between utilities that are pure play, treat wastewater or have recycling facilities, let alone determining if they are setting nonsense or meaningful climate targets. Machines don’t get these nuances. Yet.
An ESG rating requires also more than simply applying standard rules on a company. Ratings require judgment, an understanding of the specific business and strategy, the self-confidence to make a point, sector specialization to cut through marketing spin, and the stamina to stand behind a view. Machines don’t have this. Yet.
ESG ratings are not as simple as scraping what is in the public domain. Meaningful ratings should involve company dialogue. An algorithm doesn’t call up management. Yet.
You might know this example of a bot on a medical website that had all the right information, was based on the Lighthouse Open AI technology GPT-3, reacted beautifully to the patient, and yet, it somehow got it all wrong:
Bottom line: An adequate judgement takes more than the sum of the dots and a good processor. It requires connecting the dots in a smart way to form a narrative that the rating can be based on. This requires humans – but humans can greatly benefit from smart AI. It is worth noting that for alternative data sources, the use of AI is anything but trivial. In a paper published by University of St. Gallen (Switzerland) in collaboration with ISS ESG, the usability of satellite data was tested for the estimation of greenhouse gas emissions, generating some encouraging results.
A final product is still a long way away, but the results show that deep neural networks are able to identify industrial smoke plumes with an accuracy of up to 94%. The remaining mis-classifications in the presented segmentation model are likely to be confused by surface objects, cirrus clouds, or ground fog. The goal of that work, once calibrated against a range of industrial sites, will be a framework that might allow for the estimation of all sorts of industrial emissions from satellite imaging data on a global scale.
ESG in the future: Humans, augmented by machines
In summary, the future of ESG ratings is neither just human nor just machine). It is also not machines supported by humans. The future of ESG ratings is humans supported by machines.
In the recent online discussion “Man vs. Machine: Current State and the Future of ESG Ratings” between Thomas Kuh of Truvalue Labs and myself, 62.5% of the audience agreed with this conclusion:
Only the combination of highly specialized and trained ESG analysts augmented by the possibilities of Artificial Intelligence and machine learning will help generate ESG ratings that are robust enough to put investments behind them.
For fully machine-based ratings, the world isn’t ready. Yet.
By Dr. Maximilian Horster, Head of Climate Solutions, ISS ESG
This article received input from the ISS ESG AI lead Marcel Neuhäusler.