The computer conservationist
Engela Duvenage
The Hainan gibbon is the world’s rarest ape, rarest primate, and probably rarest mammal species. Once numbering around 2 000 individuals in the 1950s, this critically endangered primate's population underwent a severe decline in the late twentieth century due to habitat loss and hunting, and is now one of the most threatened species in the world. Photo by Jessica Bryant (Zoological Society of London)
Dr Emmanuel Dufourq has never seen one of the last 30 remaining Hainan gibbons on Earth, but he knows very well what their calls sound like, or at least what a spectrogram image of the highs and lows of their chatter looks like.
“Their calls are very singsong. They sing individually and then, at some point, they sing together. It’s a very pleasant sound,” says Dufourq, the African Institute for Mathematics Sciences (AIMS) Junior Chair in Data Science for Climate Resilience and a lecturer in Stellenbosch University’s (SU’s) School for Data Science and Computational Thinking and its Department of Industrial Engineering.
Hainan gibbons (Nomascus hainanus) are considered the rarest apes on earth, and among the world’s most threatened mammals. Most of these gibbons live in two square kilometres of forest on Hainan Island in the South China Sea.
A gibbon weighs about as much as a human baby of a few months. Males are almost completely black in colour, while females sport a combination of golden to buff fur and black patches on their heads.
The nearest Dufourq has come to visiting the home of these rare apes was in 2013 when he visited Vietnam to present a paper on his MSc work. The latter focused on using genetic programming for data classification, a technique in the field of artificial intelligence.
“My journey has evolved from wanting to be a computer scientist to wanting to do something impactful. And then I found ecology,” explains Dufourq, an avid nature photographer.
These days, he is something of a ‘computer conservationist’. As the leader of the Data Science for Eco-Systems (DS4ES) research group at SU, Dufourq uses his computational skills to make it easier for conservation ecologists to analyse the large volumes of animal sound, video and photo footage that they capture for their research. His group’s mission is to do this one keyboard stroke, one algorithm at a time.
“We are trying to use all the tools we have in the fields of mathematics and machine learning to support conservation ecology projects, and to assist field rangers,” he explains.
“We are one of the few research groups in Africa using machine learning for ecological outcomes. It’s really rewarding to know that, in the process, we are developing students to improve the combination of these two fields.”
Dufourq works with collaborators at, for instance, AIMS, the University of Cape Town, the University of KwaZulu-Natal, the University of Lisbon in Portugal, the University of St Andrews in the United Kingdom, and various non-profit organisations around the world.
Listen to a Hainan gibbons' call
Photo by Jessica Bryant (Zoological Society of London)
The mel scale is a scale of pitches judged by listeners to be equal in distance from one another. A small change in low frequencies is very well perceived by the human ear, but a similar change in very high frequencies is not. The mel scale adjusts for this.
Dufourq’s interest in bioacoustics and the manner in which it can be used to conserve the planet and its inhabitants was first piqued in 2019. He was finishing his PhD in applied mathematics at the University of Cape Town when a statistical ecologist told him about an interesting conundrum faced by a group of British ecologists.
“The Zoological Society of London, along with their collaborators, had collected a very large dataset of about 1,5 terabytes of gibbon call recordings – hours’ worth – but did not have anyone to process it.”
For years, researchers and field rangers had been monitoring these critically endangered primates by following them and recording the sounds they make as they go about their day. The more sound bites they captured, the more they had to analyse and code it, a typically time-consuming and largely manual exercise without which the recordings have no real value. Adding to the difficulty of the task is the fact that it requires a trained ear to distinguish the gibbons’ calls amid the general morning chorus of birdsong in the forest.
“Ultimately, it didn’t help that they had all of these recordings but couldn’t really do anything with them.”
To easily transform these recordings into something truly useful to researchers became Dufourq’s mission during his postdoctoral years. He remembers the work that went into it: “I realised that we could use machine learning and deep neural networks for sound classification – something I hadn’t considered before.”
His work has since shown that methods such as data augmentation can be used to automatically and quite accurately convert sound recordings into spectrograms, which are in turn automatically classified as images.
"Our best model converted acoustic recordings into spectrogram images on the mel frequency scale, using these to train a convolutional neural network," Dufourq and his fellow researchers wrote in a 2021 article on this topic, published in Remote Sensing in Ecology and Conservation.
“Our model is a time-saver of note,” he says.
“A post-processing step that identifies intervals of repeated calling managed to reduce an eight-hour recording, for the purposes of manual processing, to an average of only 22 minutes. Moreover, it did not miss a single calling bout over the 72 hours of test recordings.”
Dufourq believes the model can be adapted for use in the analysis of other animal sounds too. He and his students have since started working on algorithms to automatically monitor animal behaviour captured in large datasets of video footage and images. The specific focus of the students’ work ranges from the body language of endangered African penguins to the calls of Indian humpback dolphins in False Bay, understudied beaked whales, spotted hyenas, endangered birds in Malawi, and the critically endangered black-and-white ruffed lemurs in Madagascar. Some are developing an algorithm to identify and count the birds caught on webcam footage in South Africa’s national parks.
Much effort is being put into compressing the sound files that researchers collect so that it is easier to analyse in real time, as opposed to only after the recordings have been made.
The conservation of the planet is top of mind in all these efforts. “Model development and implementation must always be informed and guided by ecological objectives,” Dufourq emphasises.
“We cannot solve climate change, but we can at least solve the problem of ecologists wasting 6 months of work on processing 10 terabytes of data, when we can do it in 10 minutes.”