Artificial Intelligence can be a powerful force for good in the world. Recently, my team used AI to test new ways to preserve wildlife and make humanity’s impact on the world more sustainable by protecting biodiversity across the globe. By increasing the capabilities of neural networks, we’re able to apply AI in many areas beyond the well-known applications for autonomous vehicles and facial recognition. Working in partnership with charities and technologists at Microsoft’s “AI for Good” Hackathon, we used the Custom Vision service for fast prototyping of an AI-based solution that can assist volunteers in scaling the benefits of their manual work for wildlife protection and sustainable biodiversity.
We share this world with some amazing creatures, many of which are vulnerable to changes in their environment. To protect vulnerable wildlife, it’s vital to be able to monitor their habitats to assess whether they are leading to a decline in wildlife population numbers. According to ICUN Red List of Threatened Species, species assessed as Critically Endangered, Endangered or Vulnerable make up 14% of all birds species – and birds are not the most threatened group. A quarter of mammalian species are likely to become extinct as well as 63% of cycads (seed plants), 34% of conifers and 41% of amphibians are in the same threatened predicament.
In December 2019, Objectivity was invited to take part in “AI for Good” Hackathon organized by Microsoft together with one of the charity organizations that aims at using AI for making the world a better place - The Royal Society for the Protection of Birds (RSPB). The event took place at the Institute of Electronics and Technology in London. Together with four of my colleagues we decided to take up the challenge and explore how AI and newly-developed technologies can support the RSPB’s mission to change the fate of nature. The event is a part of Microsoft’s strategy to provide technology, resources and expertise in order to empower organizations working on creating a more sustainable world. We could not be more delighted to be a part of such an amazing undertaking!
Bird counting challenge
In order to carry out its mission, the RSPB monitors the populations of bird species in many parts of the United Kingdom. They have around 11 million species records that, among other uses, are the input for geospatial analyses aiming at geographic patterns investigations. Spotting and counting bird species is a very demanding and resource-hungry process, often comprising large teams of volunteers going into the field and huge files of drone imagery that needs to be analyzed by volunteers to estimate the number of birds in any given habitat. Automating and speeding this process up would result in a better resource allocation and an increased operating scale. This is the place where AI can make the difference by amplifying human ingenuity to effect positive change for good.
As an example of the challenge faced by volunteers, this photograph was taken in one of many cliffs of the UK’s coastline. As you can see, with your human eyes and refined human intelligence, spotting the difference between the cliff and the seagulls is simple. Try counting the number of gulls in the picture, though: how long does it take you to count them and how many times do you need to recount in order to be sure that you have counted the correct number? It’s not a big issue for a human analyst to perform the count, but it’s very time-consuming, which limits the absolute quantity of counts that can be performed and means that there are some species that aren’t counted until it’s too late to help them.
One of the images that we worked on when creating a model in Custom Vision (photo from RSPB resources)
As the hacking part of the event was pretty challenging in terms of time limit (only three hours), we decided to use Microsoft’s Custom Vision to check the possibility of detecting seagulls because it is the best option to achieve an acceptable level of accuracy in a very short period of time.
During the hackathon we had only one cliff image taken from the RSPB slide deck – quite a low number of training examples 😊, and not sufficient to train the model in Custom Vision as this tool requires at least 15 labelled images. By developing a simple python script, we divided the image and used its small parts as our input to a neural model – this small hack helped us move on and create an experiment in Custom Vision during the event. Afterwards, we got a few more images - thanks to our partners in the RSPB.
After creating a new experiment in Custom Vision, we uploaded small parts of images. As in AI machines learn from the previous experience, we need to feed them with labelled data – so we needed to mark each seagull by hand - which may be quite time-consuming (sometimes when I need to do that, I like listening to some interesting podcasts – it can help while doing such tedious work). If you are interested in data labelling tools outside the Custom Vision, I recommend you to look at LabelMe – an open source tool that is good for small projects, or Data Labelling in Azure ML.
An example of ‘ground truth’ images for training dataset – I marked each bounding box here (screen from Custom Vision)
When the training data is ready, you just need to push a ‘train’ button and wait a few moments until some pretrained model that resides under the hood of Custom Vision service is calculated.
In Custom Vision you can play with a threshold value – the probability value of the output bounding boxes that is required to consider the detected object as a prediction. I chose 50% to show the results. In this case, precision of 82.1% means that 82.1% of all the boxes predicted by the model are seagulls. As recall tells us 94.1% seagulls are detected by the model. This is a pretty good result achieved with 15 tagged images! mAP stands for mean average precision – a single value to compare different object detection models. You can learn more by visiting my colleague’s blogpost on object detection metrics!
Model’s performance metrics (from Custom Vision)
Now, we can upload a part of images that were not used during the training phase and we can qualitatively assess the model’s performance.
Model’s prediction on unseen data (from Custom Vision)
As you can see, the model produced by Custom Vision works well on the images that were not a part of the training dataset. Of course, in a real environment we should create a much bigger dataset that considers photos taken in multiple different weather conditions, camera resolution, not to mention different animal species. This would lead to better abilities for the neural network to generalize. However, with limited time (which is typical of hackathon events), the chosen approach seems to be the most suitable one for checking the feasibility of creating an object detection system.
It was a very busy yet very knowledge-enriching hackathon. I got to know more possible applications of Artificial Intelligence for making our world a better place to live in. I can see that Custom Vision as a part of Microsoft’s cognitive stack is very powerful and enables Data Scientists to create, run and validate computer vision models in a quick manner. During the event, my colleague Krzysztof Rudnicki was also having a great time running a Value Mapping exercise with the RSPB team. They worked on a potential strategy of creating value with the use of AI. Great job, guys!
Many thanks to Microsoft for running the hackathon, the RSPB for giving us the chance to learn and build using real-world challenges and, of course, many thanks to my colleagues from Objectivity: Justin, Oliver, Michał and Krzysztof!
I’m part of the Data Science Team at Objectivity and we love helping our clients get value from their data using AI, machine learning and statistical techniques.