Stony Brook University scientists have concocted creative ways to study the penguin colonies in Antarctica by using tourist photos, mashing up computer vision with satellite imagery through artificial intelligence (AI) to monitor changing patterns among the iconic seabirds.
A new study led by Heather Lynch, an endowed professor from the Department of Ecology and Evolution, and Haoyu Wu, a Ph.D. candidate researching computer vision and deep learning, shows how tourist cameras are keeping tabs on the penguin homes and shifts within their ecosystems surreptitiously via photographs. The study was published in PLOS ONE.
“Our goal here was to use this new method for science by taking hundreds of thousands of these photographs that we weren’t able to use before for our research,” Lynch said.
Using a variant of the AI models for segmenting anything, the researchers automatically identified the boundary of the penguin colony in the picture. They also mapped out a method using 3D mapping from satellite imagery to correctly identify and outline the colonies.
“The challenge was figuring out how to georeference these images in a region like Antarctica, where there are few distinct landmarks,” Wu said. “By overlaying satellite images on 3D models, we could accurately locate where each photo was taken and what it captured.”
The study highlights the potential for leveraging citizen science in remote regions where traditional monitoring methods may be costly or impractical. While satellites and aerial imagery are commonly used for environmental tracking, the inclusion of tourist photographs significantly expands the available dataset for long-term monitoring of penguin populations and their responses to climate change.
“The major challenge we had was that when downloading images from online it usually didn’t have the metadata of the location the images were taken from,” Lynch said. “Even if it did, there was so little information that we couldn’t even use it.”
The researchers acknowledge challenges such as variations in image quality and dynamic Antarctic landscapes but believe the technique has broad applications for monitoring natural environments.
The technique requires a common point in several images to be able to start piecing together the environment, but due to how the snow and the landscape are always changing, it has made it much harder to pinpoint exactly where the image was taken.
“One of our goals with this technique is to fully automate the process of getting images and processing them through, and being able to pinpoint the location of where the image was taken from,” Lynch said.
The automation of imaging and location pinpointing enables long-term, large-scale ecological studies. Continuous data over an extended period can be collected — a key feature in determining the changes occurring in penguin populations as a result of climate change and human activities. With such information, conservation efforts can be more appropriately planned for these vulnerable species.
Further research and technological refinement could enhance the accuracy of the method, opening new opportunities for collaborative environmental monitoring involving tourists, scientists and conservationists.
“This software is very simple to use; it takes all the images that you download and stores it in a drive. It then starts to analyze each image and starts to pinpoint and 3D construct the landscape to be able to pinpoint the location of where the image was taken,” Hu said.
It takes all the downloaded images and puts them into a drive. Then, it starts analyzing every single image and maps it on a 3D model, allowing researchers to use the information gathered to expand their research.
“I was surprised that my software worked. I never expected for it to work, but I’m very glad that it did,” Hu said.
