One of Bird Buddy's core features is its smart use of artificial intelligence (AI) that lets you know which birds are visiting your feeder. It’s an amazing tool that will make learning about your backyard friends much simpler and more accessible. Artificial intelligence and machine learning are amazing tools, but the result is only as good as the input you give it when you train it and how well that input matches with the actual conditions of the images you use for identification.
There are some existing models for image identification of birds that we’re already using, but our own in-house AI training project will allow us to improve significantly on their accuracy, and eventually, with the amount of data we have coming in, we’ll be able to create a model that the best ornithologists will struggle to compete with.
Bird Buddy’s already highly trained AI algorithm is capable of identifying many different bird species from all over the world and will continue to learn forever. Our AI identifies feathered visitors by sight with the help of the built-in camera.
In this article you can find more about:
- The recognition process
- ID failure: What happens if the AI doesn't recognize the bird
- How the AI deals with non-bird visitors
The recognition process
Once a feathery visitor lands on the feeder and its pictures are taken by Bird Buddy's camera module, it is time for our highly trained AI to perform its high-tech magic and recognize the species.
When this process is completed you will be able to select your favorite pictures of the identified visitor and add them to your Collections.
To give you a rough idea of the current pipeline, all images go through four stages of processing to ensure the best result:
- First, we determine whether the image is interesting or not (if there is at least one bird in the photo). What’s interesting is highly subjective, of course, so even this simple step is something we struggled with at first. We landed on what we feel is a good definition for that (a bird is in the frame and in focus, at least one eye and the beak is visible) and have now gone through all 300,000 initial images and classified them according to that criteria. What’s awesome about this is that the images that are normally interesting to a human, are also ones that AI will have a much easier time doing inference of the species on.
- Secondly, the photo goes through a bird detector model (which we also trained) that marks the bird in the photo and crops it out.
- In the next stage, we determine the species in the picture and further sort the cropped photos. With the help of advanced tagging and cropping tools, we classify them as either interesting, not interesting, or invalid (phantom detections). This is where our amazing AI trainers step in and do all the legwork. Led by Rebeka (our Head of Special Projects) and Dejan (our in-house ornithologist from the University of Ljubljana), the team tirelessly processes pictures from our AI training network.
- At least 2 thousand images per species are then introduced into the neural network. They are the material based on which the model learns to distinguish between species. This is how Bird Buddy knows to take a snapshot when a bird appears in its field of view: the potential of this technology is only limited by the data we chose to feed it, so it will only get more accurate and complex as time goes on.
So far the team has processed millions (yes, millions!) of images, fine-tuning the AI to recognize bird species that regularly visit bird feeders
This undertaking is really only aimed at improving the initial experience as the volume of images available for training the model once we go live will eclipse what we’re able to do now.
ID failure: What happens if the AI doesn’t recognize the bird?
Even though we are fairly confident that the vast majority of visitors to your feeders will be recognized most of the time, there will of course also come a time when our AI is stumped. But not for long!
In case the AI is not entirely sure about the species, you will first get asked to help it out yourself by selecting the correct bird from a list of potential matches we provide. If you don’t see the correct species on the list immediately, make sure to also tap “show me more options” to expand the list of suggestions.
If you don't see the correct bird species on the list of potential matches, you can always tap ‘Not on the list' at the bottom of the screen and manually search for the correct bird species in our database.
In case you're unsure about the species or if that specific one is not yet in our database, you can also send the picture in for review. Once the photo is tagged by our in-house ornithologists and the new bird's profile is created, it's added to your Collections and the database is expanded accordingly.
All of these steps help our AI algorithm gain new knowledge, both in expanding the database and improving its recognition accuracy.
Bird Buddy's main objective is recognizing feathery visitors, of course. However, our AI is also being trained to recognize other types of buddies that might find the feeder to be a good place for a snack break. For now, our developers have trained the neural networks to recognize squirrels. But the potential of the technology is pretty much unlimited, in the future Bird Buddy will be able to recognize other critters that visit your backyard as well.