One of Bird Buddy's core features is its bird-recognizing capability. It’s a handy tool for anyone trying to learn more about birds in a fun and accessible way. Bird Buddy's bird recognition is achieved through machine learning; the results of this process are only as good as the data input you give it when you train it. In Bird Buddy’s case, training images from our own worldwide camera network were used to create a bird-recognizing model that was then also tested in practice on Bird Buddy feeders. Bird Buddy's model is now capable of identifying many different bird species from all over the world and will continue to improve as it starts gathering ever more data.
In this article you can find more about:
- The recognition process
- ID failure: What happens if the model doesn't recognize the bird
- How the model deals with non-bird visitors
The recognition process
Once a feathery visitor lands on the feeder and its pictures are taken by Bird Buddy's camera module, it is time for Bird Buddy to run its model and recognize the species.
When this process is completed you will be able to select your favorite pictures of the identified visitor and add them to your Collections.
To give you a rough idea of the current pipeline, all images go through four stages of processing to ensure the best result:
- First, we determine whether the image is interesting or not (if there is at least one bird in the photo). What’s interesting is highly subjective, of course, so even this simple step is something we struggled with at first. We landed on what we feel is a good definition for that (a bird is in the frame and in focus, at least one eye and the beak is visible) and have now gone through all 300,000 initial images and classified them according to that criteria. What’s awesome about this is that the images that are normally interesting to a human, are also ones that AI will have a much easier time doing inference of the species on.
- Secondly, the photo goes through a bird detector model (which we also trained) that marks the birds in the photo and crops them out. There will be cases of up to 6 specimens and up to 4 different species in the same image. We have trained the custom bird detection model to detect many birds in the same image. Every detected bird in the image is then passed through our bird species classifier.
- In the next stage, we determine the species for every detection in the picture and further sort the cropped photos. With the help of advanced tagging and cropping tools, we classify them as either interesting, not interesting, or invalid (phantom detections). This is where our amazing AI trainers step in and do all the legwork.
- At least 2 thousand images per species are then introduced into the neural network. They are the material based on which the model learns to distinguish between species. This is how Bird Buddy knows to take a snapshot when a bird appears in its field of view: the potential of this technology is only limited by the data we chose to feed it, so it will only get more accurate and complex as time goes on.
So far the team has processed millions (yes, millions!) of images, fine-tuning the model to recognize bird species that regularly visit bird feeders.
This undertaking is really only aimed at improving the initial experience as the volume of images available for training the model once we go live will eclipse what we’re able to do now.
ID failure: What happens if the model doesn’t recognize the bird?
Even though we are fairly confident that the vast majority of visitors to your feeders will be recognized most of the time, there will of course also come a time when our AI is stumped. But not for long!
In case the AI is not entirely sure about the species, you will first get asked to help it out yourself by selecting the correct bird from a list of potential matches we provide. If you don’t see the correct species on the list immediately, make sure to also tap “show me more options” to expand the list of suggestions.
If you don't see the correct bird species on the list of potential matches, you can always tap ‘Not on the list' at the bottom of the screen and manually search for the correct bird species in our database.
In case you're unsure about the species or if that specific one is not yet in our database, you can also send the picture in for review. Once the photo is tagged by our in-house ornithologists and the new bird's profile is created, it's added to your Collections and the database is expanded accordingly.
All of these steps help our AI algorithm gain new knowledge, both in expanding the database and improving its recognition accuracy.
Bird Buddy's main objective is recognizing feathery visitors, of course. The potential of the technology is pretty much unlimited though, in the future Bird Buddy will be able to recognize other critters that visit your backyard as well. For now, you can collect mystery visitors.