One of Bird Buddy's core features is its bird-recognizing capability. It’s a handy tool for anyone trying to learn more about birds in a fun and accessible way. Bird Buddy's bird recognition is achieved through machine learning; the results of this process are only as good as the data input you give it when you train it. In Bird Buddy’s case, training images from our own worldwide camera network were used to create a bird-recognizing model that was then also tested in practice on Bird Buddy feeders. Bird Buddy's model is now capable of identifying many different bird species from all over the world and will continue to improve as it starts gathering ever more data.
The recognition process
Once a feathery visitor lands on the feeder and its pictures are taken by Bird Buddy's camera module, it is time for Bird Buddy to run its model and recognize the species.
When this process is completed you will be able to select your favorite pictures of the identified visitor and add them to your Collections.
To give you a rough idea of the current pipeline, all images go through four stages of processing to ensure the best result:
- First, we determine whether the image is interesting or not (if there is at least one bird in the photo). What’s interesting is highly subjective, of course, so even this simple step is something we struggled with at first. We landed on what we feel is a good definition for that (a bird is in the frame and in focus, at least one eye and the beak is visible) and have now gone through all 300,000 initial images and classified them according to that criteria. What’s awesome about this is that the images that are normally interesting to a human, are also ones that AI will have a much easier time doing inference of the species on.
- Secondly, the photo goes through a bird detector model (which we also trained) that marks the birds in the photo and crops them out. There will be cases of up to 6 specimens and up to 4 different species in the same image. We have trained the custom bird detection model to detect many birds in the same image. Every detected bird in the image is then passed through our bird species classifier.
- In the next stage, we determine the species for every detection in the picture and further sort the cropped photos. With the help of advanced tagging and cropping tools, we classify them as either interesting, not interesting, or invalid (phantom detections). This is where our amazing AI trainers step in and do all the legwork.
- At least 2 thousand images per species are then introduced into the neural network. They are the material based on which the model learns to distinguish between species. This is how Bird Buddy knows to take a snapshot when a bird appears in its field of view: the potential of this technology is only limited by the data we chose to feed it, so it will only get more accurate and complex as time goes on.
So far, the team has processed millions (yes, millions!) of images, fine-tuning the model to recognize bird species that regularly visit bird feeders.
We will be improving the species recognition feature of our AI model. Our team is actively working on retraining the model to improve its accuracy in identifying bird species.
Currently, the model has only been trained on images that were not from the actual Bird Buddy feeder. However, we are continually adding new images of species that we initially did not recognize, which will be used to train the model.
Additionally, we are exploring the possibility of having our bird expert team review a certain percentage of images from the feeders to check if the current model has made any mistakes in identifying the species. If any errors are found, they will be corrected to the right species. This is a time-consuming iterative process, but with each iteration, the model's predictions will improve and become more accurate, and the need for manual correction will decrease.
We are committed to providing our customers with the best possible experience and appreciate your patience and understanding as we work to improve the species recognition feature.