The Fruits & Vegetables App that enable anyone to recognize a foodstuff by sending a photo to the App. -----------
It can also go further by suggesting a recipe using the fruits or vegetables present in the photo.
What's the purpose of this app ?
It can be used for many purposes.Examples:
- It can be a learning tool for children
- It can be a way to help families varying their diets by having a better knowledge of fruits and vegetables
- It allows people to pick fruit during a walk without making mistakes and eat something inedible
- It allows you to have a perfect recipe with what you have in the fridge when you have no idea of what to do
How to make it real ?
Artificial intelligences (AI) are now able to recognize something from a database.Here we're going to deal with one possibility among anothers: we are going to use ml5.js which is a JavaScript library.
To Understand how to use it you can watch this video: ml5 for Beginners
Indeed neural networks can recognize the content of images.
ml5.imageClassifier() is a method to create an object that classifies an image using a pre-trained model. So ml5 with MobileNets will be able to recognize the picture of fruits or vegetables. MobileNets are a class of convolutional neural network which are trained to recognize images.
First you can copy and paste the Cdn link to the ml5 library here in an html file:
And then you have to create a javascript file to introduce your ml5 code. Here is an example:
As you can see on the java file you can also add interactivity by using the p5.js library. It enables you to directly drop your photo.
Here are examples of what you can get if you post an image of vegetable or fruit:
As you can see, you'll get the percentage of error at the same time.
To go further
The idea is that the App has a stock of recipe and can suggest a recipe with all the food products that are on the pic.For instance when you put a photo of a butternut squash on the App, it can suggest you that type of things:
"Here is a wonderful recipe using your ingredient:
Butternut Recipe 1.
This one is also very good : Butternut Recipe 2,
which will look like this: "
Here are websites you can include in the app to find recipes according to the ingredients you have: