Student Code-In is a global program that helps students grow with Open Source. It is a 2 months long Open-Source initiative which provides you the best platform to improve your skills and abilities by contributing to vast variety of Open Source Projects. In this, all the registered participants would get an exquisite opportunity to interact with the mentors and the Organizing Team.
Project idea is to implement an image recognition and classification device in their pocket with a camera in spectacles of blind people, so that it can recognize and classify the things (visuals) in front of them. It also helps to identify the people in front of them with the help of face recognition. The classified or recognized image will be converted into speech so that the Blinds can listen and understand what is in front of them with the help of a headset.
The project is developed till its image recognition and classification process, and I'm under developing its classified image text to speech process so that Blind people can know what is in front of them through their earphones connected to the device. With the help of Intel Movidius NCS we can run any complex pre-trainet Networks in the processors with low processing capability. This NCS functions as an efficient external processor for performing any complex neural architectures.
Intel Movidius Neural Compute Stick(NCS) can be used as an external GPU for the systems which cannot support for artificial intelligence based projects. They are less-cost comparing other GPU's and can be used efficiently for most of the Deep Learning, Machine Learning and Artificial Intelligence Based Projects
- To reduce the difficulties faced by the blind people in their daily life by creating a device which will tell them what is in front of them(includes identifying the people in front of them using Face recognition)
- To make the blind people forget about their disabilities.
- Intel Movidius Neural Compute Stick(NCS) 2
- Raspberry pi-3
- Raspberry pi-camera
- Headset- Click here to see how to connect your headset with Raspberry Pi-3.
- Power-Bank
The Intel Movidius Neural Compute Stick(NCS) is produced by Intel and can be run without an internet connection. This Movidius NCS's compute capability comes from its Myriad 2 Vision Processing Unit(VPU).
Profiling, tuning and compiling a DNN on a development computer with the tools are provided in the Intel and can be run without an Internet connection. The Movidius NCS's compute capability comes from its Myriad 2 VPU(Vision Processing Unit). Running Deep Learning (DL) models efficiently on a low capacity graphic processors is very difficult. Movidius allows us to optimize the operation of large models such as GoogleNet with multi-use support. It's an easy to use kit that allows you to design and implement applications such as classification and object recognition on physical products.
We can simply think of Movidius NCS's as a Graphic Processing Unit(GPU) running on a USB. Model training isn't performed on device, but a trained model can work optimally on the unit, which is intended to be used in physical environments for resting purposes.
With the help of Intel Movidius Neural Compute USB Stick with Raspberry Pi-3 we are using it for image classification and an object recognizing application. The frameworks, standards, technique used are:
It can be used in Ubuntu 16.04 or Raspberry Pi-3. It is compatible with two DNN frameworks such as TensorFlow and Caffe Movidius Myriad 2 Vision Processing Unit(VPU) works with Caffe based Convolutional Neural Networks(CNN). We can also run complex Deep Learning(DL) pre-trained models like GoogleNet, SqueezeNet, AlexNet on systems with low processing capability.
It's very simple to run inference on an image classification demo model. We can use the NC App Zoo repo for classifying an image. We need to take the graph file to activate the application of the Movidius NCS. It has a compiled GoogleNet model for ready to run. This application needs some files. The make command is used for creating the files that Movidius needs as a graph file. The graph files is a demo of image-classifier.
View the steps below for a quick application:
STEPS:
-
Step 01: For using the property of the NCSDK API add (import) the mvnc library.
import mvnc.mvncapi as mvnc
-
Step 02: You can access the Movidius NCS using an API like any other USB device. Also you can use parallel Movidius devices at once if you need more capacity to compute your model. For now, one kit is enough for this application. Select and open process.
#then look for the enumerated Intel Movidius NCS Device(), quit program if none found. devices = mvnc.EnumerateDevice(); if len(devices) == 0: print("No Devices Found"); quit; #Now get a handle to the first enumerated device and open it. device = mvnc.Device(devices[0]); device.OpenDevice();
-
Step 03: We will use a pre-trained GoogleNet model for using a compiled graph file.
#Read the graph file into a buffer. with open(GRAPH_PATH, mode = 'rb') as f: blob = f.read(); #Load the graph buffer into the NCS. graph = device.AllocateGraph(blob);
-
Step 04: We also need to do some pre-processing before loading the image into our Movidius NCS.
#Read and resize image[image size is defined during training] img = print_img = skimage.io.imread(IMAGES_PATH); img = skimage.transform.resize(img,IMAGE_DIM, preserve_range = True); #Convert RGB to BGR [skimage reads image in RGB, but Caffe uses BGR] img = img[:, :, ::-1]; #Mean subtraction and scaling [A common technique used to center the data] img = img.astype(numpy.float32); img = ( img - IMAGE_MEAN )*IMAGE_STDDEV;
-
Step 05: Use LoadTensor() to load the image into the Movidius.
#Load the image as a half-precision floating point array. graph.LoadTensor( img.astype( numpy.float16 ), 'user object' );
-
Step 06: Give the input image to the pre-trained model and get the output by using GetResult().
#Get the results from NCS output, userobj = graph.GetResult();
-
Step 07: Print the prediction of the model's output and corresponding labels. Here we also display the input image at the same time.
#Print the results print('\n ----predictions----'); labels = numpy.loadtxt(LABELS_FILE_PATH,str,delimiter = '\t'); order = output.argsort()[::-1][:6]; for i in range(0,5): print ('prediction' + str(i) 'is' + labels[order[i])); #Display the image on which inference was performed skimage.io.imshow(IMAGES_PATH); skimage.io.show();
-
Step 08: For the last step, we clear and shutdown the Movidius NCS device for using it again.
#For clear and shutting down the Movidius NCS device for using it again. graph.DeallocateGraph(); device.CloseDevice();
1. Fork this repository.
Click on the 
2. Clone the forked repository.
git clone https://github.com/<your-github-username>/Image-Recognition-and-Classification-Device-for-Blind-People-Using-Intel-NCS-2.git3. Navigate to the project directory.
cd Image-Recognition-and-Classification-Device-for-Blind-People-Using-Intel-NCS-2/4. Create a new branch.
git checkout -b <your_branch_name>5. Make changes in source code.
6. Add your changes and commit
#Add changes to Index
git add .
#Commit to the local repo
git commit -m "<your_commit_message>"7. Push your local commits to your repository.
git push -u origin <your_branch_name>Create a Pull-Request(PR) to merge your changes into the original repository(write a simple description about the changes you made)
Always feel free to file a new issue with a respective title and description on the Image-Recognition-and-Classification-Device-for-Blind-People-Using-Intel-NCS-2 repository.
If you have any great ideas which can make this project more better, you can make changes and send me a Pull Request(PR) with a respective title and description on the Image-Recognition-and-Classification-Device-for-Blind-People-Using-Intel-NCS-2 repository, I will definitely review your pull request.
.jpg) |
|---|
| AKHILDAS KS |
 Ashwini Jha Repo Link |
 Bhanvi Menghani Repo Link |
