Show
The capabilities of artificial intelligence (AI) have already gone beyond those of the human mind. Data analysis, accurate predictions, finding invisible patterns, and object recognition are some of the features modern AI solutions are embedded with. Let’s analyze the ability of AI to recognize objects and ways to use this technology in healthcare. The definition of object recognition is intuitively clear. Artificial intelligence can recognize particular objects in images, videos, or others with the guidance of specified parameters. Facial recognition software embedded in most modern smartphones is the simplest example of this technology in action. It memorizes the face of an authorized owner and compares it to the one in front of the camera to unlock the smartphone. Object Recognition Use Cases
Object Recognition vs. Object Detection vs Object TaggingImage object recognition and object detection are both computer vision tasks, while object detection is a sub-task of computer vision object recognition. The software first detects the object, then finds out whether it has the specified parameters for a particular category, and then recognizes the object with a high degree of accuracy. More advanced object recognition software can also tag objects by assigning them traits the algorithm is aware of. For example, you can identify the image of a cat and hypothetically determine its breed if you have scheduled this task. Below are the results of an object recognition test we get from each operation:
How Object Recognition Works in Machine Learning vs. Deep LearningThe task of detecting an object is pretty simple for modern algorithms. And the capabilities of machine learning are enough to cope with it successfully. However, you need more advanced technologies to create these systems, when it comes to object recognition and tagging. In these cases, the developers use neural network models or deep learning, which is the next generation of machine learning. Such projects are usually created using Python, TensorFlow object recognition API, and an OpenCV object recognition library. This library contains a pre-developed set of features for selective search for object recognition, multiple object recognition with visual attention, face recognition, voice recognition, speech recognition, and even handwriting analysis. Object Recognition TechniquesAn object recognition algorithm may use different techniques to detect, recognize and tag an object. For example, an object recognition OpenCV library allows you to use the following tactics to make the algorithm learn and work.
Object Recognition in HealthcareThere are plenty of ways to apply object recognition technology. But when it comes to healthcare, the opportunities are promisingly life-saving.
ConclusionUsing artificial intelligence, machine learning, and deep learning is quite promising for custom software development. In healthcare, it gives plenty of life-saving opportunities for early diagnosis, treatment plan control, research, and analysis. If you like to move toward innovations and better efficiency, you need a tech-savvy development partner with experience in both AI and healthcare technologies. Cprime Studios might just be the custom software development vendor you are looking for. Our up-to-date expertise in these niches allows us to create top-notch healthcare solutions powered by the most innovative technologies. You are welcome to reach out to us and find out how we can help! Last Updated on January 27, 2021 It can be challenging for beginners to distinguish between different related computer vision tasks. For example, image classification is straight forward, but the differences between object localization and object detection can be confusing, especially when all three tasks may be just as equally referred to as object recognition. Image classification involves assigning a class label to an image, whereas object localization involves drawing a bounding box around one or more objects in an image. Object detection is more challenging and combines these two tasks and draws a bounding box around each object of interest in the image and assigns them a class label. Together, all of these problems are referred to as object recognition. In this post, you will discover a gentle introduction to the problem of object recognition and state-of-the-art deep learning models designed to address it. After reading this post, you will know:
Kick-start your project with my new book Deep Learning for Computer Vision, including step-by-step tutorials and the Python source code files for all examples. Let’s get started. A Gentle Introduction to Object Recognition With Deep Learning OverviewThis tutorial is divided into three parts; they are:
Take my free 7-day email crash course now (with sample code). Click to sign-up and also get a free PDF Ebook version of the course. What is Object Recognition?Object recognition is a general term to describe a collection of related computer vision tasks that involve identifying objects in digital photographs. Image classification involves predicting the class of one object in an image. Object localization refers to identifying the location of one or more objects in an image and drawing abounding box around their extent. Object detection combines these two tasks and localizes and classifies one or more objects in an image. When a user or practitioner refers to “object recognition“, they often mean “object detection“.
— ImageNet Large Scale Visual Recognition Challenge, 2015. As such, we can distinguish between these three computer vision tasks:
One further extension to this breakdown of computer vision tasks is object segmentation, also called “object instance segmentation” or “semantic segmentation,” where instances of recognized objects are indicated by highlighting the specific pixels of the object instead of a coarse bounding box. From this breakdown, we can see that object recognition refers to a suite of challenging computer vision tasks. Overview of Object Recognition Computer Vision Tasks Most of the recent innovations in image recognition problems have come as part of participation in the ILSVRC tasks. This is an annual academic competition with a separate challenge for each of these three problem types, with the intent of fostering independent and separate improvements at each level that can be leveraged more broadly. For example, see the list of the three corresponding task types below taken from the 2015 ILSVRC review paper:
We can see that “Single-object localization” is a simpler version of the more broadly defined “Object Localization,” constraining the localization tasks to objects of one type within an image, which we may assume is an easier task. Below is an example comparing single object localization and object detection, taken from the ILSVRC paper. Note the difference in ground truth expectations in each case. Comparison Between Single Object Localization and Object Detection.Taken From: ImageNet Large Scale Visual Recognition Challenge. The performance of a model for image classification is evaluated using the mean classification error across the predicted class labels. The performance of a model for single-object localization is evaluated using the distance between the expected and predicted bounding box for the expected class. Whereas the performance of a model for object recognition is evaluated using the precision and recall across each of the best matching bounding boxes for the known objects in the image. Now that we are familiar with the problem of object localization and detection, let’s take a look at some recent top-performing deep learning models. R-CNN Model FamilyThe R-CNN family of methods refers to the R-CNN, which may stand for “Regions with CNN Features” or “Region-Based Convolutional Neural Network,” developed by Ross Girshick, et al. This includes the techniques R-CNN, Fast R-CNN, and Faster-RCNN designed and demonstrated for object localization and object recognition. Let’s take a closer look at the highlights of each of these techniques in turn. R-CNNThe R-CNN was described in the 2014 paper by Ross Girshick, et al. from UC Berkeley titled “Rich feature hierarchies for accurate object detection and semantic segmentation.” It may have been one of the first large and successful application of convolutional neural networks to the problem of object localization, detection, and segmentation. The approach was demonstrated on benchmark datasets, achieving then state-of-the-art results on the VOC-2012 dataset and the 200-class ILSVRC-2013 object detection dataset. Their proposed R-CNN model is comprised of three modules; they are:
The architecture of the model is summarized in the image below, taken from the paper. Summary of the R-CNN Model ArchitectureTaken from Rich feature hierarchies for accurate object detection and semantic segmentation. A computer vision technique is used to propose candidate regions or bounding boxes of potential objects in the image called “selective search,” although the flexibility of the design allows other region proposal algorithms to be used. The feature extractor used by the model was the AlexNet deep CNN that won the ILSVRC-2012 image classification competition. The output of the CNN was a 4,096 element vector that describes the contents of the image that is fed to a linear SVM for classification, specifically one SVM is trained for each known class. It is a relatively simple and straightforward application of CNNs to the problem of object localization and recognition. A downside of the approach is that it is slow, requiring a CNN-based feature extraction pass on each of the candidate regions generated by the region proposal algorithm. This is a problem as the paper describes the model operating upon approximately 2,000 proposed regions per image at test-time. Python (Caffe) and MatLab source code for R-CNN as described in the paper was made available in the R-CNN GitHub repository. Fast R-CNNGiven the great success of R-CNN, Ross Girshick, then at Microsoft Research, proposed an extension to address the speed issues of R-CNN in a 2015 paper titled “Fast R-CNN.” The paper opens with a review of the limitations of R-CNN, which can be summarized as follows:
A prior work was proposed to speed up the technique called spatial pyramid pooling networks, or SPPnets, in the 2014 paper “Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition.” This did speed up the extraction of features, but essentially used a type of forward pass caching algorithm. Fast R-CNN is proposed as a single model instead of a pipeline to learn and output regions and classifications directly. The architecture of the model takes the photograph a set of region proposals as input that are passed through a deep convolutional neural network. A pre-trained CNN, such as a VGG-16, is used for feature extraction. The end of the deep CNN is a custom layer called a Region of Interest Pooling Layer, or RoI Pooling, that extracts features specific for a given input candidate region. The output of the CNN is then interpreted by a fully connected layer then the model bifurcates into two outputs, one for the class prediction via a softmax layer, and another with a linear output for the bounding box. This process is then repeated multiple times for each region of interest in a given image. The architecture of the model is summarized in the image below, taken from the paper. Summary of the Fast R-CNN Model Architecture. The model is significantly faster to train and to make predictions, yet still requires a set of candidate regions to be proposed along with each input image. Python and C++ (Caffe) source code for Fast R-CNN as described in the paper was made available in a GitHub repository. Faster R-CNNThe model architecture was further improved for both speed of training and detection by Shaoqing Ren, et al. at Microsoft Research in the 2016 paper titled “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.” The architecture was the basis for the first-place results achieved on both the ILSVRC-2015 and MS COCO-2015 object recognition and detection competition tasks. The architecture was designed to both propose and refine region proposals as part of the training process, referred to as a Region Proposal Network, or RPN. These regions are then used in concert with a Fast R-CNN model in a single model design. These improvements both reduce the number of region proposals and accelerate the test-time operation of the model to near real-time with then state-of-the-art performance.
— Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, 2016. Although it is a single unified model, the architecture is comprised of two modules:
Both modules operate on the same output of a deep CNN. The region proposal network acts as an attention mechanism for the Fast R-CNN network, informing the second network of where to look or pay attention. The architecture of the model is summarized in the image below, taken from the paper. Summary of the Faster R-CNN Model Architecture.Taken from: Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks. The RPN works by taking the output of a pre-trained deep CNN, such as VGG-16, and passing a small network over the feature map and outputting multiple region proposals and a class prediction for each. Region proposals are bounding boxes, based on so-called anchor boxes or pre-defined shapes designed to accelerate and improve the proposal of regions. The class prediction is binary, indicating the presence of an object, or not, so-called “objectness” of the proposed region. A procedure of alternating training is used where both sub-networks are trained at the same time, although interleaved. This allows the parameters in the feature detector deep CNN to be tailored or fine-tuned for both tasks at the same time. At the time of writing, this Faster R-CNN architecture is the pinnacle of the family of models and continues to achieve near state-of-the-art results on object recognition tasks. A further extension adds support for image segmentation, described in the paper 2017 paper “Mask R-CNN.” Python and C++ (Caffe) source code for Fast R-CNN as described in the paper was made available in a GitHub repository. YOLO Model FamilyAnother popular family of object recognition models is referred to collectively as YOLO or “You Only Look Once,” developed by Joseph Redmon, et al. The R-CNN models may be generally more accurate, yet the YOLO family of models are fast, much faster than R-CNN, achieving object detection in real-time. YOLOThe YOLO model was first described by Joseph Redmon, et al. in the 2015 paper titled “You Only Look Once: Unified, Real-Time Object Detection.” Note that Ross Girshick, developer of R-CNN, was also an author and contributor to this work, then at Facebook AI Research. The approach involves a single neural network trained end to end that takes a photograph as input and predicts bounding boxes and class labels for each bounding box directly. The technique offers lower predictive accuracy (e.g. more localization errors), although operates at 45 frames per second and up to 155 frames per second for a speed-optimized version of the model.
— You Only Look Once: Unified, Real-Time Object Detection, 2015. The model works by first splitting the input image into a grid of cells, where each cell is responsible for predicting a bounding box if the center of a bounding box falls within the cell. Each grid cell predicts a bounding box involving the x, y coordinate and the width and height and the confidence. A class prediction is also based on each cell. For example, an image may be divided into a 7×7 grid and each cell in the grid may predict 2 bounding boxes, resulting in 94 proposed bounding box predictions. The class probabilities map and the bounding boxes with confidences are then combined into a final set of bounding boxes and class labels. The image taken from the paper below summarizes the two outputs of the model. Summary of Predictions made by YOLO Model.Taken from: You Only Look Once: Unified, Real-Time Object Detection YOLOv2 (YOLO9000) and YOLOv3The model was updated by Joseph Redmon and Ali Farhadi in an effort to further improve model performance in their 2016 paper titled “YOLO9000: Better, Faster, Stronger.” Although this variation of the model is referred to as YOLO v2, an instance of the model is described that was trained on two object recognition datasets in parallel, capable of predicting 9,000 object classes, hence given the name “YOLO9000.” A number of training and architectural changes were made to the model, such as the use of batch normalization and high-resolution input images. Like Faster R-CNN, YOLOv2 model makes use of anchor boxes, pre-defined bounding boxes with useful shapes and sizes that are tailored during training. The choice of bounding boxes for the image is pre-processed using a k-means analysis on the training dataset. Importantly, the predicted representation of the bounding boxes is changed to allow small changes to have a less dramatic effect on the predictions, resulting in a more stable model. Rather than predicting position and size directly, offsets are predicted for moving and reshaping the pre-defined anchor boxes relative to a grid cell and dampened by a logistic function. Example of the Representation Chosen when Predicting Bounding Box Position and ShapeTaken from: YOLO9000: Better, Faster, Stronger Further improvements to the model were proposed by Joseph Redmon and Ali Farhadi in their 2018 paper titled “YOLOv3: An Incremental Improvement.” The improvements were reasonably minor, including a deeper feature detector network and minor representational changes. Further ReadingThis section provides more resources on the topic if you are looking to go deeper. Papers
R-CNN Family PapersYOLO Family PapersCode ProjectsResourcesArticlesSummaryIn this post, you discovered a gentle introduction to the problem of object recognition and state-of-the-art deep learning models designed to address it. Specifically, you learned:
Do you have any questions?
...with just a few lines of python code Discover how in my new Ebook: It provides self-study tutorials on topics like: Finally Bring Deep Learning to your Vision ProjectsSkip the Academics. Just Results. See What's Inside |