Automated visual quality inspection using AI/ML in manufacturing industries that reduces scrap/rework costs

by Kaviraju, Nibedita (Co-Author)

Role of AI/ML in manufacturing to reduce costs associated with scrap and rework

In manufacturing, automatic visual quality inspection harnesses the power of AI, IIoT, and image recognition to help manufacturers improve product quality while significantly reducing the costs associated with scrap and rework.

Let us first understand what is visual quality inspection in manufacturing?

In simple words, it is the analysis or inspection of products (produced in the production lines) which is done for the purpose of quality control. Visual inspection of products can also be done for understanding proper behavior of various equipments in the Shopfloor.

In other words, it is a process that takes place at regular intervals during the shift, otherwise every N’th product is taken out for sample inspection. Note that, automated visual inspection results in the discovery of hidden defects in the product or equipment which is sometimes not possible to find through manual inspection by humans.

While visual inspection in production environment is carried-out for quality assessment of products and equipments, in non-production environments it can be used to determine whether the features indicated as “target” are present in the “actual” and thereby preventing potential negative impacts.

Most of industries consider visual inspection as very high priority activity due to the potentially high cost of any errors that may arise via inspection such as injury, loss of expensive equipment, scrapped items, rework, or a loss of customers. Such fields where visual inspection is prioritized include automotive parts, airport baggage screening, aircraft maintenance, food industry, pharmaceuticals, nuclear weapons etc.

How AI helps in automating quality inspection?

These days, AI is turning out to be a game changer with countless applications in almost every domain. It is now making its way into the area of manufacturing, allowing it to harness the power of AI and in doing so, providing automation that is faster, accurate, cheaper and more superior. This article aims to give a brief understanding of automated visual quality inspection and how the AI approach can save significant time and effort. Deep learning, computer vision and image processing (all are part of AI) are the subjects which are generally applied for developing such automated quality inspection applications.

Why can’t we just stick to manual quality inspection?
Many questions arise in our mind like, why should industries invest in AI based automatic quality inspection system development? Can’t they continue with manual inspection process without wasting time and money AI development? As we always say “old is gold”, one could argue that there are several problems in using old style of manual inspection as detailed below…

  1. Manual inspection requires presence of a quality engineer who performs assessment of the product under inspection, and passes judgement based on either training provided to him or his prior experience. Sometimes no equipment is used except the naked eye or sometimes measuring instruments are used for inspection process.
  2. The human eye is incapable of making precise measurements, especially on a very tiny scale. Even while comparing two similar objects, the eye might not notice that one is slightly smaller or larger than the other.
  3. There remains the fact that the human eye, while more technologically advanced than any mechanical or electronic camera, can be easily fooled. This does not necessarily mean that manual inspection is totally useless, but that it would be unwise to depend entirely on it.
  4. Manual inspection remains a costly venture due to the appointment of (multiple) trained individuals.

According to research, manual visual inspection errors are typically ranging from 20% to 30%. Some imperfections can be attributed to human error, while others are due to limitations of space. Certain errors can be reduced through training and practice, but cannot be completely eliminated.!

Visual inspection errors in manufacturing take one of two forms — missing an existing defect (false negative) or incorrectly identifying a defect that does not exist (false positive). Misses tend to occur much more frequently than false alarms. Misses can lead to loss in quality, while false positives can lead to unnecessary production costs and overall wastage.

A new age alternative for visual quality inspection

In manufacturing, automatic visual quality inspection harnesses the power of AI, IIoT, and image recognition to help manufacturers improve product quality while significantly reducing the costs associated scrap and reworks.

Using Deep Learning and Machine Vision, it is not only possible but quite achievable to build smart systems that perform thorough quality checks down to the finest details. Minimal physical equipment is needed to automate the visual inspection process and the process is made smarter by using deep learning. This approach typically involves steps such as image acquisition, preprocessing, feature extraction, classification etc.

AI helps to improve defect detection and hence better business outcomes…

  • 10-15% of total operating costs often associated with poor quality product (Forbes, 2018)
  • 1/3 of manufacturing executives now identify AI-driven technologies as crucial to driving customer satisfaction (Forbes, 2018)
  • $3.7 Trillion – the value that McKinsey forecasts AI-powered “smart factories” will generate by 2025.

AI/ML models are readily available for several manufacturing domain use-cases from cloud service providers (like Google, AWS) and also from 3rd party. However developing AI/ML models from scratch is very expensive and organizations can decide how to proceed based on budget and time availability. See few examples below…

Now, in next sections, let us understand few key techniques of AI & ML that can be applied in automated visual quality inspection system development…

Computer Vision and Image Processing - are they same?

Both are part of AI technology which are used while processing the data and creating a model. For instance, object recognition, which is the process of identifying the type of objects in an image, is a computer vision problem. In computer vision, you receive an image as input, and you can produce an image as output or some other type of information.

If you have noisy or blurred images, then under image processing the deblurring or denoising is done to make the object in the image clearly visible, and this shall become an input to computer vision. Therefore; image processing task normally involves filtering, noise removal, edge detection, and color processing. In entire image processing task, you receive an image as input and produce another image as an output that can be used to train the machine through computer vision algorithms.

To conclude, main difference between computer vision and image processing are the goals, but not the methods used. If the goal is to enhance the image quality for later use, then we apply image processing techniques. If the goal is to visualize like humans, like object recognition or defect detection, then we apply computer vision techniques.

Computer Vision in AI and ML - What it means?

Computer vision is simply the process of perceiving the images and videos available in the digital formats. In ML and AI, computer vision is used to train the model to recognize certain patterns and store the data into their artificial memory to utilize the same for predicting the results in real-life use. The main purpose of using computer vision technology in ML and AI is to create a model that can work itself without human intervention. The whole process involves methods of acquiring the data, processing, analyzing, and understanding the digital images to utilize the same in the real-world scenario.

Computer Vision - How it works?

Computer vision is used for deep learning to analyze the different types of data sets through annotated images showing the object of interest in an image. It can recognize the patterns to understand the visual data feeding thousands or millions of images that have been labeled for supervised machine learning algorithms training.

Here we use various software techniques and algorithms which allow computers to recognize the patterns in all the elements that relate to those labels and make the model predictions more accurate in the future. Therefore, computer vision can be utilized only with image processing through machine learning.

Computer vision involves various software techniques and algorithms that allow computers to recognize the patterns in all the elements that relate to labels and make the model predictions more accurate. Computer vision can be utilized only with image processing through ML.

Then, what is Deep Learning and how does it work?

Have you ever wondered how Google’s language translator App is able to translate entire paragraphs from one language into another in a matter of milliseconds?

How Netflix and YouTube are able to figure out our taste in movies or videos and give us appropriate recommendations?

How self-driving cars are even possible?

All of the above are product of Deep Learning and Artificial Neural Networks. Then what exactly is Deep Learning?
Deep Learning is a subset of ML, which on the other hand is a subset of AI. AI is a general term that refers to techniques that enable computers to mimic human behavior. ML represents a set of algorithms trained on data that make all of this possible. Deep Learning, on the other hand, is just a type of ML, inspired by the structure of a human brain.

Deep learning algorithms attempt to draw similar conclusions as humans would by continually analyzing data with a given logical structure. To achieve this, Deep learning technology uses neural networks containing thousands of layers which are capable of mimicking human level intelligence to distinguish anomalies, parts, and characters while tolerating natural variations in complex patterns. In this way, deep learning merges the adaptability of human visual inspection with the speed and robustness of a computerised system.

Deep learning teaches machines to do what comes naturally to humans to learn by example. New, low-cost hardware has made it practical to deploy a multi-layered “deep” neural networks that mimic neuron networks in the human brain. This gives manufacturing technology amazing new abilities to recognize images, distinguish trends, and make intelligent predictions and decisions. Starting from a core logic developed during initial training, deep neural networks can continuously refine their performance as they are presented with new images, speech, and text.

Then, what is Machine Vision?

Machine vision is the technology and methods used to provide image-based automatic inspection results of a product which is being produced on a machine. Normally, cameras are used to monitor the products moving on conveyor belts. Highly optimized AI algorithms take pictures of product, analyze them in real-time and notify other systems immediately as faulty products or if irregularities are identified. In other situations, AI algorithms monitor dangerous situations (like oil stains potentially causing failures), unusual incidents (like wrong proportions in chemical substances potentially generating issues) etc. and display on the HMI’s or Andons. Here bottom line is that, instead of forcing humans to look for hours at certain areas just to wait for events that might or might not happen, we might as well consider placing AI-powered cameras to monitor and notify us about suspicious events.

How Machine Vision (Computer Vision) and Deep Learning go hand-in-hand for automatic quality inspection system implementation?

Machine Vision has a very high optical resolution which depends upon the technology and equipment used for image acquisition. Compared to human sight, it has a ‘wider’ spectrum of visual perception with the ability to perform observations in the Ultraviolet, XRay and Infrared regions of the spectrum as well. Its benefits are…

  • Faster: Observations as well as conclusions are made extremely fast, with the speed of a computer’s speed as measured in FLOPs and also, they result in precise calculations.
  • Accurate: An automated system is capable of measuring absolute dimensions in a standardized manner.
  • Independent of Environment: Such a system can be deployed in dangerous and hazardous conditions or environments where human involvement may prove to be risky.
  • Reliable: The system is unbiased and programmable as required, following the instructions without any question.

However; due to the variation in a part’s appearance due to scaling, rotation and distortion, image quality issues generally introduce serious inspection challenges. Machine vision systems alone fail to assess such deviations between very visually similar images. Therefore deep learning-based systems in combination with machine vision implementation are well-suited for accurate visual inspections that are more complex in nature. Deep learning-based image analysis differs from traditional machine vision in its ability to conceptualize and generalize a part’s appearance at granular level.

How do I get started then?

Every manufacturer is different and every defect detection requirement is unique. It’s difficult to determine quickly that how to implement right visual inspection alogorithm that fits your quality goals and match your operating conditions. Hence start with proof-of-concept (PoC) development to determine your unique accuracy requirements and train the machine learning model accordingly. For most customers, the PoC development can be completed in 8-12 weeks at a very low cost. After certain days if it is found to be producing great results, operationalize it into a solution in your factories across machines, may be with certain customization in the algorithms as necessary. Finally, attempt to maintain and improve the ML model. On a quarterly basis, meet with your quality teams and help the model learn from any mistakes it has made. In this way, the accuracy of the output will improve over the time.

Steps to implement AI solution are given below…

1. State the problem
Visual inspection development often starts with a business and technical analysis. The goal here is to determine what kind of defects the system should detect. In this step, data Science engineer(s) must ask questions like below. Data science engineer(s) will choose the optimal technical solution based on answers they receive on these questions…

  • What is the visual inspection system environment (Shopfloor environment where equipment is placed)?
  • Should the inspection be realtime or non-realtime – mainly algorithm performance expectations?
  • How should the system notify the user(s) about detected defects – display on andons or something else?
  • Should our solution record defects detection statistics in database or so?

2. Gather and prepare data
Data science engineer(s) must gather and prepare data required to train a future model before deep learning model development starts. For manufacturing processes, it’s important to implement IoT data analytics. When talking about visual inspection models, the data is often video records, where images processed by a visual inspection model include video frames. There are several options for data gathering, but the most common are:

  • Taking an existing video recordings from customer if exists
    Taking open-source video records that are applicable to common manufacturing processes.
  • Gathering data from scratch using cameras according to deep learning model requirements.
  • The most important parameters here are the video recording quality. Note that, higher quality data will lead to more accurate results.

Once we gather the data, we prepare it for modeling, clean it, check it for anomalies, and ensure its relevance.

Visit this link to know about how much training data is normally required for developing successful AI/ML based prediction systems?

3. Develop deep learning model
The selection of a deep learning model development approach depends on the complexity of a task, required delivery time, and budget limitations. There are several approaches like below…

  • Using a deep learning model development service available in cloud platforms (likewise Google Cloud ML Engine, Amazon ML, etc.). – Such ready-made services can save both time and budget as there is no need to develop ML models from scratch. But note that these types of models are not customizable.!!!
  • Using pre-trained models – A pre-trained model is an already created deep learning model that accomplishes tasks similar to the one we want to perform. We do not have to build a model from scratch here too. A pre-trained model may not 100% comply with all of our tasks, but it offers significant time and cost savings. But it allows us to customize algorithm as per our needs, and also to improve accuracy of output.
  • Deep learning model development from scratch – This method is ideal for complex and secure visual inspection systems. The approach may be time and effort-intensive, but the results are worth it. Here, data scientists use one or more computer vision algorithms to produce the required output.

4. Train and evaluate the model
The next step after developing the visual inspection model is to train it. In this stage, data scientists validate and evaluate the performance and result accuracy of the model by providing several historical video recordings as an input. A test dataset is needed for testing the model, which is going to be small set of video recordings.

5. Deploy and improve the model
It is important to consider right software and hardware capacity in the server (based on complexity of developed ML models) to deploy your AI solution.

  • Storage, CPU and GPU capacity on the server becomes very important for storing and analyzing video frames.
  • Other hardwares you need to consider for visual inspection solution are – High resolution IP camera, Gateway, Photometer (if lighting condition Shopfloor environment is not good) and Colorimeter (if detecting color and luminance in Shopfloor environment is becoming difficult).

Deep learning models are open to improvement after deployment. A deep learning approach can increase the accuracy of the neural network through the iterative gathering of new data and model re-training. The result is a “smarter” visual inspection model that learns through increasing the amount of data during operation.

Few facts - AI based automatic visual inspection

Healthcare Domain: In the fight against COVID-19, most airports and border crossings can now check passengers for signs of the disease.

  • Real-life case is the deep learning-based system developed by the Alibaba company. The system can detect the coronavirus in chest CT scans with 96% accuracy. With access to data from 5,000 COVID-19 cases, the system performs the test in 20 seconds. Moreover, it can differentiate between ordinary viral pneumonia and the coronavirus.
  • Baidu, the large Chinese tech company, developed a large-scale visual inspection system based on AI. The system consists of computer vision-based cameras and infrared sensors that predict the temperatures of passengers. Now the technology, operational in Beijing’s Qinghe Railway Station, can screen up to 200 people per minute. The AI algorithm detects anyone who has a temperature above 37.3 degrees.

Automotive Domain: Toyota has recently agreed to a $1.3 billion settlement due to a defect that caused cars to accelerate even when drivers attempted to slow down, resulting in 6 deaths in the U.S. Using the cognitive capabilities of visual inspection systems like Cognex ViDi, automotive manufacturers can analyze and identify quality issues much more accurately and resolve them before they occur.

Textile Domain: The implementation of automated visual inspection, along with a deep learning approach, can now detect issues of texture, weaving, stitching, and color matching.

Image Source: MobiDev

14 thoughts on “Automated visual quality inspection using AI/ML in manufacturing industries that reduces scrap/rework costs”

  1. That is really attention-grabbing, You’re a very skilled blogger. I’ve joined your feed and sit up for in search of extra of your magnificent post. Additionally, I’ve shared your site in my social networks!

  2. Heya i am for the first time here. I came across this board and I find It really useful & it helped me out much. I hope to give something back and aid others like you aided me.

  4. Admiring the commitment you put into your blog and detailed information you provide. It’s awesome to come across a blog every once in a while that isn’t the same outdated rehashed information. Great read! I’ve saved your site and I’m adding your RSS feeds to my Google account.

  7. My brother suggested I might like this website. He was entirely right. This post actually made my day. You cann’t imagine just how much time I had spent for this info! Thanks!

  8. Hi there, I discovered your website by means of Google whilst searching for a similar subject, your site got here up, it seems great. I’ve bookmarked it in my google bookmarks.

  10. I was very pleased to find this web-site.I wanted to thanks for your time for this wonderful read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you blog post.

  11. I think other web site proprietors should take this web site as an model, very clean and excellent user genial style and design, let alone the content. You’re an expert in this topic!

Comments are closed.