How a partnership approach to new technology development, built on extensive industry experience, can take automation solutions to the next level.

Quantifying, understanding and verifying product quality is a critical step in any manufacturing process. Moving from a manual inspection to an automated inspection provides greater insight into the quality of the product and process by creating measurable baselines and enabling statistical process controls. This case study provides an overview and takeaways from an ATS project that involved designed flexibility, close collaboration with the customer on their evolving needs, and advanced technologies including Artificial Intelligence-based (AI) inspection.

THE PROBLEM

A customer that manufactures and fills test tubes previously performed a manual go/no-go gauge to ensure consistent product fill levels. Working with ATS, an opportunity was identified to move to an automated measurement to increase the reliability and accuracy of the measurements. This has the added benefit of providing process control statistics, which supply the data needed to improve the full process.

THE SOLUTION

The first step in designing an automated inspection system is to thoroughly evaluate the problem, how the solution will be used, and the technology that is available to solve the problem. One common technology when measuring dispensed volumes is a weight check. This is a robust method as it provides a single value that can be calibrated and directly correlated to the volume. However, in this case, the customer’s product has multiple wells, which are filled simultaneously, meaning that a weight check would provide a global measure of the volume but would not be able to provide the per-well volume measurement which is required, meaning that a weight check would provide a global measure of the volume but would not be able to provide the per-well volume measurement which is required.

After evaluating the available technology and prototyping initial concepts, an automated inspection system leveraging machine vision was agreed upon to automate the manual quality check. The machine vision system designed by ATS can be viewed as the hardware for image acquisition, and the software for image processing.

Image Processing

ATS’s extensive experience in designing and implementing automated vision systems led to the development of machine vision products that enable engineers to design and integrate the best solution for each unique problem. ATS SmartVisionTM is a rapid-development machine vision programming environment that has been developed for over 20 years with thousands of deployments in automated machines across a multitude of industry segments and environments.

Evolving Problem Description

During the project execution, through continued discussions and reviews with the customer it was determined that the system now required validation to meet the evolving use-case. ATS engaged their Validation Engineering and Systems Engineering teams to plan and execute a capability analysis and system validation at FAT (Factory Acceptance Testing). Part of the validation process included implementing recipe control to allow a user to select from a number of different part configurations.

The system was then installed at site, and product straight off the manufacturing line was run through to collect initial data. At this point, it was observed that there were air bubbles in the media when freshly filled, which caused the meniscus to be artificially high. As the customer ran the inspection system with these new challenges, every image was saved to the vision PC, allowing for review of the data and for further inspection development.

Figure 2: User interface to the vision inspection, showing the initial implementation results where bubbles were not detected or considered in the volume measurement. Note that the real image of the customer product is overlaid with a semi-representative graphic.

These images with real-world data were sent to ATS to evaluate the new challenge and potential solutions. Reviewing the hundreds to thousands of images collected, there were two observations that guided the direction for the solution:

Those two observations, paired with an abundance of data, indicated a high potential for solving this vision challenge using an AI approach.

Extendable Technology to Address New Problems

The Deep Neural Network (DNN) tool within ATS SmartVision has been developed to be able to run deep learning models within any machine vision inspection. The DNN tool can run models trained with the majority of popular deep learning frameworks, including Tensorflow, Pytorch and others. This DNN tool acts like all other graphical tools within an ATS SmartVision application, allowing AI models to be easily added to an existing application to enhance functionality as inspection requirements or product appearances evolve over time.  

ATS trained a deep learning model to detect the presence and location of air bubbles – a problem previously not possible to solve with traditional machine vision tools. One hundred images with different liquids and fill levels, from different production batches, were used to teach an object detection model to locate bubbles within the liquid.

70% of images were allocated to a training set, 15% to a tuning (validation) set and 15% to a test set. In the training set, there were 89 bubbles labeled of varying sizes and locations. Once all the images were collected, the labeling and training process took less than 8 hours. The model trained had a mean average precision of 0.966, indicating a very good overlap of the labeled bounding boxes versus the predicted bounding boxes.

The deep neural network trained to detect bubbles was then integrated into the existing vision inspection, consisting of traditional tools to take accurate measurements of the fill level. The new inspection calculated the volume in each tube, as it previously did but, if an air bubble was present, its volume would be subtracted from the tube measurement.

This code update was completed by ATS remotely and sent to the customer for deployment.

Figure 3: Illustration of ATS Deep Neural Network (DNN) tool capability, running in inference mode to output the location of detected bubbles.

Figure 4: Illustration of user interface to the vision inspection, showing the updated implementation results where bubbles are detected, and the calculated volume is subtracted from the part volume.

THE RESULTS

From the beginning of the project, ATS designed a benchtop system with the intent to be flexible in customer support and software. This flexibility allowed AI-based vision inspection updates to address new challenges with product. Following the code updates to address the product seen onsite, the customer ran parts in parallel through the manual inspection process and the ATS machine vision inspection system for a period. This was done to prove the performance and reliability of the automated vision inspection as a quality system before making the final change over from the manual inspection to the validated machine vision system.

This project provided direct value by removing the subjectivity and repeatability concerns with a manual inspection – humans are all different and make mistakes. The rich data and statistics from the system can be used to monitor quality and more quickly and proactively identify any process issues.

After the successful completion of the first project the customer was able to procure identical vision systems for other sites across their company. This helps drive consistent and standardized quality monitoring regardless of the manufacturing site. Further, in a next generation manufacturing line, this vision system can be integrated directly in-line with the automated manufacturing process to inspect parts at high-speed rates.