How Deep Learning models are revolutionizing quality assurance

When it comes to quality, we leave nothing to chance at thyssenkrupp. That's why our digital experts at thyssenkrupp Steel have developed AI-based models that take our quality assurance to the next level - for example in the production of packaging steel for food and beverage cans.

Thanks to its ideal properties for the circular economy, packaging steel is the material of the future - efficient, process-optimized, and sustainable. In Andernach, the world's largest production site for packaging steel, thyssenkrupp manufactures rasselstein® tinplate for around 400 customers in 80 countries and stands for premium-quality materials. To ensure it remains that way, the colleagues and specialists from thyssenkrupp Steel's Digital Data Lab have come up with a solution.

Intelligent edge crack detection with deep learning

Using artificial intelligence intelligently - that's the motto of Yavuz Dogan and Lasse Schmidt, Data Scientists in thyssenkrupp Steel's Digital Data Lab. Together with packaging steel expert Christoph Schirm and his colleagues from thyssenkrupp Rasselstein, they have developed a system that automatically detects and classifies production defects in packaging steel.

The idea behind is simply explained: "Defects with similar visual patterns are very likely to have similar causes," says Christoph Schirm, team leader for quality assurance systems at thyssenkrupp Packaging Steel. If these defect patterns could be specifically categorized and causes more precisely assigned, the experts assume, it would also be easier to optimize production and thus increase quality.

Automatic classification of errors

No sooner said than done? It was a long way from the initial idea to the practical solution, recalls Christoph Schirm. It quickly became apparent to him that the team would need the support of the digital experts from Duisburg to optimize the processes.  Even though the previous surface inspection system (OIS) can classify defect images into rough categories such as "cracks" or "holes," it is not able to differentiate them according to fine details such as structure and size.

"That had to be created first," explains Yavuz Dogan from the Quality Improvement and Steering (QIS) team, who is responsible for mathematics and numerical simulation at thyssenkrupp Steel. "For automatic classification, we developed an image-based deep learning model to categorize the edge defect images from the existing OIS according to visual similarity features." To speed up development, the experts relied on existing, pre-trained and already proven deep learning models.

But how can you even imagine such a model? Lasse Schmidt, Expert Data Scientist Analytics Apps at thyssenkrupp Steel, describes the basic principle: "Convolutional Neural Network, a deep learning model in the field of image analysis, works like an algorithm that tries to emulate human vision." To do this, Schmidt and Dogan trained the neural network using hand-classified learning data. They did it that way, until the model's assessments could no longer be distinguished from those of a human expert.

The goal: The model should be able to automatically assign new defect images to different types of cracks and holes depending on visual characteristics. The important thing is not that all images in a class look the same, but that the pictures have similar characteristics that suggest a common cause. In addition to the defect class, the model provides a numerical value from zero to 100% that describes how confident the model is in its assessment of the image. While high values indicate a clear class membership, very low values indicate that a defect image does not fit properly into any of the existing classes and that a new defect type might even have been found.

Challenges of surface inspection

However, as with humans, only practice makes perfect with a deep learning model. The big challenge for the digital experts was to provide their model with enough sample images for different classes so that it could learn the respective properties and recognize them in new images. To do this, they sifted and labeled numerous images. But here, too, the project team - true to their motto - used artificial intelligence to intelligently make their work easier. "We started with a few labeled sample images per defect class and used a rudimentary model to try to automatically assign the hundreds of thousands of images in the database to these defect classes," Dogan says. "This automatic assignment was randomly reviewed by the experts and - if necessary - corrected. This made it possible to iteratively expand the amount of learning data and improve the model," says Lasse Schmidt.

In the meantime, they integrated their developed deep learning model into the IT landscape. For this purpose, a new server with a powerful graphics processor was purchased. Thereby, they were able to classify the many images faster than with an ordinary computer processor. In the future, the correlation of defect classes with process parameters will support root cause analysis for the occurrence of different defect types to reduce downtime and rework costs.

Deep Learning Model Developed in 2 Weeks

The digital experts are very satisfied with the successful course of the project. "The entire project was developed entirely in-house using open source tools. With a comparable commercial development, we would end up in the six-figure range," explains Yavuz Dogan. Added to this is the incredibly fast development time of just two weeks.

The onslaught of other responsible people on Yavuz Dogan, Lasse Schmidt, and their deep learning model is correspondingly great. "Models like our deep learning model cannot be implemented elsewhere without further ado; they are customized solutions. In the meantime, we have enough methodological expertise and code modules to quickly develop appropriate solutions even for new problems," says Lasse Schmidt. To detect new types of defects, the system must be fed with a sufficiently large number of new images. Accordingly, defect images have to be classified manually in advance, the model adjusted and trained again. "Nevertheless, we are of course pleased about the many requests for image recognition algorithms and the positive response to our model for edge crack detection," says Yavuz Dogan.