Blogs

One of the major Asian automotive manufacturers wanted to explore Artificial Intelligence (AI) transformation for their factory floor to see if Artificial Intelligence could improve engine defect detection, and also track hard to read frame numbers as an automobile progressed along the assembly line.

This case study describes how the mSense product Sixth Sense helped them to improve their quality and streamline their manufacturing flow and in the process, there are no more engine assembly inspection headaches.

1. Situation- Engine Manufacturing

One of the major Asian automotive manufacturers, which makes around one million engines per year across four plants with 11 assembly lines/plant, wanted to improve:

•  Quality Assurance (QA) by using  the running engine’s sound to more thoroughly detect manufacturing  problems and
• Engine tracking through the manufacturing process, by automatically reading the frame number.

The current process was manual, labor intensive and error prone, and was not sufficient to keep up with the forecasted run rate. Customer needed a solution which improves their safety, manufacturing productivity and engine quality assurance.

2. Environment Challenges (Factory floor)

Customer’s manufacturing assembly/factory floor is compact and very noisy. There are multiple assembly lines and test stations in close proximity (10-12 feet) with engines constantly running. Each of the test stations has very limited space, and any changes to the assembly line are difficult and time consuming.  

The final test station, starts the newly manufactured automobile, sets the throttle, and a technician listens to the running engine. An experienced technician can usually hear a bad engine. 

To not impede the manufacturing run rate, a presumed bad automobile is pulled off the assembly line for further QA lab testing.   The technician needs to develop a keen ear to detect potential defects on the assembly line where several different types/models of engines with different fuel systems, exhausts etc.,  are assembled and tested. This experience is tough to gain and was not scalable.

3. Quality assurance and the manufacturing process

Unfortunately the technician listening to the engine’s sound could not catch all the bad engines.  In a total of about 2000 engines manufactured every day, post-manufacturing statistical analysis revealed that around 50 bad engines were slipping by the final inspection test.

Furthermore, to track each engine the manufacturer recorded the QR code. However, to eliminate confusion, the metal stamped 17 digit engine id number was preferred.Unfortunately, this id number was often in a poor location to read making it difficult for the technician.

4. mSense Sixth Sense Solution

Working with the manufacturer, mSense used its Sixth Sense product to train two purpose-built Deep Learning models to deploy on their factory floor:

1. Acoustic classification inference model: To classify engine sounds with background noise and
2. Character recognition inference model: To read an automobile’s Frame number in low light and varying backgrounds

Prototypes were developed within 3 months.

4a. A Sound Solution: 5X QA improvement

The Sixth Sense trained acoustic inference model improved QA by 5x detecting the bad engines that currently were missed.  A small footprint “non-invasive” hardware of 4 microphones for each final test station was able to quickly within seconds automatically detect the approximate 50 bad engines/day.

This level of accuracy was accomplished in spite of the background factory noise.   Now the technician while adjusting the throttle sees a green or red light for a good or defective engine. Furthermore in real time a QA dashboard as shown in figure 1 can be used to monitor manufacturing and spot trends before they become expensive problems.

Furthermore, mSense Sixth Sense inference models accommodate each of the type/style of engines simplifying QA across all engines.

Please see mSense demo video for more details.

4b. Frame number recognition: 6X performance

The customer for tracking purposes wanted to read the stamped frame number rather than just the barcode.  Numerous previous efforts to automatically capture the metal stamped engine id as shown in Figure 2, proved to be difficult because of the poor lighting, location and number of digits. 

mSense used a hand-held camera (similar to QR code reader) to capture the frame number’s image. Training a deep learning inference model, the image was translated to digits at a 99.0% confidence level: much better than the current manual technique. Furthermore to capture the frame number before took over 30 seconds while mSense’s solution took less than 5 seconds.

For the first time, the manufacturer had a safe and productive way of capturing this critical id number.

6. Summary: no more headaches

To summarize, mSense’s acoustic solution was able to:

• Remove the loud factory floor background noise to “listen” to just the running test engine
• The acoustic pass/fail test took seconds to complete
• The test is automatic: no technician training is needed
• The required hardware is a very small footprint/low power and is non-invasive requiring little changes to the assembly line

Our customer continues to engage with mSense in deploying these solutions across their assembly lines to accelerate their digital AI transformation resulting in improved factor floor productivity and QA.

 Over time with more automatically captured data and inference model refinement continued improvement in  engine quality will realize even higher product margins with no further headaches

For further details please contact us @ hello@msense.ai

Hidden Debt of Machine Learning Platforms

With advancement of artificial intelligence and machine learning – from smart assistants, like Alexa and Siri, to tagging our friends in the social network, it is not surprising that every aspect of our lives is being touched and revolutionized by this new technology.

It is also impacting enterprise space and driving a transformation in the manufacturing sector, paving the way for Industry 4.0 revolution

However creating and maintaining an AI/ML framework is fundamentally different from traditional software solutions. As D. Shully called out in his seminal paper ‘Hidden Technical Debt in Machine Learning Systems’[i],there is a technical price, an enterprise has to pay in order to implement and maintain AI/ ML systems, before they can reap its rewards.

Source: Hidden Technical Debt in Machine Learning System, D. Sculley et. Al¹

In this WP, we will go over the system level thinking needed to implement a functional ML system within an enterprise environment.

In conventional software development, there is an increasing focus of module isolation and code segmentation. However for ML systems, there is intricate dependence on the input signals, feature list etc, making it harder to follow the conventional software models. For example, if system has ‘N’ set of features, removing a feature could impact the entire systems, in terms of weights, connectivity etc and could require re-calibrating the entire system.

Similarly, addition of a new feature could change the dynamics of the system. So in order to create and maintain a robust AI/ ML based system, one possible option is to divide the system into multiple stages with well-defined boundaries and interfaces.

Lastly, the framework needs to integrate with the enterprise existing frameworks/ data lakes and need to be Dev Ops friendly, so that it becomes a sustainable AI/ML solution and not just a point solution for one time usage.

Following flow diagram depicts one such possible implementation, where the entire flow is divided into four different stages.

Figure 1: Potential 4 stage implementation of an Industrial AI/ML System

• Stage 1 -Sensor Network: The very first stage of an Industrial AI/ ML system could be sensor ingest/ integration stage, which collects the data from different sensors or sensor networks. These sensors could include microphones, cameras,vibration sensors etc.
  
• Stage 2 -Data Lakes: As highlighted in D. Sculley’s paper, 90% of the efforts of a machine learning system is to clean and filter input signals, perform ETL operation and make the data usable for down the stream system. So the next stage of the system is to process this sensor data, which might be different formats, like audio files, time series etc., and create a well-defined data-lake for the next stage processing.
• Layer 3:Data Labeling annotation and feature extraction: This step is needed during the model definition or training phase, and during model updates. The input data needs to labeled and the right feature sets need to be defined, so that right ML algorithms can be developed.
  • Once the models are well defined and trained,another critical step is to decide the timing and the potential trigger points,when this model should get updated.
• Layer 4: Predictive Analytics: This is the outcome of all the hard work. At this stage, data and ML trained model could perform the predictive analytics to provide valuable insights.

In the upcoming blogs, we will delve into these stages in detail and show how mSense implementation leverages this structured approach for audio classification for predictive analytics. We will also highlight the Dev Ops friendliness of mSense platform, which makes it easier to seamlessly integrate it in the enterprise framework.

In the meantime, please visit us at www.msense.ai to get more details.

---

[i] Hidden Technical Debt in Machine Learning System, D. Sculley et. Al., https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf

With advancements in new technologies, like Internet of Things (IOT), Cloud Computing and Machine Learning(ML), the modern manufacturing process has come a long way during the last few years. A new paradigm has emerged, creating Industrial 4.0 revolution, where the combination of sensors, edge data processing and ML algorithms allow manufacturing plants to not only  to digitize, automate, and optimize the work flow, but also to provide early detection of any mechanical issues.

Companies, as shown in Figure 1[i], are trying to move up the value chain, by exploiting the top 5 digital technologies – Cloud Computing, Network Control Systems, and Equipment with Embedded Sensor and Control Systems, and AI. However, they struggle with how to best utilize their existing data, pull together new sensor data and combine it with software tools, and talent to deliver on this transformative business potential.

Figure1: Production related technology leads the fourth industrial revolution¹

AI is not a very good organizational fit:

A recent PWC report [ii] highlighted that AI requires new organizational thinking. Customarily Chief Information Officers are leading most digital transformation efforts. However, as mentioned above, AI is a convergence of data, sensor physics, distributed networks/ IoT, and ML, which spans the manufacturing enterprise from business units to manufacturing operations, to data specialist, to IT.

As testimony to this new organizational thinking, Figure 2 shows the varied approaches taken from 1,000 executives’ 2019 AI plans with most predominant option being creating an internal center of excellence, which might not be the right approach, considering scarcity of talent and expertise in this area.

Figure 2: The many 2019 AI organizational approaches

You do not want a research experiment

Is an AI approach the right one to solve your problem? Most enterprises do not want to take the time and money to run a research experiment to answer this question. Instead what is needed early on is a technique to know what results AI can achieve and are all the pieces, such as sufficient data available.  

To complicate this question often AI offers a trade-off in the amount of labeled data and the result’s accuracy such as a classification confidence percentage.  Understanding if an acceptable confidence score of 60% or 70% may provide sufficient accuracy is paramount to harvesting early AI potential benefits.

The enterprise AI workflow

Getting the right people at the right time to work together will realize the full potential of AI. Leveraging an enterprise’s areas of expertise is required to integrate data, ML algorithms,and IT infrastructure into an end to end solution from manufacturing data such as sensors to the appropriate ML algorithm to realizing a tangible business ROI. 

What is needed is a software AI enterprise workflow that allows for iterative collaboration and appreciation for each step in the AI journey.  A practical enterprise workflow will automatically handle most of the tedious steps in the process while exposing critical steps such as feature engineering for review and implementation.  

Being able to capture manufacturing expertise such as factory floor inspections and converting that into labeled data for ML training to improve assembly run rates is just one example of how an integrated end to end AI workflow can realize the full potential of AI to transform businesses. 

As described in subsequent posts, mSense offers a customer proven practical AI enterprise workflow solution that accommodates the enterprise’s current organization and talent.

---

[i]https://www.eef.org.uk/~/media/c98ecc3376c44cbd86b2093cd086ca8e.pdf?la=en

[ii] https://www.pwc.com/us/en/services/consulting/library/artificial-intelligence-predictions-2019