Over the past few years, with the sharp fall in the prices of sensors and MPU and the surge in the volume of deliveries, more and more companies are trying to gain advantage by adding sensor-driven embedded AI to their products. The automobile industry is leading this trend -- there are now an average of 100 sensors per non-automatic vehicle, sending data to 30-50 microcontrols...
Over the past few years, with the sharp drop in the prices of sensors and MMU and the surge in the volume of deliveries, more and more companies have attempted to gain advantage by adding sensor-driven embedded AI to their products.
The automobile industry is leading this trend – there are now 100 sensors on average per non-automatic vehicle, sending data to 30-50 microcontrollers, which operate approximately 1 m line of code per day, generating 1 TB data per vehicle per day. The number of luxury cars is likely to be twice that, while the number of autonomous cars is expected to increase the number of sensors more significantly.
But this is not just a car trend. Industrial equipment is becoming more and more "smart," and manufacturers of rotational, back-to-back and other types of equipment are eager to increase their capabilities for state monitoring and predictive maintenance, while new consumer products, ranging from toothbrushes, vacuum cleaners to fitness monitors, are adding instrumentation and "smart."
Real-time, edge and reasonable price
Common to these applications is the need to use real-time, fluid, complex sensor data -- accelerometers, vibrations, sound, telecommunications and biometric signals -- to find the characteristics of a given event and condition, or to detect anomalies, and to do so locally on the equipment: the code that runs in solids on microcontrollers that match product prices.
There are three main challenges that need to be overcome when starting to build smart products driven by such sensors.
Challenges when using sensors with embedded AI.
Changes in goals and background
Real-time detection
Constraints - size, weight, utility, price.
The first is mutation.
The real world's data are noisy and full of variations -- which means that what you're looking for may look different in different contexts. You're facing a change in the target. Change in context is sometimes as important as change in purpose, so you need to collect as many examples and counter-examples as possible.
The second is real-time detection in firmware.
The need to be able to perform tests locally, provide users with a " real-time " experience, or trigger time-sensitive control responses in machines adds to the complexity of the problem.
And the third is binding -- physical, power, economic.
Many problems are much simpler if there is unlimited computing power. But the real world’s products have to be delivered in combination with physical size, weight, effort and cost constraints.
Traditional engineering VS machine learning
It is not easy to do all of this under the necessary constraints, to overcome mutations and to do difficult tests in real time on the edge. But it is becoming easier with the development of modern tools, including new options for mechanical learning of signals (such as Reality AI).
Of course, traditional engineering models built with tools such as Matlab are viable options for creating locally embedded detection codes. Matlab has a very powerful signal-processing toolbox that can be used to create highly complex, well-calculated detection models in the hands of engineers who really know what they are doing.
Why use machine learning?
But robotic learning is becoming a tool of choice. Why do you do that?
First, more sophisticated machine learning tools that optimize signal problems and embedded deployments can shorten the research and development cycle by a few months or even years. They can quickly get answers, quickly generate embedded codes, and allow product developers to focus on their functions rather than on the maths of testing.
But more importantly, they can often complete the tests that traditional engineering models cannot do. They can do this by using data more effectively to overcome variability. Traditional engineering methods are usually based on physical models, using data to estimate parameters, and machine learning methods can learn from these models independently. They learn how to test signatures directly from raw data and use the mechanics (mathematics) of machine learning to distinguish targets from non-targets, rather than relying on physics.
Take different approaches to different problems
Equally important, there are several different methods of machine learning for this complex data. The media are most concerned with "deep learning", a machine learning method that uses a collage and/or circular neural network layer to learn how to predict accurately from a large amount of data. In-depth learning has been successful in many cases, but it also has disadvantages -- especially because it requires very large data sets for training and, for deployment, it usually requires specialized hardware ($$$$$). If your deployment is faced with cost, size or power constraints, other methods, such as our approach in Reality AI, may be more appropriate.
There are three things to remember when building products with embedded AI.
There are three things you should know if you want to learn from machines in embedded systems.
- Use rich data, not bad data. The best machine learning method works best with well-informed data. Make sure you capture what you need.
- It's all about features. Once you have good data, the features you choose to use as input to machine learning models will be much more important than what algorithm you use.
- Prepare for compromise and trade-off. The size of the sampling rate and decision-making window that you collect will drive much of the memory and clock speed requirements of your choice. They will also affect detection accuracy. It is important to test the relationship between precision, sample rate, window size and calculation strength. The best machine learning tool in the market will make it easier for you to do that.
Machine learning: laboratory and real world
Recently, TechCrunch published a report on research at Carnegie Mellon University showing that "super-frequency smart-watch sensors use vibrations to sense gestures, objects, and positions." These people at the CMU Interactive Institute apparently modified a smart-watch operating system to capture an accelerometer size of 4 kHz (with most wearable devices having a catch rate of 0.1 kHz) and found that more data were available to detect more things. They can detect specific gestures and even judge what a person is touching or holding. (Is that an electric toothbrush, a booking machine, or a driving car steering wheel?)
"Duh!"
For those of us working in this field, including Carnegie Mellon, this is not a big revelation. "Dum! Of course, you can!" This is a very good but limited academic recognition of what many already know and are working on. Apparently, the journalist was not aware of many commercial products (among them, Myo from Thalmic Labs, using his own hardware, or some 20 other products providing smart watch tools). It seems that he has no idea of any toolbox on the market that would identify very subtle vibrations and accelerometers to detect the condition of machines (e.g., our own Reality AI for Industrial Employment Monitoring) or to detect user activity and the environment in wearable equipment (Reality AI for Consumer Products) I don't know.
Instead, I would like to use this case to illustrate some of the key questions about the use of machines to learn to produce for the real world. The difference between generalization and over-training, and laboratory trials, such as that study, and the deployment of the real world.
Generalization and Overtraining
Panorama refers to the ability to correctly identify examples not included in the original training set using machine learning or detectors. Excessive training refers to a taxonomy that has learned to identify specific examples of its training at a higher accuracy rate, but has not performed well in similar cases it has not seen before. The over-trained taxonomy has " too good " for training – in fact, it remembers the specifics of the training examples, but does not have the capacity to rediscover similar cases in the field. In laboratories, it's good when you try to determine whether something can be detected, but an overtrained sorter will never be useful in the real world.
Usually, the best way to prevent over-training is to use a training package that captures as much of the desired changes in the target and environment as possible. For example, if you want to test when a particular type of machine shows a particular condition, you include in your training data examples of many of these types of machines showing that condition, and in a series of conditions of work, loads, etc.
Detect overtraining and predict generalization
We can detect over-training and estimate the extent to which machine learning sorters or detectors are generalized. In Reality AI, our usual diagnostic method is K-Fold, which is generated by our tools.
K-Fold certification involves repeating 1) random selection of a part of the training data (e.g. 10 per cent), 2) training of the remaining part (90 per cent), 3) use of a 90 per cent training model to classify the data for retention, 4) recording of results. In general, the data set aside do not overlap, so, for example, for 10 per cent of the data set aside, 10 independent tests will be completed. The holds can be balanced across groups, and the verification can be averaged in multiple operations, but the key is that, in each iterative generation, the taxonomy is tested on data that are not part of its training. The accuracy rate is almost certainly less than what you calculated by applying the model to the training data (which we call "class separation" rather than the accuracy rate), but it will better predict how the taxonomyrs perform in the field – at least to the extent that your training set is similar to that of the real world.
Contrary to the hunch, the less divided taxonomics tend to perform better in K-Fold. It is not uncommon for training data to fall sharply in the K-fold, while the weaker taxonomics are able to maintain excellent generalizations.
Getting a high-level separation, but K-Fold is low? Your model has been overtrained and has a very poor ability to generalize, go back to the drawing board. Maybe choose a less aggressive machine learning model, or re-examine your identity choices. In reality, AI will do this automatically.
A good K-fold does not guarantee a deployable taxonomy. The only way to determine what you missed in the lab is to test in the field.
Comparison between laboratory experiments and real-world products
Laboratory experiments like the CMU study need not be too universal -- they are structured to illustrate a very specific point of view, to prove a concept, and then to move on. On the other hand, the real world’s products have to perform useful functions in a variety of unpredictable situations. For machine learning catalogues used in real world products, transversalization is essential.
But this is not the only thing. Deployment considerations are also important. It can operate in the clouds, or is it destined to operate in a processor, memory, and/or power-constrained environment? Those who gave CMU -- wish the ultra-frequency smart-watch battery continuity to an acceptable level! These issues must take into account the product design and the choice of machine learning models to support the product.
Tools such as Reality AI can help. Research and development engineers use Reality AI tools to create machine-based learning-based signal sorters and detectors for real-world products (including wearable equipment and machines) and can explore the links between sampling rates, computational strength and precision. They can train new models and run k-fold diagnostics (including others) to guard against over-training and generalization of predictability. When they are done, they can be deployed to the cloud or they can export codes that can be compiled for their specific embedded environment.
Research and development engineers who create real-world products do not aspire to a controlled environment -- over-training leads to product failure. Laboratory experiments do not face this reality. There is no reporter TechCrunch.
How to use machine learning to achieve success
In Reality AI, we see many machine learning projects that fail to produce results or are on the verge of derailing. Usually, our tools and structured approaches can help us, but sometimes not.
The following are 3 methods for successful machine learning.
Article 1: Obtain the ground truth.
The algorithm requires data and examples of what it is trying to detect, as well as of what it is not trying to detect, so that it can distinguish. This is particularly true for " supervised learning " algorithms, which must be trained in sufficient numbers to produce results. However, it also applies to " unsupervised learning " algorithms, which attempt to detect hidden relationships in the data without prior notice, as well as if there is no interest in them, no algorithm will find them.
Article 2: Plan the data.
The data should be clean and carefully planned. It means that, in order to achieve the best results, confidence in the quality of the data is essential. In the case of supervision of learning, misclassification in training data is particularly destructive -- some algorithms, such as our algorithms, can compensate for occasional misclassification in training data, but the prevailing problems are difficult to overcome.
Rule 3: Don’t overtrain.
Over-training means that machine learning models can predict training examples very accurately, but cannot be generalized into new data, leading to poor performance on the ground. Usually, this is due to the fact that data are too few or too single (i.e., they do not reflect the natural changes and mixing factors that occur in deployments), but it may also be due to inappropriate models.
Over-training is particularly harmful because it leads to false optimism and premature deployment, leading to a clear failure, which can easily be avoided. In Reality AI, our AI engineers monitor and inspect client models to prevent this unnecessary trap.
AI applications in machine health and preventive maintenance, for example
(To protect inexperienced persons, names and details have been changed.
For example, we recently had a customer who tried to set up a machine health monitoring system for a refrigerant compressor, which was installed in a system with very few leaks, and who tried to detect in advance whether the refrigerant in the pipeline had fallen to a level that put the compressor at risk – before it caused damage, overheated or closed through other mechanisms. They tried to achieve that goal through vibration data, using a small equipment with multi-axis accelerator sensors installed on the equipment.
Ideally, the client collects data using the same accelerometer under known conditions: many examples of compressors operating under a range of normal load conditions, as well as many examples of compressors operating under similar loads under unfavourable low refrigerant conditions. They then use our algorithms and tools, which contain a broad representation of the state of operation, including normal changes due to changes in load and uncontrollable environmental factors. It will also contain a range of different background noises and sufficient samples so that sensors and measuring noises are also well represented.
But they have only 10 seconds of normal compressors and 10 seconds of low refrigerants collected in the lab. This may be enough to allow engineers to begin to understand the difference between the two states -- and human engineers working in the lab may use his or her knowledge of the field of conditions in the field to begin to infer how to detect these differences in general. It is a perfect separation between training examples, showing a 100 per cent classification accuracy rate, but this result will never be extended to the real world. In order to consider all possible operational changes, the most reliable method is to include examples of all normal and abnormal conditions in the data, so that algorithms can learn by example and adjust themselves to the most robust decision-making criteria.
In reality, artificial intelligence tools automatically do this by using a variety of methods for characterization and model selection. To help detect and avoid over-training, our tools also test models with " K-fold technology " , a process that is repeated, but maintains a portion of the training data for testing. The K-Fold accuracy rate will almost never be as high as the training separation accuracy rate, but it is a better indicator of possible real world performance — at least to the extent that training data are representative.
Machine learning with embedded software is not as hard as you think.
Machine learning is difficult. Programming in embedded environments where processing cycles, memory and power are scarce is really difficult. Applying machine learning to embedded targets, um it's almost impossible, except for the simplest scene ... until now.
New modern tools, including those from Reality AI, are making it possible to use highly sophisticated machine learning models in embedded solutions, which is unprecedented. There are also important considerations.
Is the machine learning module thoroughly tested and validated?
For experienced embedded engineers, there is no doubt that the decision to submit any code modules to an embedded system product must be prudent and prudent. The solids that are pushed into the product must be solid and reliable. Usually, if it is feasible to update, the product will have a long life between the upgrades. And it is hoped that they will open the box and use it. In the case of a machine-learning sorter or detector module, the usual certification questions apply, but the attention we talk about elsewhere applies to spades. In particular, whether the system has been trained and tested for sufficiently large data in the real world to give confidence that it will maintain, unsupervised wildness.
What are the risks and trade-offs of wrong outcomes? If you know that the results of machine learning models are useful, but not perfect, can you educate clients to work within the known limits? Is safety in using examples essential and, if so, what fail-safe backups?
For example, our tools allow users to experiment, retrain, and test any amount of data before the sorter leaves a secure cloud. We even support " real-time " testing through cloud-end API, so that clients can safely test and validate the sorter or detector module before they invest their resources into solids and client equipment.
Will it translate to your embedded hardware?
Processing speed, memory, energy use, physical dimensions, costs, deployment times: all these are vital balancing elements in the design of embedded processing systems. The deployment of the sorter and the resources needed to carry out the operation -- usually within critical real-time delay limits -- will determine your design success or failure. Machine learning methods such as deep learning are very powerful, but only if you have space, power and budget in your design to support its intensive requirements. Other methods can be simplified at the time of implementation as operating sequences of standard signal processing or linear equation types, which, like ours, may not be common, but can operate comfortably in real time on cheap microcontrollers.
Do not forget mathematics: it is easy and natural to develop detectors and taxonomics on large desktops or cloud systems now using high-precision, floating-point tools. Ensuring dynamic scope and precision is sufficient to recreate similar performances on low depth, floating-point platforms, and will have significant limitations on embedding algorithms.
It was important to know what was needed, because it was pointless to spend expensive R & D on a project that could not be deployed in the targeted system.
Is there an early feedback mechanism?
Your detectors look good enough to be published, but is they perfect? The iterative generation is key to perfecting the AI cataloguer and detector. Can the designer have access to the real situation in the field? Can a pilot be established with one or more friendly clients, both to enable you to quantify the system in the real environment and, better yet, to obtain data and signals that could cause problems so that they can be incorporated into future training sets? Remember, most machine learning is data-driven. Although you have analysed the problem and synthesized a variety of test vectors in the lab, you will never cover all possible changes. I hope you will find the primary sorter or detector to be robust and able to do its job. But few good plans survive the first contact with the enemy, with data from even one or two real clients, and will soon introduce circumstances, noise, systemic errors and marginalities that are not fully planned.
How quickly can you respond with improvements?
Most companies that transport products with embedded software have a long and cumbersome (intentionally) review and quality control process to introduce new codes into production. You can't screw up, so certification, exercise and other process steps necessarily take time and money -- to ensure that only high-quality products are eliminated. This is, of course, the opposite of what you need for fast, iterative product improvements.