You Don't Even Get One Shot
Less than one-shot learning trains machine learning algorithms to recognize more objects than they have been shown.
The way in which people learn, and the way in which machines “learn” reveals a stark contrast between us and machines. After a person has seen an object — say a lion — then it is a simple task for that person to recognize other lions in the future. For a machine learning algorithm to do the same, particularly when accounting for differences in background, lighting, and pose, requires training with many thousands of example images. Training models on large data sets is laborious, expensive, and requires substantial computational resources.
A pair of researchers from the University of Waterloo are working to radically change the current paradigm in machine learning with what they call less than one-shot (LO-shot) learning. As the name implies, the method trains a model that can recognize more objects than the number of examples it was shown.
LO-shot learning was implemented by the team on the MNIST dataset of handwritten digits. Rather than take the approach of feeding the model many examples of each digit, they instead engineered a synthetic dataset based on MNIST. In their new dataset, they combined similar digits (consider the similarities between the digits “5” and “6,” for example) into single examples. These examples were given soft-labels to indicate their makeup — such as “40% probability of ‘5’, 60% probability of ‘6’.” Using only five examples, the researchers were able to train a neural network with greater than 90% accuracy at classifying the 10 digits.
While the level of accuracy achieved on such a small dataset is impressive, it must be noted that the process of generating a LO-shot dataset can be very complicated and laborious. And as the complexity of the machine learning algorithm grows, so too grow the complexities around creating the dataset.
The research is at an early stage and the team is still exploring methods to efficiently generate LO-shot datasets — whether that be algorithmically or manually. Until those details are further fleshed out, it will be difficult to evaluate if LO-shot learning will prove to be a significant advancement for the field, or more of a parlor trick in practice.
R&D, creativity, and building the next big thing you never knew you wanted are my specialties.