Attentional Factorisation Machine Model for Better Predictive Analytics
Opportunity
The massive market for Artificial intelligence (AI) continues to grow rapidly as more businesses turn to cognitive systems and AI software to create personalised customer experiences, supercharge sales and marketing efforts, and introduce automation in their manufacturing and supply chain processes. Global spending on these intelligent systems is forecast to reach US$77.6 billion by 2022 with an impressive CAGR of 37.3% between 2017-2022. This demand is driving the development of ever-smarter systems.
Factorisation machines (FM) belong to a new model class that combines the advantages of support vector machines (a popular predictor in machine learning and data mining) and factorisation models. Since its introduction in 2010, the FM model has been recognised as the most effective linear embedding method for sparse data prediction. It is, however, hindered by its same-weight modelling of all feature interactions even when not all feature interactions are equally useful and predictive. The modelling of interactions with useless features, in particular, can degrade overall performance.
Technology
This invention is a novel model named Attentional Factorisation Model (AFM). Designed as an improvement over the existing FM model, it is able to discriminate and automatically learn the importance of different feature interactions via a neural attention network without the need for any human domain knowledge.
AFM uses a new pair-wise interaction layer and an attention-aware pooling layer in neural network modelling. Its predictive analysis method involves receiving a set of predictor values comprising a plurality of features. Each feature is projected onto a dense vector representation to obtain a set of embedding vectors. From this, it calculates a set of interacted vectors with each interacted vector being an element-wise product of two embedding vectors. A weighted sum of the interacted vectors is performed using a plurality of attention scores that correspond to an interaction between a pair of features of the feature vector. This weighted sum is then projected to obtain a prediction score.
The neural network architecture of AFM
Download
