Recommendation Technology ‘Disco’

The choice of goods, services, entertainment options offered by the modern world can be overwhelming. Lots of time and effort can go into buying a vacuum cleaner, or picking a movie for the evening – a process that these days necessitates reading reviews, comparing descriptions and characteristics, asking friends for advice. Now there is not only a need to find a film, a product, or a musical track, but rather, to quickly and easily choose from millions of similar films, products, or musical tracks.

Personal recommendation systems have been created to make the selection process quicker and easier. Based on complex recommendation technologies, they aim to narrow down the selection to a smaller number of options that are likely to suit a particular person. To help people choose from large catalogues of music, videos, and products offered on its services, Yandex has developed such a technology of its own.

Before giving advice to anyone, it’s a good idea to find out what their tastes and preferences are. Yandex’s recommendation technology, Disco, uses several sources of information to find out the tastes and preferences of its users. Search queries can reveal a person’s current interests, while their gender, approximate age and occupation can be identified by Yandex’s behaviour analytics technology Crypta. This information lets Disco rule out suggestions that a person is unlikely to like – a delta blues song for a 15-year-old girl whose hobby is water aerobics, for instance. There, finally, is anonymised information that takes into account the individual user’s history on the service for which recommendations are being made. Yandex.Market, for instance, supplies information about the products that a person has been looking at, while Yandex.Music and Yandex.Radio provide information about which tracks a person has been listening to.

To fine-tune its recommendations, Disco relies on users’ signals about their preferences. Negative signals, such as skipping tracks or marking them with “I don’t like” on Yandex.Music or Yandex.Radio, mean that this kind of music shouldn’t be recommended to that user. Signals can also have different importance in making a decision whether to consider a certain type of content for future recommendations. A “like” rating and the fact that a user listened to a song from beginning to end are both positive signals, but a ‘like’ has greater weight.

Recommendation technologies, as of today, principally use three approaches to produce personally targeted recommendations. One of these approaches, content-based filtering, relies on information about items and their attributes or features – a musical track, for instance, would have the name of the album on which it was released, the artist, and genre associated with it, while a home appliance would have a list of technical characteristics and a product category. After these connections have been analysed, a user can be recommended something that is related to what they have already shown interest in. If a person often listens to progressive rock, then other tracks of this genre can be recommended. If someone recently bought a stove and a fridge by the one manufacturer and is now looking at microwaves, then other appliances by the same manufacturer might probably be of interest.

Another approach, user-based filtering, works on the tested assumption that people who share similar characteristics are often interested in similar things. Thanks to Yandex’s own behaviour analytics technology Crypta, the gender, approximate age and likely sphere of occupation of each user can be identified. Even if a person has not yet looked at, listened to or chosen anything on a service and is using it for the first time, it’s possible to offer them recommendations based on what other people with similar characteristics have been looking at, listening to or buying on this service.

The third approach to making recommendations, called collaborating filtering, uses data about users’ interactions with items, such as viewings of a video or giving “likes” to a musical track. This approach is powered by an algorithm, known as SVD, or singular value decomposition. Based on the information about the user’s known interactions, this algorithm allows predicting how they will react to other objects, for example, what rating they will give a film that’s yet to be watched.

Each approach has its advantages. The first approach allows making recommendations of rare items that few people are interested in, such as a little-known musical group. The approach considering connections between people makes it possible to offer recommendations to people who are using a service for the first time and are yet to perform any actions. Collaborative filtering with SVD can identify unexpected patterns: it might discover, for instance, that people who have an interest in inflatable swimming pools and fitness trackers buy coffee machines more frequently than anybody else.

To provide millions of users with personally targeted recommendations, Disco builds hundreds of recommendation models based on these three approaches. Each model considers specific information, such as a musical genre or a product brand, and processes a whole range of parameters to produce a list of recommendations.

To provide a user with recommendations that ideally match their interests at that particular time, recommendations from various models are processed by MatrixNet machine learning technology, which considers a plethora of factors, such as variety or popularity. More varied music can be recommended to users whose playlists include tracks of different genres and by different artists, unlike those who tend to play a single album from beginning to end. Alternatively, some people like music that’s getting airplay on radio stations, while others prefer tracks known only to a small circle of music lovers.

A final list of recommendations that a user sees in a service – on the main page of Yandex.Music or in the popular products section on Yandex.Market – is the result of MatrixNet’s work.