Yandex Blog

Our Latest Intelligent Search Update, "Andromeda"

Today, Yandex announced the latest update for its search engine.  The “Andromeda” update has been rolled out by the Yandex search team over the last year and has included over one thousand improvements and features.  The most notable of these features are:

  • Enhancing “quick answers” functionality in search results, including the introduction of Experts to help users get better answers to their queries.
  • Providing users with an easy way to recognize the most accurate and relevant sites directly within the search results.
  • Giving users a way to save searches in a visual format with the introduction of Yandex.Collections.

Andromeda builds on the prior Korolyov and Palekh updates, which improved users' search experience by better understanding complex, informal inquiries and intent.  This year’s update continues to use Yandex’s machine learning expertise to create more intuitive ways for users to find information.

"Since the previous search update, Korolyov, our share of search has been constantly growing - on desktop it increased by 2.5%, and on mobile it increased by 6%.  The quality of our search is one of the key factors for the growth of our market share.  And today we present a new large-scale update, which includes more than a thousand improvements.  Our search team has worked on them for more than a year,” said Andrey Styskin, head of Yandex.Search.

With quick answers, users receive results for their queries in a straightforward manner.  Quick answers have been available for some time for simple searches, such as checking when a particular holiday falls.  Recently, Yandex has improved the quick answers functionality to allow more complex searches, for example showing the Spanish soccer schedule and related media when “Spanish championships” is searched.  A search for “cafe” demonstrates the full capability of these recent enhancements, as users can browse and compare local cafes by price, reviews, and photos without ever leaving the Yandex search results.

The new Experts feature enhances quick answers by providing users with the most accurate and relevant information directly within the search results.  Actual human experts work with Yandex to offer advice and solutions on various subjects. Users can enter a query like “why don’t electrons fall into the nucleus of an atom,” and if they don’t see the answer in the results, they can directly ask a physics expert.  Users can see the available categories of all Experts questions and answers in one place on the Yandex.Experts page (Russian).

In the search below for “how to tell a cat he is wrong," an Expert provides the user an answer in a box to the right of the results.

Aside from improving the search results through quick answers and Experts, Yandex is also helping users ensure the information they see is from reliable sources.  Yandex’s search engineers have been continuously improving search results to ensure that the highest quality sources are presented first. The sites where users found what they were looking for, made repeat visits, and those without distracting ads are more likely to feature at the top of results.  

Part of improving the quality of search results includes new visual features to help users identify the most accurate sites directly within the results.  Unique algorithms determine how to allocate badges to high quality sites in search results.  Checkmarks are shown for Yandex websites and the sites of organizations and social media profiles verified by Yandex.  Large and popular sites will feature an icon of a green flame with their result, while those with high user engagement and repeat visits have a green crown.

The introduction of Yandex.Collections (Russian) is another major part of improving the search experience with the Andromeda update.  Collections gives users a visual way to save their searches and their favorite content.  Items can be saved to a Collection directly from the search results, whether they be links, images, movies, or locations.  Collections can be accessed on desktop and mobile devices so users can see their favorite content wherever they are.  Users can also follow public Collections; for example, if one is in the market for a new lamp, they can see the public Collections that other users have created dedicated to lamps, as shown below.  These public Collections will be available directly from the Yandex search results.

Yandex is excited to continue to enhance the user search experience and demonstrate its machine learning expertise with the Andromeda update.  Give some of the new features a try today!

New Intelligent Search Algorithm “Korolyov” 

 

Today, Yandex released a new version of its search platform, incorporating two important elements:

·      An upgraded version of a deep neural network based search algorithm called “Korolyov,” named after a town northeast of Moscow that has long served as the center of Russia’s space exploration program.

·     Incorporating Yandex.Toloka, a mass-scale crowd-sourced platform for search assessors into  Yandex MatrixNet.

“Korolyov” builds on “Palekh,”  Yandex’s first neural network based search algorithm released in late 2016.  The update improves how Yandex handles infrequent and complex queries, known as long-tail queries, in two distinct ways. 

First, “Korolyov” is better at understanding user intent than its predecessor because it examines the entirety of web pages rather than just their headlines. Second, “Korolyov” can scale to analyze a thousand times more documents in real time than “Palekh.”

Like all modern AI-based systems, “Korolyov” improves itself with each incremental data point. Yandex’s position as the largest search engine in Russia creates a positive feedback loop for our deep neural network algorithm, which leads to superior search results for our users.  

"Korolyov" results feed into MatrixNet, Yandex’s proprietary machine learning ranking algorithm, where a number of other ranking factors are considered before results are returned to a user.  

Recently, MatrixNet started incorporating data from Yandex.Toloka, Yandex's crowdsourcing platform, in addition to anonymized user data to train the machine learning algorithms. 

These updates help us to enhance the quality of our search services for our users.  To learn more about “Korolyov,” you can watch the event broadcast in Russian only

Yandex Rolls Out Multifunctional App for Android Users in Russia

With more than 85% of the smartphone market in Russia, Android continues to be the country’s favourite operating system. The great variety of devices within a $200 price range supporting this platform, among other things, has contributed to Android’s popularity with Russians who are happy to compromise some of their smartphone’s technical quality, such as memory space, for its affordability. ‘There’s an app for that’ doesn’t really do it for a lot of budget smartphone users.

To make life easier for the owners of budget Android-based devices we have offered them an all-in-one application for their everyday needs – from current weather, currency exchange rates or traffic conditions to what’s on in the cinema around the corner or the shortest way to the nearest bank or restaurant. We have reissued our search app for Android to provide our users with a one-tap access to our key services on their mobile devices.

The new Yandex app has expanded its functions beyond search to include quick access to email, news, maps, city navigation, taxi booking, or any other service available in Yandex’s product range. To use any of these services one doesn’t even need to have a corresponding app on their phone – the refurbished search app will take them to the mobile version of the service at Yandex.ru. The new Yandex app allows the owners of low-budget Android-based smartphones to enjoy the full mobile experience without having to compromise anything.

First announced in 2011, the Yandex search app now has a weekly user audience of over three million. The new app can be downloaded from Google Play. The users of Yandex.Search in Russia, Ukraine, Belarus and Kazakhstan will be upgraded to the new Yandex app when they update their current version.

Forgetting the right to search

The State Duma Committee on Information Policy and Communications has discussed a bill that requires search engine operators to delete hyperlinks to illegal or unreliable information, or even reliable information that refers to events that happened three years ago or more, from their search results on requests from individuals and without a court order.

Internet search is our core business. In more than 15 years in this market, we have put colossal human and financial investments into our search engine, first and foremost, to offer our users search results that are complete, unbiased and useful. If this bill is passed in its current form, a search engine based on these principles will be difficult or even impossible to develop. That is why we feel it is important for us to offer commentary on this bill.

According to its authors, this bill enables any individual to control distribution of unreliable or outdated personal information on the internet. In principle, this gives people a right, which is based on one of the most basic human rights – the right to privacy, including the right to control access to information about oneself. Unfortunately, the procedure offered in this bill does not stop information from being distributed online, but contradicts the basic principles of law and current legislation.

The current law does not permit limiting a person's right to access reliable information. The Constitution of the Russian Federation guarantees everyone the right to freely seek, obtain, transfer, produce and disseminate information by any lawful means (Article 29). The Federal Act ‘On Information, Information Technologies, and Information Protection’ also stipulates an individual’s or organization’s right to search and obtain any information in any form from any source (Article 8). This is exactly what a search engine does – searches for information available through any public source. This bill ignores the right to search for information.

The limitations introduced by this bill reflect imbalance between private and public interests. The need to seek and obtain information often falls within public interest and concerns public figures, whose actions can have an impact on the general public or private lives. This bill impedes people's access to important and reliable information, or makes it impossible to obtain such information. If this bill is passed, the information about a clinic or a doctor, a school or a teacher one is considering to choose, may be impossible to find.

In addition, the procedure for requesting a search engine to remove hyperlinks introduced in this bill opens the door to numerous opportunities for misuse, as it doesn't require any evidence or justification. A search engine, on the other hand, is required to delete an undefined number or hyperlinks to indeterminate web pages. This loophole can very conveniently be used by unscrupulous businesses to undermine their rivals, or by criminals to facilitate fraud.

But even if we assume that it is possible to equal adequate information with inadequate or illegal information in the right to be searched for, one question remains: who will study the information which is searched for, and decide whether it is legal, adequate, relevant or reliable? The bill assigns this role to search engines, while the functions of the court or law enforcement agencies are given to individual commercial organizations. Failure to comply with this role is punished with penalties and litigations.

This bill also ignores the basic principles of information technology and information search. It gives any person the right to request a search engine operator to stop providing hyperlinks to web pages that contain specific information, but it does not require this person to say which hyperlinks should be removed. All they have to do is provide the information, hyperlinks to which they want to be removed. Instead of deleting hyperlinks to specific web pages from search results, a search engine is expected to stop retrieving a piece of information on any search terms and regardless of its location on the internet. For this to become plausible, a search engine operator would have to find all pages containing this information that might appear in any place in search results triggered by any search term that a human mind can come up with. This step alone would take eternity. The next steps would require a search engine operator to make sure that these pages do contain the information hyperlinks to which were requested to be removed, and then confirm that this information is indeed inadequate or older than three years old. It is obvious that this is an impossible task.

Even though the list of flaws of this bill can go further, it doesn't make sense discussing them all at a point when the stipulated procedure itself contradicts the law and is technically impossible.

The current bill is much less well thought through than the Google Spain v ARPD, González (C-131/12) decision by the Court of Justice of the European Union, which has been widely criticized, and which the Russian bill has often been likened to.

The links to be removed from search results mandated by the ruling of the Court of Justice of the European Union are specific, lead to specific information and appear on a narrow class of search terms. Hyperlink erasure is also considered on a case-by-case basis to make sure it does not limit access to important information or alter the balance between private and public interests.

Nokia’s Android Phones Come to Russia Fitted with Yandex Apps

Nokia’s Android smartphone, Nokia X, announced at the recent Mobile World Congress, comes to the Russian and Belarusian markets equipped with the essential search functionality provided by Yandex. The Finnish company’s first in the X-series of phones premieres today on these markets with the Yandex.Search app already preinstalled and the Yandex search engine as a built-in search provider in the phone’s browser.

Buyers of Nokia’s new product in these countries will be able to funnel mobile apps for all their needs through our alternative Android app store, Yandex.Store, which is now featured in Nokia Store. Nokia’s range of Yandex-enhanced Nokia X smartphones will continue with Nokia X+ and XL with Yandex as a default search provider and Yandex.Store preinstalled.

Nokia’s smartphones have always been popular in Russia and the CIS, not in the least, because of the company’s ability to cater for the specific needs of their customers in each of the markets. It’s only natural for Yandex, who always placed a top priority on relevance of service to a specific user in a specific location, and Nokia to join efforts to make sure the user experience of their customers is locally-relevant, smooth and seamless, even when they go from desktop to mobile, or from one mobile device or OS to another. Nokia’s decision to choose Yandex’s products for their new phones in Russia and Belarus ensures comfortable user experience for their customers in each of these countries, who already know Yandex and its products very well.

We’ve been partnering with Nokia since 2011, when Yandex became a default search provider on Nokia’s Lumia 800 and Lumia 710 distributed in Russia. We’re now happy to have rekindled our old relationship through our support of the alternative Android ecosystem. The love of freedom of choice is what brings Yandex and Nokia together in this partnership.

Facebook ‘Firehose’ Comes to Yandex

One of the biggest changes to have taken place in the internet recently is, without doubt, the rise of social networks. They’ve become so popular that it’s now difficult to imagine the internet without them. Millions of people turn to online social networks every day to catch up with friends and family, share news and opinions, or just have a laugh. Like it or not, they’ve become a part of daily life.

It follows that information about the hot topics in social networks is an important factor for a search engine in answering users’ questions. The intensity of discussion on any subject in social media is proof of the topic’s relevance, or “hotness” if you will. A search engine has to take this into consideration.

Here at Yandex, we’ve always said that our specialisation is information-search services, aggregation and the structuring of content. We don’t compete with anybody in the sphere of social networks; instead we seek to collaborate with all the players. We see one of our key tasks as being the creation of social search services, using content from all the popular social networks in equal measure. We want to provide users with the possibility to receive answers that take into account the information that can be gathered from these resources. This would allow a user to find an old friend without having to register on every single social network one after another. It would also allow a user to tap in to all the discussions of some interesting event all together in one place.

We’re already working with Twitter in such a way, and we index status updates in LiveJournal, VK and others.

Today we’re announcing another important step in this direction: Facebook has granted us full access to its “firehose” of public data. This means that now, not only can Yandex search for people and company pages on Facebook, it can also search for content marked “Public” from users in Russia, Ukraine, Belarus, Kazakhstan and Turkey. Of course, anything users mark as “Private” will remain off-limits.

At present, Facebook posts from users in Russia, Ukraine, Belarus and Kazakhstan pop up in search results only on the Blogs part of Yandex Search, but soon they will be added to the service’s main Search page, giving users even fresher answers to their questions about recent and current events. Along with answers to such queries, Yandex will add up-to-date articles and videos, among other things that have had great resonance among Facebook users. In addition, the popularity of materials in the social network will be taken into consideration when ranking search results.

Well, search has been personalised already. How about the rest of the internet?

At Yandex we’ve long been striving to tailor search results especially for every individual user – and we can already do it pretty well.

Our Personalised Search fetches results and delivers search suggestions individually for each user based on the many things we know about them – including their geographical location, language preferences, search history and clicks in search results. The user's search history tells the search engine what may be currently relevant for this particular user, and whether he or she would appreciate getting search results in English, for instance. Our MatrixNet machine-learning algorithms allow our search engine to look at users as live, multi-faceted human beings: gender, age, sphere of activity and domestic status are just some of the qualities it knows how to consider when delivering personalised search results and suggestions.

Well, naturally we couldn’t stop there, and we started thinking about how to take this great idea one step further. Once we’d developed personalised search, another idea arose: if we can personalise search results, why not personalise the whole internet? Introducing …. (drum roll, please) …. Atom!

Atom is one of Yandex’s new technology concepts. It allows any web resource to be adapted (or personalised) for nearly any person, even if they have not visited that web resource before but have a search history at Yandex.

For example, a site selling package tours is more likely to satisfy a user (and make a sale) if its main page only shows those tours that are likely to be of most interest to that user, based on his or her past behavior online. If a site can work out how to reconfigure its front page or catalogue according to the interests of any given person – and deliver what’s needed right at the start -- then it follows that the person will return to the site again and again.

How does it all work? We “talk” to a site through an API, telling the site what to show, in what order, in what priority, for each individual. We’d like to emphasise that we don’t give third-party sites any private information about the user – none of their cookies, nor their search history. We process all that information ourselves.

At present Atom exists on the level of a concept that we will be developing over the next few years together with the internet community. It’s an ambitious plan, which will work only if it gets the support of everybody – users, web site owners, web masters. 

And who wins? First of all – users, who will get only relevant and useful information on their PC or tablet or smartphone screen. Imagine a newswire website where all new items are interesting for everybody. Nothing to be left unread. Or an e-commerce service delivering not only recommendations based on their own statistics, but considering much more extensive behaviour of a user in the internet. A personal internet – for each, their own – is coming. That would be the huge shift in upcoming years or even decades. Stay tuned!

Yandex Takes Leap With Its New Interactive Search Platform - First In Turkey

Yandex has always been there to give answers. But then there comes a moment when giving answers becomes ‘not enough’. A large portion of the five billion searches that come from almost 100 million Yandex users each month are looking for a solution rather than an answer – people want to pay a bill, check-in for a flight, book a show or a visit to the dentist.

Using all our knowledge and expertise, all these years we have been developing products and technologies to figure out what it is exactly that our visitors are looking for and offer this to them in our search results. But even the most relevant of search results doesn’t always give a solution. Even if you can see in search results a link to the website where you can potentially book your movie ticket, you still need to click through to this site and possibly even browse a few pages before you find the booking form.

We have been thinking about offering web users a shortcut precisely to what they need, and we came up with a new concept of a goal-oriented search platform. 

Yandex's new search platform embodies our new approach to search and website engagement, and features an attention-based search results page, where each search result is a standalone block of interactive information – an Island. These blocks are the first step to the user’s search goal and can be anything from factual information to purchase buttons or order forms.


While web users can instantly see and choose the best and most relevant solution to their problem, Islands enable website owners to directly connect with their visitors and enjoy handpicked target audience. Any website would like to see an interested visitor and about 15-20% websites on the Russian internet are using interaction to improve the quality of website experience for their visitors. Our new search platform embraces website owners catering for the interests of internet users. We are offering them an opportunity to choose what information from their site should be featured on Yandex’s search results page. Through our Yandex.Webmaster service, using a free and easy toolset, website owners can markup their pages and see what their Island will look like on Yandex’ search results page right now. Or see whether they need an Island of their own at all.

Right now, Islands are triggered by the same search algorithms as regular snippets, but we are working on giving websites an opportunity to tune their interactive snippets to specific search parameters, such as search terms, user’s search history, their location, and even their device. Prospectively, Yandex users looking for ‘The Great Gatsby’ on their personal computer will see a ‘film review’ Island, those searching for the same on their mobile device will find an Island with the show times at the closest cinema and a booking form, while an Internet-connected TV viewer will receive an Island with a pay-per-view page to purchase this film. This is the next step in Yandex Islands’ future development.

Yandex’s ranking is not affected by Islands, as such. A website’s natural position, however, does depend on the quality of the user experience it provides. The better is the website, the more popular it is with the users, the higher is the ranking. It is quite possible that web users may like a website more because it has an interactive snippet, but the mere fact of having such a snippet won’t propel a website to the top of the search results page. 

By introducing the goal-oriented search, Yandex facilitates interaction between web users and websites. Our mission is to offer solutions to people’s problems by openly sourcing the best of the web and delivering it to web users. We would like to attract the actual content generators and publishers and see them actively participate in this process.

We are rolling out our new experimental search platform, Islands, first, for the eight million of Yandex users in Turkey. Yandex Islands is now publicly available at yandex.com.tr on personal computers and tablets in this country. The mobile version is expected to see the light later this summer. This is the first time we are launching a new search platform on a relatively new market, without first trying it out at home.

The Turkish market is not only new, it also has a young and open-minded demographic, which, we believe, is hard to scare by innovation. The new search platform’s success is largely in the hands of website owners and web publishers. We will focus on educating them about the benefits and possibilities of Yandex Islands in a series of free workshops, which will be open for all and where any website owner can learn how to make an Island of their own. Detailed instructions for creating interactive Yandex snippets for specific website categories, such as hotels, restaurants, airlines, or car services, will soon appear on the Yandex Islands specification pages

Our users in Russia, Ukraine, Belarus and Kazakhstan will see their own Yandex Islands some time later this year.

It May Get Really Personal – we have rolled out our second-generation personalised search program

We all know what it takes to understand another person. It’s a lot. Even if all you need to understand is what a person is looking for online. We have been trying to do this for years. A person’s interests and preferences give a good clue as to what they want to find. We used to look into a user’s search history as far as a few months back to choose for them the search results that would be most relevant to their scope of interests.

Now, we have added to our search algorithm a search history of a few seconds – searches within the current search session. We can now deliver results and search suggestions based on the ‘full picture’ of the user’s search behaviour.


Updating our knowledge of users’ interests once a day allows us to understand their more-or-less stable interests, such as a love of books or football, or that they speak Russian and live in Saint Petersburg. More than half of all searches on Yandex, however, are about something that interests the searcher at the very moment of searching and stops interest them the moment after. To be able to cater to such momentary searches, we now analyse search sessions in real time.

Search queries begin to influence search results within seconds. The search engine can figure out whether a person is looking for a book or a film even before they have finished typing [The Great Gatsby] in the search box.

 

 

 

With the new personalised search program, we can offer relevant results even to those web users who don’t have any search history on Yandex. To instantaneously react to changes in users’ search behavior, we created a real-time data processing system, which processes more than 10 terabytes of data a day, continuously correcting its knowledge of users’ needs.

To make search results as personal as they can be, we first learnt to take into account users’ language preferences and permanent interests. Now, we have learnt how to tell current search intents of our users from their search footprint and give them what they are looking for. Personally.

Search results based on current search interests are available on all Yandex domains globally for all 93 millions of our users. Personalisation that takes into account long-term or medium-term interests works best for searches in Russian, but, just like all previous editions of Yandex’s Personalised Search, the latest version will learn to fetch personlised search results in other languages as well as it receives more search queries in these languages. 

Read more about our Personalised Search program in Products and Technologies.

Yandex roots out malware on the router level with Yandex.DNS

Those of our users who take their internet safety for granted, can keep doing just that with our new internet security solution Yandex.DNS. The Domain Name System (DNS) structures information on the world’s interconnected computers by associating it with names – conventional webpage addresses for users and sequences of digits for computer programs – and continually running and updating a ‘catalogue’ of these names. This structure allows people to access webpages through their browsers. By having one’s own, customised DNS, a web user can choose which webpages can be in the scope of their browsing. With three security-level modes to choose from, Yandex.DNS installed on a personal computer, mobile phone or router lets users customise internet experience for each member of their family on each internet-connected device in their household. 

  • Yandex.DNS Standard makes sure the user has a fast and unfaltering access to webpages each time they go online 
  • Yandex.DNS Secure filters out infected and phishing websites 
  • Yandex.DNS Family blocks adult content websites in addition to infected and phishing websites

To start receiving Yandex.DNS protection on their personal computers or mobile devices, web users in Russia, Ukraine, Kazakhstan and Belarus can follow easy and simple step-by-step instructions on the service’s website (available in Russian). When installing Yandex.DNS at the router level, the Secure mode is offered by default, with an opportunity to choose the Family or Standard security levels. One mode can be selected for all devices in the user’s network or each device can have its own security level – the Family adult content filter for kids’ mobiles, the Secure mode for dad’s iPad and zero protection for the piggybacking neighbour.

Owners of Zyxel's Keenetic series home routers can download Yandex.DNS to their devices right now. D-Link's DIR-615/K2 and DIR-620 routers are currently in the pipeline with other router manufacturers expressing an interest in giving their customers an opportunity to install Yandex.DNS and protect their networks. To take the installation pain off our users’ shoulders, we are working on getting Yandex.DNS preinstalled as firmware on home routers shipped in Russia and neighbouring markets.

Yandex has long been waging war on internet threats, developing a whole range of products and solutions to help web users have a safe internet experience. Our very own, behaviour-based anti-virus technology detects malware on 4,300 websites and helps us show to our users about 8,000,000 malware warnings every day and protect them from malware that spreads via Java applets. We use our own adult content filter in our search results. Integrating our anti-virus technologies, Yandex.Browser provides safe browsing for internet users, with Opera and Firefox also enjoying our malware protection via the Safe Browsing API. We can also notify website owners about malware threats on their pages via our Yandex.Webmaster service and help them fix the problem.

Yandex.DNS incorporates all of our malware protection experience. It shares database of malicious and adult content websites with Yandex Search and Yandex.Browser.