Since we started our search engine in 1997, we’ve sought to connect people with the information most relevant to them. Over the years, we’ve updated Yandex Search with smart features and more informative results, making it as easy as possible for our users to navigate the mass of information that is the internet. Today, we’re excited to announce the latest advancements for Russia’s largest search engine. The Vega update brings 1,500 improvements to Yandex Search that help our 50 million daily search users in Russia find the best solutions to their queries. The most significant improvements from Vega pair machine learning with human knowledge and double our search index to deliver users better quality results at lightning speed, and we're excited to share more on the major updates with today's announcement.
With Vega, our search team has also introduced a uniquely human element to our users' search experience. We've updated the ranking algorithm with neural networks trained on data provided by real experts in several fields, providing users with even higher quality solutions to their searches. We're also connecting people with answers to their searches from qualified experts, through our new question-and-answer service, Yandex.Q.
The Vega updates not only make Yandex Search smarter, but also enable it to search across more of the web. Our search index is now twice as large thanks to a clever method for grouping similar web pages into “clusters.” We’re also now presenting these smarter results with as few clicks as possible thanks to improvements that will help bring users information faster, such as pre-rendering technology for mobile users.
"Our new search update combines our latest technologies with human knowledge,” says Andrey Styskin, Head of Yandex Search. “At Yandex, it's our goal to help consumers and businesses better navigate the online and offline world. With this new search update, users across the RuNet are helping us do just that. By contributing their knowledge, experts are enhancing our algorithms and helping our Search users, who continue to grow; over the past year, Yandex’s search share on Android in Russia rose 4.8% to 54.7% in early December.”
Machine learning with a personal touch
We’ve used machine learning in our search engine for a decade now, with this year marking our tenth year since adding our MatrixNet algorithm to Yandex Search. Machine learning remains a critical part of our search engine, but having the human element is vital, too; after all, who better than ourselves to understand what we’re looking for on the internet? People, or “assessors,” have long helped train our machine learning platforms through our crowdsourcing platform, Yandex.Toloka. Using our search result evaluation guidelines, the assessors in Yandex.Toloka complete tasks that help us find the most relevant results for specific queries.
With Vega, we've updated our ranking algorithm with a system where specialists in their respective fields now evaluate the work of these assessors. The professionals appraising the assessors range from IT administrators for data queries to hydrologists for searches relating to rivers. The expert assessors use over a hundred criteria to evaluate the work of the assessors, and our deep learning neural networks then receive these expert evaluations. By training our machine learning algorithms with expert assessments, our search engine learns to rank relevant information higher in results thanks to the work of a highly qualified group of individuals.
In addition to improving the quality of our search results with expert crowdsourcing, we’re also integrating answers to queries from real experts right within our results page. Yandex.Q is a service that combines our Yandex.Experts tool, introduced in last year’s Andromeda update, with content from the popular Russian Q&A service TheQuestion, which we acquired earlier this year. After a seven-month testing period, we’re now rolling Yandex.Q out to all of our search users. Yandex.Q features over a million answers from experts across many fields. Users can ask their questions in the Yandex.ru search field, and expert responses will appear at the top of the results page. For example, someone looking for information on Alexander Pushkin can see answers from a literary critic, or a search on the behaviors of seals will result in a response from the head of the National Arctic and Antarctic Museum.
Improving our search index with clusters
Each of our updates provides our search engine with smart tools for understanding complex queries and finding relevant results. With our 2016 Palekh update, Yandex Search began using neural networks to better process long-tail keywords, which are highly specific, multi-word searches. Our Korolyov update added the ability to search entire web pages rather than just titles, and could better understand the intent of queries. Last year’s update, Andromeda, enhanced the results page with tools to gauge the quality of a result and “quick answers,” which presents answers from experts on various topics.
Vega builds on our previous updates with intelligent improvements in how our search engine processes queries. Our algorithms use neural networks to now group pages into clusters based on their similarity. When a user enters a query, it's searched among the most relevant cluster of pages, rather than our entire index. Since using the clusters frees up more computing power for our search engine, we've doubled the size of our index without slowing the speed of searches. We can also better connect users with results for infrequent queries, as our search algorithms find rare web pages for the clusters along with highly visited sites.
Based on our search quality metric, which considers, among other factors, document relevance and source reliability, Vega's performance is three times better than Korolyov and six times better than Palekh.
Smarter results at lightning speed
The updates in Vega not only process search queries faster, but we’re also delivering people information quicker, with as few clicks as possible. Since March, Yandex mobile users on Android have been searching with pre-rendering technology, which predicts the user’s query and selects relevant results as the user is typing.
Thanks to this feature, now millions of users can instantly find the information they’re looking for on their mobile devices. The number of queries with an instant result has grown over 20% over the last year, and we’re planning to rollout pre-rending to more of our mobile users soon.
We’ve also made our Turbo technology universally accessible so that content providers can create fast-loading versions of their sites. Turbo-powered websites are light versions of full websites, which load about 15 times faster than regular websites. Turbo sites help us connect people with information regardless of how they access the web, as the quick-loading sites are especially useful for slow connections and mobile sites. As content providers expand Turbo to more of our search results, our users see relevant solutions for their queries quicker than ever. Users are now loading 75% of the lightweight Turbo sites in less than a second.
Our Vega update continues to make Yandex Search the smartest and quickest way for Russian internet users to connect with the most relevant information on the web. By pairing our machine learning expertise with the knowledge of real experts, our search engine brings together the best of artificial and human intelligence to connect people with the most relevant information. Give some of our best search features a try today at Yandex.ru!