Yandex Blog

Forgetting the right to search

The State Duma Committee on Information Policy and Communications has discussed a bill that requires search engine operators to delete hyperlinks to illegal or unreliable information, or even reliable information that refers to events that happened three years ago or more, from their search results on requests from individuals and without a court order.

Internet search is our core business. In more than 15 years in this market, we have put colossal human and financial investments into our search engine, first and foremost, to offer our users search results that are complete, unbiased and useful. If this bill is passed in its current form, a search engine based on these principles will be difficult or even impossible to develop. That is why we feel it is important for us to offer commentary on this bill.

According to its authors, this bill enables any individual to control distribution of unreliable or outdated personal information on the internet. In principle, this gives people a right, which is based on one of the most basic human rights – the right to privacy, including the right to control access to information about oneself. Unfortunately, the procedure offered in this bill does not stop information from being distributed online, but contradicts the basic principles of law and current legislation.

The current law does not permit limiting a person's right to access reliable information. The Constitution of the Russian Federation guarantees everyone the right to freely seek, obtain, transfer, produce and disseminate information by any lawful means (Article 29). The Federal Act ‘On Information, Information Technologies, and Information Protection’ also stipulates an individual’s or organization’s right to search and obtain any information in any form from any source (Article 8). This is exactly what a search engine does – searches for information available through any public source. This bill ignores the right to search for information.

The limitations introduced by this bill reflect imbalance between private and public interests. The need to seek and obtain information often falls within public interest and concerns public figures, whose actions can have an impact on the general public or private lives. This bill impedes people's access to important and reliable information, or makes it impossible to obtain such information. If this bill is passed, the information about a clinic or a doctor, a school or a teacher one is considering to choose, may be impossible to find.

In addition, the procedure for requesting a search engine to remove hyperlinks introduced in this bill opens the door to numerous opportunities for misuse, as it doesn't require any evidence or justification. A search engine, on the other hand, is required to delete an undefined number or hyperlinks to indeterminate web pages. This loophole can very conveniently be used by unscrupulous businesses to undermine their rivals, or by criminals to facilitate fraud.

But even if we assume that it is possible to equal adequate information with inadequate or illegal information in the right to be searched for, one question remains: who will study the information which is searched for, and decide whether it is legal, adequate, relevant or reliable? The bill assigns this role to search engines, while the functions of the court or law enforcement agencies are given to individual commercial organizations. Failure to comply with this role is punished with penalties and litigations.

This bill also ignores the basic principles of information technology and information search. It gives any person the right to request a search engine operator to stop providing hyperlinks to web pages that contain specific information, but it does not require this person to say which hyperlinks should be removed. All they have to do is provide the information, hyperlinks to which they want to be removed. Instead of deleting hyperlinks to specific web pages from search results, a search engine is expected to stop retrieving a piece of information on any search terms and regardless of its location on the internet. For this to become plausible, a search engine operator would have to find all pages containing this information that might appear in any place in search results triggered by any search term that a human mind can come up with. This step alone would take eternity. The next steps would require a search engine operator to make sure that these pages do contain the information hyperlinks to which were requested to be removed, and then confirm that this information is indeed inadequate or older than three years old. It is obvious that this is an impossible task.

Even though the list of flaws of this bill can go further, it doesn't make sense discussing them all at a point when the stipulated procedure itself contradicts the law and is technically impossible.

The current bill is much less well thought through than the Google Spain v ARPD, González (C-131/12) decision by the Court of Justice of the European Union, which has been widely criticized, and which the Russian bill has often been likened to.

The links to be removed from search results mandated by the ruling of the Court of Justice of the European Union are specific, lead to specific information and appear on a narrow class of search terms. Hyperlink erasure is also considered on a case-by-case basis to make sure it does not limit access to important information or alter the balance between private and public interests.

New Concept Yandex.Browser Boosts Privacy and Launches as Beta

The minimalist concept version of our Yandex.Browser launched at the end of last year to respond to the highly interactive nature of contemporary web browsing is now available as a beta version, which is designed to also address the rising demand for personal privacy.

To meet the expectations of those users who would like to have more control over their digital footprint, we’re now rolling out a much more private beta version of the experimental Yandex.Browser, available in 15 languages, including English, German, Portuguese, Spanish and French.

Unlike in most browsers, sending the information about users’ behaviour to the developer (i.e. Yandex) in the private version of Yandex.Browser is disabled by default. While sharing browsing history and web cache can in principle be disabled in other browsers, this opportunity isn’t normally offered as a default option. Sharing users’ information helps developers better understand their behaviour and offer them a better browsing experience. The problem is that the right to make a decision whether to share this information is effectively removed from the user – few can find a pathway to customised privacy settings in a browser.

In addition to data sharing disabled by default, Yandex.Browser provides the ‘Stealth Mode’ option, which blocks analytics cookies, sharing plugins etc. This mode is activated by clicking a button conveniently located right next to the browser’s ‘smartbox’, a combined search and address bar, at the top of the screen. The source code of the built-in blocking extension was developed by AdGuard and is available on Github for anyone to see.

 

Safe browsing, as well as search suggestions appearing in the browser’s smartbox, is the feature indispensible for contemporary browsing that relies on sharing user information, albeit in an anonymised form. The safe browsing technology allows us to warn the user about unsafe websites. Each fraudulent or potentially harmful website that we identify in the process of indexing more than 30 million webpages every day is logged in our proprietary cloud database. Every time a user is about to visit a website, the website’s address is automatically checked against this database to see if it might be there and whether a warning should be shown.

We have modified the safe browsing technology to use it in our privacy-conscious version of Yandex.Browser. Instead of sending the full address of a website the user is about to visit to Yandex in order to check this address against the database of potentially harmful websites, Yandex.Browser only uses a fraction of a ‘hash’ of this website, which is checked against a ‘hashed’ database of potentially harmful websites on the user’s own computer. The browser uploads a ‘shell’ of this database to the user’s computer at the first launch. This database is then ready to be ‘filled’ with fractions of ‘hashed’ website addresses the user intends to visit. To keep this database updated and the user safe, Yandex.Browser synchs the database on the user’s computer with Yandex’s cloud database every hour, using only fractions of each hash.

Search suggestions in the browser’s smartbox give instant answers to users’ search queries without redirecting them to search results pages. The users of Yandex.Browser can at the first launch choose a default search provider from a selection of three, which varies depending on the user’s location.

To generate search suggestions, predict search terms and offer instant results without redirecting the user to the search results page, the browser has to share the search terms with the search provider as the user enters them into the browser’s smartbox. This option is enabled by default in Yandex.Browser. Although this type of data sharing can be disabled in settings, its benefits massively outweigh privacy risks. Also, web users have an opportunity to add any search provider they trust to the browser and set it as the default.

One of the flagship features of the new Yandex.Browser is rich search suggestions, which instantly take the user directly to the website, or even specific page on a specific website, via a widget and bypassing the search results page. Similarly, simple, straightforward searches in the smartbox of the new Yandex.Browser will retrieve simple and straightforward results right in the browser.

Other automated features essential for the contemporary web surfing, such as sending crash reports, resolving web navigation errors, or the autofill function, involve sharing users’ information in one form or another. These features remain enabled by default. The user has full control over this aspect of their browsing experience and can disable any or all of these features.

The beta version of experimental Yandex.Browser retains its minimalist look to offer the user unhindered experience interacting with the website. Browser tabs can now be toggled within groups, while tab groups can be moved within windows. Website information, the smartbox and favourite websites are hidden when not in use and can be summoned by clicking on the website’s header in Yandex.Browser.

Just like the alpha version of experimental Yandex.Browser, the beta version is available for Windows and OS X and can be downloaded at browser.yandex.com.

Yandex Labs collaborated with Carnegie Mellon University to build a new personalized TV experience

At Yandex Labs, we had a chance to work on a 3-month practicum with students and Professor Ian Lane from Silicon Valley department of Carnegie Mellon University. The project was ambitious but also fun: we wanted to build a new TV experience - personalized and interactive. We developed an application for TV that shows personalized content on a TV screen and allows the users to easily manipulate and interact with the content using hand gestures. The app is still a prototype and is not available for download, but we made this video to share our ideas with you.

The app brings users’ social network streams to their TV screens and allows them to navigate over this information using hand gestures. It is built on Mac OS X platform and we used Microsoft Kinect for gesture recognition.

The application features videos, music, photos and news shared by the user’s friends on social networks in a silent ‘screen saver’ mode. As soon as the user notices something interesting on the TV screen, they can easily play, open or interact with the current media object using hand gestures. For example, they can swipe their hand horizontally to flip through featured content, push a “magnetic button” to play music or video, move hands apart to open a news story for reading and then swipe vertically to scroll through it.

To train gesture recognition, the Carnegie Mellon students together with Professor Ian Lane evaluated several machine learning techniques, which included Neural Networks, Hidden Markov Models and Support Vector Machines (SVM), with SVM showing 20% better accuracy. They put a lot of effort in building a real training set – they collected 1,500 gesture recordings, each gesture sequenced into 90 frames, and manually labeled from 4,500 to 5,600 examples of each gesture. By limiting the number of gestures to be recognized at any given moment and taking into account the current type of content, the students were able to significantly improve the gesture recognition rate.

We have been thinking of controlling a social application with gestures for quite a while. When we found a team of like-minded enthusiasts, we took this opportunity and did a nearly three-month research project. The results of this effort were quite impressive and now we are looking whether we can implement them in a real life application.

Future of search, who really owns your content and why personalisation can backfire

Watch Yandex's CTO discuss all these and other important issues at DLD's Business and Technology Conference, TES 2013

Ilya Segalovich, Yandex CTO talks to Gerhard Thomas, Managing Director of Burda Direkt Services about the future of search technologies, who should own the user-generated content and who really owns it, and why strong personalisation is bad.

 

Yandex Rolls Out Experimental Social Recommendation Tool – Wonder

Social networks have become the main source of information for many of us. We get news from Twitter and Facebook, learn about new places on Foursquare and Instagram, discover new music on Facebook via Spotify and Rdio. We receive information through our news feeds which change so quickly that we often lose track of what has just caught our attention. To help ourselves retrieve interesting bits from our social networking space, we have built an app for iPhone and iPod Touch that can respond to our questions using the information about our friends’ activity on Facebook, Instagram, Foursquare and Twitter – places they visited, music they listened to, or news they read – and called it Wonder

Wonder supports natural language, which is English so far, and features a voice user interface built using the speech recognition and text-to-speech conversion technologies provided by Nuance Communications, Inc. (NASDAQ:NUAN). Making the most of your social networking has never been easier and more effective:

- If you are looking for a proven sushi place in New York, you can just ask: what sushi restaurants do my friends go to in New York?
- When you are looking for coffee shops in a new area, you can ask: coffee shops nearby.
- If you need to catch up with your friends on a Friday night, just ask: where do my friends party?
- You know your friend John has a good taste for music, ask: what music does John listen to?
- Feel like listening to electronic music, ask: I wonder what electronic music are my friends listening to?
- Want to catch up on news, ask: news shared by my friends.

The information that the user receives in Wonder in response to their questions comes from their friends across a number of social networks. For example, a location-related question brings visual stacks of pictures together with friends’ tips from Facebook, Instagram and Foursquare. The user only has to tap on a stack to unfold the pictures and swipe through them.

When in a crowded place and uncomfortable to speak to their phone in public, the user can type – there is the keyboard button under the activated microphone icon. To facilitate the input, Wonder provides personalized suggestions of friends’ names, cities they visited, place categories, and music genres.

Wonder is integrated with a number of web services (Last.fm, Foursquare’s venue database, iTunes), which allows users to get more information and consume what they have found without leaving the application. For example:

Tap on a restaurant name to see the menu and call to book a table.



Tap on the album title to read the artist’s bio or tap on the album cover to preview a song and buy it on iTunes.

The user can start asking questions in Wonder a few minutes after they have provided access to their social network accounts and the first portion of data has been collected. The more data is collected over time, the better the answers.

Wonder is implemented at Yandex Labs using Yandex’ technologies for natural language understanding and social data collector. The natural language understanding unit translates questions in English into structured database queries. The social data collector is built from scratch, it periodically fetches, cleans and stores social data into a database designed to run structured queries. Read more about the app’s mechanics here.

Wonder is an experiment. Currently, it supports three domains (places, music and news) and only a few types of questions. The app’s scope will be expanded based on users’ feedback. Made by Yandex’s Palo Alto team with the U.S. audiences in mind, the app may become a welcomed and long-awaited tool for the users of social networks in Russia, such as VK, the country’s largest and most popular social networking website. 

Wonder can be downloaded for free from the U.S. Apple App Store. We would love to hear from you - feel free to share your perceptions here

More screenshots of Wonder app are available at Yandex.Fotki photo album.

Visit the app's website at wonder.yandex.com.

From Feature Phone to Smartphone with Yandex.Disk

Yandex has rolled out a utility app to help new buyers of iOS or Android smartphones to seamlessly transfer their data from their old Symbian, Java or WindowsMobile phone to the new device.

The mobile data transfer utility hinges on Yandex's free cloud storage service Yandex.Disk, which was launched as a beta in April 2012 and became available to a wider audience in September the same year. Yandex.Disk is a godsend to anyone who appreciates an opportunity to store their files online and access them from any internet-enabled device. The service users can upload files in any format and use up to 20 GB of free storage space for personal documents, photos, music, or videos. Yandex.Disk also automatically saves email attachments in the users’ Yandex.Mail accounts to the additional storage space allocated specifically for this purpose.

The owners of iOS or Android smartphones access their files stored on Yandex.Disk, easily and instantly, via the Yandex.Disk app for iOS and Android, which complements the web-based service. In addition to using an opportunity to upload, download or manage their files on Yandex.Disk from their smartphones, they can also share files via mobile phone at a minimal cost – instead of sending the whole file the app forwards just a link to the file uploaded to the service. Syncing files from phone to computer is also easy – files added to the Yandex.Disk app on the user’s phone can be instantly viewed on a PC or laptop. The app also allows taking photos with the phone’s camera and saving them directly to Yandex.Disk. 

The new Yandex.Disk utility downloaded to any smartphone operating Symbian, Java or WindowsMobile sends the user's contacts to their new Android or iOS phone. Users of Symbian or WindowsMobile systems, in addition to contacts, can transfer their bookmarks, text messages and call history to their new Android device. 

This utility is an add-on to the Yandex.Disk service, which offers users a safe and easy way to sync their data across multiple devices. Downloaded to a Symbian, Java or WindowsMobile phone, it gathers the user’s data and sends it over to the web-based Yandex.Disk. The only thing the user needs to do next is download the Yandex.Disk app to their new Android phone for their contacts, bookmarks, text messages and call history to appear on the new phone. To transfer personal contacts from their old phone to an iOS phone, the user only needs to enable contact sync via CardDAV in settings. 

The new Yandex.Disk utility facilitates users' transition from a feature phone to the latest generation smartphone devices. It offers the most convenient and user-friendly way to transfer data between different mobile platforms and gives users continuity of access to their information regardless of which device they use.

Currently, the Yandex.Disk utility is released with the Russian language user interface for customers in Russia, Ukraine, Belarus and Kazakhstan. 

The Yandex.Disk app is available in English and can be downloaded from Yandex, the App Store or Google Play

To app developers, Yandex offers the Yandex.Disk API, which is also available in English.