The Year 2016 On A Rice Grain

If we consider all domains of machine learning, the year was definitely a huge success. For instance, recurrent networks are now all over the place, either stand-alone or as part of other networks. In the domain of images, all cool kids are now using some variant of generative adversarial networks (GAN) and reinforcement learning is so popular that you cannot avoid it. In other words, whenever a method works with “natural” data, like images, videos, documents or ‘environments’, a lot of progress has been made. Of course there was also progress in other domains, for instance information retrieval or recommender systems, but not in such a profound way. And this brings to the domain, we are mostly concerned about which is personalization.

First, even if finding the needle in the digital haystack gets more and more important, for instance product search/recommendation, there were very few papers about it. One reason might be that experiments require a sufficiently large data set which contains valuable information and not many are willing to give away data like this for free. Another reason might be that most items -in general- cannot be described in a uniform way. For instance, two product descriptions in different languages can be considered very similar, but are totally differently encoded. Furthermore, the level of details might strongly vary and thus, content-based methods might fail. So, we are back at colloborative filtering which has its own weaknesses and strengths.

Stated differently in contrast to images/video or audio, the quality of a model mainly consists of how the input data is encoded into features and since we do not know in advance if the features are powerful enough for the problem we try to solve, the model is often simply not powerful enough. We already argued that those models could be enriched with data from other domains, like image data, a photo of the product or summary of the video/song, but this multi-modal approach has not been widely explored yet and needs a lot of more research.

Without a doubt we made some progress to build personalized services, but compared others the steps seem to be very tiny and sometimes even targeted backwards. It’s hard to tell what could improve the situation, because even such a simple thing like a taxonomy can fail when a couple of parties is involved. There are some approaches for open knowledge bases but they are far from being complete or easy to use and the rest of the knowledge bases have been acquired by companies which now control the data and the access to them.

This might all sound very darkly but also underlines that this calls for a fundamental rethink. For instance, instead of encoding all required knowledge in the features, it could make sense to have some kind of memory that allows us to access pre-existing knowledge that is not restricted to a specific problem. Like the problem that a feature like “david-cronenberg” is not just a word, but describes a concept that implies a lot of facts, like (david-cronenberg,is-a,director) or (david-cronenberg,directed,scanners). In other words, we need an efficient way to re-use or at least integrate existing knowledge in a consistent way to improve the expressiveness of models with limited features.

In a nutshell, maybe we need to throw away most of the old baggage and start -almost- from scratch again to get rid of all the restrictions that have been piled up over the years. Of course this is easier said than done, but researchers already demonstrated that this is possible in other domains. However, it might also be possible that somebody already came up with a great idea but is not able/willing to publish it since it is owned by some company. But one is for sure, namely that we
are still at the begin of very long, but interesting journey.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s