A Canticle For Theano

Back in 2010 there was a presentation about a new computation framework at SciPy and the name was Theano. At this time, even if it was not the only one available framework, there was also Torch for example, it provided a lot of new and very powerful features. Furthermore, it provided an interface very similar to the popular numpy framework, which is very versatile and provides everything you need to build machine learning algorithms, especially neural nets.

The idea to describe an algorithm in a purely symbolic way and let the framework optimize this expression into a computational graph was something new on the one hand, but it also allowed to execute code on other devices like the GPU, which can be much faster, on the other hand. And the cherry on top of this delicious cake was something called automatic differentiation. In contrast to other frameworks, or if in case you implement algorithms by hand, you can define an arbitrary loss function and let the framework do the calculation of the gradient. This has several advantages, including the error-prone process to derive the gradient manually, but also to perform the required steps in a deterministic way with the help of the computational graph.

In other words, Theano allows you to implement everything that you can imagine as long as it is continuous and differentiable. Of course this does not come for free, since the framework is a low-level library, it requires a solid understanding of the whole work flow and also to code all the functionality yourself. But once you have done this, it allows you rapid prototyping of different algorithms since you don’t need to manually derive the gradients and also you don’t have to care for the optimization of the data flow since Theano optimizes the graph for you.

More than five years ago, we started to use Theano, because we love python, machine learning, including neural nets, and the way Theano does the things. In this time, we never encountered any serious bugs or problems and we were also able to use it on a broad range of devices including, CPUs, GPUs and embedded ones. We implemented innumerable algorithms in this time, from rather straightforward nets to deep nets with dozens of layers. But we did not only use it for research, we also used it to build real-world applications which were deployed and are still in use.

In other words, without Theano, the world of machine learning would definitely not be the same. For instance, recent frameworks were likely to be inspired by the features Theano provided. So, the question is, why such a framework, with all the awesomeness never managed it to get the attention it deserved? Was it too difficult to use? Does it require the backup of a major company? Was it too close to academia? Missing marketing? We don’t have the answers and now that we development of Theano will stop soon, the future is very uncertain. Sure, it is open source, but we still need people that are willing to spend time to improve it and to coordinate the further development. But even if we are a little skeptical about the future of Theano, version 1.0 is still a very useful piece of software that can be used to design and deploy your models.

What’s left to say? We would like to thank all the contributors and authors of Theano for their hard work. Theano is an amazing piece of software that allows everyone to express their thoughts as algorithms in a straightforward and efficient way. With it, it was possible to build cutting edge products and to use GPUs even if you were not an CUDA expert.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s