Something that is really missing in my opinion is a good opensource OpenCL [1] runtime for regular x86 CPUs using both muticore optims (a la openmp) and vector instructions (SSE). Bonus points if your OpenCL runtime can also leverage the nouveau and/or the open source ATI drivers.
Having such a runtime available as a standard lib packaged in all linux distribs would make it interesting for lowlevel vector math libs such blas / lapack / atlas and convex optimization solvers to have an implementation based on OpenCL kernels that would work (almost) as fast as the currently manually optimized C + simd intrinsics code but could also 10 to 100 as fast whenever a GPU is available on the machine without having to re-compile anything.
Some advanced and very promising machine learning algorithms (e.g. algorithms of the deep learning family) can really benefit from the computing power of vector processors such as the GPUs.
Right now everybody who wants to be able to gain the perf boost of GPUs uses CUDA but:
- it's only useful on NVidia GPU machines (e.g. not on Amazon EC2 machines for instance)
- it's not opensource hence has to be manually installed on every single machine you want to use it (no apt-get install from the official repos). This which makes it a heavy dependency for your pet machine learning library: be prepared to support CUDA installation problems on your project mailing list.
Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Theano features:
Yes I know and use theano for some deep learning experiments. This is really a great tool. But theano will probably never be considered a default dependency for common machine learning libs as long as there is no good OpenCL runtime opensource and pre-packaged in major posix distros (linux and bsd, osx already has opencl by default but not opensource hence cannot be reused in linux AFAIK).
Having such a runtime available as a standard lib packaged in all linux distribs would make it interesting for lowlevel vector math libs such blas / lapack / atlas and convex optimization solvers to have an implementation based on OpenCL kernels that would work (almost) as fast as the currently manually optimized C + simd intrinsics code but could also 10 to 100 as fast whenever a GPU is available on the machine without having to re-compile anything.
Some advanced and very promising machine learning algorithms (e.g. algorithms of the deep learning family) can really benefit from the computing power of vector processors such as the GPUs.
Right now everybody who wants to be able to gain the perf boost of GPUs uses CUDA but: - it's only useful on NVidia GPU machines (e.g. not on Amazon EC2 machines for instance) - it's not opensource hence has to be manually installed on every single machine you want to use it (no apt-get install from the official repos). This which makes it a heavy dependency for your pet machine learning library: be prepared to support CUDA installation problems on your project mailing list.
[1] http://en.wikipedia.org/wiki/OpenCL