I just want to thank you for your incredible work. I'm a fellow countryman migrating a Python program that fetches financial data from a large amount of sources, that's become very hard to maintain, to Elixir. The BEAM fits my use case perfectly, and Elixir has been an absolute joy to work with and debug.
Off-topic: hi Jose, what's your expectation on how much more raw performant Erlang can become in the next 1-2 years?
PHP ~3x in performance from 5.x to 7.x. LuaJIT is even faster over vanilla Lua.
There were multiple attempts at creating an Erlang JIT for well over a decade now and last year BeamAsm was released but in everyday use it seems to net about a 10-15% boast.
I ask because it'd be great if we didn't have to create so many Rust NIFs in order to address Erlang raw slowness.
Curious what your take is on speeding up raw Erlang (please note, I'm not talking about concurrency & I do truly love Exilir/Erlang/OTP)
I am not informed enough to give actual estimates. :)
Although your estimates for JIT improvements seem low on my experience. Compilation times are consistently half of what they were before and test suites range between 33% to 50% faster. Even Whatsapp reported server efficiency increased by 25% and many other factors are likely at play there (source: https://twitter.com/wcathcart/status/1385253969522413568).
I _assume_ we will continue seeing improvements on this front, especially if at some point we start doing cross module optimizations!
Very impressive library! Is it still in the developmental stage, or do you know of anyone using it for real yet?
Also, I love livebook! I keep a tab with it open and it has replaced my use of iex. Can you talk a little bit about how you decided to embark on building that, too? An ML library and framework is a big enough job already, how did you decide to add Livebook to the list?
This release means we are getting closer to prod usage but Nx requires one of the existing compilers/backends to shine. So I am hoping we will see production uses once we officially release Torchx/EXLA packages too.
---
Curiously, I had two ideas for the name Livebook:
1. An interactive book to teach LiveView. LiveView is a new approach for developing web apps and I thought it could be worth exploring different non-traditional ways of teaching it too!
2. Livebook as code notebooks (what we got!)
It just feels a great fit for the platform, considering Phoenix, LiveView, Distributed Erlang, etc. So why not? :) And I am curious to see if and how we can contribute new ideas to the already extensive ecosystem of code notebooks!
Are there any specific advantage that Elixir being on BEAM VM brings to Machine Learning and Data Science? I am assuming BEAM VM's actor model fits well with the GPU / Tensor usage ? ( I have zero knowledge on ML and Data Science )
Personally I think this is a great move from Elixir community and especially coming directly from the creator. Some people may be dismissive ( as with at least half of HN ) But I see this with Ruby where there is a chicken and egg problem. The community stuck with Web or Rails Development only. And no one has incentive to kickstart the languages ecosystem in different directions.
So instead of Elixir being another language for Web / Phoenix, it could now ( or in the future ) be used as ML with some level of official support.
I was particularly delighted to see how you built `defn` on top of existing metaprogramming functionality. This kind of thing has lead me to think of Elixir as a Lisp in Ruby's clothing running on the BEAM. :) I hope that's not too inaccurate. Were there any delightful "aha!" moments you got while building this?
In general notes, it is an effort to bring more use cases to Elixir, the Erlang VM, and the ecosystem. But it is an area I have been interested on (back in 2009 my master thesis was actually within Machine Learning). When I met Sean (through the Genetic Algorithms in Elixir book), our interests aligned and we started working on it!
