It is the 21st of August and as for the last couple of months, this means that there is a new REEF release. Given that we have recently been accepted into the Apache Incubator, this is one of if not the last release before we transition development into the Apache infrastructure. And it is a good one:
- Wake moved from Netty 3.x to 4.x. As Netty changed quite a bit between these two releases, this was a huge amount of work lead and done by our friends at Seoul National University. Way to go!
- REEF has the beginnings of the upcoming infrastructure for Evaluator-preserving Driver restarts. This is huge for developers of long running applications. The work shall be completed in one of the next releases. Stay tuned.
- We clarified the CLASSPATH handling to allow applications to overwrite, following the Hadoop tradition of putting the user’s CLASSPATH entries first (also this).
- For once, we have all the logs from all the various logging frameworks used by our dependencies come out in the same stream.
- In our constant refactoring quest, we shrunk EvaluatorManager by 200 lines and solved a heisenbug in the process: Very infrequently, we forgot about Evaluators in the Driver right before it received the last heartbeat, leading to the infamous “Unknown Evaluator” crash of the Driver.
We found and fixed that last issue when scaling our tests to the 1000 Evaluator mark, which is a new record for REEF applications.
The release is making its way to Maven Central as I am writing this. More detailed changes can be found in the milestones on GitHub (REEF 0.7, Tang 0.7, Wake 0.7)
