Gradient Flow #18: Forecasting & Groupthink, Interpreting NLP, Ray Ecosystem

Sep 24, 2020

This edition has 640 words which will take you about 4 minutes to read.

“If you want to go fast, go alone. If you want to go far, go together.” - African Proverb.

Data Exchange podcast

Using machine learning to modernize medical triage and monitoring systems Kira Radinsky is Chairwoman & Chief Technology Officer at Diagnostic Robotics, a startup using AI to build a medical-grade triage and clinical-predictions platform. She was one of the pioneers in using alternative data sources to augment forecasting models. Her earlier work includes models to predict social unrest as well as disease outbreaks. Diagnostic Robotics was already building tools for clinical triage and they very quickly were able to tune their systems for the battle against COVID-19.
Connecting Reinforcement Learning to Simulation Software Max Pumperla, deep learning engineer at Pathmind and a contributor to many open source projects in data science and machine learning (including Hyperopt and Keras). At Pathmind, he is helping bring reinforcement learning to software used for simulations. Many companies already use simulation software and RL lets them tackle more complex real-world scenarios.

Our 2020 NLP Industry Survey Report is now available as a free download. If you are using natural language technologies, make sure you read this report.
One Simple Graphic: companies that offer deep neural network accelerators
The History, Status, and Future of FPGAs Speaking of accelerators, here’s a balanced take on the class of chips that remain an option for machine learning training and inference. Turing Award Winner Dave Patterson has a funny take on FPGAs.
Comparing open source hyperparameter tuning libraries (Hyperopt, Optuna, and Tune) Max Pumperla is also a maintainer of Hyperopt. We recently discussed the state of open source tools for hyperparameter tuning. In this short video he candidly admits that Tune is what he would use today: “For me, and I say this as a Hyperopt maintainer, Tune is the clear winner down the road.” Max forgot to include Determined, which also has a great HPO offering as part of its end-to-end, open source model development platform.
POLARIS: The Distributed SQL Engine in Azure Synapse A 10-minute video summary of a VLDB paper. Companies are really beginning to rally around the Lakehouse paradigm which we introduced earlier this year.

Horovod on Ray Initially released in 2017, Horovod has become one of the most popular open source packages out of Uber. It’s used for scaling deep learning training, specifically data-parallel distributed training. As of the latest release, Ray can now be used to execute Horovod jobs “without needing to coordinate the workers by hand”. Travis Addair who leads the Horovod project at Uber, will give a talk at the Ray Summit on Distributed Deep Learning with Horovod on Ray. Horovod and Seldon are among a growing number of popular libraries that have integrations with Ray for distributed execution.
Interpreting Natural Language Processing Models In this recent Simons Institute talk, Yonatan Belinkov of Technion describes much needed new interpretability tools in an age when end-to-end learning using neural models dominates most NLP tasks. He goes through recent research into how linguistic structures (syntax, POS, morphology, …) can be used to understand neural language models. Reminiscent of work in explaining computer vision and speech models, layers of deep language models appear to correspond to a hierarchy of language properties (higher layers correspond to more sophisticated linguistic units). For more on recent research in NLP, the upcoming NLP Summit features progress in speech recognition (Bo Li of Google) and new tools for testing NLP models (Marco Túllio Ribeiro of MSR).

Blood and Oil One of my favorite books of 2020 is this balanced portrait of the Crown Prince of Saudi Arabia.
Uncharted: How to Navigate the Future This excellent new book provides real-world examples of forecasting, experimentation, scenario planning and beyond. It’s also full of practical tips for how to implement these various methodologies.
TikTok reveals details of how its algorithm works to reporters
Welcome to your Bland New World Ben Schott describes venture backed direct-to-consumer brands as “ironic normcore trifle” for the rich, and “fleeting facsimile of prosperity” for hipsters in Silver Lake, Brooklyn, the Mission District, Shoreditch, and beyond.
How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence

If you enjoyed this newsletter please support our work by encouraging your friends and colleagues to subscribe:

Ben Lorica edits the Gradient Flow newsletter. He is co-chair of the Ray Summit, chair of the NLP Summit, and host of the Data Exchange podcast. You can follow him on Twitter @BigData. This newsletter is produced by Gradient Flow.