Gradient Flow #38: Large Language Models, Infinite Laptop, Overhyping AI
This edition has 455 words which will take you about 3 minutes to read.
“Yesterday I was clever, so I wanted to change the world. Today I am wise, so I am changing myself.”- Rumi
Data Exchange podcast
Training and Sharing Large Language Models Connor Leahy, AI Researcher at Aleph Alpha GmbH, and founding member of EleutherAI, a collective of researchers and engineers building resources and models for researchers who work on natural language models. We discussed the state of natural language research, the rise of large language models, embeddings, and a new role that Connor and others refer to as “prompt engineer”.
Neural Models for Tabular Data Sercan Arik, Research Scientist at Google Cloud AI, and his collaborators recently published apaper on TabNet, a deep neural network architecture for tabular data. It uses sequential attention to select features, is explainable, and based on tests and actual deployments, TabNet outperforms or is on par with other models on classification and regression problems.
Featured Virtual Event
Our friend Paco Nathan is part of a strong roster of speakers at the upcoming Datanova For Data Scientists. Paco will moderate a panel on the data lakehouse, an emerging data management paradigm.
Data & Machine Learning Tools and Infrastructure
Combine the development experience of a laptop with the scale of the cloud I compiled observations from keynotes at the recent Ray Summit. This post will be of interest to anyone building AI applications or machine learning platforms.
Scaling Apache Pulsar to 10 PB/day Splunk finally details how it uses Pulsar
AugLy from Facebook AI Research A new open source library for data augmentation in computer vision and natural language models. Data augmentation tools expand your labeled training datasets and is particularly useful for model validation and improving model robustness.
Flat Data This clever open source project simplifies routine data acquisition and cleanup tasks. Since it runs on GitHub Actions, you won’t need specialized infrastructure to use it.
Facebook’s Prophet: a mostly negative review As Prophet’s co-creator, Sean Taylor notes, it is very challenging to build a forecasting tool that generalizes across many domains and problems. I’ve had decent luck using Prophet in combination with an optimization tool like Ray Tune.
How Edge Observability will enable companies to push compute to the edge New generation of tools combine distributed stream processing and federated machine learning capabilities for intelligent, dynamic, and automated data routing to optimal destinations.
[Image by Armin Forster from Pixabay.]
Funding Updates
Recommendations
AI promised a revolution in radiology, so far it hasn’t delivered on the initial hype
The Epic Sepsis Model poorly predicts sepsis This is a penalized logistic regression model included as part of Epic’s EHR and currently in use at hundreds of hospitals throughout the U.S.
GitHub Co-pilot This AI tool co-developed with OpenAI, provides programmers with suggestions for whole lines or entire functions right inside their editor.
The World for Sale: Money, Power and the Traders Who Barter the Earth’s Resources An eye-opening tour through the world of major commodity traders.
Stepping Back from Speaking A must-read and incredibly honest post from Martin Fowler.
Closing Short:
If you enjoyed this newsletter please support our work by encouraging your friends and colleagues to subscribe: