Welcome to the Daft blog
Join us as we explore innovative ways to handle multimodal datasets, optimize performance, and simplify your data workflows.
VLAs are dead, long live World Action Models - a summary of Jim Fan's Robotics End Game talk
Jim Fan argues robotics will follow the exact LLM playbook - and VLAs are already being replaced by World Action Models.
Daft v0.7.8: Dashboard Heatmaps, Vectorized GroupBy, and Native Image Hashing
Daft's query dashboard now shows you exactly where time is going. Slow operators light up red, completed nodes turn green, and arrows trace the data flow through your pipeline. No more guessing which
Open-sourcing 43 Billion Tokens of SEC EDGAR
Datamule, Teraflop AI, and Eventual collaborated to release the SEC-EDGAR dataset containing 590 GB of data, spanning 8 million samples and 43 billion tokens from all major filings in the SEC EDGAR database.
Daft v0.7.7: Parquet Cache Regression Fixed, df.shuffle(), and Coalesce Short-Circuit
Daft v0.7.7 fixes a parquet streaming regression that made aggregations 2-4x slower, adds df.shuffle() for ML data prep, and makes coalesce short-circuit per the SQL spec.
Daft v0.7.6: Every Major Lake Format, O(1) Scalars, and Swordfish Plan Caching
Daft natively reads and writes every major open lake format — Iceberg, Delta Lake, Hudi, and now Apache Paimon. Plus O(1) scalar columns, fingerprint-based plan caching in Swordfish, and production observability.
Daft UDF Patterns: Four Patterns, One Notebook
Row-wise, generator, async, and stateful UDFs — one notebook, one dataset, runnable side by side.
GPU Inference with @daft.cls
Run GPU models on millions of rows without OOM. Real patterns from ByteDance, Essential AI, and more.
Stateful UDFs with daft.cls: Python Classes that Scale
Turn any Python class into a distributed operator. Hold models, connections, and clients across rows with one decorator.
Daft v0.7.5: A Plugin System, 5x Faster Parquet, and a Real-Time Query Debugger
Native Extensions via Stable C ABI, Live Query Dashboard, and 2-5x faster Parquet Reads on Nested Types
Stateless UDFs with daft.func - four patterns, one decorator
Row-wise, async, generator, and batch UDFs in Daft — one decorator, zero boilerplate, local or distributed.
