Hello folks! Over the last 2 years, I've helped 35+ companies improve their AI products. I distilled my approach into this guide, which covers error analysis, synthetic data, eval-human alignment, involving domain experts, optimizing # of experiments & more I appreciate you receiving this, but if you want to stop, simply unsubscribe • • • Read in browser for best experience 👉(web version has extras like videos, images, tables and more) 👈 A Field Guide to Rapidly Improving AI Products...
over 1 year ago • 10 min read
Thoughts On A Month With Devin Impressions of Devin after giving it 20+ tasks. In March 2024, a new AI company burst onto the scene with impressive backing: a $21 million Series A led by Founders Fund, with support from industry leaders including the Collison brothers, Elad Gil, and other tech luminaries. The team behind it? IOI gold medalists - the kind of people that solve programming problems most of us can’t even understand. Their product, Devin, promised to be a fully autonomous software...
over 1 year ago • 8 min read
nbsanity - Share Notebooks as Polished Web Pages in Seconds Transform your GitHub Jupyter notebooks into beautiful, readable web pages with a single URL change. No setup required. I’ve long believed that Jupyter Notebooks are an excellent medium for technical writing, combining live code, visualizations, and narrative text in a single document. However, sharing notebooks in a way that’s both beautiful and accessible has always been a challenge. While GitHub’s notebook viewer is functional, it...
over 1 year ago • 4 min read
Hello folks! Wanted to share a new blog post with ya'll. Below are the details: Building an Audience Through Technical Writing: Strategies and Mistakes Published on November 30, 2024 People often find me through my writing on AI and tech. This creates an interesting pattern. Nearly every week, vendors reach out asking me to write about their products. While I appreciate their interest and love learning about new tools, I reserve my writing for topics that I have personal experience with. One...
over 1 year ago • 5 min read
Hello folks! Wanted to share a new blog post with ya'll. Here is a preview of the post - clink the link below for the full version. Creating a LLM-as-a-Judge That Drives Business Results Published on October 29, 2024 Earlier this year, I wrote Your AI product needs evals. Many of you asked, “How do I get started with LLM-as-a-judge?” This guide shares what I’ve learned after helping over 30 companies set up their evaluation systems. The Problem: AI Teams Are Drowning in Data Ever spend weeks...
almost 2 years ago • 1 min read
Concurrency For Starlette Apps (e.g FastAPI / FastHTML) Published on October 11, 2024 For the best experience, read this blog post on the web. Motivation: we often want to call LLMs in Starlette based apps (FastHTML, FastAPI, etc.) apps, and we don’t want to block the server on network calls to APIs. This post documents my explorations of various approaches that can run tasks in the background without blocking the main process. You can see this notebook here. Using A SQL Database As A Queue...
almost 2 years ago • 7 min read
Hello folks! Wanted to share a new blog post with ya'll. Below are the details: An Open Course on LLMs, Led by Practitioners Published on July 29, 2024 Today, we are releasing Mastering LLMs, a set of workshops and talks from practitioners on topics like evals, retrieval-augmented-generation (RAG), fine-tuning and more. This course is unique because it is: Taught by 25+ industry veterans who are experts in information retrieval, machine learning, recommendation systems, MLOps and data...
about 2 years ago • 2 min read
