Datasette Cloud, Datasette 1.0a3, llm-mlc and more
Plus a tool for building annotated presentations, ChatGPT-generated, prompts included
In this newsletter:
Datasette Cloud, Datasette 1.0a3, llm-mlc and more
How I make annotated presentations
Weeknotes: Plugins for LLM, sqlite-utils and Datasette
Plus 15 links and 3 quotations and 9 TILs
Datasette Cloud, Datasette 1.0a3, llm-mlc and more - 2023-08-16
Datasette Cloud is now a significant step closer to general availability. The Datasette 1.03 alpha release is out, with a mostly finalized JSON format for 1.0. Plus new plugins for LLM and sqlite-utils and a flurry of things I've learned.
Datasette Cloud
Yesterday morning we unveiled the new Datasette Cloud blog, and kicked things off there with two posts:
Welcome to Datasette Cloud provides an introduction to the product: what it can do so far, what's coming next and how to sign up to try it out.
Introducing datasette-write-ui: a Datasette plugin for editing, inserting, and deleting rows introduces a brand new plugin, datasette-write-ui - which finally adds a user interface for editing, inserting and deleting rows to Datasette.
Here's a screenshot of the interface for creating a new private space in Datasette Cloud:
datasette-write-ui
is particularly notable because it was written by Alex Garcia, who is now working with me to help get Datasette Cloud ready for general availability.
Alex's work on the project is being supported by Fly.io, in a particularly exciting form of open source sponsorship. Datasette Cloud is already being built on Fly, but as part of Alex's work we'll be extensively documenting what we learn along the way about using Fly to build a multi-tenant SaaS platform.
Alex has some very cool work with Fly's Litestream in the pipeline which we hope to talk more about shortly.
Since this is my first time building a blog from scratch in quite a while, I also put together a new TIL on Building a blog in Django.
The Datasette Cloud work has been driving a lot of improvements to other parts of the Datasette ecosystem, including improvements to datasette-upload-dbs and the other big news this week: Datasette 1.0a3.
Datasette 1.0a3
Datasette 1.0 is the first version of Datasette that will be marked as "stable": if you build software on top of Datasette I want to guarantee as much as possible that it won't break until Datasette 2.0, which I hope to avoid ever needing to release.
The three big aspects of this are:
A stable plugins interface, so custom plugins continue to work
A stable JSON API format, for integrations built against Datasette
Stable template contexts, so that custom templates won't be broken by minor changes
The 1.0 alpha 3 release primarily focuses on the JSON support. There's a new, much more intuitive default shape for both the table and the arbitrary query pages, which looks like this:
{
"ok": true,
"rows": [
{
"id": 3,
"name": "Detroit"
},
{
"id": 2,
"name": "Los Angeles"
},
{
"id": 4,
"name": "Memnonia"
},
{
"id": 1,
"name": "San Francisco"
}
],
"truncated": false
}
This is a huge improvement on the old format, which featured a vibrant mess of top-level keys and served the rows up as an array-of-arrays, leaving the user to figure out which column was which by matching against "columns"
.
The new format is documented here. I wanted to get this in place as soon as possible for Datasette Cloud (which is running this alpha), since I don't want to risk paying customers building integrations that would later break due to 1.0 API changes.
llm-mlc
My LLM tool provides a CLI utility and Python library for running prompts through Large Language Models. I added plugin support to it a few weeks ago, so now it can support additional models through plugins - including a variety of models that can run directly on your own device.
For a while now I've been trying to work out the easiest recipe to get a Llama 2 model running on my M2 Mac with GPU acceleration.
I finally figured that out the other week, using the excellent MLC Python library.
I built a new plugin for LLM called llm-mlc. I think this may now be one of the easiest ways to run Llama 2 on an Apple Silicon Mac with GPU acceleration.
Here are the steps to try it out. First, install LLM - which is easiest with Homebrew:
brew install llm
If you have a Python 3 environment you can run pip install llm
or pipx install llm
instead.
Next, install the new plugin:
llm install llm-mlc
There's an additional installation step which I've not yet been able to automate fully - on an M1/M2 Mac run the following:
llm mlc pip install --pre --force-reinstall \
mlc-ai-nightly \
mlc-chat-nightly \
-f https://mlc.ai/wheels
Instructions for other platforms can be found here.
Now run this command to finish the setup (which configures git-lfs
ready to download the models):
llm mlc setup
And finally, you can download the Llama 2 model using this command:
llm mlc download-model Llama-2-7b-chat --alias llama2
And run a prompt like this:
llm -m llama2 'five names for a cute pet ferret'
It's still more steps than I'd like, but it seems to be working for people!
As always, my goal for LLM is to grow a community of enthusiasts who write plugins like this to help support new models as they are released. That's why I put a lot of effort into building this tutorial about Writing a plugin to support a new model.
Also out now: llm 0.7, which mainly adds a new mechanism for adding custom aliases to existing models:
llm aliases set turbo gpt-3.5-turbo-16k
llm -m turbo 'An epic Greek-style saga about a cheesecake that builds a SQL database from scratch'
openai-to-sqlite and embeddings for related content
A smaller release this week: openai-to-sqlite 0.4, an update to my CLI tool for loading data from various OpenAI APIs into a SQLite database.
My inspiration for this release was a desire to add better related content to my TIL website.
Short version: I did exactly that! Each post on that site now includes a list of related posts that are generated using OpenAI embeddings, which help me plot posts that are semantically similar to each other.
I wrote up a full TIL about how that all works: Storing and serving related documents with openai-to-sqlite and embeddings - scroll to the bottom of that post to see the new related content in action.
I'm fascinated by embeddings. They're not difficult to run using locally hosted models either - I hope to add a feature to LLM to help with that soon.
Getting creative with embeddings by Amelia Wattenberger is a great example of some of the more interesting applications they can be put to.
sqlite-utils-jq
A tiny new plugin for sqlite-utils, inspired by this Hacker News comment and written mainly as an excuse for me to exercise that new plugins framework a little more.
sqlite-utils-jq adds a new jq()
function which can be used to execute jq programs as part of a SQL query.
Install it like this:
sqlite-utils install sqlite-utils-jq
Now you can do things like this:
sqlite-utils memory "select jq(:doc, :expr) as result" \
-p doc '{"foo": "bar"}' \
-p expr '.foo'
You can also use it in combination with sqlite-utils-litecli to run that new function as part of an interactive shell:
sqlite-utils install sqlite-utils-litecli
sqlite-utils litecli data.db
# ...
Version: 1.9.0
Mail: https://groups.google.com/forum/#!forum/litecli-users
GitHub: https://github.com/dbcli/litecli
data.db> select jq('{"foo": "bar"}', '.foo')
+------------------------------+
| jq('{"foo": "bar"}', '.foo') |
+------------------------------+
| "bar" |
+------------------------------+
1 row in set
Time: 0.031s
Other entries this week
How I make annotated presentations describes the process I now use to create annotated presentations like this one for Catching up on the weird world of LLMs (now up to over 17,000 views on YouTube!) using a new custom annotation tool I put together with the help of GPT-4.
A couple of highlights from my TILs:
Catching up with the Cosmopolitan ecosystem describes my latest explorations of Cosmopolitan and Actually Portable Executable, based on an update I heard from Justine Tunney.
Running a Django and PostgreSQL development environment in GitHub Codespaces shares what I've learned about successfully running a Django and PostgreSQL development environment entirely through the browser using Codespaces.
Releases this week
openai-to-sqlite 0.4 - 2023-08-15
Save OpenAI API results to a SQLite databasellm-mlc 0.5 - 2023-08-15
LLM plugin for running models using MLCdatasette-render-markdown 2.2.1 - 2023-08-15
Datasette plugin for rendering Markdowndb-build 0.1 - 2023-08-15
Tools for building SQLite databases from files and directoriespaginate-json 0.3.1 - 2023-08-12
Command-line tool for fetching JSON from paginated APIsllm 0.7 - 2023-08-12
Access large language models from the command-linesqlite-utils-jq 0.1 - 2023-08-11
Plugin adding a jq() SQL function to sqlite-utilsdatasette-upload-dbs 0.3 - 2023-08-10
Upload SQLite database files to Datasettedatasette 1.0a3 - 2023-08-09
An open source multi-tool for exploring and publishing data
TIL this week
Processing a stream of chunks of JSON with ijson - 2023-08-16
Building a blog in Django - 2023-08-15
Storing and serving related documents with openai-to-sqlite and embeddings - 2023-08-15
Combined release notes from GitHub with jq and paginate-json - 2023-08-12
Catching up with the Cosmopolitan ecosystem - 2023-08-10
Running a Django and PostgreSQL development environment in GitHub Codespaces - 2023-08-10
Scroll to text fragments - 2023-08-08
How I make annotated presentations - 2023-08-06
Giving a talk is a lot of work. I go by a rule of thumb I learned from Damian Conway: a minimum of ten hours of preparation for every one hour spent on stage.
If you're going to put that much work into something, I think it's worth taking steps to maximize the value that work produces - both for you and for your audience.
One of my favourite ways of getting "paid" for a talk is when the event puts in the work to produce a really good video of that talk, and then shares that video online. North Bay Python is a fantastic example of an event that does this well: they team up with Next Day Video and White Coat Captioning and have talks professionally recorded, captioned and uploaded to YouTube within 24 hours of the talk being given.
Even with that quality of presentation, I don't think a video on its own is enough. My most recent talk was 40 minutes long - I'd love people to watch it, but I myself watch very few 40m long YouTube videos each year.
So I like to publish my talks with a text and image version of the talk that can provide as much of the value as possible to people who don't have the time or inclination to sit through a 40m talk (or 20m if you run it at 2x speed, which I do for many of the talks I watch myself).
Annotated presentations
My preferred format for publishing these documents is as an annotated presentation - a single document (no clicking "next" dozens of times) combining key slides from the talk with custom written text to accompany each one, plus additional links and resources.
Here's my most recent example: Catching up on the weird world of LLMs, from North Bay Python last week.
More examples:
Prompt injection explained, with video, slides, and a transcript for a LangChain webinar in May 2023.
Coping strategies for the serial project hoarder for DjangoCon US 2022.
How to build, test and publish an open source Python library for PyGotham 2021
Video introduction to Datasette and sqlite-utils for FOSDEM February 2021
Datasette—an ecosystem of tools for working with small data for PyGotham 2020.
Personal Data Warehouses: Reclaiming Your Data for the GitHub OCTO speaker series in November 2020.
Redis tutorial for NoSQL Europe 2010 (my first attempt at this format).
I don't tend to write a detailed script for my talks in advance. If I did, I might use that as a starting point, but I usually prepare the outline of the talk and then give it off-the-cuff on the day. I find this fits my style (best described as "enthusiastic rambling") better.
Instead, I'll assemble notes for each slide from re-watching the video after it has been released.
I don't just cover the things I said in the the talk - I'll also add additional context, and links to related resources. The annotated presentation isn't just for people who didn't watch the talk, it's aimed at providing extra context for people who did watch it as well.
A custom tool for building annotated presentations
For this most recent talk I finally built something I've been wanting for years: a custom tool to help me construct the annotated presentation as quickly as possible.
Annotated presentations look deceptively simple: each slide is an image and one or two paragraphs of text.
There are a few extra details though:
The images really need good
alt=
text - a big part of the information in the presentation is conveyed by those images, so they need to have good descriptions both for screen reader users and to index in search engines / for retrieval augmented generation.Presentations might have dozens of slides in - just assembling the image tags in the correct order can be a frustrating task.
For editing the annotations I like to use Markdown, as it's quicker to write than HTML. Making this as easy as possible encourages me to add more links, bullet points and code snippets.
One of my favourite use-cases for tools like ChatGPT is to quickly create one-off custom tools. This was a perfect fit for that.
You can see the tool I create here: Annotated presentation creator (source code here).
The first step is to export the slides as images, being sure to have filenames which sort alphabetically in the correct order. I use Apple Keynote for my slides and it has an "Export" feature which does this for me.
Next, open those images using the annotation tool.
The tool is written in JavaScript and works entirely in your browser - it asks you to select images but doesn't actually upload them to a server, just displays them directly inline in the page.
Anything you type in a textarea
as work-in-progress will be saved to localStorage
, so a browser crash or restart shouldn't lose any of your work.
It uses Tesseract.js to run OCR against your images, providing a starting point for the alt=
attributes for each slide.
Annotations can be entered in Markdown and are rendered to HTML as a live preview using the Marked library.
Finally, it offers a templating mechanism for the final output, which works using JavaScript template literals. So once you've finished editing the alt=
text and writing the annotations, click "Execute template" at the bottom of the page and copy out the resulting HTML.
Here's an animated GIF demo of the tool in action:
I ended up putting this together with the help of multiple different ChatGPT sessions. You can see those here:
HTML and JavaScript in a single document to create an app that lets me do the following...
JavaScript and HTML app on one page. User can select multiple image files on their own computer...
Write a JavaScript function like this: executeTemplates(template, arrayOfObjects)...
Cleaning up the transcript with Claude
Since the video was already up on YouTube when I started writing the annotations, I decided to see if I could get a head start on writing them using the YouTube generated transcript.
I used my Action Transcription tool to extract the transcript, but it was pretty low quality - you can see a copy of it here. A sample:
okay hey everyone it's uh really
exciting to be here so yeah I call this
court talk catching up on the weird
world of llms I'm going to try and give
you the last few years of of llm
developments in 35 minutes this is
impossible so uh hopefully I'll at least
give you a flavor of some of the weirder
corners of the space because the thing
about language models is the more I look
at the more I think they're practically
interesting any particular aspect of
them anything at all if you zoom in
there are just more questions there are
just more unknowns about it there are
more interesting things to get into lots
of them are deeply disturbing and
unethical lots of them are fascinating
it's um I've called it um it's it's
impossible to tear myself away from this
I I just keep on keep on finding new
aspects of it that are interesting
It's basically one big run-on sentence, with no punctuation, little capitalization and lots of umms and ahs.
Anthropic's Claude 2 was released last month and supports up to 100,000 tokens per prompt - a huge improvement on ChatGPT (4,000) and GPT-4 (8,000). I decided to see if I could use that to clean up my transcript.
I pasted it into Claude and tried a few prompts... until I hit upon this one:
Reformat this transcript into paragraphs and sentences, fix the capitalization and make very light edits such as removing ums
This worked really, really well! Here's the first paragraph it produced, based on the transcript I show above:
Okay everyone, it's really exciting to be here. Yeah I call this talk "Catching Up on the Weird World of LLMs." I'm going to try and give you the last few years of LLMs developments in 35 minutes. This is impossible, so hopefully I'll at least give you a flavor of some of the weirder corners of the space. The thing about language models is the more I look at them, the more I think they're practically interesting. Focus on any particular aspect, and there are just more questions, more unknowns, more interesting things to get into.
Note that I said "fractally interesting", not "practically interesting" - but that error was there in the YouTube transcript, so Claude picked it up from there.
Here's the full generated transcript.
It's really impressive! At one point it even turns my dialogue into a set of bullet points:
Today the best are ChatGPT (aka GPT-3.5 Turbo), GPT-4 for capability, and Claude 2 which is free. Google has PaLM 2 and Bard. Llama and Claude are from Anthropic, a splinter of OpenAI focused on ethics. Google and Meta are the other big players.
Some tips:
OpenAI models cutoff at September 2021 training data. Anything later isn't in there. This reduces issues like recycling their own text.
Claude and Palm have more recent data, so I'll use them for recent events.
Always consider context length. GPT has 4,000 tokens, GPT-4 has 8,000, Claude 100,000.
If a friend who read the Wikipedia article could answer my question, I'm confident feeding it in directly. The more obscure, the more likely pure invention.
Avoid superstitious thinking. Long prompts that "always work" are usually mostly pointless.
Develop an immunity to hallucinations. Notice signs and check answers.
Compare that to my rambling original to see quite how much of an improvement this is.
But, all of that said... I specified "make very light edits" and it clearly did a whole lot more than just that.
I didn't use the Claude version directly. Instead, I copied and pasted chunks of it into my annotation tool that made the most sense, then directly edited them to better fit what I was trying to convey.
As with so many things in LLM/AI land: a significant time saver, but no silver bullet.
For workshops, publish the handout
I took the Software Carpentries instructor training a few years ago, which was a really great experience.
A key idea I got from that is that a great way to run a workshop is to prepare an extensive, detailed handout in advance - and then spend the actual workshop time working through that handout yourself, at a sensible pace, in a way that lets the attendees follow along.
A bonus of this approach is that it forces you to put together a really high quality handout which you can distribute after the event.
I used this approach for the 3 hour workshop I ran at PyCon US 2023: Data analysis with SQLite and Python. I turned that into a new official tutorial on the Datasette website, accompanied by the video but also useful for people who don't want to spend three hours watching me talk!
More people should do this
I'm writing this in the hope that I can inspire more people to give their talks this kind of treatment. It's not a zero amount of work - it takes me 2-3 hours any time I do this - but it greatly increases the longevity of the talk and ensures that the work I've already put into it provides maximum value, both to myself (giving talks is partly a selfish act!) and to the people I want to benefit from it.
Weeknotes: Plugins for LLM, sqlite-utils and Datasette - 2023-08-05
The principle theme for the past few weeks has been plugins.
Llama 2 in LLM via plugins
I added the ability to support models other than the OpenAI ones to my LLM command-line tool last month. The timing on this could not have been better: Llama 2 (the first commercially usable version of Meta's LLaMA language model) was released on July 18th, and I was able to add support to prompting it via LLM that very morning thanks to the llm-replicate plugin I had released the day before that launch.
(I had heard a tip that a new exciting LLM was about to be released on Replicate, though I didn't realize it was Llama 2 until after the announcement.)
A few days ago I took that a step further: the new llm-llama-cpp plugin can now be used to run a GGML quantized version of the Llama 2 model directly on your own hardware.
LLM is available in Homebrew core now, so getting Llama 2 working is as simple as:
brew install llm
llm install llm-llama-cpp llama-cpp-python
llm llama-cpp download-model \
https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/resolve/main/llama-2-7b-chat.ggmlv3.q8_0.bin \
--alias llama2-chat --alias l2c --llama2-chat
Then:
llm -m l2c 'Tell me a joke about a llama'
I wrote more about this in Run Llama 2 on your own Mac using LLM and Homebrew - including instructions for calling Llama 2 using the LLM Python API as well.
Plugins for sqlite-utils
My sqlite-utils project, similar to LLM, is a combined CLI tool and Python library. Based on requests from the community I adding plugin support to it too.
There are two categories of plugins so far: plugins that add extra commands to the sqlite-utils
CLI tool, and plugins that add extra custom SQL functions that can be executed against SQLite.
There are quite a few plugins listed in the sqlite-utils plugins directory already.
I built sqlite-utils-shell in time for the initial launch, to help demonstrate the new system by adding a sqlite-utils shell
command that opens an interactive shell enabling any SQL functions that have been installed by other plugins.
Alex Garcia suggested I look at litecli by Amjith Ramanujam, which is a much more sophisticated terminal shell for SQLite, incorporating auto-completion against tables and columns.
I used that to build a better alternative to my sqlite-utils-shell
plugin: sqlite-utils-litecli, which lets you run the following command to get a full litecli
shell with all of the custom SQL functions from other plugins:
sqlite-utils litecli mydatabase.db
datasette-auth-tokens and dclient
Meanwhile, in Datasette land... I've been investing more time building Datasette Cloud, the SaaS cloud hosted version of Datasette.
The Datasette 1.0 alphas introduced a write API. I wanted a mechanism for Datasette Cloud users to be able to setup automatic imports of data into their instances, taking advantage of that API.
This meant I needed an API key mechanism that allowed tokens to be both created and revoked interactively.
I ended up building that into the existing datasette-auth-tokens
plugin, released in preview in the datasette-auth-tokens 0.4a0 alpha.
I've been quietly working on a new CLI utility for interacting with Datasette instances via the API, called dcloud
. I shipped dcloud 0.2 with a new dclient insert
command that can read CSV, TSV or JSON data and write it to an external Datasette instance using that new 1.0 write API.
I'll have more news to share about Datasette Cloud soon!
Large Language Model talk at North Bay Python
On Sunday I gave the closing talk at North Bay Python, titled Catching up on the weird world of LLMs.
I tried to summarize the last few years of development in the field of LLMs in just 40 minutes. I'm pretty happy with how it turned out! I've since published a full annotated transcript of the talk, with slides, additional links and notes - so even if you don't want to watch the full talk you can still read through a thorough summary of what I covered.
I've given a few of my talks this treatment now and I really like it - it's a great way to unlock as much value as possible from the time I spend putting one of these things together.
Examples of this format:
Catching up on the weird world of LLMs - from this Sunday.
Prompt injection explained, with video, slides, and a transcript for a LangChain webinar in May 2023.
Coping strategies for the serial project hoarder for DjangoCon US 2022.
How to build, test and publish an open source Python library for PyGotham in November 2021
Datasette—an ecosystem of tools for working with small data for PyGotham 2020.
Personal Data Warehouses: Reclaiming Your Data for the GitHub OCTO speaker series in November 2020.
Redis tutorial at NoSQL Europe - this was the first time I put together annotated slides like this, for a three hour tutorial on Redis presented at NoSQL Europe back in 2010.
This time round I built a small tool to help me assemble the notes and alt attributes for the video - I hope to write more about that soon.
Blog entries these weeks
Releases these weeks
llm-llama-cpp 0.1a0 - 2023-08-01
LLM plugin for running models using llama.cppdatasette-upload-dbs 0.2 - 2023-08-01
Upload SQLite database files to Datasettesqlite-utils-litecli 0.1.1 - 2023-07-26
Interactive shell for sqlite-utils using liteclillm-gpt4all 0.1.1 - 2023-07-25
Plugin for LLM adding support for the GPT4All collection of modelsdclient 0.2 - 2023-07-24
A client CLI utility for Datasette instancesllm 0.6.1 - 2023-07-24
Access large language models from the command-lineasgi-replay 0.1a0 - 2023-07-24
Record and replay ASGI web page loadssqlite-utils-shell 0.2 - 2023-07-24
Interactive shell for sqlite-utilssqlite-utils-dateutil 0.1 - 2023-07-24
Date utility functions for sqlite-utilssqlite-migrate 0.1a1 - 2023-07-23
A simple database migration system for SQLite, based on sqlite-utilssqlite-utils 3.34 - 2023-07-22
Python CLI utility and library for manipulating SQLite databasesllm-replicate 0.3 - 2023-07-20
LLM plugin for models hosted on Replicatesymbex 1.3 - 2023-07-19
Find the Python code for specified symbolsdatasette-auth-tokens 0.4a0 - 2023-07-17
Datasette plugin for authenticating access using API tokens
TIL these weeks
Checking if something is callable or async callable in Python - 2023-08-04
axe-core and shot-scraper for accessibility audits - 2023-07-30
Exploring the Overture Maps places data using DuckDB, sqlite-utils and Datasette - 2023-07-27
Protocols in Python - 2023-07-26
Using pytest-httpx to run intercepted requests through an in-memory Datasette instance - 2023-07-25
TIL 2023-07-26 Protocols in Python:
Datasette currently has a few API internals that return sqlite3.Row
objects. I was thinking about how this might work in the future - if Datasette ever expands beyond SQLite (plugin-provided backends for PostgreSQL and DuckDB for example) I'd want a way to return data from other stores using objects that behave like sqlite3.Row
but are not exactly that class. …
Quote 2023-07-26
Much of the substance of what constitutes “government” is in fact text. A technology that can do orders of magnitude more with text is therefore potentially massively impactful here. [...] Many of the sub-tasks of the work of delivering public benefits seem amenable to the application of large language models to help people do this hard work.
TIL 2023-08-04 Checking if something is callable or async callable in Python:
I wanted a mechanism to check if a given Python object was "callable" - could be called like a function - or "async callable" - could be called using await obj()
. …
Link 2023-08-04 You can stop using user-scalable=no and maximum-scale=1 in viewport meta tags now: Luke Plant points out that your meta viewport tag should stick to just "width=device-width, initial-scale=1" these days - the user-scalable=no and maximum-scale=1 attributes are no longer necessary, and have a negative impact on accessibility, especially for Android users.
Link 2023-08-06 Python cocktail: mix a context manager and an iterator in equal parts: Explanation of a neat trick used by the Tenacity Python library, which provides a mechanism for retrying a chunk of code automatically on errors up to three times using a mixture of an iterator and a context manager to work around Python's lack of multi-line lambda functions.
TIL 2023-08-08 Scroll to text fragments:
I ran a Google search this morning for s3-credentials simon willison
and when I clicked on the top result it jumped me to a highlighted snippet of text on the page, despite that page not having relevant anchor links. …
Link 2023-08-09 Llama from scratch (or how to implement a paper without crying): Brian Kitano implemented the model described in the Llama paper against TinyShakespeare, from scratch, using Python and PyTorch. This write-up is fantastic - meticulous, detailed and deeply informative. It would take several hours to fully absorb and follow everything Brian does here but it would provide multiple valuable lessons in understanding how all of this stuff fits together.
Link 2023-08-09 Datasette 1.0a3: A new Datasette alpha release. This one previews the new default JSON API design that's coming in 1.0 - the single most significant change in the 1.0 milestone, since I plan to keep that API stable for many years to come.
Link 2023-08-10 Getting creative with embeddings: Amelia Wattenberger describes a neat application of embeddings I haven't seen before: she wanted to build a system that could classify individual sentences in terms of how "concrete" or "abstract" they are. So she generated several example sentences for each of those categories, embedded then and calculated the average of those embeddings.
And now she can get a score for how abstract vs concrete a new sentence is by calculating its embedding and seeing where it falls in the 1500 dimension space between those two other points.
TIL 2023-08-10 Running a Django and PostgreSQL development environment in GitHub Codespaces:
Helping people setup development environments (and fix them when they break) can be incredibly frustrating. I'm really excited about cloud-based development environments such as GitHub Codespaces for exactly this reason - I love the idea that you can get a working environment by clicking a green button, and if it breaks you can throw it away and click the button again to get a brand new one. …
TIL 2023-08-10 Catching up with the Cosmopolitan ecosystem:
I caught up with some of the latest developments in the ecosystem around Justine Tunney's cosmopolitan and Actually Portable Executable (APE) projects this week. They are absolutely fascinating. …
Link 2023-08-11 Shamir Secret Sharing: Cracking war story from Max Levchin about the early years of PayPal, in which he introduces an implementation of Shamir Secret Sharing to encrypt their master payment credential table... and then finds that the 3-of-8 passwords needed to decrypt it and bring the site back online don't appear to work.
Link 2023-08-11 Dependency Management Data: This is a really neat CLI tool by Jamie Tanna, built using Go and SQLite but with a feature that embeds a Datasette instance (literally shelling out to start the process running from within the Go application) to provide an interface for browsing the resulting database.
It addresses the challenge of keeping track of the dependencies used across an organization, by gathering them into a SQLite database from a variety of different sources - currently Dependabot, Renovate and some custom AWS tooling.
The "Example" page links to a live Datasette instance and includes video demos of the tool in action.
Link 2023-08-12 llm-mlc: My latest plugin for LLM adds support for models that use the MLC Python library - which is the first library I've managed to get to run Llama 2 with GPU acceleration on my M2 Mac laptop.
TIL 2023-08-12 Combined release notes from GitHub with jq and paginate-json:
Matt Holt asked: …
Link 2023-08-12 deno_python: A wildly impressive hack: deno_python uses Deno's FFI interface to load your system's Python framework (.dll/.dylib/.so) and sets up JavaScript proxy objects for imported Python objects - so you can run JavaScript code that instantiates objects from Python libraries and uses them to process data in different ways.
The latest release added pip support, so things like 'const np = await pip.import("numpy")' now work.
Link 2023-08-13 Lark parsing library JSON tutorial: A very convincing tutorial for a new-to-me parsing library for Python called Lark.
The tutorial covers building a full JSON parser from scratch, which ends up being just 19 lines of grammar definition code and 15 lines for the transformer to turn that tree into the final JSON.
It then gets into the details of optimization - the default Earley algorithm is quite slow, but swapping that out for a LALR parser (a one-line change) provides a 5x speedup for this particular example.
Link 2023-08-14 Write about what you learn. It pushes you to understand topics better.: Addy Osmani clearly articulates why writing frequently is such a powerful tool for learning more effectively. This post doesn't mention TILs but it perfectly encapsulates the value I get from publishing them.
TIL 2023-08-15 Storing and serving related documents with openai-to-sqlite and embeddings:
I decide to upgrade the related articles feature on my TILs site. Previously I calculated these using full-text search, but I wanted to try out a new trick using OpenAI embeddings for document similarity instead. …
TIL 2023-08-15 Building a blog in Django:
We launched the Datasette Cloud blog today. The Datasette Cloud site itself is a Django app - it uses Django and PostgreSQL to manage accounts, teams and soon billing and payments, then launches dedicated containers running Datasette for each customer. …
Quote 2023-08-15
Someone asked me today if there was a case for using React in a new app that doesn't need to support IE.
I could not come up with a single reason to prefer it over Preact or (better yet) any of the modern reactive Web Components systems (FAST, Lit, Stencil, etc.).
One of the constraints is that the team wanted to use an existing library of Web Components, but React made it hard. This is probably going to cause them to favour Preact for the bits of the team that want React-flavoured modern webdev.
It's astonishing how antiquated React is.
TIL 2023-08-16 Processing a stream of chunks of JSON with ijson:
A follow-up to Using OpenAI functions and their Python library for data extraction and Using the ChatGPT streaming API from Python. If I have a stream of chunks of a larger JSON document, how can I output full individual JSON objects as soon as they are available? …
Link 2023-08-16 Welcome to Datasette Cloud: We launched the Datasette Cloud blog today! The SaaS hosted version of Datasette is ready to start onboarding more users - this post describes what it can do so far and hints at what's planned to come next.
Link 2023-08-16 Introducing datasette-write-ui: a Datasette plugin for editing, inserting, and deleting rows: Alex García is working with me on Datasette Cloud for the next few months, graciously sponsored by Fly. We will be working in public, releasing open source code and documenting how to build a multi-tenant SaaS product using Fly Machines.
Alex's first project is datasette-write-ui, a plugin that finally lets you directly edit data stored inside Datasette. Alex wrote about the plugin on our new Datasette Cloud blog.
Quote 2023-08-16
llama.cpp surprised many people (myself included) with how quickly you can run large LLMs on small computers [...] TLDR at batch_size=1 (i.e. just generating a single stream of prediction on your computer), the inference is super duper memory-bound. The on-chip compute units are twiddling their thumbs while sucking model weights through a straw from DRAM. [...] A100: 1935 GB/s memory bandwidth, 1248 TOPS. MacBook M2: 100 GB/s, 7 TFLOPS. The compute is ~200X but the memory bandwidth only ~20X. So the little M2 chip that could will only be about ~20X slower than a mighty A100.
Link 2023-08-16 An Iowa school district is using ChatGPT to decide which books to ban: I'm quoted in this piece by Benj Edwards about an Iowa school district that responded to a law requiring books be removed from school libraries that include "descriptions or visual depictions of a sex act" by asking ChatGPT "Does [book] contain a description or depiction of a sex act?".
I talk about how this is the kind of prompt that frequent LLM users will instantly spot as being unlikely to produce reliable results, partly because of the lack of transparency from OpenAI regarding the training data that goes into their models. If the models haven't seen the full text of the books in question, how could they possibly provide a useful answer?
Link 2023-08-16 Running my own LLM: Nelson Minar describes running LLMs on his own computer using my LLM tool and llm-gpt4all plugin, plus some notes on trying out some of the other plugins.