Simon Willison

Open source developer building tools to help journalists, archivists, librarians and others analyze, explore and publish their data. https://datasette.io and many other #projects.

Public Key

npub13v97j0kknscwnf5pt87nsn7cxzxwfwl3dsu7ss8qsq7ukmqgwg8q84w5eu

NIP-05 Address

[email protected]

Profile Code

nprofile1qqsgkzlf8mtfcv8f56q4nlfcflvrpr8yh0ckcw0ggrsgq0wtdsy8yrspz4mhxue69uhhyetvv9ujumt0wd68ytnsw43q2zqhnf

Publishing to

relay.mostr.pub

Author Public Key

npub13v97j0kknscwnf5pt87nsn7cxzxwfwl3dsu7ss8qsq7ukmqgwg8q84w5eu

Show more details

Last Notes

2024-08-26 19:46:37

- reply

@npub1r05…qp54 I shipped a fix for the tool, the axis display correctly now - thanks!
https://cdn.masto.host/fedisimonwillisonnet/media_attachments/files/113/029/571/845/705/456/original/801a5e70e147ad36.png

2024-08-17 00:09:39

- reply

@npub172v…689n huh... that's the case, but now you mention it there's no reason I shouldn't start a habit of using "python -m pytest" instead

2024-08-13 18:41:16

- reply

@npub1074…z2j6 does that use the GPU?

2024-08-09 06:40:37

- reply

@npub1n7h…8755 I’ve also recently started using the VS Code feature where you can select a bunch of lines and click the little sparkle icon and give it a prompt telling it how to modify the selected code

2024-08-06 06:59:47

- reply

@npub19en…k643 I wouldn’t expect it to!

2024-08-01 23:52:20

- reply

As for the original set of questions, we're nowhere near closer to answering them!
The earliest mentions we have found of this August 1st "World Wide Web Day" were on August 1st 2008, both on Twitter and in at least one news story, and possibly in various "fun things about today" calendars that people were consulting then
Nothing at all prior to 2008 yet

2024-08-01 21:15:42

- reply

@npub1kg2…ckz0 weird how we keep finding references from 2008 but nothing before that

2024-08-01 19:59:16

- reply

@npub1skj…mjh7 I just found two earlier than that - Twitter from 2008 https://fedi.simonwillison.net/@simon/112888061517636624

2024-08-01 18:13:14

- reply

Anyone got access to LexisNexis or similar news archive search? Would be useful to run a phrase search and fine the very first news articles that referenced it

2024-07-31 05:59:17

- reply

@npub1e7a…tc2x whoa I think it is! GitHub profile photo matches LinkedIn: https://www.linkedin.com/in/paulgauthier/

2024-07-31 05:54:39

- reply

@npub13zy…3zww I know I shouldn't let this bother me so much, but /blog.html is a REALLY bad URL for a blog. And no permalinks for posts, no RSS feed. That's a very bad blog.

2024-07-30 08:27:13

- reply

@npub1wrc…0xlc the one product category I _really_ don't want companies to deprecate is a proprietary hosted database that they encourage me to build my own business on top of!

2024-07-30 08:06:37

- reply

Here's a handy GitHub repo that tracks AWS's uncommon (but evidently increasingly frequent) backwards compatibility breaks: https://github.com/SummitRoute/aws_breaking_changes
It even provides an Atom feed, neatly powered by the GitHub releases feature

2024-07-30 08:04:16

I have a mental model of AWS as almost never breaking anything (my S3 bucket has worked for 15+ years now), but I may need to update that model.
AWS quietly deprecated both AWS CodeCommit (Git hosting) and AWS QLDB (a blockchain database thing) in the past few weeks. Notes here: https://simonwillison.net/2024/Jul/30/aws-codecommit-quietly-deprecated/

2024-07-29 21:52:48

I'm fascinated by how difficult LLM-based AI systems are proving to use
Makes me realize that the Star Trek ship computers are actually a pretty good representation of this: you need to have a very high level of experience to know to say things like "Computer: reroute auxiliary power to the forward shield emitters"
Much like getting ChatGPT et al to reliably-enough perform useful tasks for you!

2024-07-24 00:28:40

- reply

@npub1470…h68t there is one significant change in terms of openness - Llama 3.1 dropped the license term from Llama 3 that says you can't use it to help create or improve other models

2024-07-20 20:27:43

- reply

We bought these three beauties for $45 for the lot - $25 for the white one and $10 each for the other two
Next time I want to go back with a pickup truck!
https://cdn.masto.host/fedisimonwillisonnet/media_attachments/files/112/820/227/051/326/678/original/ea63a821cdc54180.jpeg
https://cdn.masto.host/fedisimonwillisonnet/media_attachments/files/112/820/227/773/891/853/original/deefaa50e06b7982.jpeg

2024-07-19 02:15:48

Weeknotes: GPT-4o mini, LLM 0.15, sqlite-utils 3.37 and building a staging environment https://simonwillison.net/2024/Jul/19/weeknotes/

2024-07-18 20:23:53

My notes on today's release of GPT-4o mini: https://simonwillison.net/2024/Jul/18/gpt-4o-mini/
The biggest news is the price: this is cheaper even than Claude 3 Haiku, at just 15c per million input tokens / 60c per million output, a 99% reduction on the prices of GPT-3 Da Vinci from 2022!
Also notable: the model has 128,000 input tokens and 16,000 output tokens, twice the output tokens of Claude 3.5 Sonnet, making it a better fit for translation and transformation tasks where output length more closely matches the input

2024-07-18 19:51:45

Randomly reminded that Mrs. Davis (the show about the nun going to war with an AI) is my favourite fictional depiction of AI since Person of Inferest, and just a wildly entertaining and unpredictable show
... and the reveal at the end about the origins of the AI is the most relatable scene about software engineering I've ever seen in any medium

2024-07-17 05:32:07

I'm speaking at DjangoCon this year! My talk title is "How to design and implement extensible software with plugins" - full abstract here: https://simonwillison.net/2024/Jul/17/djangocon-us-2024-talks/
https://mastodon.social/@djangonews/112799357740006213

2024-07-16 19:07:28

Two new Mistral models today: Codestral Mamba is an Apache 2 licensed 7B code model using a Mamba architecture (not transformers!), and MathΣtral is for "math reasoning and scientific discovery"
My notes here, including an update to my llm-mistral plugin: https://simonwillison.net/2024/Jul/16/codestral-mamba/

2024-07-11 03:31:44

- reply

@npub1pfe…e2dm the publication already dropped the writer's names after they complained, now they have made-up names with bylines against LLM-reworded versions of the old content instead!

2024-07-11 01:51:37

Wow this is some diabolical slop... TUAW closed in 2015, now a new company has purchase the brand (but not the content) and then rewrote the old content with LLMs and put up AI generated images in place of the original authors! https://www.theverge.com/2024/7/10/24195858/tuaw-unofficial-apple-tech-blog-ai-web-orange-khan

2024-07-08 17:36:05

- reply

@npub1j4z…gfym for a super-low-effort version you could dump it all into a private (or public) GitHub repository and use their search tool, that's pretty good these days

2024-07-06 21:03:17

This may be the nicest thing anyone has ever said about my software! I'm going to be chuffing about this all weekend
https://mastodon.me.uk/@fantasticlife/112738153601043450

2024-07-04 07:53:06

- reply

The code is all open source, so you can see a commit log of recent tweaks I've made on GitHub https://github.com/simonw/simonwillisonblog/commits/1d8b51655fe39324d2522ed9bb7a0393249796e6/
I wrote about the larger changes in my blog birthday post last month https://simonwillison.net/2024/Jun/12/homepage-redesign/

2024-07-04 07:47:41

- reply

@npub16te…hs2y only very loosely! I would be delighted if someone else figured out the details before I did

2024-07-03 02:20:18

- reply

@npub13m5…lese thanks, fixed!

2024-07-02 22:12:12

Today in LLM-enabled side quests: I got Claude 3.5 Sonnet to build me a web UI for visually comparing PDF files using a couple of short prompts - result is here: https://tools.simonwillison.net/compare-pdfs
Further notes (including the prompts and a link to the full Claude transcript): https://simonwillison.net/2024/Jul/2/compare-pdfs/
https://cdn.masto.host/fedisimonwillisonnet/media_attachments/files/112/718/713/164/446/957/original/2b9f013d42cb8331.png

2024-07-02 07:06:31

- reply

@npub1eug…vkjz technically not a lot, but it won't have its fate tied to the whims of Google's ongoing funding - Chris talks more about the rationale for the project here: https://youtu.be/k9edTqPMX_k

2024-07-01 19:59:46

The new Ladybird initiative is exciting: a new us non-profit initially backed by $1m to support development of Ladybird, an open source web browser that isn't based on a fork of something older and that doesn't depend on Google for funding! https://simonwillison.net/2024/Jul/1/the-ladybird-browser-initiative/

2024-06-29 20:46:59

- reply

@npub1cxy…sd92 last year they took a few days after the conference to slice up the livestream into separate videos, I expect they'll do that soon

2024-06-29 07:40:57

PyCon US 2024 talk videos have started to be released on YouTube - a bunch of the tutorials from the conference are now available on the PyCon channel https://youtube.com/@pyconus

2024-06-27 21:30:00

- reply

@npub1vqk…x7gx that's why I describe it as a challenge for the whole industry - regaining trust is going to be a lot of hard work

2024-06-27 19:08:47

- reply

Here's the full annotated write-up of my keynote from yesterday: https://simonwillison.net/2024/Jun/27/ai-worlds-fair/
Topics include:
- GPT-4 has competition! GPT-4 class models are becoming a commodity
- The AI trust crisis
- We still haven't solved prompt injection
- The best models are now free to consumers
- Slop, and taking accountability for our AI-assisted content

2024-06-26 20:17:36

- reply

@npub1h85…7jhr I don't want to add Rich as a core dependency but I'd love to have it work as a plugin
I think Datasette needs a new plugin hook to support that, want to hang out on Discord and figure out the details?

2024-06-26 18:17:15

I'm a late addition to the AI World's Fair morning lineup - I'll be speaking on the main stage (and the YouTube livestream) in a few minute's time https://www.youtube.com/live/5zE2sMka620?si=smOeQA9LCijwFbaa

2024-06-26 08:35:18

- reply

@npub1k0j…e8k9 that sadly doesn't work for polyfill.js because the whole idea there was to serve different JavaScript based on the user-agent header, so browsers that support features don't get sent polyfills

2024-06-26 00:31:30

Worth grepping your source code for "polyfill.io" and taking urgent measures to remove that code if you're linking it into your site - the domain name apparently now intermittently serves malicious JavaScript
My notes here: https://simonwillison.net/2024/Jun/25/polyfill-supply-chain-attack/ - or read this article https://sansec.io/research/polyfill-supply-chain-attack

2024-06-25 20:19:07

Published some notes on the new Claude Projects feature, which lets Claude Plus users create a Claude with custom instructions and up to 200,000 tokens of additional data https://simonwillison.net/2024/Jun/25/claude-projects/

2024-06-24 23:51:40

The BBC appear to have restricted The Archers to only listeners located in the UK.
The relevant Facebook group spotted this 42 minutes ago https://www.facebook.com/groups/1133707723696454/posts/2135091543558062
This is going to flare up into an international incident
https://cdn.masto.host/fedisimonwillisonnet/media_attachments/files/112/673/809/841/912/106/original/b59f9545951a0f08.png

2024-06-24 20:52:46

- reply

@npub1hvy…56uc taking down an old bug tracker without at least carefully archiving the contents feels like a sacrilegious act of vandalism to me!

2024-06-24 20:51:18

- reply

@npub1d8a…tcc0 I do that pretty often, I think it's a good pattern - example here https://github.com/simonw/datasette/blob/c2e8e5085b5ccc49121feaf11b57eb7d5af53bd9/datasette/renderer.py#L110

2024-06-24 18:16:33

- reply

@npub1ach…vx0y that's one of the reasons this works so well, the issue thread link can show up in git blame years later

2024-06-24 17:24:55

Often the best code documentation is a commit message that links to an issue thread full of notes and screenshots

2024-06-24 06:59:03

- reply

@npub1pfe…e2dm I first came around to them in Python when I realized that they added a succinct and unambiguous level of API documentation that might not get clearly captured otherwise

2024-06-24 06:45:41

It took me an embarrassingly long time to realize that type hints (in Python and in TypeScript) aren't so much about avoiding accidental type errors at runtime... they're much more about enabling really good editor autocomplete

2024-06-24 00:11:07

- reply

@npub1vs2…wqqp it can run Tetris! https://www.youtube.com/watch?v=Ms1Drb9Vw9M&t=1376s

2024-06-23 19:34:08

- reply

@npub1g3e…ca46 I should hope not!
WebAssembly code can't access a network unless you explicitly expose a method to it inside the sandbox that can do that

2024-06-23 18:42:48

- reply

@npub1g3e…ca46 that's the selling point of WebAssembly: it's genuinely a robust sandbox, proven in the hostile adversarial environment faced by Chrome and Safari and Firefox since they shipped it in 2017

2024-06-23 18:40:34

Claude's new "artifacts" feature - where LLM-generated HTML and JavaScript is shown in an iframe directly within their UI - may be the inspiration I need to finally get over my security paranoia about iframe sandboxing
The iframe sandbox= attribute has existed for more than a decade! And I imagine it's used by ad networks billions of times a day, so it should have very robust browser implementations by now

2024-06-23 18:22:32

Here's a brilliant hack... llama.ttf https://fuglede.github.io/llama.ttf/ is a font with a LLM embedded in it (using WebAssembly, which apparently can be used in fonts now)
You type "Once upon a time!!!!!!!!!!" and those exclamation marks are rendered to show the LLM generated text, using a tiny 30MB model
I wrote more notes on how it works (including a link to another font that can be used to play Tetris) here: https://simonwillison.net/2024/Jun/23/llama-ttf/

2024-06-22 16:11:47

- reply

@npub1zu8…htr2 the kids these days figured out that Instagram and TikTok don't allow links and Twitter and LinkedIn downrank them in "the algorithm" 😕

2024-06-22 16:07:17

- reply

@npub1eee…e209 I love the idea of thinking if it "sounds" right, I'm going to start trying that myself

2024-06-22 16:03:29

- reply

@npub1lwu…6w4j fantastic!

2024-06-22 16:02:50

- reply

@npub1j08…7sse of course GDS would have a thoughtful page about this!

2024-06-22 04:33:54

- reply

(I'm increasingly suspicious that hardly anyone ever clicks on inline links in text like this, but that's not going to stop me from including them!)

2024-06-22 04:32:32

Anyone else spend way too much time thinking about the grammar of linking to things?
In this fragment I'm not at all confident about the "backs up that data" link, maybe I should have included "as JSON" in that link...
Anyone seen a comprehensive style guide that addresses this?
Extract from https://simonwillison.net/2024/Jun/21/search-based-rag/#ingredients-for-rag
https://cdn.masto.host/fedisimonwillisonnet/media_attachments/files/112/657/927/145/698/410/original/266a67593b28a274.jpeg

2024-06-21 23:21:56

Here are my detailed notes (and video) on implementing search-based RAG question answering using Claude 3.5 Sonnet, https://val.town/ and Datasette
I extracted the notes and code from the livestream I recorded with Val.Town founder Steve Krouse yesterday morning, where we hacked everything together in about an hour https://simonwillison.net/2024/Jun/21/search-based-rag/
You don't need fancy vector search to build RAG! Good old-fashioned BM25 full-text search (as seen in SQLite) is more than capable on its own

2024-06-20 21:25:59

Spotted this at the bottom of https://www.anthropic.com/news/claude-3-5-sonnet which is a much more clear statement on privacy than I've seen from most other model providers:
> One of the core constitutional principles that guides our AI model development is privacy. We do not train our generative models on user-submitted data unless a user gives us explicit permission to do so. To date we have not used any customer or user-submitted data to train our generative models.

2024-06-20 21:07:15

- reply

@npub1jsv…nekl At least it had an actual relevant punchline! Most LLM jokes I've tried don't even get that far

2024-06-20 21:06:41

- reply

@npub1y4p…vsfq I've never looked at how it interacts with other tools to be honest - it manages a virtual-environment-per-tool for me and I let it do whatever it wants to do

2024-06-20 20:18:53

It looks like the new Claude 3.5 Sonnet model is the new best-available LLM - same price as GPT-4o but longer context and appears to be more capable. Some notes here: https://simonwillison.net/2024/Jun/20/claude-35-sonnet/
The https://claude.ai/ UI to it can now build and then render on-demand web apps - here's what I got when I prompted "Build me a web app that teaches me about mandelbrot fractals, with interactive widgets"
https://cdn.masto.host/fedisimonwillisonnet/media_attachments/files/112/650/323/147/733/593/original/d2d9d0afce245ab7.mp4

2024-06-20 07:03:44

- reply

Some fascinating details about the development of the underlying machine learning models in this post https://www.holovaty.com/writing/machine-learning-thoughts/

2024-06-20 06:29:46

- reply

If you haven't seen Soundslice before here's a neat demonstration of the player, which synchronizes video and sheet music https://www.soundslice.com/slices/RXTDc/course-preview-5904/
The next level magic is the sheet music editor, but you'll have to create a free account to try that out

2024-06-20 05:59:08

- reply

@npub18sf…x6dg thanks, hadn't tried either of those - added them to my TIL as alternatives

2024-06-20 04:35:35

New TIL: Running Prettier against Django or Jinja templates https://til.simonwillison.net/npm/prettier-django

2024-06-19 06:59:07

Weeknotes: Datasette Studio and a whole lot of blogging https://simonwillison.net/2024/Jun/19/datasette-studio/

2024-06-18 16:45:00

- reply

@npub1klj…n44l love that!

2024-06-17 23:58:31

- reply

@npub1dmp…w6du I'm constantly paranoid about whether daylight savings time has kicked in for different parts of the world

2024-06-17 23:23:27

What are people's favourite timezone meeting planner tools? I want a nice cleanly designed website where I can get a page that shows the proposed meeting time in multiple locations such that I can link people directly to that page

2024-06-17 20:24:52

- reply

I included a colophon at the end talking about the tools I used to help put together the annotated version of the talk (made trickier by the fact that this talk was all demos, no slides so I couldn't start with the slide deck) https://simonwillison.net/2024/Jun/17/cli-language-models/#llm-colophon

2024-06-17 18:55:12

Here are detailed notes (an annotated presentation) to accompany the video of the talk I gave last week on using Large Language Models on the command-line https://simonwillison.net/2024/Jun/17/cli-language-models/

2024-06-16 05:12:27

- reply

@npub1h7e…cldv Google had CSP headers for Bard but turned out that exposed an exfiltration vector via AppScript https://simonwillison.net/2023/Nov/4/hacking-google-bard-from-prompt-injection-to-data-exfiltration/

2024-06-16 00:48:54

While searching my blog for any previous mentions of Phanpy I was delighted to find that I linked to previous work by @npub1jc6…7v67 21 years ago when they created icons for the Phoenix browser (the previous name for Firefox) https://simonwillison.net/2003/Feb/6/aBetterPhoenixIcon/

2024-06-14 16:32:02

- reply

@npub1ssf…50fw bit far from California for me!

2024-06-13 22:29:16

- reply

@npub104j…6ku4 wow, that's huge! What happens if you exceed $50 of CDN traffic in a month on the free plan, do your sites go offline until you add billing details? (That would make total sense to me)

2024-06-13 05:50:09

The https://phanpy.social/ timeline browsing experience is genuinely the first time I've actively enjoyed scrolling through my Mastodon timeline, it feels really fresh and the way it groups boosts together is a huge improvement on the default chronological lists from other tools

2024-06-13 05:47:43

- reply

@npub1ztm…t758 does the prompt to ChatGPT include details on how much each firm being covered has paid the analyst firm in "advisory fees"?

2024-06-12 22:37:33

- reply

@npub1ah5…zc28 I use it for when I want a single place with issues that live in multiple repositories - I've used it heavily in the past, not really using it actively at the moment but I imagine I'll get back to it soon

2024-06-12 22:24:51

- reply

The GitHub issue for this is representative of how I use GitHub issues across all of my projects - open for about half an hour, dropped in a couple of screenshots and a linked commit, closed it again https://github.com/simonw/simonwillisonblog/issues/438

2024-06-12 22:01:17

Celebrated my blog's 22nd birthday (!) with a small redesign of the homepage https://simonwillison.net/2024/Jun/12/homepage-redesign/

2024-06-12 17:15:12

Generative AI Is Not Going To Build Your Engineering Team For You by @npub1s6r…lrw5 is an absolute barnstormer of an essay, if you only read one thing today it should be this:
https://stackoverflow.blog/2024/06/10/generative-ai-is-not-going-to-build-your-engineering-team-for-you/

2024-06-11 17:57:51

Wrote up some of my own notes on Apple's Private Cloud Compute and foundation language models https://simonwillison.net/2024/Jun/11/

2024-06-11 00:56:59

- reply

Here are some of the technical details of the Private Cloud Compute platform and WOW that is some cutting edge computer science https://security.apple.com/blog/private-cloud-compute/

2024-06-10 23:15:33

- reply

@npub1l5a…7axw that is an excellent idea!

2024-06-10 22:23:49

Wrote up some initial thoughts on the Apple Intelligence WWDC announcements today:
- On-device models that can outsource to Apple’s servers with cryptographic guarantees of privacy are really interesting
- Restricting image generation to three approved styles (sketch, illustration and animation) avoids a whole host of complicated ethical concerns
- App Intents look like a minefield for prompt injection
https://simonwillison.net/2024/Jun/10/apple-intelligence/

2024-06-10 18:16:08

- reply

I'm particularly interested in hearing from people who have a SQLite version earlier than 3.24.0 (released 2018-06-04), which is when they added upsert support - but I'm interested in all answers since it's going to be fun to have a better idea of what's out there in the wild

2024-06-10 18:06:28

I want to run a quick informal survey on SQLite versions people have in their Python environments... could you run the following command in a terminal and reply with the output?
python -c 'import sys, sqlite3, platform; print(sqlite3.connect(":memory:").execute("select sqlite_version()").fetchone(), sys.version, platform.platform())'
I get this:
('3.45.2',) 3.10.14 (main, Mar 19 2024, 21:46:16) [Clang 15.0.0 (clang-1500.3.9.4)] macOS-14.1.1-arm64-arm-64bit

2024-06-10 08:02:12

- reply

@npub1gx0…9g2h the distance floor thing is hard because you can't really pick a number and use that - the relevance scores are so unpredictable that I can't figure out how to pick a useful cutoff

2024-06-09 03:29:09

- reply

@npub1808…98hl hah, getting it to compile a C program that fires SQL queries through SQLite SELECT queries to add two numbers together is a delightfully obtuse way to solve that problem!

2024-06-09 02:21:13

- reply

And thanks to my blog's tags I can see that I've been linking to stuff by @npub19jj…zmpa for just shy of 21 years https://simonwillison.net/tags/kellanelliottmccrea/

2024-06-09 02:19:48

- reply

Blogged about @npub19jj…zmpa's new link blog on my link blog https://simonwillison.net/2024/Jun/9/a-link-blog-in-the-year-2024/

2024-06-08 21:28:09

- reply

This isn't a big surprise - I've had it write me SQLite extensions in C for use with Python in the past - but being able to write standalone C programs that don't use any Python at all feels like it could be a lot of fun https://simonwillison.net/2024/Mar/23/building-c-extensions-for-sqlite-with-chatgpt-code-interpreter/

2024-06-08 21:26:42

This is fun: it turns out ChatGPT Code Interpreter can use gcc to compile and run C programs that link in the SQLite library and use it to execute SQL queries! https://chatgpt.com/share/197de2aa-935e-488e-84ca-145d3f7349ac
https://cdn.masto.host/fedisimonwillisonnet/media_attachments/files/112/582/642/728/200/474/original/28da4f114410ed9f.jpeg

2024-06-08 21:06:56

- reply

@npub1jzt…0d6s yeah I imagine they had them produce a LOT of those 15s samples, but it does sound like they then picked just one that gave them the effect they wanted for the voice in question

2024-06-08 20:16:04

Published some notes on OpenAI Voice Engine TTS on my blog
It turns out even their flagship TTS custom voices were created using just 15 second samples from paid voice actors!
(On top of untold quantities of undocumented training data for the underlying model)
https://simonwillison.net/2024/Jun/8/how-voice-engine-works/

2024-06-08 18:46:29

- reply

@npub1fvn…sghp wow, that's some impressive production values for ten years ago!

2024-06-08 02:17:02

What are some of the best Python tutorial videos you've seen? Ideally relatively short (5-20 minutes) but I'm interested in anything which you think illustrates a really great teaching style that makes the most of the video format