kero

Co-founder & CTO @meilisearch. Rust developer and ex-student from @42born2code.

Public Key

npub1xyypr4cduzmn6zwx9yuxjq2utyhrdnxpjz422pxc6y0hwaa652uqmy6vyg

NIP-05 Address

Profile Code

nprofile1qqsrzzq36ux7pdeap8rzjwrfq9w9jt3kenqep249qnvdz8mhw7a29wqpz3mhxue69uhhyetvv9ujuerpd46hxtnfduqs6amnwvaz7tmwdaejumr0dsjlpamt

Publishing to

relay.damus.io

nos.lol

Author Public Key

npub1xyypr4cduzmn6zwx9yuxjq2utyhrdnxpjz422pxc6y0hwaa652uqmy6vyg

Show more details

Last Notes

2026-03-13 13:03:46 GMT

Hey @nprofile…kj4x 👋 I plan to release a blog post about how we largely sped up embedding indexing with Meilisearch by patching LMDB. Would you mind taking a look at it before I publish (next week), please? Have a nice day. https://github.com/Kerollmops/blog/issues/23

2026-02-20 16:03:13 GMT

So cool 🤩 The @nprofile…jjs5 recording of Many is available. He talked about the language support and how we handle that in Meilisearch. I recommend it as a short start-of-the-weekend treat 🍬 https://fosdem.org/2026/schedule/event/GS7B8Y-meilisearch-laguage-support/

2026-02-07 11:13:24 GMT

- reply

@nprofile…kj4x based on my LMDB wrapper: heed! Nice 👌

2026-01-26 10:41:31 GMT

Happy New Year, @nprofile…kj4x 🎉
Did you have any time to review my proposals for adding nested read transactions?
It is working perfectly in Meilisearch (in production across 500+ instances) and is a non-breaking addition that unlocks tons of performance gains on multi-core machines (every machine these days). I made it C99-compatible by using mutexes, so it would be cool to take a look at my patch and tell me if you think I can improve anything.
Have a nice day 🌵
https://bugs.openldap.org/show_bug.cgi?id=10395#c9

2025-12-25 16:12:02 GMT

- reply

@nprofile…jg6v I don't even jnderstand what it is but looking at the tagged personality I don't know if I want to dive deeper...

2025-12-09 08:04:19 GMT

- reply

@nprofile…jg6v Ho! But it's committed to mdb.master3. However, it seems fairly easy to port on mdb.master.

2025-12-09 08:02:03 GMT

- reply

@nprofile…jg6v this looks very good actually! As we are working on replicated sharding at Meilisearch we may have a use for this new feature 👀

2025-12-09 07:55:26 GMT

- reply

@nprofile…jg6v yes! That's the article of Nate and I did the integration into Meilisearch + the LMDB shell around the vector store 💪

2025-12-09 07:54:02 GMT

Hey Mastodon people 👋 It’s been a while! This morning, we published a new article about how we made Meilisearch’s semantic search much faster with hannoy. Hannoy is a new #lmdb disk-based HNSW vector store that is much more perf. Now, it’s the default backend! #rustlang https://blog.kerollmops.com/from-trees-to-graphs-speeding-up-vector-search-10x-with-hannoy

2025-11-30 14:28:55 GMT

- reply

@nprofile…jg6v would it enable a way to get an estimated amount of entries between two keys? Because, I can see a use for Meilisearch where we use different algorithms depending on this number 🤔

2025-11-25 07:46:18 GMT

- reply

@nprofile…jg6v I mean all those posts are from the same persons (me) and Quentin, CEO or Meilisearch 😅

2025-11-08 10:39:12 GMT

Hey @nprofile…jg6v 👋 Have you seen this article? I am interested in what you think. I'm wondering if it would be easy to introduce transactions and MVCC. It seems that it could easily done with a good tracking with txnIDs and some locks, no? https://blog.wilsonl.in/blobd/

2025-09-10 09:38:53 GMT

Hey @nprofile…2cpy 👋 What do you think about this "new" kv-store? It seems to me that is is good but doesn't support, by default, durability on point-write, RMW operations as the operations are done in memory. The kv-store answers the user directly and without appending to a WAL. But it seems strange to me, maybe I missed something. I'm particularly curious about the reason why LMDB is not listed in the benchmarks. https://www.vldb.org/pvldb/vol18/p4910-kanellis.pdf

2025-08-21 07:21:07 GMT

- reply

@nprofile…2cpy Thanks for the explanation. So, the current implementation of the MDB_FIXED_MAP flag is unusable as any insertion into a database could break the mmap addresses, right?

2025-08-21 06:56:47 GMT

Hey @nprofile…2cpy, another LMDB question: I was thinking about using MDB_FIXED_MAP to store raw pointers to other entries from another entry but I don't think it will work as if I rewrite the pointed entry it will move somewhere else but it'll also happen if I modify an entry that is on the same page because LMDB rewrites entire leaf and overflow pages. Am I right? If so, what's the benefit of using MDB_FIXED_MAP?

2025-08-11 07:15:25 GMT

- reply

@nprofile…nwut Ho! That's a strange trick by @nprofile…2cpy to "compensate" the ugly design of C/C++ function interfaces and design... This can lead to a lot of issues. I don't know if we ever encountered an issue on Meilisearch running on Windows due to this 🤔

2025-08-04 06:13:11 GMT

- reply

@nprofile…2cpy, at Meilisearch we use LMDB for 6 years and we had only one or two bugs and another one was on our end. #LMDB rocks and that's it 💪

2025-07-21 10:34:14 GMT

- reply

@nprofile…2cpy yes, I was thinking about the same. Being able to defer balancing but you'll still need to rebalance the tree in a synchronous way from time to time and that'll be a "stop the world" operation. I think that an append-only WALL or SST format could benefit from io-uring. Sending many _APPEND this buffer to this file_ requests could work. Anyway, you still need to organize the data to search in it after...

2025-07-21 06:52:34 GMT

Hey @nprofile…2cpy 👋
I was reading this article and was wondering if LMDB would have any use of io_uring to batch syscalls when possible 🤔 LMDB writes either in an allocated buffer to be written to disk or directly in the mmapped file. Only the former uses direct syscalls. However, everytime a new entry is inserted the B+tree must be rebalanced, right? Batching is therefore complex as each page depends on another one, making the operation inherently sequential.
https://blog.canoozie.net/async-i-o-on-linux-and-durability/

2025-07-10 06:42:45 GMT

- reply

@nprofile…2cpy ...6 years ago...

2025-06-30 06:13:35 GMT

- reply

@nprofile…2cpy 600 lignes and most of it is boilerplate? That seems ok to me. However, you're right, they probably have good reasons: unshared benchmarks.

2025-06-29 17:47:32 GMT

- reply

@nprofile…2cpy yeah, you're right. The OS will load pages as needed and the only thing that changes now is the page format (leaf and overflow pages). However, this take you quote still need actual benchmarks. Tomorrow, I could also right a 10 word tweet saying the opposite...

2025-06-29 09:56:01 GMT

- reply

@nprofile…2cpy I don't know but there are some missing benchmarks here. Anyway, I don't think it's untrue as I think using Go to perform direct C calls requires more CPU operations due to the syscalls not having the right "shape" (stack and registers). Go needs to switch to C-mode or something because they have their own function calling format to support out-of-the-box green threading 🤔

2025-06-18 07:16:38 GMT

https://trifectatech.org/blog/bzip2-crate-switches-from-c-to-rust/

2025-06-10 09:06:34 GMT

- reply

@nprofile…2cpy generated by AI (claude code it seems). CI is broken, and they are not even using the latest version of heed (0.22)... I'll probably never try this tool on any production database at Meilisearch...

2025-05-10 08:07:39 GMT

Hey @nprofile…2cpy 👋 I wrote you there to ask you about LMDB intrinsics. Can you answer there, please? It's about being able to process database entries in parallel. https://github.com/meilisearch/heed/pull/334

2025-03-29 12:02:48 GMT

It happened! @meilisearch.com reached 50k stars on GitHub 🤩 we are still the second most-started search engine. Elasticsearch, we are coming for you. #rustlang
https://cdn.fosstodon.org/media_attachments/files/114/245/614/490/099/566/original/19c1d09c8e3a0386.png

2025-03-20 15:20:12 GMT

Today is a sad day 🥺 We will no longer use bors and switch to GitHub Merges Queues on @meilisearch.com. #rustlang https://github.com/meilisearch/meilisearch/pull/5447

2025-03-18 22:27:06 GMT

@nprofile…293t, I just read your original article about using c2rust to convert bzip2 to Rust, and I was wondering if you would recommend it to convert the LMDB source code. It does a lot of platform-specific operations, but it is also quite small (mdb.c, lmdb.h, mild.c, and mild.h only). Do you have more information than me on the converting tool from C to safe Rust? https://arxiv.org/abs/2412.15042

2025-03-18 16:50:44 GMT

- reply

> working on a rust language experiment
What do you mean by that? Will it be a macro or something more deep that modifies the compiler with keywords and gotos?

2025-03-18 11:18:47 GMT

Hey @nprofile…293t, I am reading your recent article on zlib-rs, "zlib-rs is faster than C," and I was wondering if you tried to use the fallthrough libraries proposed at the end of this article to fill the Rust switch syntax gap? https://huonw.github.io/blog/2025/03/rust-fallthrough/

2025-03-17 16:53:16 GMT

We have worked hard on Meilisearch for the last 2 years and have cooked you smt amazing! We mixed chocolate and some AI toppings to release Hybrid Search, something new on the market. Be ready and subscribe to our upcoming Product Hunt event 🛎️
https://www.producthunt.com/products/meilisearch-cloud
https://cdn.fosstodon.org/media_attachments/files/114/178/811/077/609/494/original/87ae9a4e1f43b3a9.png

2025-03-15 10:15:20 GMT

Just released heed & heed3 v0.22, my Rust #lmdb wrapper, which improves the doc, supports dup sort custom comparators, support the PREV_SNAPSHOT flag, better support RoTxn with TSL, and use the latest LMDB version. Meilisearch is already using it! Go star it 🌟 https://github.com/meilisearch/heed/releases/tag/v0.22.0

2025-03-15 10:14:48 GMT

Just released heed & heed3 v0.22, my Rust LMDB wrapper, which improves the doc, supports dup sort custom comparators, support the PREV_SNAPSHOT flag, better support RoTxn with TSL, and use the latest LMDB version. Meilisearch is already using it! Go star it 🌟 https://github.com/meilisearch/heed/releases/tag/v0.22.0

2025-03-10 09:14:42 GMT

- reply

@nprofile…2cpy I don't understand your point about libraries aborting a process. LMDB does precisely that in case an assertion is not valid. Panicking is the same thing as the mdb_cassert function being run in non-debug programs. The main difference is that panics can be watched in #Rust. You can see an example in Meilisearch: Sometimes, users have disk corruptions, and LMDB aborts the process. Shouldn't it return an MDB_PANIC error instead?
https://cdn.fosstodon.org/media_attachments/files/114/137/362/828/227/516/original/cefeba2f07d5266f.png

2025-02-26 18:06:56 GMT

- reply

@nprofile…2cpy nice, based on heed3 my wrapper over LMDB with encryption-at-rest support 💪 Everything is Rust idiomatic and well documented.

2025-02-26 12:05:28 GMT

- reply

@nprofile…2cpy I'm not only talking about the superior performances of the Rust impl but mainly about the memory safety of it and the non-performance-loss of it. So, why not using the safe one when there are no downside to do that? This paper from Inria x Microsoft also shows that using Rust is safer and doesn't impact performances. https://arxiv.org/abs/2412.15042

2025-02-26 08:56:20 GMT

Rust is the future, that's it! https://www.phoronix.com/news/Zlib-rs-0.4.2

2025-02-19 08:24:35 GMT

@nprofile…2cpy Hey 👋 I am wondering if you could fix LMDB by applying my patch because it no longer compiles on macOS due to the F_FULLSYNC non-existing macro. If you don't, I'll have to patch the compilation of LMDB in heed to alias F_FULLFSYNC with F_FULLSYNC. https://bugs.openldap.org/show_bug.cgi?id=10296#c6

2025-01-15 18:59:10 GMT

- reply

@nprofile…2cpy I love reading this kind of news! It's just showing that we did the right choice at Meilisearch 👏

2024-12-20 16:37:29 GMT

I wonder if the io_uring stuff being released by Jens Axboe, for example, could be used by #LMDB or if this key-value store must wait for previous writes to be done for the next ones to be performed (like a chain). What do you think @nprofile…2cpy? Obviously, not applicable to MDB_WRITEMAP.

2024-11-10 09:50:53 GMT

- reply

@npub1y0e…mm9n when you say "not in shared memory" do you mean that the write buffers are only visible by the program owner of the write txn? Like the write buffers comes from malloc? And if so, would it make sense to restrict the parallel readers (my proposal) to only be available to this program?

2024-11-09 12:52:00 GMT

- reply

@npub1y0e…mm9n I'am asking this because right now, in Meilisearch, we want to read the content in LMDB because it is already sorted (and why spent time sorting the entries by ourselves when LMDB already did it?). So, we collect all data pointers and associate them to the keys to then read them in parallel. Same technique as this one: https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy#reading-the-user-item-nodes-from-different-threads

2024-11-09 12:44:48 GMT

@npub1y0e…mm9n Hey, I know that I already asked this some times ago but what would be required to be able to read uncommitted pages with multiple read transactions? What I was thinking is that it coule be possible to "freeze" (disallow writes) with the write transactions ans from there:
- spawn read transactions
- on the txnid of the write txn
- and read pages from the in-memory buffer first then disk
What do you think?

2024-11-09 11:48:08 GMT

- reply

@npub1y0e…mm9n I also wanted to rewrite LMDB in Rust. It is a very complex codebase with a lot of c defines in every single function for every single supported OS. However, Rust has something that C doesn't have : real types. This article showing that sanakirja, an LMDB-inspired key value store, is much faster than LMDB when it comes to store constant-sized entries inspired me to start this quest. Then I actually looked at the LMDB codebase... https://pijul.org/posts/2021-02-06-rethinking-sanakirja/

2024-11-08 18:52:47 GMT

- reply

@npub1y0e…mm9n @npub16l0…syup there is no support for mmap in redb. He removed it for a file read/write support only. You can read my concern there. https://github.com/cberner/redb/pull/536#issue-1639871059

2024-10-16 07:12:33 GMT

- reply

@npub1svn…xpzg if you plan to use it in Rust just go straight to heed 😉 https://github.com/meilisearch/heed

2024-09-08 09:10:49 GMT

- reply

@npub13j3…fq0x Hey! I think you can look at our doc and deploy a Meilisearch on your infra. However, I highly recommend using our cloud. Why do you want it self hosted ?

2024-06-23 16:22:08 GMT

- reply

@npub1ed8…umh5 Ok, but it depends on the value size again. If the values are just u8 it's ok, but if it's UUIDs it can be slower. Just try it. And if you are interested in Rust, go try heed. https://github.com/meilisearch/heed

2024-06-22 15:42:25 GMT

- reply

@npub1ed8…umh5 that is quite fast to move in LMDB entries with a cursor, however, it depends on the data and the cursor operations you plan to use.

2024-06-10 22:02:02 GMT

- reply

@npub13j3…fq0x it was a joke, actually! It's because we are THE tool that Apple could have used to do the RAG for their new Apple Intelligence software 🙃

2024-05-07 19:25:29 GMT

- reply

@npub13j3…fq0x Wow! Are you using Meilisearch for a Spotify-like app? 🤩

2024-03-08 17:07:13 GMT

- reply

@npub15uf…ws6c You can take inspiration from mine. Or even just fork it! https://blog.kerollmops.com/using-rust-and-the-github-ci-for-my-blog

2024-02-15 22:53:45 GMT

- reply

@npub1y0e…mm9n You are right: a new language cannot fix logical bugs. However, not having compiler errors doesn't mean any logical bugs. And you probably guessed it, but I'll share those links. According to Google and Microsoft, 70% of "security" bugs (the worst ones) can be fixed by memory-safe languages.
https://old.reddit.com/r/rust/comments/gpdorw
https://old.reddit.com/r/rust/comments/apm5dp

2024-02-15 16:12:36 GMT

- reply

@npub1y0e…mm9n What I find interesting is that you are talking about the errors reported by a 2008 C compiler. We all know it evolved with plenty of new checks to ensure that the C-based industry stops shipping bugs everywhere. You and I know that you are a good programmer but, you tell it yourself, "such an engineer does not exist". Just think about it: making sure you cannot create invalid programs :point_down: is easier than making sure everybody is as good as you are. https://youtu.be/TRphkdL7Hz8

2024-01-18 10:27:33 GMT

Hey @npub1y6w…v0td 👋
We want to use your StatsAlloc crate at Meilisearch. I modified it a little bit and would love it if you could take a look at it.
https://github.com/neoeinstein/stats_alloc/pull/9

2023-12-24 08:20:48 GMT

Just before Christmas I offer you this article on how the @meilisearch team implemented filtered #DiskAnn on arroy and integrated it in the engine 🎁
Wouldn't it be cool to do fast and relevant #hybridSearch?
blog.kerollmops.com/meilisearch-expands-search-power-with-arroy-s-filtered-disk-ann

2023-12-16 09:14:33 GMT

You all asked for it! I just released the second part of my blog series on Arroy: #DiskANN-inspired library. https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy

2023-12-12 14:22:13 GMT

Something big is coming #meilisearch
https://cdn.fosstodon.org/media_attachments/files/111/567/886/263/849/352/original/8c5ff10ae10acabf.png

2023-12-08 16:31:10 GMT

Today, I want to discuss the beginning of our journey in building our homemade #DiskANN data structure and how we made it work on top of #lmdb. We were highly inspired by the @npub1554…tpt5 work. We made it safer, much faster and added a bunch of features to it. https://blog.kerollmops.com/spotify-inspired-elevating-meilisearch-with-hybrid-search-and-rust

2023-12-06 22:31:13 GMT

- reply

@npub1y0e…mm9n @npub1a56…jgz2 It seems that you have more positive feedback on real products related to AI (LLMs and ANN) than with this trap 😅 https://mastodon.social/@hyc/111510228308723803

2023-12-01 18:59:00 GMT

- reply

@npub1kzt…guag Thanks for helping us with the SIMD-optimized distance functions ☺️

2023-12-01 16:17:15 GMT

What if I tell you that Meilisearch is about to ship with a homemade #DiskANN vector store? Spotify/Annoy highly inspired our new library. Arroy allows you to do Approximate Nearest Neighbors search in hundreds of millions of vectors. https://github.com/meilisearch/arroy

2023-11-29 12:32:12 GMT

Hey #LMDB community 👋 I would like to know if there is a way to specify that the keys in an LMDB database will always be of the same length. Like an MDB_FIXEDKEYSIZE flag?

2023-11-26 22:47:45 GMT

- reply

@npub18xs…mug0 Whoops! Makes me think about a possibility to have a servo-based Arc browser! I like that!

2023-08-23 13:05:10 GMT

- reply

@npub1acg…p35c I just released the new v0.20.0-alpha.4 version of heed my #LMDB Rust wrapper. I advise you to try it out and tell me if everything works for you. https://github.com/meilisearch/heed/releases/tag/v0.20.0-alpha.4

2023-08-17 18:01:47 GMT

My first blog website ever 🤗
https://blog.kerollmops.com/using-rust-and-the-github-ci-for-my-blog

2023-08-14 09:17:19 GMT

Hey @npub1y0e…mm9n, we want to, zero-downtime swap an #LMDB environment that comes from an S3 with one that is already on disk (and opened). My first idea was to open the incoming env, open a write txn on the dst one and clear and copy the entries into the existing one but I fear about the MDB_TXN_FULL error. The second idea is to close the env (awaiting running txns) and swap the env files. This can produce a downtime. Do you recommend anything else?

2023-08-08 10:50:24 GMT

I don’t know when but I’ll probably ban people making me loose time by using incompetent LLM’s to code… Here is an example contributor using ChatGPT to work on my #LMDB Rust wrapper… But the documentation is clear. https://github.com/meilisearch/heed/pull/201#discussion_r1286562660

2023-08-06 21:41:59 GMT

- reply

@npub1zea…rzh5 @npub15s5…xszk Unfortunately, lmdb-rkv is always above heed on lib.rs, even if it's no more maintained... https://lib.rs/search?q=lmdb

2023-08-06 21:05:15 GMT

- reply

@npub1acg…p35c @npub1xts…kk5s Would be better if reproduced from scratch in C but if you know the last operation that corrupts the database maybe do this last one in C if possible. Providing the pre-corrupted database and the last operations breaking it could probably help diagnosis.

2023-08-06 20:36:46 GMT

- reply

@npub15s5…xszk Just go with #LMDB. Only a single file, faster than SQLite for key-value storage and zerocopy read. I maintain a Rust wrapper called heed and Meilisearch is running on too of that, works perfectly 🎉
I recently updated the Wikipedia page. https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database

2023-08-06 20:30:22 GMT

- reply

@npub1xqt…3ygc Nice work 🎉 I confirm that. I am my own user on heed and I find a lot of new interresting features whenever I need them!

2023-08-06 20:22:19 GMT

- reply

@npub1acg…p35c @npub1xts…kk5s
If you can repro the LMDB issue you got in C, you can create a new issue on the OpenLDAP ITS. https://bugs.openldap.org/

2023-08-06 20:20:49 GMT

- reply

@npub1acg…p35c @npub1xts…kk5s
I worked on bringing the DUPSORT LMDB feature to heed today. I will continue with the DUPFIXED later. I will release a new alpha version later next week, probably 🙂 https://github.com/meilisearch/heed/pull/200

2023-08-03 19:35:03 GMT

- reply

@npub1y0e…mm9n @npub1acg…p35c Can we request anything on the issue tracker? Because we will think about it again. I am thinking about the memory aligned values MDB_FIXEDMAP… but there are probably more useful features like having a database flag that makes it possible for LMDB to have a callback to ask for the key and value length (without forcing LMDB to store it) 🤔

2023-08-03 12:44:00 GMT

- reply

@npub1y0e…mm9n @npub1acg…p35c That’s wonderful! Do you mean keys longer than 511 bytes by default? Is it something important? Will it be possible to store values of 4GiB when using the DUPSORT option?

2023-08-03 08:22:07 GMT

- reply

@npub1acg…p35c @npub1xts…kk5s If you need anything, tell me, create an issue and I will see if I have time to work on it. I am working on this big v0.20 alpha refactor so let’s break everything 🏺

2023-08-02 22:11:59 GMT

- reply

@npub1acg…p35c It doesn't for now but once I merge this PR I will be able to continue the design and work on exposing this feature. A company started working on that but we figured out that #LMDB was restraining DUP values to 511 bytes... https://github.com/meilisearch/heed/pull/197

2023-08-02 21:51:52 GMT

- reply

@npub1xts…kk5s @npub1acg…p35c I advise you to use heed that I maintain. We use it in Meilisearch. You should try the latest v0.20-alpha versions. https://lib.rs/crates/heed