kero

Co-founder & CTO @meilisearch. Rust developer and ex-student from @42born2code.

Public Key

npub1xyypr4cduzmn6zwx9yuxjq2utyhrdnxpjz422pxc6y0hwaa652uqmy6vyg

NIP-05 Address

Profile Code

nprofile1qqsrzzq36ux7pdeap8rzjwrfq9w9jt3kenqep249qnvdz8mhw7a29wqpz3mhxue69uhhyetvv9ujuerpd46hxtnfduqs6amnwvaz7tmwdaejumr0dsjlpamt

Publishing to

relay.damus.io

nos.lol

Author Public Key

npub1xyypr4cduzmn6zwx9yuxjq2utyhrdnxpjz422pxc6y0hwaa652uqmy6vyg

Show more details

Last Notes

2025-03-29 12:02:48 GMT

It happened! @meilisearch.com reached 50k stars on GitHub 🤩 we are still the second most-started search engine. Elasticsearch, we are coming for you. #rustlang
https://cdn.fosstodon.org/media_attachments/files/114/245/614/490/099/566/original/19c1d09c8e3a0386.png

2025-03-20 15:20:12 GMT

Today is a sad day 🥺 We will no longer use bors and switch to GitHub Merges Queues on @meilisearch.com. #rustlang https://github.com/meilisearch/meilisearch/pull/5447

2025-03-18 22:27:06 GMT

@nprofile…293t, I just read your original article about using c2rust to convert bzip2 to Rust, and I was wondering if you would recommend it to convert the LMDB source code. It does a lot of platform-specific operations, but it is also quite small (mdb.c, lmdb.h, mild.c, and mild.h only). Do you have more information than me on the converting tool from C to safe Rust? https://arxiv.org/abs/2412.15042

2025-03-18 16:50:44 GMT

- reply

> working on a rust language experiment
What do you mean by that? Will it be a macro or something more deep that modifies the compiler with keywords and gotos?

2025-03-18 11:18:47 GMT

Hey @nprofile…293t, I am reading your recent article on zlib-rs, "zlib-rs is faster than C," and I was wondering if you tried to use the fallthrough libraries proposed at the end of this article to fill the Rust switch syntax gap? https://huonw.github.io/blog/2025/03/rust-fallthrough/

2025-03-17 16:53:16 GMT

We have worked hard on Meilisearch for the last 2 years and have cooked you smt amazing! We mixed chocolate and some AI toppings to release Hybrid Search, something new on the market. Be ready and subscribe to our upcoming Product Hunt event 🛎️
https://www.producthunt.com/products/meilisearch-cloud
https://cdn.fosstodon.org/media_attachments/files/114/178/811/077/609/494/original/87ae9a4e1f43b3a9.png

2025-03-15 10:15:20 GMT

Just released heed & heed3 v0.22, my Rust #lmdb wrapper, which improves the doc, supports dup sort custom comparators, support the PREV_SNAPSHOT flag, better support RoTxn with TSL, and use the latest LMDB version. Meilisearch is already using it! Go star it 🌟 https://github.com/meilisearch/heed/releases/tag/v0.22.0

2025-03-15 10:14:48 GMT

Just released heed & heed3 v0.22, my Rust LMDB wrapper, which improves the doc, supports dup sort custom comparators, support the PREV_SNAPSHOT flag, better support RoTxn with TSL, and use the latest LMDB version. Meilisearch is already using it! Go star it 🌟 https://github.com/meilisearch/heed/releases/tag/v0.22.0

2025-03-10 09:14:42 GMT

- reply

@nprofile…2cpy I don't understand your point about libraries aborting a process. LMDB does precisely that in case an assertion is not valid. Panicking is the same thing as the mdb_cassert function being run in non-debug programs. The main difference is that panics can be watched in #Rust. You can see an example in Meilisearch: Sometimes, users have disk corruptions, and LMDB aborts the process. Shouldn't it return an MDB_PANIC error instead?
https://cdn.fosstodon.org/media_attachments/files/114/137/362/828/227/516/original/cefeba2f07d5266f.png

2025-02-26 18:06:56 GMT

- reply

@nprofile…2cpy nice, based on heed3 my wrapper over LMDB with encryption-at-rest support 💪 Everything is Rust idiomatic and well documented.

2025-02-26 12:05:28 GMT

- reply

@nprofile…2cpy I'm not only talking about the superior performances of the Rust impl but mainly about the memory safety of it and the non-performance-loss of it. So, why not using the safe one when there are no downside to do that? This paper from Inria x Microsoft also shows that using Rust is safer and doesn't impact performances. https://arxiv.org/abs/2412.15042

2025-02-26 08:56:20 GMT

Rust is the future, that's it! https://www.phoronix.com/news/Zlib-rs-0.4.2

2025-02-19 08:24:35 GMT

@nprofile…2cpy Hey 👋 I am wondering if you could fix LMDB by applying my patch because it no longer compiles on macOS due to the F_FULLSYNC non-existing macro. If you don't, I'll have to patch the compilation of LMDB in heed to alias F_FULLFSYNC with F_FULLSYNC. https://bugs.openldap.org/show_bug.cgi?id=10296#c6

2025-01-15 18:59:10 GMT

- reply

@nprofile…2cpy I love reading this kind of news! It's just showing that we did the right choice at Meilisearch 👏

2024-12-20 16:37:29 GMT

I wonder if the io_uring stuff being released by Jens Axboe, for example, could be used by #LMDB or if this key-value store must wait for previous writes to be done for the next ones to be performed (like a chain). What do you think @nprofile…2cpy? Obviously, not applicable to MDB_WRITEMAP.

2024-11-21 12:32:26 GMT

@nprofile…r0qg Hello, I sent you multiple emails from my meilisearch email address and never get an answer. I was asking for a DB-Engines account for Meilisearch. Would it be poasible to get one, please? Thank you 😊

2024-11-10 09:50:53 GMT

- reply

@npub1y0e…mm9n when you say "not in shared memory" do you mean that the write buffers are only visible by the program owner of the write txn? Like the write buffers comes from malloc? And if so, would it make sense to restrict the parallel readers (my proposal) to only be available to this program?

2024-11-09 12:52:00 GMT

- reply

@npub1y0e…mm9n I'am asking this because right now, in Meilisearch, we want to read the content in LMDB because it is already sorted (and why spent time sorting the entries by ourselves when LMDB already did it?). So, we collect all data pointers and associate them to the keys to then read them in parallel. Same technique as this one: https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy#reading-the-user-item-nodes-from-different-threads

2024-11-09 12:44:48 GMT

@npub1y0e…mm9n Hey, I know that I already asked this some times ago but what would be required to be able to read uncommitted pages with multiple read transactions? What I was thinking is that it coule be possible to "freeze" (disallow writes) with the write transactions ans from there:
- spawn read transactions
- on the txnid of the write txn
- and read pages from the in-memory buffer first then disk
What do you think?

2024-11-09 11:48:08 GMT

- reply

@npub1y0e…mm9n I also wanted to rewrite LMDB in Rust. It is a very complex codebase with a lot of c defines in every single function for every single supported OS. However, Rust has something that C doesn't have : real types. This article showing that sanakirja, an LMDB-inspired key value store, is much faster than LMDB when it comes to store constant-sized entries inspired me to start this quest. Then I actually looked at the LMDB codebase... https://pijul.org/posts/2021-02-06-rethinking-sanakirja/

2024-11-08 18:52:47 GMT

- reply

@npub1y0e…mm9n @npub16l0…syup there is no support for mmap in redb. He removed it for a file read/write support only. You can read my concern there. https://github.com/cberner/redb/pull/536#issue-1639871059

2024-10-16 07:12:33 GMT

- reply

@npub1svn…xpzg if you plan to use it in Rust just go straight to heed 😉 https://github.com/meilisearch/heed

2024-09-08 09:10:49 GMT

- reply

@npub13j3…fq0x Hey! I think you can look at our doc and deploy a Meilisearch on your infra. However, I highly recommend using our cloud. Why do you want it self hosted ?

2024-06-23 16:22:08 GMT

- reply

@npub1ed8…umh5 Ok, but it depends on the value size again. If the values are just u8 it's ok, but if it's UUIDs it can be slower. Just try it. And if you are interested in Rust, go try heed. https://github.com/meilisearch/heed

2024-06-22 15:42:25 GMT

- reply

@npub1ed8…umh5 that is quite fast to move in LMDB entries with a cursor, however, it depends on the data and the cursor operations you plan to use.

2024-06-10 22:02:02 GMT

- reply

@npub13j3…fq0x it was a joke, actually! It's because we are THE tool that Apple could have used to do the RAG for their new Apple Intelligence software 🙃

2024-05-07 19:25:29 GMT

- reply

@npub13j3…fq0x Wow! Are you using Meilisearch for a Spotify-like app? 🤩

2024-03-08 17:07:13 GMT

- reply

@npub15uf…ws6c You can take inspiration from mine. Or even just fork it! https://blog.kerollmops.com/using-rust-and-the-github-ci-for-my-blog

2024-02-15 22:53:45 GMT

- reply

@npub1y0e…mm9n You are right: a new language cannot fix logical bugs. However, not having compiler errors doesn't mean any logical bugs. And you probably guessed it, but I'll share those links. According to Google and Microsoft, 70% of "security" bugs (the worst ones) can be fixed by memory-safe languages.
https://old.reddit.com/r/rust/comments/gpdorw
https://old.reddit.com/r/rust/comments/apm5dp

2024-02-15 16:12:36 GMT

- reply

@npub1y0e…mm9n What I find interesting is that you are talking about the errors reported by a 2008 C compiler. We all know it evolved with plenty of new checks to ensure that the C-based industry stops shipping bugs everywhere. You and I know that you are a good programmer but, you tell it yourself, "such an engineer does not exist". Just think about it: making sure you cannot create invalid programs :point_down: is easier than making sure everybody is as good as you are. https://youtu.be/TRphkdL7Hz8

2024-02-04 09:45:49 GMT

- reply

@npub1y0e…mm9n Can you share the link, please? Just want to show that to some friends!

2024-01-18 10:27:33 GMT

Hey @npub1y6w…v0td 👋
We want to use your StatsAlloc crate at Meilisearch. I modified it a little bit and would love it if you could take a look at it.
https://github.com/neoeinstein/stats_alloc/pull/9

2023-12-24 08:20:48 GMT

Just before Christmas I offer you this article on how the @meilisearch team implemented filtered #DiskAnn on arroy and integrated it in the engine 🎁
Wouldn't it be cool to do fast and relevant #hybridSearch?
blog.kerollmops.com/meilisearch-expands-search-power-with-arroy-s-filtered-disk-ann

2023-12-16 09:14:33 GMT

You all asked for it! I just released the second part of my blog series on Arroy: #DiskANN-inspired library. https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy

2023-12-12 14:22:13 GMT

Something big is coming #meilisearch
https://cdn.fosstodon.org/media_attachments/files/111/567/886/263/849/352/original/8c5ff10ae10acabf.png

2023-12-08 16:31:10 GMT

Today, I want to discuss the beginning of our journey in building our homemade #DiskANN data structure and how we made it work on top of #lmdb. We were highly inspired by the @npub1554…tpt5 work. We made it safer, much faster and added a bunch of features to it. https://blog.kerollmops.com/spotify-inspired-elevating-meilisearch-with-hybrid-search-and-rust

2023-12-06 22:31:13 GMT

- reply

@npub1y0e…mm9n @npub1a56…jgz2 It seems that you have more positive feedback on real products related to AI (LLMs and ANN) than with this trap 😅 https://mastodon.social/@hyc/111510228308723803

2023-12-01 18:59:00 GMT

- reply

@npub1kzt…guag Thanks for helping us with the SIMD-optimized distance functions ☺️

2023-12-01 16:17:15 GMT

What if I tell you that Meilisearch is about to ship with a homemade #DiskANN vector store? Spotify/Annoy highly inspired our new library. Arroy allows you to do Approximate Nearest Neighbors search in hundreds of millions of vectors. https://github.com/meilisearch/arroy

2023-11-29 12:32:12 GMT

Hey #LMDB community 👋 I would like to know if there is a way to specify that the keys in an LMDB database will always be of the same length. Like an MDB_FIXEDKEYSIZE flag?

2023-11-26 22:47:45 GMT

- reply

@npub18xs…mug0 Whoops! Makes me think about a possibility to have a servo-based Arc browser! I like that!

2023-09-07 15:37:48 GMT

- reply

@npub1y0e…mm9n I just found an email describing incremental LMDB backups. Where can I find more information?
https://www.openldap.org/lists/openldap-technical/201407/msg00122.html

2023-08-23 13:05:10 GMT

- reply

@npub1acg…p35c I just released the new v0.20.0-alpha.4 version of heed my #LMDB Rust wrapper. I advise you to try it out and tell me if everything works for you. https://github.com/meilisearch/heed/releases/tag/v0.20.0-alpha.4

2023-08-17 18:01:47 GMT

My first blog website ever 🤗
https://blog.kerollmops.com/using-rust-and-the-github-ci-for-my-blog

2023-08-14 09:17:19 GMT

Hey @npub1y0e…mm9n, we want to, zero-downtime swap an #LMDB environment that comes from an S3 with one that is already on disk (and opened). My first idea was to open the incoming env, open a write txn on the dst one and clear and copy the entries into the existing one but I fear about the MDB_TXN_FULL error. The second idea is to close the env (awaiting running txns) and swap the env files. This can produce a downtime. Do you recommend anything else?

2023-08-08 10:50:24 GMT

I don’t know when but I’ll probably ban people making me loose time by using incompetent LLM’s to code… Here is an example contributor using ChatGPT to work on my #LMDB Rust wrapper… But the documentation is clear. https://github.com/meilisearch/heed/pull/201#discussion_r1286562660

2023-08-06 21:41:59 GMT

- reply

@npub1zea…rzh5 @npub15s5…xszk Unfortunately, lmdb-rkv is always above heed on lib.rs, even if it's no more maintained... https://lib.rs/search?q=lmdb

2023-08-06 21:05:15 GMT

- reply

@npub1acg…p35c @npub1xts…kk5s Would be better if reproduced from scratch in C but if you know the last operation that corrupts the database maybe do this last one in C if possible. Providing the pre-corrupted database and the last operations breaking it could probably help diagnosis.

2023-08-06 20:36:46 GMT

- reply

@npub15s5…xszk Just go with #LMDB. Only a single file, faster than SQLite for key-value storage and zerocopy read. I maintain a Rust wrapper called heed and Meilisearch is running on too of that, works perfectly 🎉
I recently updated the Wikipedia page. https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database

2023-08-06 20:30:22 GMT

- reply

@npub1xqt…3ygc Nice work 🎉 I confirm that. I am my own user on heed and I find a lot of new interresting features whenever I need them!

2023-08-06 20:22:19 GMT

- reply

@npub1acg…p35c @npub1xts…kk5s
If you can repro the LMDB issue you got in C, you can create a new issue on the OpenLDAP ITS. https://bugs.openldap.org/

2023-08-06 20:20:49 GMT

- reply

@npub1acg…p35c @npub1xts…kk5s
I worked on bringing the DUPSORT LMDB feature to heed today. I will continue with the DUPFIXED later. I will release a new alpha version later next week, probably 🙂 https://github.com/meilisearch/heed/pull/200

2023-08-03 19:35:03 GMT

- reply

@npub1y0e…mm9n @npub1acg…p35c Can we request anything on the issue tracker? Because we will think about it again. I am thinking about the memory aligned values MDB_FIXEDMAP… but there are probably more useful features like having a database flag that makes it possible for LMDB to have a callback to ask for the key and value length (without forcing LMDB to store it) 🤔

2023-08-03 12:44:00 GMT

- reply

@npub1y0e…mm9n @npub1acg…p35c That’s wonderful! Do you mean keys longer than 511 bytes by default? Is it something important? Will it be possible to store values of 4GiB when using the DUPSORT option?

2023-08-03 08:22:07 GMT

- reply

@npub1acg…p35c @npub1xts…kk5s If you need anything, tell me, create an issue and I will see if I have time to work on it. I am working on this big v0.20 alpha refactor so let’s break everything 🏺

2023-08-02 22:11:59 GMT

- reply

@npub1acg…p35c It doesn't for now but once I merge this PR I will be able to continue the design and work on exposing this feature. A company started working on that but we figured out that #LMDB was restraining DUP values to 511 bytes... https://github.com/meilisearch/heed/pull/197

2023-08-02 21:51:52 GMT

- reply

@npub1xts…kk5s @npub1acg…p35c I advise you to use heed that I maintain. We use it in Meilisearch. You should try the latest v0.20-alpha versions. https://lib.rs/crates/heed