kero

Co-founder & Tech Lead @meilisearch. Rust developer and ex-student from @42born2code.

Public Key

npub1xyypr4cduzmn6zwx9yuxjq2utyhrdnxpjz422pxc6y0hwaa652uqmy6vyg

NIP-05 Address

[email protected]

Profile Code

nprofile1qqsrzzq36ux7pdeap8rzjwrfq9w9jt3kenqep249qnvdz8mhw7a29wq5xkrqj

Publishing to

relay.mostr.pub

Author Public Key

npub1xyypr4cduzmn6zwx9yuxjq2utyhrdnxpjz422pxc6y0hwaa652uqmy6vyg

Show more details

Last Notes

2024-06-23 18:22:08

- reply

@npub1ed8…umh5 Ok, but it depends on the value size again. If the values are just u8 it's ok, but if it's UUIDs it can be slower. Just try it. And if you are interested in Rust, go try heed. https://github.com/meilisearch/heed

2024-06-11 00:02:02

- reply

@npub13j3…fq0x it was a joke, actually! It's because we are THE tool that Apple could have used to do the RAG for their new Apple Intelligence software 🙃

2024-05-07 21:25:29

- reply

@npub13j3…fq0x Wow! Are you using Meilisearch for a Spotify-like app? 🤩

2024-03-08 18:07:13

- reply

@npub15uf…ws6c You can take inspiration from mine. Or even just fork it! https://blog.kerollmops.com/using-rust-and-the-github-ci-for-my-blog

2024-02-15 23:53:45

- reply

@npub1y0e…mm9n You are right: a new language cannot fix logical bugs. However, not having compiler errors doesn't mean any logical bugs. And you probably guessed it, but I'll share those links. According to Google and Microsoft, 70% of "security" bugs (the worst ones) can be fixed by memory-safe languages.
https://old.reddit.com/r/rust/comments/gpdorw
https://old.reddit.com/r/rust/comments/apm5dp

2024-02-15 17:12:36

- reply

@npub1y0e…mm9n What I find interesting is that you are talking about the errors reported by a 2008 C compiler. We all know it evolved with plenty of new checks to ensure that the C-based industry stops shipping bugs everywhere. You and I know that you are a good programmer but, you tell it yourself, "such an engineer does not exist". Just think about it: making sure you cannot create invalid programs :point_down: is easier than making sure everybody is as good as you are. https://youtu.be/TRphkdL7Hz8

2024-02-04 10:45:49

- reply

@npub1y0e…mm9n Can you share the link, please? Just want to show that to some friends!

2024-01-18 11:27:33

Hey @npub1y6w…v0td 👋
We want to use your StatsAlloc crate at Meilisearch. I modified it a little bit and would love it if you could take a look at it.
https://github.com/neoeinstein/stats_alloc/pull/9

2023-12-24 09:20:48

Just before Christmas I offer you this article on how the @meilisearch team implemented filtered #DiskAnn on arroy and integrated it in the engine 🎁
Wouldn't it be cool to do fast and relevant #hybridSearch?
blog.kerollmops.com/meilisearch-expands-search-power-with-arroy-s-filtered-disk-ann

2023-12-16 10:14:33

You all asked for it! I just released the second part of my blog series on Arroy: #DiskANN-inspired library. https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy

2023-12-08 17:31:10

Today, I want to discuss the beginning of our journey in building our homemade #DiskANN data structure and how we made it work on top of #lmdb. We were highly inspired by the @npub1554…tpt5 work. We made it safer, much faster and added a bunch of features to it. https://blog.kerollmops.com/spotify-inspired-elevating-meilisearch-with-hybrid-search-and-rust

2023-12-06 23:31:13

- reply

@npub1y0e…mm9n @npub1a56…jgz2 It seems that you have more positive feedback on real products related to AI (LLMs and ANN) than with this trap 😅 https://mastodon.social/@hyc/111510228308723803

2023-12-01 19:59:00

- reply

@npub1kzt…guag Thanks for helping us with the SIMD-optimized distance functions ☺️

2023-12-01 17:17:15

What if I tell you that Meilisearch is about to ship with a homemade #DiskANN vector store? Spotify/Annoy highly inspired our new library. Arroy allows you to do Approximate Nearest Neighbors search in hundreds of millions of vectors. https://github.com/meilisearch/arroy

2023-11-29 13:32:12

Hey #LMDB community 👋 I would like to know if there is a way to specify that the keys in an LMDB database will always be of the same length. Like an MDB_FIXEDKEYSIZE flag?

2023-09-07 17:37:48

- reply

@npub1y0e…mm9n I just found an email describing incremental LMDB backups. Where can I find more information?
https://www.openldap.org/lists/openldap-technical/201407/msg00122.html

2023-08-23 15:05:10

- reply

@npub1acg…p35c I just released the new v0.20.0-alpha.4 version of heed my #LMDB Rust wrapper. I advise you to try it out and tell me if everything works for you. https://github.com/meilisearch/heed/releases/tag/v0.20.0-alpha.4

2023-08-17 20:01:47

My first blog website ever 🤗
https://blog.kerollmops.com/using-rust-and-the-github-ci-for-my-blog

2023-08-14 11:17:19

Hey @npub1y0e…mm9n, we want to, zero-downtime swap an #LMDB environment that comes from an S3 with one that is already on disk (and opened). My first idea was to open the incoming env, open a write txn on the dst one and clear and copy the entries into the existing one but I fear about the MDB_TXN_FULL error. The second idea is to close the env (awaiting running txns) and swap the env files. This can produce a downtime. Do you recommend anything else?

2023-08-08 12:50:24

I don’t know when but I’ll probably ban people making me loose time by using incompetent LLM’s to code… Here is an example contributor using ChatGPT to work on my #LMDB Rust wrapper… But the documentation is clear. https://github.com/meilisearch/heed/pull/201#discussion_r1286562660

2023-08-06 23:41:59

- reply

@npub1zea…rzh5 @npub15s5…xszk Unfortunately, lmdb-rkv is always above heed on lib.rs, even if it's no more maintained... https://lib.rs/search?q=lmdb

2023-08-06 23:05:15

- reply

@npub1acg…p35c @npub1xts…kk5s Would be better if reproduced from scratch in C but if you know the last operation that corrupts the database maybe do this last one in C if possible. Providing the pre-corrupted database and the last operations breaking it could probably help diagnosis.

2023-08-06 22:36:46

- reply

@npub15s5…xszk Just go with #LMDB. Only a single file, faster than SQLite for key-value storage and zerocopy read. I maintain a Rust wrapper called heed and Meilisearch is running on too of that, works perfectly 🎉
I recently updated the Wikipedia page. https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database

2023-08-06 22:30:22

- reply

@npub1xqt…3ygc Nice work 🎉 I confirm that. I am my own user on heed and I find a lot of new interresting features whenever I need them!

2023-08-06 22:22:19

- reply

@npub1acg…p35c @npub1xts…kk5s
If you can repro the LMDB issue you got in C, you can create a new issue on the OpenLDAP ITS. https://bugs.openldap.org/

2023-08-06 22:20:49

- reply

@npub1acg…p35c @npub1xts…kk5s
I worked on bringing the DUPSORT LMDB feature to heed today. I will continue with the DUPFIXED later. I will release a new alpha version later next week, probably 🙂 https://github.com/meilisearch/heed/pull/200

2023-08-03 21:35:03

- reply

@npub1y0e…mm9n @npub1acg…p35c Can we request anything on the issue tracker? Because we will think about it again. I am thinking about the memory aligned values MDB_FIXEDMAP… but there are probably more useful features like having a database flag that makes it possible for LMDB to have a callback to ask for the key and value length (without forcing LMDB to store it) 🤔

2023-08-03 14:44:00

- reply

@npub1y0e…mm9n @npub1acg…p35c That’s wonderful! Do you mean keys longer than 511 bytes by default? Is it something important? Will it be possible to store values of 4GiB when using the DUPSORT option?

2023-08-03 10:22:07

- reply

@npub1acg…p35c @npub1xts…kk5s If you need anything, tell me, create an issue and I will see if I have time to work on it. I am working on this big v0.20 alpha refactor so let’s break everything 🏺

2023-08-03 00:11:59

- reply

@npub1acg…p35c It doesn't for now but once I merge this PR I will be able to continue the design and work on exposing this feature. A company started working on that but we figured out that #LMDB was restraining DUP values to 511 bytes... https://github.com/meilisearch/heed/pull/197

2023-08-02 23:51:52

- reply

@npub1xts…kk5s @npub1acg…p35c I advise you to use heed that I maintain. We use it in Meilisearch. You should try the latest v0.20-alpha versions. https://lib.rs/crates/heed