Lead Core Lightning, Standards Wrangler, Bitcoin Script Restoration ponderer, coder. Full time employed on Free and Open Source Software since 1998. Joyous hacking with others for over 25 years.
Public Key
npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s NIP-57 Address
npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s@npub.cash
Profile Code
nprofile1qqs0zuj4s6jq9sr2ajqc69rc53d25rwpd3afcjrfm97r2qek69hcuscpz3mhxue69uhhyetvv9ujuerpd46hxtnfduurfv0s
Author Public Key
npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Show more details
Published at
2024-05-23T14:45:45+02:00 Event JSON
{
"id": "8905ad38afc59608887fb5b68e2fd699d0535e79836b76d146753c9d665923e0" ,
"pubkey": "f1725586a402c06aec818d1478a45aaa0dc16c7a9c4869d97c350336d16f8e43" ,
"created_at": 1716468345 ,
"kind": 0 ,
"tags": [
[
"alt",
"User profile for Rusty Russell"
]
],
"content": "{\"name\":\"Rusty Russell\",\"display_name\":\"Rusty Russell\",\"website\":\"https://rusty.ozlabs.org\",\"about\":\"Lead Core Lightning, Standards Wrangler, Bitcoin Script Restoration ponderer, coder. Full time employed on Free and Open Source Software since 1998. Joyous hacking with others for over 25 years.\",\"lud16\":\"npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s@npub.cash\",\"nip05\":\"[email protected] \",\"picture\":\"https://en.m.wikipedia.org/wiki/Rusty_Russell#/media/File%3ARusty_Russell-lca2011%2Bcrop.jpg\"}" ,
"sig": "63135fec5258c76ed52005efbdebf52b5af83cee6c24ba376cb8cba97eb65cf45ab74ee13aae54bf5e0ec91d83f5ab71d89d6e369dd66b6ab5a71bccbb6c1681"
}
Last Notes npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Last three weeks I've been working on improving payments. Critical to this are five things: 1. Better payment routing, a-la min-cost-flow. 2. Better low-level payment injection API 3. Simpler, more reliable implementation. 4. Better simulation and realistic testing. 5. Better gossip maintenance. For the first, I took the heart off renepay to make "askrene" the payment oracle, which provides "getroutes" for other plugins to use. A primitive version shipped in the current release, but it's already getting better as I start to use it. Today I finished "injectpaymentonion" which takes incoming payments the same way we deal with incoming HTLCs, unifying things that were previously difficult, like self-pay and bolt12 blinded paths that begin at our own node. Tying these together is a new plug-in called "xpay". This calls getroutes then injectpaymentonion, then iterates as results come back, feeding back information to askrene as we learn more about the network. To figure out how well this actually performs, I've got a new format for representing topology, with minimal information (no node names, just channel capacities, details and topology). A simple tool compresses this from a node's gossip store (and decompress back for use). The result is small enough that we can include graph snapshots on our repository for testing. Then I have written a fake "channeld" which simulates payments through this graph (it knows all the secret keys). This needs to be enhanced with simulating capacity of the channels (deterministically, based on a seed value) and realistic network delays, but it already does MPP timeouts and checks fees and cltv delays. This should allow me to test this system's performance in a way that mirrors what people actually see, and also measure it against what we have now. It may uncover bugs, but even better it will let me know where to optimize (min cost flow can definitely be a CPU hog!). And @nprofile…zu7w is working on ideas for more aggressive gossip gathering: if we're missing part of the map we cannot expect to have reliable payments! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell So, askrene (the routing oracle plugin) has support for layers, each containing information about channels (also, can add/remove channels and nodes, but that's mainly for special-effects for particular payments). It would not be hard to import/export entire layers: I've speculated that LSPs or large players may want to issue "weather reports" which could be incorporated this way. There are two issues: 1. Needs a standard format. This is easy, let's come up with something? 2. Privacy of those contributing to the data. This is hard, I don't know how to fuzz/exclude/delay data to avoid revealing the individual payments, but someone surely does? This is absolutely a service you could charge for, too. Perhaps a free version with very rough data, and a tier with more complete data? npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I'm working on a new pay plugin called xpay (working title). It's designed to use the new (very WIP) askrene plugin, which provides routes. Actually, most of the work is there, fixing and enhancing that. Especially diagnosis: *why* did it give a weird/no route? Common issues are: there's a good channel but it's offline, or doesn't have capacity, or you don't have enough capacity at all, or the recipient doesn't. We've had issues where gossip is missing, too: detecting when that happens is an open research question, but we'll start probing more aggressively. There have been many complaints about pay in the wild, and the code has grown quite complex. I've heard good things about Boltz's mpay, which is far simpler, so I'm inspired to go in that direction. Moving the routing (which *is* complex!) out is the first step. To be honest, I should have taken a direct interest in this earlier. I considered pay a simple problem, or at least not one I needed to concern myself with. But better late than never! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Started working my way though the classic "How to Win Friends and Influence People" with my 10-yo. I should have re-read this years ago! It's a classic for a reason. All self-help books have to spend 50% of their time convincing you it's worth reading, and the remainder is the actual content. This is why the title is so off-putting to many, BUT the book itself is a gentle admonition on the requirements for nuanced empathy in human interaction, with a very reasoned approach. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell 1. I'm always interested in open source things. 2. You're in a bubble inside a bubble. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Fixed just now, Works For Me! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Always enlightening to hear directly from Gloria! https://fountain.fm/episode/TmC6KGSxTKYt8X9Zb2zY #nevent1q…esel npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell ? I don't know anyone involved. But this title is poor judgement, and if nobody says anything maybe that will become a norm. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell "Gauche at best, unhinged at worst" then. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Your title is offensive, the question is, is Jeff ok with it? If he is, perhaps it's edgy and clever. If he isn't, it's gauche at best, unhinged at worst. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Yes! I'll ping you. What TZ are you RN? npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell They're fixing it: after some testing, apparently this one wasn't our fault (on CLN v24.08rc3) Surprised me too! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I tried (and somewhat succeeded) in taking a week off work. Unfortunately, I did rework the nostrdb cursor code, because it annoyed my sensibilities of how iteration across protocols should be written. I'm still not happy with the results, but I did find some bugs! And reading code by changing stuff and reading the compiler errors has another problem: I still don't know exactly what this code is trying to *do*! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Audio should be fine. There's one graph, I think. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell #gsr I had a quick call with Brink engineers, on my Script work. Now I just need to polish things a bit more and actually release a prototype that people can sink their brains into... https://brink.dev/blog/2024/08/22/eng-call-great-script-restoration/ npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell And of course, since it's making a tx already it might be a good time to cross-splice funds from once channel to another... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Absolutely. You can theoretically do that already, but I don't want to have to do the math, I want to just say "open a channel to X" and have it figure out where to get the funds. It might be optimal to use onchain funds, or even close down a low-utilization channel entirely into the new one. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Then it depends entirely on who will watch? Neither Schiff's regular audience nor Bitcoiners are going to be persuaded. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Are you going to change anyone's mind? npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell #cln #dev #someday Wallet "Intents" With anchor channels, lightning nodes were forced to get smarter about tx construction. (Kind of funny, nobody really wanted to write a Bitcoin wallet, but here we all are). You have a deadline, user wants to spend as little as possible, you have to use CPFP to boost the commitment tx, and bring your own fees for any HTLC txs (which are now single input/output ANYONECANPAY|SINGLE). I did the minimal things to get this to work, but it brings me back to the question of how these things *should* work. I don't think the primary interface for a wallet should be "spend these utxos to create this output" but "do this by this time, with this budget cap". The wallet should figure out how to do it. For example: sources include onchain funds, splicable channels and even closing channels. What if the wallet code had a rough "cost" for each of these? Sinks include splicing in, onchain outputs (maybe to cold storage?) or new channels. And there are also specific txs you might want. It should combine them opportunistically (and later, pull them apart if priorities change). There are several problems though. The first is complexity: this is not trivial to get all the cases correct, and all the combinations. The second is related: understandability and debugging is hard too: what did it do, what did it decide *not* to do, and was the result optimal? i.e. why? But mainly, how to present this to the user? Or the plugins that they use to direct it? They need to know that funds are coming to them (especially since we low-ball fees for non-urgent things). Plugins will want to see PSBTs before they get signed, so they can do clever things (coinjoin, open new channels, combine in other ways). The upside of all this is maximum efficiency, and a side-helping of more confusing transaction graphs. Both of which help everyone. This is currently an idle thought: it's not going to reach the top of my TODO this year, for sure! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell The fundamental difference is between defined benefit (pension scheme) and defined contribution (super). The latter *is* thought of as an entitlement, but the separate accounting makes that closer to the truth? Ofc, poor overall economic performance will hurt super (in Australia, this means if property prices ever go down, we're screwed). I suspect the govt will raid the outliers during some crisis ("who needs $10m of super?" they will cry) probably using exit taxation. This political risk from here to my retirement is significant, which is why I've never topped up mine. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Yeah I've been reading your parsing code. We have to talk 🤣 npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell If you put it that way: I guess I'm also betting 2M sats it won't work :) npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Australia transitioned away from defined benefit schemes in the eighties and early nineties, and has increased retirement age (not a legal concept here, but when you can access a pension or your benefits) has been increasing too, from 55 when I was young to 67 currently. I'm not a huge fan of the resulting system in its details, but it's a sign that people and politicians used to care about longer-term thinking. (Interestingly, these reforms were from the Left, not Right, which maybe explains why they happened?). npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell @nprofile…6h5g put his hand up first, hes taking a look. I think the coding might be the easy part though: getting the PR merged is another thing altogether... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Simple rework PR up. I also want to update your BOLT11 parser to the latest (it's buggy). I need to think harder about your cursor API: my instinct is to split it into two types, one for reading, one for writing. But you do a lot of lookahead in your parsing, so the classic "mark cursor invalid if we hit an error" pattern doesn't fit. Hmm.... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Let's see what I can do after lunch then. Fun! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Me! I wrote an Android app years ago, but not my jam. Someone who knows what they're doing can probably do this in 1/100th of the time I can... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Any Android devs out there? I will pay 2 million sats for someone to implement the option to customize Signal notifications for *reactions* separately from new messages. Conditions: 1. Ping me first! It's not a race, you gotta get public agreement from me to enter. 2. Gotta get it merged and shipped so I can install it on my Android phone. 3. You have to have fun! References: https://community.signalusers.org/t/mute-custom-reaction-notifications/14931/20 npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Yeah, it's a well-kept secret unfortunately. Rough skimming code review: 1. bool is your friend, int is for old people :( 2. You should use ccan/ directly not in pieces: easier to update. 3. Your cursor API makes me cry, can I rewrite it? It's going to hurt somebody. 4. Do you want neatening pull requests? npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Reviewing. You should have used tdb, of course, but as Tridge said, you don't need an excuse to rewrite something! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell What format/how do I create "queries"? npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Otoh, I can't see your replies in gossip. So... Otherwise I'd be posting my adventures in trying to install notedeck here... (Cargo gets egui-winit, which wants a newer rustc (!!) than the Ubuntu install, so now I'm trying to use rustup. Which doesn't do anything until you read the docs...) npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Wow. I take it back. gossip is not clunky! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Linux desktop nostr apps? I'm using gossip, but it feels clunky. Prefer desktop for writing detailed posts: mobile is optimized for consumption, not production :( npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell It's 50 years since the greatest book on programming was published! Elements of Programming Style. A few years back, when I couldn't find mine, I swept up a half dozen copies and Niftynei mailed them to me. https://wikipedia.org/wiki/The_Elements_of_Programming_Style npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell In those cases, iterate. Almost everywhere else became non-dups, so will now assert if you try to insert a duplicate... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell #cln #dev So, we've had this annoying intermittent bug where UTXO spends would get missed. Sometimes it meant that we would keep gossip for channels which had been spent, and sometimes we'd miss opportunities to sweep funds (more concerning!). Eventually I started to suspect our (my!) hash table implementation. It's extremely efficient, but if it had some bug it could explain the issues: it has a random seed for the hash function, so weird corner cases would appear random. I wrote some random churn tests, nothing. I could get more elaborate, of course, but then something else happened. Shahana wrote some code to create all our new documentation examples, which involved getting nodes into all kinds of weird states, and hit a strange bug. I tracked it down to a case where the recovery code was putting a new peer into the hash table, where one already exists. Easy bug fix, but it made me wonder: were we doing this elsewhere? My hash table code allows duplicate keys just fine. But it's actually unusual to want that, and there are APIs (get, delkey) which only handle the first one vs getfirst, getnext which are fully generic. So, I wondered. Did we make this mistake anywhere else? I bit the bullet and split the APIs: up front you now declare what type of hash table you want (duplicate keys or nodups) and you don't even get the deceptive APIs for each case. As you might expect, the only code which had a problem was the various places where we watch UTXOs. You can absolutely be watching for the same thing in multiple places, and indeed the code was not iterating, but only handling the "first" one. And this was all my own code, front to back. Mea culpa. APIs matter. The natural use of an API should be the correct one. And of course "don't patch bad code, rewrite it" a-la Elements of Programming Style. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Sorry for the delay, I've been dwelling on this a little. I think that self-custody is easier than many alternatives, unlike with cash: I use Green wallet myself which shows how you can have support while still having ultimate control. I believe Casa have a similar product for larger custody. I hope to see this kind of offering mature, and be attractive beyond the DIY crowd: Bitcoin insurance will be necessarily limited, there are going to be more high profile custody disasters. Also, there's a possibility of generational attitude change: if fiat becomes noticably worse, many people will expend effort looking after their money. However, that's somewhat of an aside. The question was not those who choose not to custody, but those who can't. What guarantees can we give them, if we can't give them a UTXO? I think, and hope, we can do better than "you all need to trust one of these few people". But I agree this will only get its due attention when enough people are in that group: for now, the low hanging fruit is improving lightning and other L2s which can simply fall back to a UTXO each. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell In that model, you never have full custody. Someone sends money into the mint and they hold it on your behalf. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I've had both, but my Logitech trackball is at least 20 years old and still going strong... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell (Hi, Lyn, long time fan!) I'd push back on this, a little. There are definitely cases where I won't have a UTXO, but all the cryptographic solutions I've seen to handle others' failure (malice or incompetence) devolve to me obtaining a UTXO. In times (and places) of high trust people may forgo this security, but if you can afford it, why would you? "Verify, don't trust" is our entire motto at Blockstream, and while I am not my employer, it resonates. In my mind, forgoing cryptographic protection for your Bitcoin, if enough people do it, is akin to detaching your issued currency from gold. Seemingly a minor technical detail which has no immediate effect, perhaps. Obviously nobody would be foolish enough to do that, though! 😬 npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell So far I've got the RPI3, RPI5, my Intel laptop and my AMD build machine. Neither of the last two are recent, but I'm not really interested in speed records. There's some variance: the RPI5 wins at SHA256 for example. The RPI3 loses at everything:) There's real variance in the "fast" ops between RPi and x86. I'll post more when I'm at my desktop. Importantly, I'm benchmarking the actual opcodes, run through the script interpreter. I really want to benchmark the complete set to make sure there are no surprises... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Jeremy Rubin asked on Twitter what was happening with #GSR. Good q! After far too much fiddling with benchmarks I now have preliminary numbers. Budget is 5200 varops per weight. Fast ops (compare, zero fill, copy) cost 1 varop per stack byte. SHA256 costs 10 per byte. Everything else costs 2 per byte. I need to clean up my benchmarks so everyone can run them, and get "on your machine the worst case validation would be <X> seconds, doing OP_<Y>". That's concrete and gives us a chance to find any wild machines which are unexpectedly slow, and gives a tangible worst case, which should allow fruitful discussion I also need to write code to answer "what input size (if any) would cause <this script> to exhaust it's varops budget?". This again enables us to think concretely about my thesis (yet to be proven to my satisfaction!) that it's possible to have a budget which allows any reasonable scripts not to worry about it. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Yes, just need a round tuit... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I somewhat agree: "trust somebody" can be quite efficient. But it's prone to bad incentives (undeclared fractional reserve) which can drive out good actors (who are not as profitable) *then* collapse. I know @nprofile…rt7g has thought about this a fair bit, and has a rotating ecash solution which may address it well enough? npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Not really, that I can tell. The custodian you are trusting there is the miners. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell It's hard to get exact numbers, but a significant fraction of the world will not have enough money at one time to spend a single UTXO. This is clear from the uneven distribution of wealth and the inevitable rise in Bitcoin fees. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Indeed, that's the current effort, in every direction. But it definitely has limits. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell British vs American English, sorry. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Lazy content-free post. Blocked. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Thanks, AI. Blocked. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Yes. If you're going to use a custodian, ecash is a win. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Liquid is a "trust someone" solution. There are precautions to avoid targeting particular users (confidential amounts and assets) similar to ecash, but it's still a custodian. We can certainly argue it's a "good enough" custodian, given the number, reputation and legal consequences of failure, but it's still a custodian. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell First up, I want to recognize that this is an uncomfortable topic! Bitcoin is inevitably changing towards user-pays, and that's not all positive. But facts we don't like are still facts: can't engineer a solution if we can't think about the problems. There are three kinds of bitcoiners. A. Those who can afford any fee. B. Those who can afford a UTXO, but not often. C. Those who can't afford a UTXO. Nobody worries about the A group (and in the early days, that was everyone). Obviously Lightning (my area!) caters to the B group, and we want it to be as large as possible. To do this we can (1) make lightning as resiliant as we can so onchain spends are rare, (2) make bitcoin as efficient as possible so we can cram as much as we can into what we have. (1) Making lightning more resilient and reliable is engineering. Lots of people working on this, even before we get soft-forks which could help further. (2) More efficiency has two benefits: obviously if your own onchain spends are 20% smaller, that's 20% cheaper. But if *everyone's* onchain spends are 20% smaller, that means fees are lower *for everyone* too (and it's non-linear). So we really care about all Bitcoin usage! Some things are obvious wins: Taproot so you can avoid even putting the script onchain in many cases, FROST so you can cram your 2 of 3 or other scheme into a single key and signature. We know we want to get more aggressive with sharing one signature across multiple inputs (Cross Input Signature Aggregation), but that needs a lot more research, and a soft-fork. But even with all these, the math is clear: some people, even if you somehow gave them their wealth in a UTXO, it couldn't afford its own fees to spend. The C group is real. Spoiler alert: we don't have an answer for this! But let's look at some approaches people have tried. Firstly, there are attempts to move these people into the B group: give them long enough that maybe fees will reach a point they can afford. This seems unlikely to me: 1. As fees increase everyone will start doing the work to take advantage of low fee times, and that itself means that low-fee times won't be so low. 2. These schemes tend to increase onchain footprints, so they need fees to drop a lot to overcome that (typical is 2x the transaction size, so you need fees to halve to gain anything). 3. If you really can't afford the fee, you probably also can't afford to wait. 4. You still haven't actually dealt with those who really, really can't afford the fees. Ever. Another suggestion is that someone (e.g. a lightning service provider) will lock up funds which would cover fees, in case something goes wrong. This doesn't work economically, because nobody is paying $100 for a $5 user (not at scale), but it doesn't even work mathematically: the reason some people will have small UTXOs is because there are not enough sats for 10 billion people with any realistic distribution. There are two basic approaches left: 1. Group people, so they fall into the B category (i.e. onchain tx is possible, but expensive). 2. Trust someone, but rely on incentives. 1. Grouping people is possible, but they need to work together if somenthing goes wrong. So grouping inside a community is probably better than grouping with randos. For example, there are various tree-of-transaction schemes where you go onchain only if the coordinator fails/goes rogue, and how much it costs you depends on whether anyone near you in the tree pays to get themselves out. These are basically free if nothing goes wrong (one UTXO required for thousands of users!). But this is subject to ghettoization, where the coordinator makes sure all the C people are grouped together, knowing none of them can afford the transactions they need to get their funds back. It's particularly bad because the coordinator can insert its own fake "whales" to make it look like it's not ghettoized. You can play with incentives here, too: more research needed. The details matter! 2. Relying on incentives. As a simple example, lightning-connected e-cash mints. They can't rug individuals very easily, they have to rug everyone together (or go fractional and rug the last ones to exit). Maybe with enough anonymity and reputation, these would be Good Enough. More ambitious would be a single UTXO held for multiple people by a coordinator. Can we make it so that if a coordinator is dishonest, you can force them to burn your funds? Maybe burn more than your funds (ie. a bond)? Won't get your money, but it aligns incentives so they're not motivated to rug you. The details here really matter! There's a cute scheme which has been proposed where the coordinator pays a temporary bond, and asserts that they actually have everyone's signature to transfer the funds. If nobody challenges within a week, they get the bond back and the funds move. If someone challenges, all the signatures are put onchain, and if they're not all valid, the bond gets half-burned and half-given to the (successful) challenger. This is hard to make work, though. Someone needs to get the money to challenge (hard if you don't have the money in the first place, plus it's hard to prove to someone you *didn't* sign something!), and then make sure nobody gets the challenge bond before them (in particular, a dishonest coordinator, seeing the game is up, completes the successful challenge *themselves* and gets half their bond back), and make sure someone can't grief and delay the settlement indefinitely or bankrupt the coordinator. More research needed, here, too. Summary A longer post than I had expected to write. And it's buried in the middle of a thread nobody will read. (I do this sometimes. I suck at marketing I guess!) Sub-fee bitcoin amounts will have tradeoffs, involving trusting someone who has more money than you (at least, in someone's competence, even if their *financial* incentives can be made to match yours). This is difficult to build well, and not a very exciting thing to build today, so it hasn't really happened (custodial things are much, much easier!). This is also a key reason I believe we need to make Bitcoin more expressive: if we can do *more* with our own UTXOs, we can build better solutions. And by "we" I mean "someone smarter than me" of course! Feedback welcome! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I know, I kept waiting too! Let me get on my desktop so I can answer properly... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell On the lite end, Blockstream Green, too... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Not a fair assessment: scammers always want you to reassess. "Maybe he's Satoshi this time!" After getting flooded by waves of bullshit in the form of altcoins, I chose not to waste my time. Maybe I missed some genuine innovations, but I also saved a lot of time and sanity. With such a white-hot scamfrenzy, many of these people's introduction to "Bitcoin" was probably someone pushing an affinity scam. That's a big barrier to overcome, no ego required. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I'm disappointed that @nprofile…kr48 never got an answer to his question about what Bitcoin looks like to those who can't afford a UTXO. This is an important question, maybe *the* important question. But it also underscores how much he diverged from his "pleb everyman" origins: this is very much not a newcomer question! Guess I won't be on WBD to answer it, either!* *Spoiler: I don't know the answer, but I can describe the possibilities and issues people can (and are!) exploring... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Your willingness to enforce your supreme knowledge of right and wrong is WHAT MAKES BITCOIN GREAT npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell 🤣 Bitcoin: it's all about telling other people how to use their genitals! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell We are definitely wrong about some things! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Fifteen years! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I am sometimes haunted by the phrase I heard log ago from Eben Moglen: It's wrong to be right too soon. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Lol, everyone is female except for birth defects :) "There's a lot at stake". I disagree: there are a lot of people very upset at others' choices. It's a moral panic. I've heard all this before, too. Gotta go pray the gay away... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Yes! You can *add* pubkeys! You create a secret key, S1, give miner the pubkey P1 (= GxS1). Miner generates a secret key S2, calculates P2 (= GxS2) and pubkey P1 + P2. If it's good, gives you S2, you sum that with S1 to get your secret key S1+S1. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Ignoring, for the moment, that real biology is more complex than that: Messing with people's societal expectations has always been a part of the hacker ethos. Coming from a conservative background myself, I used to find this shocking, too! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Just got a "call" from "Ledger Live Support". MtGox bringing them all out, I guess. Be careful! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell People keep talking about stable channels. But few people need worry about -10% BTC prices, they worry about -70% prices, which is when they fail. I don't see anyone discussing their service limits here. They're a great business opportunity: if bitcoin goes up, you profit, if it goes down you declare bankrupcy and start again! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell HWI, USB, Linux. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell https://bitcoinmagazine.com/culture/reflections-on-bitcoin-culture BUT BITCOIN WAS SUPPOSED TO DEFEND MY IMPLIED PRONOUNS!!!! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I would have answered very differently in my youth. Diplomacy, empathy, social situations, relationships: these improve with practice! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell "non-binary" is a gender term, surely... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell As I've aged, I have come to terms with people not asking my permission or seeking my approval. Surprisingly often, my disapproval was just glass shards I was feeding myself. So I stopped. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Gender, as in "which box do I put you in" has always had exceptions. This is new to many. But I am old and sheltered enough to remember wrestling with homosexuality's existence, too. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Good point: where are the hot gay Bitcoiners at? npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell You're right! I should have said "a significant number". npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Most of my friends are non-Bitcoiners. A significant fraction are non-binary, an even larger number are neurodiverse, and many are half my age: a great variety of fascinatingly different humans! It's OK to be weird, and it makes our world more interesting. If that makes me outside someone's "ethos of Bitcoin" I think they misunderstand what Bitcoin is for. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell #cln #dev Finally catching up with the latest BOLT12 spec, and damn, Phoenix just announced BIP 353 support. Gotta code faster! npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I just spent way too many hours reworking this part of the spec: I would *really* appreciate feedback on it! https://github.com/lightning/bolts/pull/1179 npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell You should seriously consider this. Annoying as it is to throw away working code, this code is *annoying* in a way few things in the specs are. I really wish there was a way to use partial onions for this instead, but I can't make it work with the addition of the payload at the end (and you'd have to use exact values down each path, but that's probably ok). Even my new code doesn't do padding, delaying, dummy hops or fake node IDs like everyone would like. That will probably be in 24.11... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell #cln #dev Wrapping up the great blinded path catchup, I hit a snag. Paying an invoice where we ourselves are nominated as the head of the blinded path. I solved this case elegantly for onion messages, but actual payments are not so easy. In an ideal world, our code would have been written so we could have either the RPC or an incoming HTLC trigger this code. But unwrapping and forwarding are only written for incoming HTLCs, and reworking all that is a major task. I may revisit this one day, as such code would transparently allow self-payment which currently has a special path. So instead we are going to have another special path for this case :( I thought about simply unwrapping the onion inside the pay plugin for this, but it requires ECDH using the node key which we don't expose through RPC (and I'm reluctant to expose). Hoping to finish soon as this is a large part of getting offers to production ready (vs the current experimental config option). npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell You don't need to *ask* a public node to route for you, they can't really tell you're using them this way. The web->LN issue is real: no certs, no web for you! Getting an invoice without waking the node seems to require PTLCs. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Usually found at the intersection of any fan base and bitcoin. For example, why Apple is going to get into Bitcoin... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell There's a thing I saw often in the early Linux days and still see in Bitcoin communities. People discover Bitcoin and try to wedge it into their existing special interest and extol the crossover as a discovery (rather than a construction). I call this "Bitcoin fan-fiction". It's weird, awkward and very very human. It's also usually very easy to spot in others, never in ourselves 💛 npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Yep, spec changed recently, but the change isn't in CLN yet. Embarrassing, but the arguments for the change were quite compelling*. And it's the reason the final spec has not been merged. *It also happens to reduce the size of the minimal BOLT12 tattoo! Why? Um, no reason... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Lunduke should always be taken with a large pinch of salt... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell You assertions about what people will do is ungrounded: people do this currently. Notably, Bitcoin currently has inflation, so one could argue that the transition which will occur is the change! I agree with you, but mainly because I believe changing would be unethical. But the that would also apply if the original design was the other way. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell You don't need to reach to philosophy to reject this. There was a deliberate decision early on to support the network via usage fees, not via holding fees: otherwise some non-zero inflation would have been the cleanest solution. It does, of course, imply a third stage of Bitcoin's economic journey, which creates uncertainty. But you can't say people weren't warned! https://rusty-lightning.medium.com/the-three-economic-eras-of-bitcoin-d43bf0cf058a npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell #dev #CLN I've spent the last few workdays completely reworking our onion message code. This was scattered in various places and I wanted to unify it, and also written several years ago and I'd forgotten how the protocol actually works! onion messages are *double* encrypted; this is the main source of confusion! At the high layer, they're a series of nested encrypted calls ("onionmsg_tlv" in the BOLT 4 spec), so each recipient decrypts and hands it on: this is exactly the same as we use for payment information. But inside that is *another* encrypted blob (onionmsg_tlv.encrypted_recipient_data), which requires a tweak which was handed to you alongside the onion, for you to decrypt (into an "encrypted_data_tlv"). Inside that is all the information about where to send next, any restrictions, and allows you to calculate the *next* tweak to hand on (it can also override the next tweak). The double encryption is necessary because there are *three* actors here: Alice wants Bob to send her a message, without revealing her identity. So she gives Bob a "blinded path" which goes via Charlie: this path contains Charlie's pubkey (where to start the path), a blinding tweak, and two encrypted blobs for Alice to put into each layer of the onion message. The first an encrypted blob which Charlie can read, which contains her pubkey so he knows where to send it next. The second is her own, and contains a secret specific to the purpose of this message, so Bob can't play games trying to use this blinded path for anything else ("hey, are you the same node as this previous payment?") or use a different blinded path for this purpose. She can also add dummy hops (we don't yet), which she will simply absorb, to obscure the path length from Bob. You can add padding to make the hops indistinguishable (we don't yet). Bob puts the actual stuff he wants to send Alice into the final onion call (often including his own blinded reply path!), along with the encrypted blob. Importantly, even if Bob were sending a message *not through a blinded path* he would use the same double-encrypted format: that's so Charlie can't tell whether a blinded path is being used or not, even though it's slightly less efficient. Crypto is cheap these days, too. Now, if Alice gives Bob a blinded path to Charlie and Charlie is Bob's peer, he can simply send the onion and the first blinding tweak to Charlie. But if Alice needs to send the message via Dave to Charlie, she needs to prepend a step. That's not quite possible, naively, because blinding tweaks are generated *forwards*, and she needs Charlie to get the right blinding tweak from Dave, and Alice has no way of making that happen. So inside Dave's encrypted blob, she uses next_blinding_override to tell Dave to hand that blinding override to Charlie instead of the normal one. I just implemented this for Core Lightning (previously we would simply connect to the first node, which is privacy-compromising and should only be done as a last resort). These blinded paths have some nice properties: you can't use part of them (you don't know the blinding factor except for the first one, so you can't start in the middle, and you can't replace any data), you need to use all of them. They can contain timelimits to avoid easy probing, too: a classic measure would be to see if the path fails when a given node is down, but that takes time. The spec insists all errors within the blinded path are the same, and originate from the entry: this loses some analytical power on failure, but makes probing harder. The entry point is supposed to add a random delay (we don't yet!). There may still be implementation differences, but they're hard for Bob to probe (and Alice doesn't need to, as she set up the path). npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell ChaumedToMeetYou BlindedByTheSats EllipticalTrainer (for Signet) SatoshiMince npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell The lesson of sigops was exactly that limits should be based on weight, so you don't need to optimize for two different constraints on block construction. Same deal here: more vbytes, more varops budget. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Now the good news. The basic idea is still intact: the speed is indeed proportional to operand size, so the basic idea is correct, it's just the costing of each operation is different. And we could simply say "what's the slowest operation, we'll assume everything is that slow", but comparing (OP_VERIFY, OP_EQUALS), copying (OP_DUP, OP_CAT) and zero-initializing (OP_UPSHIFT) are pretty common operations, so treating them as lower cost is both fairly simple and amply justified by the numbers. The evidence so far says we can divide into "fast" (copy, compare, initialize) and "slow" (everything else), where slow is twice as expensive as fast. This overcharges a little, but our thesis is still that we should have ample budget for any resonable operations, and I hope to prove that once I have some realistic placeholder values for the model. Indeed, we can certainly further optimize various opreations using SSE/AVX/NEON. That's awesome, but I've only gone so far as to assume a 64-bit processor which can perform these operations 64-bits at a time: again, such optimizations would mean we're overcharging for some operations, but my "we have so much budget we don't care" thesis means it doesn't matter. If it becomes commonplace to do (say) multiplications on 4096-bit values, I'm sure someone will do the work to shave a few microseconds off block processing times. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Bad new first. The "bytes touched" model does not actually reflect real costs, even on the RPi 3 (the lowest machine specs I'm measuring). Why? Caching is real, and these kind of linear operations which Bitcoin Script consists of are the most optimal case for readahead, write combining and all the tricks modern CPUs employ. A bytes touched model would have AND-in-place (A &= B, where A and B are 4MB long) be twice as expensive as invert (A = ~A): on my laptop they're actually comparable. The slowest operation? 1 bit shift in place! Doing this to two 4MB blobs: Rpi3: 16ms, vs ADD (twice) at 12ms. Laptop: 2.0ms vs 1.1ms. This is because the data dependency cuts down on the ability to pipeline, as the next value depends on the previous. This is interesting, and more work is needed on finding the worst-case add for similar reasons: if you never carry (i.e. overflow on each addition) the branch prediction will be correct, so I will be benchmarking different patterns (carry every second one? Every eighth?) to find the worst case in real life. Given that we're actually CPU constrained, not memory bandwidth constrained, you might be able to guess the fastest operations. Those optimized to use SIMD instructions! So memcmp, memcpy and memset etc are blindingly fast: my simple loop-coded memset on RPi3 takes 17msec for two 4MB blobs, but memset takes 4.1! (1.8 vs 0.45 on my laptop). npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell #Bitcoin #dev #GSR At https://btcplusplus.dev/ in Austin I presented my work on restoring Bitcoin Script. Script had an emergency amputation in 2010 as there were no limits on resource consumption; restoring it means solving this problem. My proposal is a runtime limit, similar to Taproot's runtime sigops limit, called "varops". How much you get depends on your transaction size (ie how much you paid), similar to sigops. But how much should operations cost? I had a cost model based on "worst-case bytes touched" for each operation. I've spent the last few weeks doing increasingly precise micro-benchmarks on my laptop, my build machine, and various Raspberry Pi. There's bad news, and good news.... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell I couldn't find anything on the OpenSats site about how long LTS grants are? A year, three, five, ten? Paid monthly, annually, time-locked? npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Python: it just works, they said! We've generally had less problem with Rust, though that's still optional. Though OMG is it slow... npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell Yes, I understand this feeling. Not quite how I would have expressed it, but there are genuine insights in here: https://ludic.mataroa.blog/blog/i-will-fucking-piledrive-you-if-you-mention-ai-again/ npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell 12 words is more of a super power. npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s Rusty Russell For us, it's often how slow the CI is. cpuset can be used to reproduce this...