Rusty Russell

https://rusty.ozlabs.org

Lead Core Lightning, Standards Wrangler, Bitcoin Script Restoration ponderer, coder. Full time employed on Free and Open Source Software since 1998. Joyous hacking with others for over 25 years.

Public Key

npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s

NIP-05 Address

[email protected]

NIP-57 Address

npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s@npub.cash

Profile Code

nprofile1qqs0zuj4s6jq9sr2ajqc69rc53d25rwpd3afcjrfm97r2qek69hcuscrrzqtu

Publishing to

nos.lol

relay.damus.io

Author Public Key

npub179e9tp4yqtqx4myp35283fz64gxuzmr6n3yxnktux5pnd5t03eps0elz4s

Show more details

Last Notes

2024-06-19 09:10:48

- reply

12 words is more of a super power.

2024-06-18 01:25:23

- reply

For us, it's often how slow the CI is. cpuset can be used to reproduce this...

2024-06-17 13:06:54

- reply

Yeah, we need to handle (non-trivial) blinded paths in offers. I've assigned this to me for next release.

2024-06-16 14:05:34

- reply

We try hard not to do that! Backwards compatibility should always be the default

2024-06-16 01:12:46

#dev #CLN
One of the fun things after a release is that I sweep the code and remove things which are now past the end of their deprecation cycle, and then update to the latest BOLT specs. The former is usually trivial (this time there were only two things to remove, and nobody will care). The latter can be more interesting, and was.
Our code contains quotes from the BOLT they are implementing, so when we update the version it reports what has changed. Sometimes it's merely a typo fix, but since last release we actually made a pile of old features ASSUMED. This includes "option_static_remotekey" which we supported in CLN since January 2019 release, meaning you will only see channels without this if they were opened more than 5 years ago. We still support these existing channels, but all support was removed for opening new ones.
The other option is "option_anchor_outputs": we were the only ones to ever deploy this, and then only in experimental mode, because zero-fee anchors quickly replaced it.
Unfortunate to I don't know how many (if any!) such channels there are!
So what do we do about existing channels? I can enable the currently-experimental code to migrate such channels, which works if both ends are CLN. But it's not clear how much further I should go: my spec proposal for upgrade has been in limbo for years, and at some point it's far cheaper to pay a handful of people to reopen their channels than it is to do the engineering required for migration.
It's a few versions away yet, but at some point we will have to refuse to upgrade if the user has such a channel, in the hope they will contact us if it happens.

2024-06-15 02:34:35

Great podcast: I gain thoughtful insight from an expert in a field I'm not an expert on.
Good podcast: I gain minor insight into how to discuss a field of my expertise.
Bad podcast: Someone who knows no more than me about the subject.
It's ok to not know something! But I wouldn't go on a podcast and discuss AI. Or business cases for Bitcoin, nuclear power, economic incentives, politics, nostr...

2024-06-12 02:01:25

Sneak peeking at new bolt12.org re-wrought by @npub1spr…8s72 (especially Stephen DeLorme, Brandon Lucas and the Bitcoin Design Community incl @npub14vn…s0ny). So.... pretty....

2024-06-11 11:43:52

- reply

This is nothing to do with Taproot or Segwit. You are very confused.

2024-06-11 11:42:31

- reply

No segwit, no lightning.
You really don't understand Bitcoin, so I'm going to balance your opinion appropriately.

2024-06-11 11:17:44

- reply

How did this "spam" affect you, if you're not complaining about fees?

2024-06-11 10:53:30

- reply

BTW "It is not possible to delegate the work without delegating the key (supposedly)" is wrong: you can definitely do this! Various vanity address generators did this back in the day, for Bitcoin.

2024-06-11 04:47:29

- reply

"Bitcoin is feature complete."
You keep repeating this mantra. When did this happen? Before or after we did a soft-fork to enable lightning? It seems you are unaware of this?
Because there are things like vaults which do require additional features. Similarly multi-party lightning.
Your absolutist position seems quite hard to justify, though I'm trying to give you an much credit as I can, lest this become Twitter.

2024-06-11 04:35:12

- reply

"Segwit and Taproot helped produce the spam that's currently attacking the network."
This is simply wrong. Segwit increased block size, so you could pack 4MB instead of 1MB into blocks. This worst-case was well understood. And indeed, most nodes didn't notice (block relay is not the majority bandwidth for default nodes).
Without segwit, the spam would have driven up fees much more: they were clearly prepared to spend the money!

2024-06-11 00:20:47

@npub1tea…gq5u Verifying My Public Key: "rusty_twit"

2024-06-10 02:08:26

There are always many things you can do with your time, but implementing silent payments (https://github.com/bitcoin/bips/blob/master/bip-0352.mediawiki) for Core Lightning is rising in my to-do list.
Unfortunately, we are all PSBT internally, so we need support so we can receive, which the authors argue (sensibly) should be the first priority for wallets. There's a delving post on this, so I did my bit by annoying Ava Chow into looking at it. We can probably hack in some experimental thing if we need for now, but proper support on libwally would smooth the path for Blockstream Green in future, which frankly is a much bigger win than CLN support.
Looking forward to the day when they're not "silent payment addresses" but simply "Bitcoin addresses".

2024-06-09 09:27:06

- reply

Um so, I need a *third* nostr client?
If only nostr messages were authenticated somehow so I could send instructions to the site. Or y'know, send a one-time password to log in like humans do.

2024-06-09 09:04:07

- reply

Everything I press seems to go to "Choose a login method" and then??? I have Amethyst on Android, Gossip on desktop...

2024-06-09 08:28:51

One day I will reach the pinnacle of nostr technical capabilities, and be able to access my npub.cash funds. Right, @npub12rv…85vg?

2024-06-09 06:13:28

- reply

I'm delighted to hear that!
There's an old proposal called "eltoo" (or, these days, called "ln-symmetry") where there's no penalty phase if one side accidentally publishes an old state: you just fix it up instead. This does some cool things:
1. No more risky backups, where you can lose all your funds by restoring an old backup.
2. You can now have more than 2 parties sharing a channel, and build on top of that.
3. We can reduce latency for payments in some cases, making it twice as fast.
4. It's generally simpler: you only need to remember the last state, as you can fixup and old ones
As to not knowing who I am, that's totally fair! I would much rather people evaluate a proposal on its merits, not the reputation of the proposer. As a non-expert, it can be impossible to distinguish someone who is good at their job, from someone who is good at self-promotion :(

2024-06-09 05:47:00

- reply

That's the whole point, though? You said "use lightning" and I said "we needed a protocol change to do that".
To get here, we needed changes. To get to our full potential we need to continue. Obviously it depends on the specifics of the proposal, but I would urge you to keep an open mind.

2024-06-09 05:23:33

#nevent1q…s9y5
I am who I am. My voice may well get drowned out by those who are louder, better resourced or more persuasive. But as an engineer working in this space, I need to tell the raw unfiltered truth as best I see it, and I hope others feel the same?

2024-06-09 05:21:32

#nevent1q…g9jr
He explicitly argued that a miner who spends $1B on mining infra should not have their investment undermined by protocol changes. It was pretty concrete!
They would definitely argue that a protocol change which increased space efficiency would have a short-term effect on fee revenue, right? It seems a pretty clear-cut case where Saylor's model is directly against most people's understanding of what we should improve.

2024-06-09 05:14:08

- reply

The balance shifts with fees. I think Lightning is the current obvious go-to as fees rise, and for Lightning 5 txs a month seems a *lot*, so I think that there is at least another order of magnitude here, before we get to shared UTXOs or similar mechanics...

2024-06-09 05:01:55

- reply

For better it worse, the sophistication of scripting doesn't seem to matter: if you give $1M to the first non-coinbase tx in a block, eventually miners will write systems to take that money :(
If you write any protocol where you let miners choose the winner, you get this problem if the effect of winning is economically significant. Of course you shouldn't create protocols that stupid, but when there's a race to collect money from gullible people, engineering very much takes a back seat.
The real defence seems to be that economically useful actions on Bitcoin can outspend stupidity over the long term. I certainly hope this continues, and I suspect (though cannot prove!) enabling more people to use Bitcoin economically will further tip the balance in its favor.

2024-06-09 04:41:58

- reply

That is an amazing and inflammatory take, and why people do often avoid serious discussion on such topics :(

2024-06-09 04:39:34

- reply

What changed, then? You would not have enabled lightning?
There is a change we want to make lightning simpler and more efficient as well as multi-party. It's not a complex change: in fact there are several ways we could do this. But I assure you that the reduced Script we have today does not allow " everything else... built on layers" :(

2024-06-09 04:22:32

- reply

This approach leads us to an increasingly complex and awkward scripting system, as we add more and more special cases which must be maintained forever.
If we can restore script as it was originally, as a general programming language, we don't have to try to "pick winners", and you don't have to ask permission to do what you want with your own UTXOs. Doing this safely and cleanly is my current area of research.
My favourite thing about this approach is that following soft forks become less about "is we had this new feature we could do X” and more "this would make existing scripts more efficient and allow us to do X% more in a block" which is a much more quantifiable assessment.

2024-06-08 16:40:24

- reply

I certainly haven't felt his support of lightning, but perhaps I missed it? It seems like he very much would have opposed the changes which made it practical, and now he sounds opposed to changes which would make it more effective. Hard to tell, since the conversation was in analogies, not specifics though!

2024-06-08 16:13:55

- reply

Indeed! But if I try to argue the analogy, I might say "Satoshi glued the ailerons into a fixed position as an emergency stabilization procedure and we've been flying like that ever since". But there's no evidence that any conclusions we you might draw from that are actually useful in real life!

2024-06-08 05:16:10

- reply

Thanks! I'll copy to self host when I'm back at a keyboard!

2024-06-08 05:11:58

I listened to the What Bitcoin Did Saylor podcast, and I really want to respond, though that may be unwise. But I want thoughtful, fearless content in my feed, so I should start making some, right?
Firstly, while analogies can provide useful guide rails for understanding, listening to people *arguing* using analogies makes you stupider. Debate the thing itself, not the words about the thing: it hurts my head to even think about doing this, so I won't.
Let's set my priors first: I assume we're talking about technically solid, well-vetted, backward compatible protocol changes: this is the minimum bar.
I don't wholesale agree with Saylor's "don't threaten anyone's investment" hard limit. This has happened multiple times in the past, from the dust limit breaking SatoshiDice, enabling Lightning threatening miner fees (real or not), and segwit breaking stealth ASICBoost. These interests can, and will, stand up for themselves and will compete against other benefits of changes.
To be explicit: I consider any protocol change which makes block space usage more efficient to be a win!
Obviously Saylor is invested in Bitcoin the asset, and can afford to do all his business onchain in any conceivable scenario. His projection of a Bitcoin world in which there are 100,000 companies and governments who use Bitcoin as the base layer is interesting:
1. This does not need "smart contracts", just signatures. By this model, Bitcoin Script was a mistake.
2. It can work if Bitcoin does not scale and is incredibly expensive to spend and hold. By this model, the consumer hardware wallet industry is a dead-end and needs to pivot to something else (nostr keys, ecash?)
3. You could do this with gold, today? Bitcoin here is simply an incremental, not fundamental, improvement. I think this is suggestive, though: that such a network would not be long-term stable, and very much subject to capture.
4. In this view, Saylor is simply a gold bug with first mover advantage, shilling his bags. That's fine, but it's important to understand people's motivations.
5. This vision does not excite me. I wouldn't have left Linux development to work on making B2B commerce more efficient. I wouldn't get up at 5:30am for spec calls, and I sure as hell wouldn't be working this cheap.
I believe we can make people's UTXOs more powerful, and thus feel a moral responsibility to do so. This gives them more control over their own money, and allows more people to share that control. I assume that more people will do good things than stupid things, because assuming the other way implies that someone should be able to stop them, and that's usually worse.
I believe the result will be a more stable, thus useful, Bitcoin network. I am aware that this will certainly benefit people with very different motivations than me (Saylor).
Thanks for reading, and sorry for the length!

2024-06-08 03:30:42

#dev #scriptrestoration
Took some time off this week, and it was slow anyway. I have ordered a proper power supply for my RPi3 as compiling Bitcoin is crashing it (which bodes badly for actually running the benchmarks!).
After some head-scratching results I have returned to micro-benchmarks. My initial model treated every read and write as the same cost, but I'm investigating alternatives. In particular, the ubiquity of SIMD instructions and write combining in modern processors means some operations are *much* faster than a naive "pull 64 bits into a register, write it out". I wouldn't care about beating the model generally, but:
1. I have to make sure my benchmarks are measuring the right thing, and
2. OP_DUP, OP_EQUALS and the like are so common that if they really are twice as fast as the naive versions on every platform, it's worth considering whether the model should accommodate them explicitly.
So, I'm now breaking them down into micro benchmarks for each case explicitly. There's so much data here, particularly when considering different size operand limits, that I'm going to have to produce a series of graphs to illustrate (and hopefully illuminate!) the results.

2024-06-02 04:34:09

- reply

Australian, so in feel you. In reverse order of importance:
3. I tend to have to hit the ground running, so I unlimit my caffeine and sugar intake when overseas.
2. Fly business overnight: being able to rest, even if not sleep properly, really makes a difference.
1. Half the stress is anticipation: I try to travel very Zen. Once I'm at the first airport, I chill. Don't have any explicit goals for working on the flight, don't stress security, delays, etc. Calm and stillness.
Also when I travel for pleasure with the family, I have an ugly Hawaiian shirt I wear to emphasize to myself that this is *not* a business trip! Sets the holiday mood early...

2024-06-02 01:05:17

- reply

But there's a well-known business model where you only allow one-way trade for your tokens. It's very lucrative if you have volume, as people over buy and then can't trade out easily. Think parking meter minutes, in-game currency, mobile phone credits, gift cards, airline miles....

2024-06-01 06:17:32

- reply

I don't see any evidence in your post, just hope?

2024-06-01 01:23:09

- reply

I don't usually comment, but since I'm trying to do nostr-only content...
Ecash can be helpful or a hindrance. Basically, if the norm becomes closed non-fungible systems, where you cannot get out (think gift cards, arcade tokens) that's a clear lose. Unfortunately running a system like that is great for the person running it.
On the other hand, if the expectation is fungibility: that you can seamlessly lightning out 24x7, and mints which don't do this are widely eschewed, then the promise is great.
I hope for the latter, but fear the former. If the pioneers set the social norms correctly, it might just work though.

2024-06-01 00:51:12

- reply

Still do, but since my workload no longer resolves around email I sometimes go a week without reading it at all.
I also deliberately don't have access to my (non-Blockstream) email on mobile.

2024-06-01 00:49:22

- reply

Never was. Runes are CLN's simplified alternative to LND's macaroons :)

2024-05-31 14:08:48

#dev #cln
This morning Shahana reminded me of an idea we had earlier to create "categories" for commands, so runes can filter on that basis. But choosing the categories is a bit of a research project which we never got around to, so we sat down and did it: described using keywords of each documented command, then tried to come up with a list.
The original motivation was the "read-only" commando rune: originally it allowed all list commands, and the getinfo command, except not listdatastore because that gave you access to the master rune itself! That's no longer true, but the point stands: some commands are read-only but reveal information which could allow escalation. ,(This is why when we added the command to list runes we called itshowrunes, not listrunes!).
When plugins add commands, you want it to be intuitive. If a rune allows the pay command, that won't allow "mpay" for example: the rune probably wants to say "you can send lightning payments" and cover all the commands that are needed, or can be used, to do that.
Anyway, the exercise resulted in a first list of categories, which we will play with as we implement: they need to be in documentation, as well as accessable to runes. And Shahana suggested the term "genus" (well , genera) instead of "category".

2024-05-31 01:56:30

#dev #cln
Yesterday I brunched with a friend, that turned into lunch. I try to catch up with a friend every week or so, to keep my work hours contained.
Nonetheless, I continued my work on connectd: it now limits the rate it will stream gossip (1MB per second should be enough for anyone!) but this doesn't apply to responses to gossip queries, which get handed off to gossipd to answer. Connectd has the gossip store open to stream it: if it simply switches to our gossmap library it can answer these queries itself, and rate limit the processing correctly.
It also means that it will be able to look up node announcements for connections itself, rather than having them fed from lightningd as now. This simplifies things, and even more once I have connectd automatically reconnect to important peers, rather than waiting for lightningd to tell it to reconnect.

2024-05-29 11:03:55

#dev #cln
Today was a good day! On this morning's call with Alex we discovered the cause of a INVALID_ONION error which Elle had just reproduced: LND will still send legacy onions of it doesn't see a node's announcement. We removed support for that years ago, after it was removed from the spec. Still, before breakfast I had a working and tested implementation to sneak into rc2. Not happy about it, but interop wins over my feelings unfortunately!
https://github.com/ElementsProject/lightning/pull/7352
I also finished code to properly send reestablish messages for closed channels: we weren't doing this properly since some other code changes last release, preferring to send the last error we had. Which was just as well, because doing so was triggering a corner case where we could delay on chain cleanup on older versions (embarrassingly, my logs show this happened in my own node in Feb 2022, with v0.10.2: I thought it was broken because I messed with my DB using an experimental version!).
https://github.com/ElementsProject/lightning/pull/7353
Finally, I wrote a simple benchmark to vaguely simulate the slowdown/high CPU large nodes like Boltz are seeing: I wanted to this since Michael let me strace and profile their connnectd in Austin. The actual optimization was surprisingly trivial, and makes the time spent in poll() down to noise. As a bonus, I also implemented and tested gossip rate limiting (1MB per second). PR coming tomorrow!

2024-05-27 11:45:14

#dev #cln
More work on askrene today. The infrastructure basically done, now I'm stealing Eduardo's min cost flow code.
But even that isn't trivial to drive. You get the min cost flow but the resulting path might be too expensive, or too unlikely, or too slow, and you need to tweak parameters (fees vs probability vs delay) then ask again. The current code does a binary search for this over a mu value from 0 to 127, so 7 times. I need to play with this, as I suspect we can shortcut this if we're "good enough".
Once you've got that, you look at htlc_max and htlc_min that had been published on the channels: you might have violated that too! If you hit the min, you remove that route and block the channel from future attempts. If you hit the max,, you reduce the amount you send down that route. Then you have some sats you still need to send, so you ask the min cost flow solver for the solution for just that part. This is actually unusual though: most channels route most payments.
The question is, how much goes inside the pay oracle, and how much can we punt to the caller? I think the caller can only specify the max fee and max delay, and the oracle needs to give the highest probability routes within those constraints.
I hope to have something tomorrow, but it's going to need a lot of tests! Fortunately this should be easier to test than a complete pay plugin.

2024-05-25 14:43:32

Just confirmed that I'm heading to Cheatcode in Sydney in October! Don't know what I'll speak about, but I'm sure there will be something happening...
https://www.cheatcode.co.uk/

2024-05-25 01:04:53

#cln #dev
Shahana brought a request to delay startup if bitcoind had gone backwards, rather than refusing to start. This is good for umbrel and like apps where it can happen (disk corruption causes bitcoind to have to resync). Seems simple enough, but that code proved really awkward to work on: so much so it took over two hours for me to get it working without breaking other cases.
"Don't patch bad code, rewrite it" said Kernighan and Plauger. So I spent my afternoon reworking the chain topology startup code to something I wasn't embarrassed by: it now simply and straightforwardly asks for the three pieces of information it needs from bitcoind at startup, then performs initialization. Then later it starts the various polling loops (for fees, and for new blocks).
Adding code to delay is now trivial. And the next person who touches this code will have a much more pleasant time without all the scar tissue which was left from previous incremental changes.

2024-05-24 01:56:59

With remote interviews the norm these days and I've taken to using a "runaway" mode where I talk long-form rather than let the interviewer drive.
Maybe I should slow down, and try to answer each question with two different takes to let the interviewer choose the path?
#nevent1q…38w3

2024-05-23 14:15:50

- reply

I sit, I think, I type. Just me so far: it's too in flux for anyone to contribute without getting frustrated that I rewrote it again!

2024-05-23 10:11:07

#dev #cln
Shifted gears today: there's a bug that Christian Decker brought to my attention. There's a Greenlight node where onchaind hasn't finished spending a HTLC tx, which shouldn't happen.
Weirdly, I have one of these on my node, but that didn't hugely surprise me as I have done experimental upgrades in the past and edited the db by hand on at least one occasion 😬
But now I know it's real, time to find out why. Since Greenlight restarts nodes all the time, it's pretty clearly a failure across restarts.
This code used to be trivial: on restart we would start processing the chain back at the earliest outstanding channel close. This was the exact same as if it was happening the first time, only faster. But replaying up to 100 blocks at startup is slow, so instead we recorded events on the DB and replay from there on startup. And I never really liked that code, but it *was* much faster.
But I would rather be simple than fast (as long as we're not preventing the rest of startup completing), and since there's a bug, it's a chance to simplify.
We *do* need to save the tx which closes the channel, and that's simple and fine. But then on restart we can simply walk through blocks starting with the one containing the funding spend, and tell onchaind about all spends. This is how onchaind works: we follow all txs which spend the channel output, and their children, and as an optimization it tells us to ignore txs it doesn't care about.
This didn't actually take that long to code, and as a nice side effect it will even work on pruned nodes, as in the latest release we now ask bitcoind to refetch old blocks if they're pruned.
I'll report what happens when I actually run it on my node tomorrow: onchaind does spam my debug logs with a huge number of messages right now, and it has been closed for 120,000 blocks!

2024-05-23 00:06:33

- reply

Erk, this comment is so HN:
bogwog 1 day ago | prev | next [–]
Nostr would be so cool if it wasn't associated with bitcoin. I know the protocol has nothing to do with crypto currencies whatsoever, but the people who use it are predominantly in that world.
This is a cool project though, so nice job!

2024-05-22 15:41:00

- reply

Yes! Once the basic work is done, there will be plenty of work considering what other things make sense, such as OP_TXHASH or CSFS, and I'm leaving that to others to wrangle...

2024-05-22 10:33:36

- reply

Figure I'll write what I work on each day, why not?

2024-05-22 09:20:41

Today, I got back to working on "askrene" for Core Lighting. The idea is to extract the core logic of the experimental "renepay" plug-in into an Oracle you can ask for routes (and provide feedback on what happened). This is much more composable: anyone can write a plugin which uses this information, *or* an alternate pathfinding plugin to replace it.
It's built around the idea of "layers" which are where information lives: you tell it what layers to use when you ask for a set of routes. This has many uses: you might have route hints or blinded paths you want (or have to!) use. You might want to constrain a payment to a particular channel for rebalancing, etc.
In theory, these layers can be exported and imported: you can share information about the state of the network between nodes. I'm sure there's a pile of obfuscation needed to preserve privacy in this case, but it's an interesting idea...

2024-05-22 02:26:25

Wow, nostr is giving very "welcome to Living, now configure your modeline" 90s vibes. Lots of clients, most abandoned, very easy to "choose wrong" and not be able to do things like NIP-46. And yes, you're going to have to learn what that is :)

2024-05-22 02:15:01

- reply

Nope, looks like that's a client not a server. Trying 'cargo fillmydisk' on gossip, which seems to be a popular client. Which will probably need an ssh tunnel because, y'know, web...

2024-05-22 01:56:47

- reply

Except, Amethyst on Android. No NIP-7, no NIP-46. I guess most people use a web extension? Hmmm, trying nkcli, wish me luck!

2024-05-21 23:46:33

- reply

Literally founded by core developers, so no surprise or conspiracy there...

2024-05-21 12:39:59

- reply

I gotta write some code for that! Hard to do on mobile...

2024-05-21 12:27:07

Since returning from Bitcoin++ in Austin, I've been trying to alternate between working on Lightning one week, and Bitcoin hacking on the other.
Mainly, I've been finishing my Script interpreter cpp modification so I can make more comprehensive benchmarks. I've also given in and ordered a raspberry pi 3, 4 and 5 to benchmark.
I hope to answer the key questions on what a realistic budget for large stack objects should be. In particular, there are four reasonable limits on their size (I'm not considering changing the 1000 stack element limit, since that seems roomy!)
1. 520 bytes. We are here now, but as Script gets better that will be very limiting.
2. 520,000 bytes: simply limit the total stack size, not individual entries.
3. 4,000,000 bytes. This allows you to put the largest possible transaction on the stack. But you need a total limit of at least twice this, so you can make a copy: you can't do much in script without doing this!
4. 400,000 bytes. If number 3 is too much of a reach, this means you can put any standard tx on the stack. Again, you want a total limit of at least twice this.
My laptop, with something stupid like 26MB of level 3 cache, and my build machine with 32MB, can handle this fine. But what about the Pi?
I'll find out next week...