Motivation

With Taproot activation coming soon, now is a good time to look at BIP-118, now known as Sighash Anyprevout and previously as Sighash Noinput. This soft fork enables a powerful new mechanism for replacing presigned transactions in off-chain protocols. The new Lightning Network state update protocol Eltoo, for example, was designed around the existence of this transaction replacement mechanism.

In this blog post I will describe how to use the Bitcoin Core functional test framework to test a set of Bitcoin Taproot transactions required by the eltoo protocol. These tests use a fork of Bitcoin Core with an initial implementation of BIP-118 that builds on Taproot. In the Addendum I also summarize the relevant motivations and features of Anyprevout, eltoo and Taproot to give context for understanding these eltoo transactions, as well as the detailed Bitcoin scripts and transactions themselves.

The target audience for this write-up is anyone who wants to understand BIP-118, eltoo and Taproot better at the transaction and script level. You should be able to build Bitcoin Core from source and understand Python scripts. Following the steps below you will be able to see eltoo transactions confirmed on a local regtest network. My hope is that this will help people understand how these technologies work and give context for reviewing BIP-118. More people understanding and creating transaction tests for BIP-118 and eltoo will help find problems and improvements for both. More review and testing is critical before BIP-118 can be adopted by the Bitcoin network and eltoo by the Lightning network.

Tests

I recommend that you confirm your system is setup to build Bitcoin Core and run the functional tests. If you have never done this before, then Jon Atack has a good summary of how to compile bitcoin core and run the tests. Confirm your ability to run the functional tests (Step 8. in Jon’s summary) before proceeding.

You can now follow the steps again, but from my fork of the Bitcoin Core repository and using the eltoo-anyprevout branch:

git clone --single-branch --branch eltoo-anyprevout https://github.com/remyers/bitcoin.git bitcoin-eltoo-anyprevout
cd bitcoin-eltoo-anyprevout
git checkout eltoo-anyprevout

Now rebuild bitcoind patched with BIP-118 and run the new eltoo functional tests via the python script, for example:

./autogen.sh 
... follow any other steps required to build for your system
make
test/functional/simulate-eltoo.py

The resulting output should end by showing that all tests passed:

Success!
2021-07-27T10:08:10.249000Z TestFramework (INFO): Stopping nodes
2021-07-27T10:08:10.505000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_yjbnqj84 on exit
2021-07-27T10:08:10.505000Z TestFramework (INFO): Tests successful

Tests Explained

Taproot tests

The file /test/functional/simulate-eltoo.py runs two tests. The first is test_bitcoinops_workshop_tapscript which goes through a short Taproot test based on the Schnorr Taproot Workshop created by Bitcoin Optech. I highly recommend these tutorials, but note that some changes in the Taproot BIP occurred after the workshop - specifically the change to “x-only” public keys which reduce the size of keys by one byte. You can also check out their recent Preparing for Taproot series.

eltoo tests

You should read both the BIP-118 specificaton and the eltoo paper to understand these tests at a high level. The Anyprevout, eltoo and Taproot sections below also summarize details relevant to understanding these eltoo transaction tests.

The second test, test_tapscript_eltoo performs the following tests after initializing some addresses, generating some coinbase outputs on the regtest network and funding the first eltoo state output.

All of the other transactions before the divider line are presigned transactions that in eltoo would occur without being immediately confirmed. Note that these tests use a normal Schnorr key and not a 2 of 2 multisig key created with something like MuSig2 for the internal Taproot key called pubkey_AB; except for how the public key and signatures are generated there should be no functional difference in how it is used in the test scripts.

  1. The first set of tests uses the function test_transaction to test if a transaction is valid without actually submitting it to the mempool. The first test checks if the update1_tx transaction is valid. We also confirm that settle1_tx when signed with APO can not be rebound to update0_tx but that it can be rebound to a different state if incorrectly signed with APOAS. Because APOAS does not commit to the exact script of the output this would allow any of the presigned Settle transactions to be bound to any of the Update transactions. The script then commits update1_tx into a block and generates CSV_DELAY number of blocks so that settle1_tx will be valid.

  2. The next section starts by modifying settle1_tx and update2_tx to rebind them to the specific update1_tx transaction that has now been confirmed on the blockchain. Because these transactions were signed with the APOAS and APO sighashes they do not commit to the transaction ID of a specific transaction. After these transactions are tested with test_transaction, the script commits to update2_tx to advance the channel to state 2.

  3. We then create the update1b_tx transaction which attempts to spend the update2_tx output back to channel state 1. This transaction uses the same pregenerated signature used by update_1 but rebinds it to update2_tx instead of update0_tx. When update1b_tx is tested, it fails with the error message, 'non-mandatory-script-verify-flag (Locktime requirement not satisfied)' because the signature commits to a transaction with a locktime of 1, which is less than the locktime encoded in the spending script of update2_tx and so the script fails.

  4. The next set of tests request we commit settle2_tx which can only spend from update_2 because it uses the APO, and not APOAS, sighash. The APO sighash must be used because although the update_2 script and amount are known, the exact transaction id of update_2 was not known when it was signed and can change based on if update_2 was bound to update_0 or update_1.

  5. The final tests demonstrate how to spend the HTLCs outputs of settle2_tx using either a timeout via htlc_refund_tx or a hash preimage via htlc_claim_tx.

Fees

Off-chain protocols must presigned transactions well in advance of when they are committed to a block. This means signers may not know the amount of fees to add for a transaction to be accepted into a block before a timeout critical to the protocol expires. The ability to select transactions fees at the time a transaction is committed reduces the risk a pre-signed transaction has too low a fee.

In these eltoo tests, HTLC refund and claim transactions are only signed by the channel partner collecting the HTLC and can include an additional fee input and change output. They also do not need to use an Anyprevout sighash. Update transactions also have a single input and output, so by using SIGHASH_SINGLE can add a fee input and change output just before commiting to the blockchain. Only the Settle transactions needs to allocate part of the channel balance in advance as transaction fees. The Challenges section discusses how this might be solved for eltoo and other off-chain protocols.

BIP-118 Unit tests

In addition to high level functional tests for eltoo, there are also low-level tests added to src/tests/sighash_tests.cpp to exercise the new BIP-118 sighashes. They are implemented by the tests sighash_anyprevout_legacy and sighash_anyprevout_taproot. Review and expansion of theses tests would be very helpful.

Next Steps

These transaction tests are only a very initial attempt at implementing eltoo so as to better understand how it works with BIP-118 and Taproot. Adding additional features will bring them closer to an actual working implementation.

PTLCs

Tapscript uses Schnorr signatures which allow us to replace Hash Time Locked Contracts with Point Time Lock Contracts via adapter signatures. This will require modifying our htlc_claim_tx transaction to take a modified signature only and not a preimage.

PTLCs have three main advantages: they enhance privacy by removing the common preimage hash from transactions, transactions get smaller (which saves on transaction fees) and it also allow for “stuckless payments” as well as other more advanced features like escrowed payments.

Layered Commitments

AJ Towns has proposed a modification of the eltoo protocol called Layered Commitments as a way to reduce the CLTV expiry delta of eltoo which currently must include the CSV expiry time of its Update transaction which occurs before the Settle transaction becomes valid. Layered Commitments offer a way for each HTLC (or PTLC) to commit to either the Claim or Refund path before the CSV expiry time has expired and thus remove it from the CLTV expiry delta.

Multiparty Channels

A key feature of eltoo is the ability to support multiparty channels which can be used to create Channel Factories as originally described by Burchert, Decker and Wattenhofer in Scalable funding of Bitcoin micropayment channel networks. The current Poon-Dryja Lightning update scheme requires each channel partner to hold a different transaction, but using eltoo allows all channel partners to hold the same set of transactions for each update. This works because in eltoo newer state updates can spend from older ones, but Poon-Dryja channel updates assign a penalty to one side or the other if an old update is used. Assigning a penalty to enforce increasing state updates is difficult if you have more than two parties in each channel.

Extending these tests to explore channels with multiple parties will help confirm implementation details of this channel type. We would like to identify any problems or possible optimizations needed to make Channel Factories, and off-chain transactions generally, practical using multiparty channels.

Musig

To refine message passing in our future eltoo simulation we should replace the single party Taproot internal key used by our eltoo transactions with a 2 of 2 multisignature Schnorr key using something like the MuSig2 protocol. This will also include exploring how the PTLC signing protocol can be combined with the Schnorr multisignature protocol. This step should also include a thorough examination of how to use a multisignature signing protocol with the least communication overhead while preserving its safety.

Simulation

Once we have finalized the eltoo transactions we can update the simulation code originally developed for the segwit (SIGHASH_NOINPUT) version of eltoo. A higher level simulation will help when creating more complicated tests and evolve to include mock message passing and other logic to more closely emulate the full Lightning protocol.

Challenges

There are currently two main challenges identified for an eltoo implementation using BIP-118. The first is how to avoid long CLTV expiry delta values; it is hoped that the Layered Commitments scheme points the way to a solution. A potential downside of Layered Commitments though is that it removes the ability to use SIGHASH_SINGLE to defer adding transaction fee inputs and outputs to an eltoo Update transaction until it is committed to the blockchain.

The second challenge is how to generally have the ability to add transaction fee inputs and change outputs to eltoo transactions when they are commited to the blockchain instead of negotiating these values in advance to be paid from the channel balance. Current methods like RBF and CPFP could possibly help. Another suggested mechanism would be to replace SIGHASH_SINGLE with a mechanism to commit to a subset of outputs. Some ideas around this have recently been discussed in the “A Stroll through Fee-Bumping Techniques : Input-Based vs Child-Pay-For-Parent” thread on the bitcoin-dev mailing list. In particular the SIGHASH_GROUP proposal uses the new Taproot annex to specify a range of outputs committed to by a particular input.

Summary

Implementing basic tests of the transactions required to implement eltoo using BIP-118 has been helpful for understanding both the BIP-118 and eltoo proposals and for being able to give useful feedback and review. Help is needed to implement the next steps outlined above; in the process we will likely uncover new challenges in addition to those already identified so far. BIP-118 requires a relatively small amount of code changes, but unlocks a huge design space for off-chain protocols of which eltoo is just one example. More people running and extending these tests will increase the amount of feedback and review and lead to faster consensus for adopting BIP-118.

Acknowledgements

I’d like to thank Christian Decker for his patient mentorship on all things eltoo related and AJ Towns for his help understanding the intricacies of the taproot versions of BIP-118 and eltoo. Thanks also to goTenna for their support of my Lot49 related Lightning development efforts and to the Chaincode Labs Summer Residency which allowed me to dedicate an entire summer to learning about Bitcoin and implementing my first eltoo simulation. Finally, my recent Bitcoin work was also made possible by a generous gift from the Human Rights Foundation Bitcoin Dev Fund.

Addendum

Anyprevout

BIP-118 proposes a soft fork that allows transactions to not just spend a specific previous transaction output, but instead any transactions output that uses the same signing key, in the case of SIGHASH_ANYPREVOUTANYSCRIPT (APOAS), or that spends an output with a specific amount and script in the case of SIGHASH_ANYPREVOUT (APO). Use of these two sighashes are enabled only for taproot script path spends of taproot transactions that reference a public key prepended with a new public key type. Some key features of BIP-118 are summarized below.

Taproot Key Type

To spend an output using a signature signed with APO or APOAS the tapscript must include a 32-byte schnorr public key prepended with the new ANYPREVOUT key type or alternately the single Anyprevout key type byte without a following 32 byte schnorr public key. If only the key type byte is used, then it will be substituted with the taproot internal public key. This removes the largest piece of data from the tapscript for the most common spending condition and yields a significant transaction fee saving. The example eltoo transactions below make use of this feature.

Transaction Hash

The core changes of BIP-118 are to how transaction hashes are generated in the SignatureHashSchnorr method of bitcoin/src/interpreter.cpp . This table summarizes which fields are hashed differently for the two new sighashes:

  SIGHASH_ANYPREVOUT (APO) SIGHASH_ANYPREVOUTANYSCRIPT (APOAS)
Prevout Excluded Excluded
Amount Included Excluded
ScriptPubkey Included Excluded
Sequence Included Included
Tapleaf Hash Included Excluded
Sighash Anyonecanpay

If either sighash APO or APOAS are used, then the signature behaves as if SIGHASH_ANYONECANPAY is used and allows additional inputs to be added to the transaction without invalidating the signature used for the input signed with these new sighashes. This feature is used by the eltoo transactions to add an additional input for transaction fees.

Sighash Single or All

Sighash APO or APOAS must be combined with either SIGHASH_SINGLE or SIGHASH_ALL to indicate whether the signature commits to only a single output at the same index as the input, or all outputs in the transaction. The eltoo Update transaction is signed with both SIGHASH_ANYPREVOUT and SIGHASH_SINGLE which allows for the addition of both a transaction fee input and change output.

Prevout

Both APO and APOAS ignore the prevout field of the output being spent. This enables the creation of pre-signed transactions that do not spend only a specific transaction that has already been confirmed on the blockchain.

Amount

A transaction signed with APO, but not APOAS, commits to the specific amount of the output being spent. This creates a way to restrict which outputs can be spent from when the APO sighash is used. An input signed with APOAS has more flexibility to allow protocols that “consolidate a group of UTXOs with the same spending condition into a single UTXO” (see BIP-118 ref 3) and it also enables Layered Commitments.

ScriptPubkey

Only an input signed with APO commits to the exact script of the output being spent. An input signed with APOAS can spend any input that has matching spending conditions, even if the hash of the exact script is not the same. For example, the script for an eltoo state update transaction includes the number of the state. This means that the hash of the script will be different for each state. Spending conditions, such as the key used to sign, can be used to constrain which output can be spent when APOAS is used to create the signature.

Sequence

Both APO and APOAS commit to the sequence number of the specific input being spent. This allows a way to restrict which transactions can be spent when these sighashes are used.

Tapleaf Hash

APOAS does not commit to the root script of a transaction, defined by the ScriptPubkey of the output it spends, or the leaf script defined by the tapleaf hash of the spending transaction. APO does commit to a specific leaf script so must commit to a matching tapleaf hash.

Unchanged Fields

These fields are committed to in the same way both with or without BIP-118:

  • hash type: bit field used to sign, eg. SIGHASH_ANYPREVOUT | SIGHASH_ALL
  • version: of the spending transaction
  • locktime: of the spending transaction
  • spend type: bit field that indicates key vs. script spend and whether an annex is present
  • annex hash: hash of the annex, but only if an annex is present for the transaction
  • key type: indicates the transaction is signed with a normal Taproot, Anyprevout or undefined key type.
  • outputs hash: a hash of either all (SIGHASH_ALL) outputs in the spending transaction or just the output at the same index as the input of the spending transaction being signed (SIGHASH_SINGLE).
  • code separator position: included for all script path spends

See anyprevout.xyz and Bitcoin Optech for more discussion about the advantages and other uses of BIP-118.

eltoo

The eltoo channel update protocol uses floating transactions that can spend from any older channel state, but not a newer one, using the anyprevout sighash. The eltoo protocol has three primary improvements over the current channel update mechanism used by Lightning:

  1. no penalty for publishing an old channel state
  2. no need to store old state information
  3. compatible with multiparty channels

It may also be possible to reduce the need to preselect transaction fees for uncooperative close transactions.

I have based my tests on the taproot eltoo sketch from AJ Towns, but without chaperone signatures (ie. the ‘X CHECKSIGVERIFY’ in his sketch).

Overview

The Taproot eltoo scripts and transactions are pre-signed but not broadcast to the network except in the case of a non-cooperative close of the channel. Everytime the payment channel is updated, two new transactions are generated and pre-signed by both parties: one to immediately update the state of the channel and another to settle the value after a delay that gives time for a newer state to override it. When an update transaction is confirmed by the Bitcoin network, it can be spent, and effectively replaced, by a newer update, but not an older one, within a window of time after it was broadcast. After the relative timelock of an update transaction has expired, the corresponding settlement transaction can then spend its value and create outputs for settled value spendable by each of the two parties and inflight value as HTLC outputs that can be claimed by one party if they have a hash preimage, or refunded to the other after a timeout.

An update transaction uses the APOAS sighash to allow it to spend from any previous update transaction. A settlement transaction uses the APO sighash to bind it to a specific update transaction based on its script, which contains the state number, but not a specific transaction because the update transaction’s id can not be known when signing. A transaction to claim or refund an HTLC and collect the settled value only requires one party to sign so these can be signed after the settlement transaction has been confirmed in a block and does not need the APO or APOAS sighash.

APO, APOAS eltoo transaction diagram

Scripts

Scripts for the update, settle, claim and refund transactions are described below. The update transaction is used to immediately spend an earlier update transaction with a lower state number. The settle transaction spends from a corresponding update transaction after a relative timelock has expired. A claim transaction immediately spends from the HTLC output of a specific settle transaction if presented with the correct hash preimage. A refund transaction can spend from the HTLC output of a specific settle transaction after a certain absolute clock time.

Update

The first two lines of this script check that the corresponding transaction has been signed by a Schnorr key used as the Taproot internal key. This internal key should be a 2 of 2 multisig (eg. using MuSig2) key and the signature should be generated by both channel parties. In these tests though we use a Schnorr key and signatures generated in the standard way for simplicity. OP_1 is used for a single byte anyprevout key that is replaced by the internal Taproot public key. The ctlv_start_time + state value represents the state number of the channel and is increased every time the channel is updated. As per BIP-65, OP_CHECKLOCKTIMEVERIFY fails if the spending transaction has a nLocktime field with a value from an earlier state.

OP_1	 
OP_CHECKSIGVERIFY
<cltv_start_time + state>
OP_CHECKLOCKTIMEVERIFY

Note that OP_CHECKLOCKTIMEVERIFY leaves the state constant on the stack, but because it is the last opcode there is no need to add an OP_DROP.

Settle

The first two lines of this script check that the corresponding transaction has been signed by both parties in the same way as the update script. The csv_delay constant represents a relative block delay before which this transaction will not be valid. As per BIP-112, OP_CHECKSEQUENCEVERIFY fails if the spending transaction has an nSequence field value not greater than the csv_delay and so can only be included in a block after that delay.

OP_1
OP_CHECKSIGVERIFY
<csv_delay>
OP_CHECKSEQUENCEVERIFY

Note that OP_CHECKSEQUENCEVERIFY leaves the delay constant on the stack, but because it is the last opcode there is no need to add an OP_DROP.

Claim

The input to this script is both a signature and a preimage value that when hashed matches the preimage_hash value encoded in the script. The OP_EQUALVERIFY opcode fails if they do not match. The pubkey constant is provided by the payee of the HTLC to ensure they can generate the signature to claim the HTLC output immediately if they learn the preimage by forwarding the payment successfully.

OP_HASH160
<preimage_hash>
OP_EQUALVERIFY
<pubkey>
OP_CHECKSIG
Refund

The pubkey constant is provided by the payer of the HTLC to ensure only they can generate the signature to claim the HTLC output if the payee has not forwarded the payment successfully. The expiry constant defines the time after which the payer can reclaim the HTLC balance. The OP_CHECKLOCKTIMEVERIFY opcode fails if the nLocktime value of the spending transaction does not exceed the expiry constant. The transaction can not be mined into a block until the median time past of the last 10 blocks exceeds the nLocktime of the transaction as per BIP-65 and BIP-113.

<pubkey>
OP_CHECKSIGVERIFY
<expiry>
OP_CHECKLOCKTIMEVERIFY

Transactions

Update

An Update transaction spends an old payment state to a new state. The transaction has the following field settings:

  nLocktime: <cltv_start_time + state>
  nVersion = 2
  Input nSequence: 0
  sighash = SINGLE | ANYPREVOUTANYSCRIPT
  Output 0: scriptPubkey = <state_address>

The nLocktime value is checked by the script to ensure it can only spend earlier states. The state_address is a Taproot address that encodes an Update script at the new state that can only be spent by specific Settle transaction for that state. A signature created by both channel partners in advance needs to be provided to spend an Update transaction to the output of a newer Update transaction. An Update transaction can spend any Update transaction with a lower state immediately. These transactions are signed with APOAS which allows the spending of an Update output from any transaction that is signed with the internal Taproot key for the channel and satisfies the script condition of a larger nLocktime.

Settle

A Settle transaction spends from a specific Update transaction with a matching scriptPubkey and amount. The transaction has the following field settings:

nLocktime: <cltv_start_time + state + 1>
nVersion = 2
Input nSequence: CSV_DELAY
sighash = ALL | ANYPREVOUT
Output 0: scriptPubkey = <settled_amount_address_A>
Output 1: scriptPubkey = <settled_amount_address_B>
Output 2...n: scriptPubkey = <HTLC_addresses>

The nSequence value of the field is checked by the script to ensure it can only spend from an Update output after CSV_DELAY blocks have been mined. This gives time for the other party to spend from the current Update output with an Update with a later state. This transaction is signed in advance by both parties with APO which means the Update output it spends from must have a matching scriptPubkey and amount. Because the scriptPubkey of the Update transaction is hashed from the script, and the script contains the state number, that ties a Settle transaction to a specific Update transaction. Once the Settle transaction is confirmed, then the settled amounts can be claimed immediately by each channel participant. The HTLC outputs can only be claimed by the payee (or refunded to the payer) based on the HTLC conditions.

Claim

A Claim transaction spends from an HTLC output of a confirmed Settle transaction. Because it only requires one party to sign it and spends from a settled transaction we do not need to use APO or APOAS to sign it.

nLocktime: 0
nVersion = 2
Input nSequence: DEFAULT_NSEQUENCE
sighash = SINGLE | ANYONECANPAY
Output 0: scriptPubkey = <settled_amount_address_payee>

To spend an HTLC output with this transaction requires a signature from the HTLC payee and also the preimage for the HTLC. There is no time lock and this output can be spent immediately. The settled amount they control and any HTLC outputs they can collect can be combined into a single transaction with fees deducted from the output.

Refund

A Refund transaction spends from an HTLC output of a confirmed Settle transaction. Because it only requires one party to sign it and spends from a settled transaction we do not need to use APO or APOAS to sign it.

nLocktime: <htlc_expiry>
nVersion = 2
Input nSequence: DEFAULT_NSEQUENCE
sighash = SINGLE | ANYONECANPAY
Output 0: scriptPubkey = <settled_amount_address_payer>

To spend an HTLC output with this transaction requires a signature from the HTLC payer. The median time stamp of the last 11 blocks must also must be past htlc_expiry time. A transaction that spends the settled amount they control and any HTLC outputs they can collect can be combined into a single transaction with fees deducted from the output.

Taproot

The Update and Settle tapscripts are encoded as different leaves of a taptree computed for each eltoo state. The Update tapscript is presented to spend from an eltoo transaction to a newer eltoo state transaction. The Settle tapscript is presented to spend to the corresponding settle transaction for that state. The address of each HTLC output of the settle transaction encodes a Claim and Refund script as different leaves of a taptree. Presenting the Claim script and correct preimage is required to spend the output, or presenting the Refund script after the htlc_expiry time.