Consensus - CometBFT

Blockchains use a variety of different consensus algorithms to determine the content of the next block in the blockchain. While algorithms like Proof of Work and Proof of Stake are the most common and well known, they are far from the only ones available and in use.

The goal of Botanix Labs is to build a Proof of Stake for the Botanix EVM. At this stage in our development, we implement a consensus layer based on CometBFT that will be modified as we go along the development path and introduce new features such as staking, slashing etc.

CometBFT in short

CometBFT is a blockchain application platform; it provides the equivalent of a web-server, database, and supporting libraries for blockchain applications written in any programming language. Like a web-server serving web applications, CometBFT serves blockchain applications.

More formally, CometBFT performs Byzantine Fault Tolerant (BFT) State Machine Replication (SMR) for arbitrary deterministic, finite state machines.

The following information is based on their official documentation which can be visited here: https://docs.cometbft.com/

What is CometBFT?

CometBFT is software for securely and consistently replicating an application on many machines. By securely, we mean that CometBFT works as long as less than 1/3 of machines fail in arbitrary ways. By consistently, we mean that every non-faulty machine sees the same transaction log and computes the same state. Secure and consistent replication is a fundamental problem in distributed systems; it plays a critical role in the fault tolerance of a broad range of applications, from currencies, to elections, to infrastructure orchestration, and beyond.

The ability to tolerate machines failing in arbitrary ways, including becoming malicious, is known as Byzantine fault tolerance (BFT). The theory of BFT is decades old, but software implementations have only became popular recently, due largely to the success of “blockchain technology” like Bitcoin and Ethereum. Blockchain technology is just a reformalization of BFT in a more modern setting, with emphasis on peer-to-peer networking and cryptographic authentication. The name derives from the way transactions are batched in blocks, where each block contains a cryptographic hash of the previous one, forming a chain.

CometBFT consists of two chief technical components: a blockchain consensus engine and a generic application interface. The consensus engine, which is based on Tendermint consensus algorithm, ensures that the same transactions are recorded on every machine in the same order. The application interface, called the Application BlockChain Interface (ABCI), delivers the transactions to applications for processing. Unlike other blockchain and consensus solutions, which come pre-packaged with built in state machines (like a fancy key-value store, or a quirky scripting language), developers can use CometBFT for BFT state machine replication of applications written in whatever programming language and development environment is right for them.

CometBFT is designed to be easy-to-use, simple-to-understand, highly performant, and useful for a wide variety of distributed applications.

CometBFT vs Bitcoin, Ethereum, etc

Tendermint consensus algorithm, adopted by CometBFT, emerged in the tradition of cryptocurrencies like Bitcoin, Ethereum, etc. with the goal of providing a more efficient and secure consensus algorithm than Bitcoin’s Proof of Work. In the early days, Tendermint consensus-based blockchains had a simple currency built in, and to participate in consensus, users had to “bond” units of the currency into a security deposit which could be revoked if they misbehaved -this is what made Tendermint consensus a Proof-of-Stake algorithm.

Since then, CometBFT has evolved to be a general purpose blockchain consensus engine that can host arbitrary application states. That means it can be used as a plug-and-play replacement for the consensus engines of other blockchain software. So one can take the current Ethereum code base, whether in Rust, or Go, or Haskell, and run it as an ABCI application using CometBFT.

ABCI

The Application BlockChain Interface (ABCI) allows for Byzantine Fault Tolerant replication of applications written in any programming language.

Motivation

Thus far, all blockchains “stacks” (such as Bitcoin) have had a monolithic design. That is, each blockchain stack is a single program that handles all the concerns of a decentralized ledger; this includes P2P connectivity, the “mempool” broadcasting of transactions, consensus on the most recent block, account balances, Turing-complete contracts, user-level permissions, etc.

Using a monolithic architecture is typically bad practice in computer science. It makes it difficult to reuse components of the code, and attempts to do so result in complex maintenance procedures for forks of the codebase. This is especially true when the codebase is not modular in design and suffers from “spaghetti code”.

Another problem with monolithic design is that it limits you to the language of the blockchain stack (or vice versa). In the case of Ethereum which supports a Turing-complete bytecode virtual-machine, it limits you to languages that compile down to that bytecode; while the list is growing, it is still very limited.

In contrast, CometBFT's approach is to decouple the consensus engine and P2P layers from the details of the state of the particular blockchain application. They do this by abstracting away the details of the application to an interface, which is implemented as a socket protocol.

Intro to ABCI

CometBFT, the “consensus engine”, communicates with the application via a socket protocol that satisfies the ABCI, the CometBFT Socket Protocol.

To draw an analogy, let’s talk about Bitcoin. Bitcoin is a cryptocurrency blockchain where each node maintains a fully audited Unspent Transaction Output (UTXO) database. If one wanted to create a Bitcoin-like system on top of ABCI, CometBFT would be responsible for

Sharing blocks and transactions between nodes
Establishing a canonical/immutable order of transactions (the blockchain)

The application will be responsible for

Maintaining the UTXO database
Validating cryptographic signatures of transactions
Preventing transactions from spending non-existent transactions
Allowing clients to query the UTXO database.

CometBFT is able to decompose the blockchain design by offering a very simple API (i.e. the ABCI) between the application process and consensus process.

The ABCI consists of 3 primary message types that get delivered from the core to the application. The application replies with corresponding response messages.

The messages are specified here: ABCI Message Types.

The FinalizeBlock message is the work horse of the application. Each transaction in the blockchain is finalized within this message. The application needs to validate each transaction received with the FinalizeBlock message against the current state, application protocol, and the cryptographic credentials of the transaction. FinalizeBlock only prepares the update to be made and does not change the state of the application. The state change is actually committed in a later stage i.e. in commit phase.

The CheckTx message is used for validating transactions. CometBFT’s mempool first checks the validity of a transaction with CheckTx, and only relays valid transactions to its peers. For instance, an application may check an incrementing sequence number in the transaction and return an error upon CheckTx if the sequence number is old. Alternatively, they might use a capabilities based system that requires capabilities to be renewed with every transaction.

The Commit message is used to compute a cryptographic commitment to the current application state, to be placed into the next block header. This has some handy properties. Inconsistencies in updating that state will now appear as blockchain forks which catches a whole class of programming errors. This also simplifies the development of secure lightweight clients, as Merkle-hash proofs can be verified by checking against the block hash, and that the block hash is signed by a quorum.

There can be multiple ABCI socket connections to an application. CometBFT creates four ABCI connections to the application; one for the validation of transactions when broadcasting in the mempool, one for the consensus engine to run block proposals, one for creating snapshots of the application state, and one more for querying the application state.

ABCI++

ABCI++ is a major evolution of ABCI (Application Blockchain Interface). Like its predecessor, ABCI++ is the interface between CometBFT (a state-machine replication engine) and the actual state machine being replicated (i.e., the Application). The API consists of a set of methods, each with a corresponding Request and Response message type.

The methods are always initiated by CometBFT. The Application implements its logic for handling all ABCI++ methods. Thus, CometBFT always sends the Request* messages and receives the Response* messages in return.

All ABCI++ messages and methods are defined in protocol buffers. This allows CometBFT to run with applications written in many programming languages.

You can read more about ABCI++ in the CometBFT official documentation: https://docs.cometbft.com/v0.38/spec/abci/

Consensus Overview

CometBFT adopts Tendermint consensus, an easy-to-understand, mostly asynchronous, BFT consensus algorithm. The algorithm follows a simple state machine that looks like this:

Participants in the algorithm are called validators; they take turns proposing blocks of transactions and voting on them. Blocks are committed in a chain, with one block at each height. A block may fail to be committed, in which case the algorithm moves to the next round, and a new validator gets to propose a block for that height. Two stages of voting are required to successfully commit a block; we call them pre-vote and pre-commit.

There is a picture of a couple doing the polka because validators are doing something like a polka dance. When more than two-thirds of the validators pre-vote for the same block, they call it a polka. Every pre-commit must be justified by a polka in the same round. A block is committed when more than 2/3 of validators pre-commit for the same block in the same round.

Validators may fail to commit a block for a number of reasons; the current proposer may be offline, or the network may be slow. Tendermint consensus allows them to establish that a validator should be skipped. Validators wait a small amount of time to receive a complete proposal block from the proposer before voting to move to the next round. This reliance on a timeout is what makes Tendermint consensus a weakly synchronous algorithm, rather than an asynchronous one. However, the rest of the algorithm is asynchronous, and validators only make progress after hearing from more than two-thirds of the validator set. A simplifying element of Tendermint consensus is that it uses the same mechanism to commit a block as it does to skip to the next round.

Assuming less than one-third of the validators are Byzantine, Tendermint consensus algorithm guarantees that safety will never be violated - that is, validators will never commit conflicting blocks at the same height. To do this it introduces a few locking rules which modulate which paths can be followed in the flow diagram. Once a validator precommits a block, it is locked on that block. Then,

it must prevote for the block it is locked on
it can only unlock, and precommit for a new block, if there is a polka for that block in a later round

Stake

Not applicable to the Botanix Federation

This text excerpt is part of the official CometBFT documentation, however, at this point in time we are not implementing staking yet but instead rely on 15 chosen partners to be the Orchestrators.

In many systems, not all validators will have the same “weight” in the consensus protocol. Thus, we are not so much interested in one-third or two-thirds of the validators, but in those proportions of the total voting power, which may not be uniformly distributed across individual validators.

Since CometBFT can replicate arbitrary applications, it is possible to define a currency, and denominate the voting power in that currency. When voting power is denominated in a native currency, the system is often referred to as Proof-of-Stake. Validators can be forced, by logic in the application, to “bond” their currency holdings in a security deposit that can be destroyed if they’re found to misbehave in the consensus protocol. This adds an economic element to the security of the protocol, allowing one to quantify the cost of violating the assumption that less than one-third of voting power is Byzantine.

CometBFT on Botanix

TODO: insert any details on the implementation of CometBFT on Botanix