The core objective of the Hyperscale Alpha consensus protocol is to integrate deterministic safety into Bitcoin’s (Nakamoto) consensus protocol while preserving its advantages. In this article, I will explain how that works.

This is the second in a series of articles walking you through the design principles and technical details of the Hyperscale Alpha consensus protocol.

If you have not read the first article in the series which detailed the design principles of Hyperscale Alpha, here's a link where you can start from.

Nakamoto consensus offers weak liveness guarantees and probabilistic safety through its proof-of-work sybil resistance mechanism and longest chain rule. Although this method enables high decentralization and ensures continued network progress under challenging conditions, it falls short of providing the deterministic safety guarantees of Byzantine Fault Tolerant (BFT) protocols.

The objective of Hyperscale Alpha is to integrate deterministic safety into Bitcoin’s consensus model while preserving its advantages. 

This article lays out the hybrid consensus mechanism designed to combine the strengths of Nakamoto consensus with the robust safety guarantees of classical BFT protocols. 

Motivation for Incorporating Deterministic Safety to Nakamoto Consensus Protocol

The development of the Hyperscale Alpha consensus protocol is the culmination of a decade of R&D. The core goal behind this work is to create a consensus protocol that ensures continuous network progress under adverse conditions while supporting a highly decentralized / uncapped validator set and fast finality.

Additionally, maintaining atomicity of execution and commitment with a sharded topology can only be achieved with deterministic safety. Probabilistic safety alone is not enough, because while small, there is always a probability that the record of a transaction in one of the shards involved could be undone. The other shards involved in the transaction would still believe it happened, while the compromised shard would have no record of its existence, constituting a safety failure. 

To achieve this, it is important to start with a consensus protocol that balances both liveness (keeping the network active) and safety (ensuring correct outcomes).

Therefore the development of a consensus protocol with the following properties is required:

1. Weak Liveness Guarantee: This ensures that the network can continue to make progress even in the presence of network asynchrony or node failures.
2. Deterministic Safety: This ensures that the network will not produce conflicting outputs or reach an invalid state.
3. High Decentralization: This ensures the ability to support a large number of validators (potentially hundreds of thousands) across multiple shards.
4. Leaderless Design: This helps to avoid bottlenecks and single points of failure associated with leader-based systems.
5. Fast Finality: Confirm transactions rapidly, ideally within a few seconds

The Nakamoto consensus protocol, with its inherent liveness and safety guarantees, provides a solid foundation for this.

By incorporating the strong deterministic safety guarantees of Byzantine Fault Tolerant (BFT) protocols (which have no liveness guarantees) into the Nakamoto consensus protocol, you can take the first step towards building a consensus mechanism that leverages the strengths of both approaches.‍

The question is, how would you combine these two?

Cassandra Consensus: The Best of Both Worlds - A Hybrid Consensus Protocol

Alright, let's break down how the Cassandra Consensus works by retro-fitting it to Bitcoin. This is a consensus protocol that uses both a proof-of-work variant and proof-of-stake as sybil resistance mechanisms.

“Miners” still create blocks by solving computational puzzles via proof-of-work. This keeps block production decentralized and prohibitively expensive for attackers to gain control of the network.

Miners can lock up some of their tokens as "stake." The amount of tokens they stake determines their voting power when the network needs to agree on something.

When a miner creates a new block, instead of just relying on the computational puzzle to determine which proposal should be accepted and agreed upon, as in the strongest chain rule, the miner's stake is also a deciding factor. 

Ok, I already hear your critics: 

As of now there is no such consensus protocol out there combining two of these Sybil resistance mechanisms at the same time in the manner we propose in Cassandra. And I know there are obvious prejudices about proof-of-work due to what we see in Bitcoin now - the centralization of mining pools, the excessive energy consumption, etc., 

But everything in the world comes with trade-offs, which if managed intelligently, can mitigate each other to some degree.

Let's go deeper! 

Redefining Nakamoto Consensus with Deterministic Safety

The approach to incorporating deterministic safety into Nakamoto Consensus protocol involves two main phases:

Phase 1: Hybrid Proof-of-Work and Proof-of-Stake Model

The first phase introduces an implicit voting mechanism and a hybrid proof-of-work / proof-of-stake model. This approach aims to maintain the existing structure of Nakamoto consensus while adding a layer of deterministic safety.

Here’s how it works:

Step 1: Stake Registration

This step establishes the foundation for the proof-of-stake element of the system, allowing participants to declare their commitment to the network and gain voting influence. 

‍

Network participants register their stake by placing it in a designated address. 

This action serves two primary purposes: 

1. It signals the participant's intention to act as a validator in the network, and 
2. It quantifies their voting weight in the consensus mechanism.

The amount of token staked directly correlates to the participant's voting influence in the network. For example, if a miner stakes 1% of the total registered stake, their implicit votes in the consensus process will carry a weight of 1%. This proportional representation ensures that participants with larger stakes have a correspondingly larger say in the network's decision-making process, aligning economic incentives with network security.

An important point to remember for later; not all parties voting with stake will be miners and vice versa, not all miners may have stake and vote.

Step 2: Block Production

Miners continue to produce blocks through proof-of-work, similar to the original Bitcoin protocol, maintaining the decentralized nature of block production and the Sybil resistance provided by proof-of-work. Additionally, now each block is associated with the miner's registered stake.

‍

‍

One key difference in Hyperscale Alpha's block production compared to Bitcoin is the potential for shorter block intervals. 

While Bitcoin uses a 10-minute average block time, Hyperscale Alpha aims for faster block production, potentially as short as 5 seconds. This shorter interval allows for quicker transaction confirmations and a more responsive network.

Step 3: Implicit Voting

Implicit voting is a mechanism that leverages the existing structure of the blockchain to create a voting system without requiring additional communication overhead.

‍

‍

When a miner produces a block on top of a previous block, this action can be interpreted as an implicit vote for that block and its entire history. Subsequent blocks built on top of this chain are also counted as implicit votes for the earlier blocks.

Essentially, by choosing to build upon a specific block, the miner is saying, "I agree with all the blocks that constitute this canonical chain."

This approach is both efficient and elegant, as it utilizes the natural behavior of miners in the network to express their agreement with the blockchain's state.

Step 4: Quorum Formation

After a certain number of blocks, a quorum can be formed based on the accumulated stake of the miners who have implicitly voted for a particular block. This quorum represents a supermajority agreement on the validity of that block.

‍

‍

The quorum formation process begins by examining the chain of blocks that have been mined since a specific block, let's call it block N. The system then calculates the total stake associated with the miners who produced these blocks.

For example, if miner A with 1% stake mines a block N+1 directly on top of block N, and miner B with 2% stake mines the next block N+2, their combined stake of 3% counts towards the quorum for block N. This process continues as more blocks are added to the chain.

The accumulation of stake is monitored until it reaches a supermajority of the total registered stake in the network. This supermajority threshold is set at two-thirds (66.67%) of the total stake, as it is the minimum threshold at which the system can protect against adversarial actors and dishonest voters when applying deterministic safety.

Once the accumulated stake reaches or exceeds this threshold, a quorum is formed for block N. This quorum represents a significant portion of the network's economic stake agreeing on the validity of block N and its history.

This step is important because:

1. It provides a stronger guarantee of block finality than proof-of-work alone, as it requires both computational work and economic stake to confirm a block.
2. It creates a checkpoint in the blockchain beyond which the network agrees not to reorganize, enhancing the overall security of the system.
3. It allows the network to achieve a form of deterministic safety, where once a quorum is formed, the block and its predecessors are considered irreversible under normal network conditions.
4. It helps mitigate potential attacks by requiring an adversary to control both a majority of the network's hash power and a significant portion of the stake to successfully alter the blockchain's history.

Step 5: Rollback Prevention/Deterministic Safety

The final step in Phase 1 introduces a mechanism for rollback prevention, effectively providing deterministic safety to the network prior to the block with the most recently generated quorum.

Once the quorum is established, the system implements a rule that prevents any rollback beyond this point. This means that even if a longer, stronger chain were to appear that conflicts with transactions before this point, the network would reject it because a super-majority of economic security has already pledged itself to another version of history. This rule effectively creates a "point of no return" in the blockchain, beyond which transactions can be considered final with certainty.

This step addresses one of the key limitations of traditional Nakamoto consensus: the lack of absolute finality for transactions.

Additionally, phase 1 has created a hybrid Sybil resistance model where an attacker would need to control both the majority of hash power and the majority of stake to fully compromise the network.

To sum it up: Phase 1 enhances the Nakamoto consensus protocol with the following benefits:

1. It maintains the weak liveness guarantee, allowing the network to continue operating even if a large portion of miners go offline.
2. It maintains probabilistic safety, ensuring that blocks that have been produced but not yet have a quorum are still difficult to attack or undo, requiring vast amounts of compute power.
3. It introduces a deterministic safety mechanism through implicit voting and quorum formation.
4. It creates a more robust security model by requiring an attacker to control both the majority of hash power and the majority of stake to compromise the network.
5. It allows for a gradual transition from pure proof-of-work to a hybrid model, as miners can choose to participate in staking or continue mining without staking.

However, this approach also introduces some trade-offs:

• The finality time is relatively long, potentially requiring a large quantity of blocks depending on the distribution of stake, to be produced, extending block N for deterministic safety.
• The implicit voting mechanism, while elegant, may not be as robust as explicit voting in classical BFT protocols. It assumes mining on top of a block implies agreement, which may not always hold true in adversarial scenarios.
• The system may still be vulnerable to certain types of attacks during the transition period between probabilistic and deterministic safety.
• As the number of miners/validators increases, the quorum formation process may become complex and computationally expensive, potentially limiting scalability.

Given these limitations, a phase 2 is necessary to address these issues and retain the benefits of the hybrid model.

Phase 2: Explicit Voting and Quorum Certificates

To address the limitations of phase 1 and further improve the safety and finality time of the system, the system incorporates a more sophisticated approach in phase 2. 

In this phase, the protocol retains the hybrid proof-of-work and proof-of-stake model but adds a layer of explicit communication between voting validators. This communication occurs through a side channel, separate from the blockchain, allowing for more efficient vote collection and quorum formation.

The introduction of quorum certificates provides a clear and verifiable record of consensus, enhancing the system's overall security and reliability.

Here’s how it step-by-step works:

Step 1: Explicit Voting and Side-Channel Communication

In this process, the first step is block production through proof-of-work mining, just like in phase 1.

Following the block production and broadcast, the protocol introduces explicit voting through a side-channel communication mechanism. Validators, who may or may not be miners, participate in this explicit voting process.

The key thing to understand here is the difference between implicit and explicit voting:

‍

• Implicit voting is when miners with stake make blocks on top of previous blocks, which indirectly shows they agree with those blocks.
• Explicit voting, on the other hand, is when validators directly cast votes for specific blocks through this separate communication channel.

When a block is produced and broadcast to the network, validators begin to collect and evaluate the proposed blocks. They use a separate communication channel, distinct from the main blockchain network, to exchange their votes on which block they consider to be the strongest or most valid for the current round.

Step 2: Quorum Certificates

As validators receive votes from their peers, they begin to form a picture of the network's consensus. When a validator observes that a supermajority (two-thirds) of the total stake has voted for a particular block, they can create a quorum certificate for that block.

‍

Quorum certificates serve as cryptographic proof that a supermajority amount of stake controlled by a set of validators have agreed on a particular block, enhancing the security and finality of the consensus process.

Step 3: Certificate Chaining

In this step, quorum certificates are included in subsequent blocks, creating a chain of certificates that reference previous blocks. This embeds them directly in the blockchain making them tamper resistant as they are secured by future quorum certificates.

‍

This chaining mechanism serves two primary purposes: 

1. It propagates the quorum certificate to other parts of the network that may not have received it through the side-channel communication, and
2. It creates a verifiable sequence of consensus decisions that can be used to establish finality.

Step 4: Two-Phase Commit

When a block contains a quorum certificate for its parent block, and that parent block contains a quorum certificate for its parent, a chain of two consecutive quorum certificates is formed. This chain of two quorum certificates is what allows the protocol to commit a block with finality.

For example, consider the following sequence:

• Block A is produced
• Block B is produced, containing a quorum certificate for Block A
• Block C is produced, containing a quorum certificate for Block B

‍

To achieve deterministic safety, two consecutive quorum certificates are required:

• The first certificate indicates that a supermajority has seen and agreed on a block.
• The second certificate confirms that a supermajority has seen the first certificate and agrees with it.

In this scenario, when Block C is observed, Block A can be committed with finality. This is because the quorum certificate in Block C proves that a supermajority has seen and agreed with Block B, which in turn contains proof that a supermajority agreed with Block A.

This two-phase commit process ensures that not only has a supermajority agreed on a block, but a supermajority has also seen and accepted that agreement. This double confirmation is what provides the strong safety guarantees

Once two consecutive quorum certificates are observed for a block, it can be considered final with deterministic safety.

This approach allows for faster finality (within a few seconds) while maintaining the weak liveness guarantee of the original Nakamoto consensus.

This approach provides a lot of improvements over phase 1:

Faster Finality: Deterministic safety can be achieved within two block rounds, potentially reducing finality time to 10 seconds or less (assuming 5-second block times). Generally the propagation of blocks to a supermajority of observers should be similar in duration to the propagation of a of votes to a supermajority of observers.

Stronger Safety: The use of explicit votes and quorum certificates provides stronger safety guarantees than implicit voting as it exhibits a stronger element of objectivity.

Flexible Architecture: Quorum certificates can be embedded in blocks or transmitted separately, allowing for optimization based on network conditions.

Maintains Weak Liveness Guarantees: The system retains the ability to make progress under network asynchrony, as blocks can still be produced through proof-of-work even if voting is delayed.

‍

This two-phase method allows the integration of deterministic safety into the Nakamoto consensus protocol. Very cool, we think!

But there are tradeoffs too.

Mixing proof-of-work for creating blocks with proof-of-stake for explicit voting and quorum certificates brings some practical issues, particularly when trying to scale it to accommodate a large number of validators. 

Before we go into the tradeoffs of this approach, I know at least some of you might have one question in mind:

Why combine Proof of Work with Proof of Stake rather than going fully with Proof of Stake?

Contrary to conventional wisdom, this hybrid approach that combines elements of proof-of-work and proof-of-stake offers a lot of advantages over a fully proof-of-stake system.

Backstopped Security

Operating a miner and producing proofs-of-work introduces a sunk cost into the system. Even if a particular miner’s block isn’t accepted into the network, there is a cost of producing that proof-of-work, primarily through electricity usage, which is irrecoverable.

If we look at any particular block N and assume the proposing miner spends $1 of electricity to produce it, and there are 1000 other miners participating, then approximately $1001 of electricity may have been spent trying to produce a valid block in that period.

Effectively that block, and every block following it, is backed by $1001 of economic security. Unless they are consistently very lucky, a single miner would have to spend the equivalent in electricity on average to produce that same block, and consequent ones, by mining solo.

Over the course of a number of blocks, and assuming that each miner produces a block which is accepted, that sunk cost reduces via the rewards for mining. But importantly, and assuming a status quo in the system for simplicity here, if each block produced costs a miner $1, then at block N+1000, block N is still backed by $1001 of economic security.

This property is very useful when considering proof-of-stake, as it solves a number of inherent security issues and allows us to relax some requirements of participation such as high initial stake required to operate a validator.

Multiple Opportunities for Participation

As a participant in the system I can choose the role I want to play, or even change my strategy as time goes on. For example, if I have very low powered hardware, I can participate only in voting with stake. Likewise if I have no ability to stake yet, I can be a miner, be rewarded if any of my proposed blocks are accepted and later also become a validating participant by staking those rewards.

The ability for participants to contribute in various ways also reduces the barrier to entry significantly, as typically I either need to have lots of compute to participate in mining, or large amounts of stake to initiate a validator.

Delegated proof-of-stake is a solution to the latter, where I pledge my stake to a validator operated by a third-party and there is no minimum requirement to do so. However, there is risk associated with this solution vs operating my own validator, as I have no control over the operation of the validator I am staked to. If it should suddenly turn dishonest, my stake is in jeopardy via slashing should the validators' illicit actions be discovered by the network.

Thanks to the proof-of-work element and its backstopping of security, initial validator staking requirements can be very low, opening the doors to participation and also fostering greater decentralization.

Solves the “nothing at stake” Problem

In a pure proof-of-stake system, an attacker with a significant stake could potentially generate multiple block proposals at no cost, trying different combinations of transactions or block headers to find one that maximizes their chances of being selected as the block producer for future rounds. This process, known as "grinding," could potentially allow an attacker to gain an unfair advantage in block production. 

By incorporating proof-of-work into the consensus mechanism, this hybrid model introduces a computational cost to block proposal generation. This cost serves as a deterrent against grinding attacks. 

Here, the honest participants who want to propose a block only need to perform a minimal amount of computation. They simply gather valid transactions, create a block, perform the proof-of-work, and broadcast it to the network. 

However, for an adversary attempting a grinding attack, the situation is quite different. To have a chance of their proposals being accepted, they would need to generate a large number of block proposals, each requiring a proof-of-work solution. This process becomes computationally expensive and time-consuming, effectively making grinding attacks impractical.

Along with that, if an attacker wants to double-spend or perform other malicious actions, they need to control the proposal generation for an extended period. The only way to ensure this control is by generating a sequence of proposals into the future that include them as the proposer. This requires solving the proof-of-work puzzle for each proposal, which becomes prohibitively expensive as the attacker tries to grind through a huge amount of different possibilities.

Enhanced Double Spend Protection

Lets consider the case of a double spend attack with proof-of-work only and how incorporating proof-of-stake and explicit voting prevent this attack.

Consider an adversary with significant compute who is mining and producing proofs-of-work. If the compute resources they control is 33% or greater than the network total, with some tricks such as selfish mining, they can generate an alternative chain of blocks which has the possibility to become longer and stronger than the rest of the miners are producing collectively.

The attacker begins by sending funds to a merchant, such as Circle, issuer of one of the largest, most widely used stable coin USDC. The attacker then waits for the transaction to be confirmed on the network. However, their goal is to spend these same funds again elsewhere, effectively double-spending.

With pure proof-of-work, the attacker waits until Circle has credited their account, they withdraw the funds from Circle to elsewhere, perhaps a bank account or more likely a different blockchain entirely. Then they broadcast their longer, stronger, secret alternative chain to the network and take advantage of the strongest chain rule. 

All the nodes in the network now consider the attackers chain of blocks to be the correct one, so undo all of the transactions on the chain they had which included the Circle transaction, and now follow the attackers chain which excludes the Circle transaction.

The inclusion of proof-of-stake makes this attack much harder to execute, as now the attacker has to ensure that it not only has an ability to control block production, but must control at least 34% or more of all stake in the network.

For the attack to be successful now, the chain including the transaction to Circle must not receive a supermajority of votes or form a quorum, otherwise the transaction to Circle will become final. After this point no matter how strong the attacker’s alternate chain of blocks is, the network, including Circle, will reject it.

The only way for the attacker to ensure this doesn’t happen while they are waiting for Circle to credit their account and move the funds is to control 34% or more of total stake and abstain from voting on the chain that includes the transaction to be double spent. Then, even if the remaining 66% of the stake in the network is honest and votes, it is just shy of the supermajority required.

This holds the hybrid consensus in probabilistic safety mode which is governed by the strongest chain rule in the same way as the original Nakamoto consensus and the attack can be executed in the same way.

Ultimately, for the dishonest actor, this attack scenario results in a costly, resource-intensive process with a high risk of failure making the attack economically unattractive.

This hybrid design of proof-of-work and proof-of-stake effectively deters potential attackers by making dishonest behavior prohibitively expensive.

Are We There Yet? Not Quite

Now we have a consensus protocol with liveness, probabilistic and deterministic safety depending on network condition, along with a much stronger security model thanks to the hybridization of proof-of-work and proof-of-stake.

It doesn’t resemble Bitcoin or the original Namamoto consensus much anymore, but it does retain all the fundamental elements and properties which Nakamoto consensus introduced.

We’re not quite done yet, there are some remaining looming problems.

One concern is the use of proof-of-work in the same fashion as Bitcoin. The system could become a high-energy-consuming system in the same manner and eventually centralize the whole network into three or four huge mining operations which control the block production.

A second concern is if the number of participants becomes very high. This is great for decentralization, but a real problem if we wish to keep finality low. A low finality requirement means that the proof-of-work can not be too computationally expensive or difficult to solve.

With a large number of mining participants, this would result in a large portion of them discovering a valid proof-of-work in a for the next block. This then results in a huge amount of block proposals being communicated around the network, adding overhead, which itself then works against the ability to satisfy fast finality.

How are we going to solve this final problem? Jump over to part 3 in the series to find out!

‍