It is very likely that if you are reading this book, you already have heard about blockchain and have some fundamental appreciation of its enormous potential. If not, then let me tell you that this is a technology that has promised to positively alter the existing paradigms of nearly all industries including, but not limited to, the IT, finance, government, media, medical, and law sectors.
This chapter is an introduction to blockchain technology, its technical foundations, the theory behind it, and various techniques that have been combined together to build what is known today as blockchain.
In this chapter, the theoretical foundations of distributed systems are described first. Next, the precursors of Bitcoin are presented. Finally, blockchain technology is introduced. This approach is a logical way of understanding blockchain technology, as the roots of blockchain are in distributed systems and cryptography. We will be covering a lot of ground quickly here, but don't worry—we will go over a great deal of this material in much greater detail as you move throughout the book.
With the invention of Bitcoin in 2008, the world was introduced to a new concept, which revolutionized the whole of society. It was something that promised to have an impact upon every industry. This new concept was blockchain; the underlying technology that underpins Bitcoin.
Some describe blockchain as a revolution, whereas another school of thought believes that it is going to be more evolutionary, and it will take many years before any practical benefits of blockchain reach fruition. This thinking is correct to some extent, but, in my opinion, the revolution has already begun. It is a technology that has an impact on current technologies too and possesses the ability to change them at a fundamental level.
Many prominent organizations all around the world have moved on from the proof-of-concept stage and are already writing production systems utilizing blockchain. The disruptive potential of blockchain has now been fully recognized. For example, Decentralized Finance (DeFi) has recently emerged as a new disruptive idea that aims to decentralize the existing financial system. DeFi offers individuals more control over their assets, allowing them to fully own and operate their financial strategy on blockchain without requiring any intermediaries. This is already challenging traditional finance by creating decentralized platforms for trading, investment, insurance, payments, and lending.
A glimpse of the potential of DeFi can be seen at https://defipulse.com. A noticeable fact is the amount of total value locked in the system, which is almost 5 billion US Dollars.
However, some organizations are still in the preliminary exploration stage, though they are expected to progress more quickly as the technology matures.
If we look at the last few years, we notice that, in 2013, some ideas started to emerge that suggested that blockchain may have the potential for application in areas other than cryptocurrencies. Around that time, the primary usage of blockchain was in cryptocurrency space such as Bitcoin and Litecoin, and many new coins emerged during that period.
Cryptocurrency can be defined as a digital currency that is secured by cryptography.
The following graph shows a broad-spectrum outline of the year-wise progression and adoption trends of blockchain technology. The years shown on the x axis indicate the range of time in which a specific phase of blockchain technology falls. Each phase has a name that represents the stage at which the technology was reached, and this is shown on the x axis starting from the period of IDEAS AND THOUGHTS in 2013 to eventually MATURITY AND FURTHER PROGRESS, expected in 2025. The y axis shows the level of activity, involvement, and adoption of blockchain technology. The graph shows that, by roughly 2025, blockchain technology is expected to become mature and have a high number of users:
Figure 1.1: Blockchain technology adoption and maturity
The preceding graph shows that, in 2013, ideas and thoughts emerged regarding other usages of blockchain technology apart from cryptocurrencies. Then, in 2014, some research and experimentation began, which led to proofs of concept, further research, and full-scale trial projects between 2015 and 2017. In 2018, we saw real-world implementations. Already many projects are underway and set to replace existing systems; for example, the Australian Securities Exchange (ASX) is soon to become the first organization to replace its legacy clearing and settlement system with blockchain technology.
More information on this topic can be found at https://www.asx.com.au/services/chess-replacement.htm.
Another recent prominent example is that of a production-ready project implemented by Santander, where the first end-to-end blockchain bond has been issued. This is a significant step toward the mainstream adoption of blockchain.
You can read more about this at http://www.santander.com/csgs/Satellite/CFWCSancomQP01/en_GB/Corporate/Press-room/2019/09/12/Santander-launches-the-first-end-to-end-blockchain-bond.html.
It is expected that, during 2020, more research will be carried out, along with an increasing interest toward the regulation and standardization of blockchain technology. After this, production-ready projects and off-the-shelf projects will be available utilizing blockchain technology from 2020, and by 2021 mainstream production-level usage of blockchain technology is expected.
Progress in blockchain technology almost feels like the internet dot-com boom of the late 1990s.
In the next few years, research in the scalability of blockchains, where blockchains will be able to handle a large number of transactions similar to traditional financial networks, is expected to become more mature. Also, during the first few years of the 2020s, we will see more production-level usage of blockchain addressing issues such as privacy, decentralized identity, and some progress toward the decentralized web (or internet decentralization). Note that such solutions already exist but are not in mainstream use. The mainstream usage of such technologies is expected from the start of 2020.
It is expected that, at the start of the next decade, research in such areas will continue along with the adoption and further maturity of blockchain technology. Finally, in 2025, it is expected that the technology will be mature enough to be used on a day-to-day basis by, less tech-savvy people. For example, blockchain networks can be used as easily and naturally as consumers using the internet now. Further research is expected to continue even beyond this point. Please note that the timelines provided in the chart are not strict and may vary, as it is quite difficult to predict when exactly blockchain technology will become mature. This graph is based on the progress made in recent years and the current climate of research, interest, and enthusiasm regarding this technology, which can be extrapolated to predict that blockchain will progress to become a mature technology by 2025.
Interest in blockchain technology has risen quite significantly over the last few years. Once dismissed simply as "geek money" from a cryptocurrency point of view, or as something that was just not considered worth pursuing, blockchain is now being researched by the largest companies and organizations around the world. Millions of dollars are being spent to adopt and experiment with this technology. This is evident from recent actions taken by the European Union, where they have announced plans to increase funding for blockchain research to almost 340 million Euros by 2020.
Interested readers can read more about this at https://www.irishtimes.com/business/technology/boost-for-blockchain-research-as-eu-increases-funding-four-fold-1.3383340.
Another report suggests that global spending on blockchain technology research could reach 9.2 billion US Dollars by 2021.
More information regarding this can be found at https://bitcoinmagazine.com/articles/report-suggests-global-spending-blockchain-tech-could-reach-92-billion-2021/.
Also, the interest in blockchain within academia is astounding, and many educational establishments—including prestigious universities around the world—are conducting research and development on blockchain technology. There are not only educational courses being offered by many institutions, but academics are also conducting high-quality research and producing a number of insightful research papers on the topic. There are also a number of research groups and conferences around the world that specifically focus on blockchain research. This is extremely useful for the growth of the entire blockchain ecosystem. A simple online search of "blockchain research groups" would reveal hundreds, if not thousands, of these research groups.
There are also various consortiums such as Enterprise Ethereum Alliance (EEA) at https://entethalliance.org and Hyperledger at https://www.hyperledger.org, which have been established for research, development, and the standardization of blockchain technology.
Moreover, a large number of start-ups are providing blockchain-based solutions already. A simple trend search on Google reveals the immense scale of interest in blockchain technology over the last few years.
Especially since early 2017, the increase in the search term "blockchain" is quite significant, as shown in the following graph:
Figure 1.2: A popularity graph for the search term "blockchain," based on data from Google trends
It should be noted that the decrease shown at the end of the graph does not necessarily directly translate into the decreased interest in blockchain technology, but it simply captures the fact that the search term was searched fewer times as compared to late 2017 and early 2018. This could be simply due to the fact that people interested in blockchain technology have already understood enough about blockchain to no longer warrant searching for it on Google. We do see, however, a significant and continued interest in blockchain technology.
Various benefits of this technology have already been envisioned, such as decentralized trust, cost savings, transparency, and efficiency. However, there are multiple challenges too that are actively being researched on blockchain, such as scalability and privacy.
In this book, we are going to see how blockchain technology can help to bring about the benefits mentioned previously. You are going to learn what exactly blockchain technology is and how it can reshape businesses, multiple industries, and indeed everyday life by bringing about a plenitude of benefits such as efficiency, cost savings, transparency, and security. We will also explore what distributed ledger technology (DLT) is, decentralization, and smart contracts, and how technology solutions can be developed and implemented using mainstream blockchain platforms such as Ethereum and Hyperledger. We will also investigate what challenges need to be addressed before blockchain can become a mainstream technology.
We'll also take a critical look at blockchain; Chapter 21, Scalability and Other Challenges, is dedicated to a discussion of the limitations and challenges of blockchain technology.
We shall begin our exploration of blockchain by looking at distributed systems in the following section. This is a foundational paradigm used within blockchain, and we must have a firm grasp on what distributed systems are before we can meaningfully discuss blockchain in detail.
Understanding distributed systems is essential to our understanding blockchain, as blockchain was a distributed system at its core. It is a distributed ledger that can be centralized or decentralized. A blockchain is originally intended to be and is usually used as a decentralized platform. It can be thought of as a system that has properties of the both decentralized and distributed paradigms. It is a decentralized-distributed system.
Distributed systems are a computing paradigm whereby two or more nodes work with each other in a coordinated fashion to achieve a common outcome. It is modeled in such a way that end users see it as a single logical platform. For example, Google's search engine is based on a large distributed system; however, to a user, it looks like a single, coherent platform.
A node can be defined as an individual player in a distributed system. All nodes are capable of sending and receiving messages to and from each other. Nodes can be honest, faulty, or malicious, and they have memory and a processor. A node that exhibits irrational behavior is also known as a Byzantine node after the Byzantine Generals problem.
The Byzantine Generals problem
In 1982, a thought experiment was proposed by Lamport et al. in their research paper, The Byzantine Generals Problem, which is available here:
https://www.microsoft.com/en-us/research/publication/byzantine-generals-problem/
In this problem, a group of army generals who lead different parts of the Byzantine army is planning to attack or retreat from a city. The only way of communicating among them is via a messenger. They need to agree to strike at the same time in order to win. The issue is that one or more generals might be traitors who could send a misleading message. Therefore, there is a need for a viable mechanism that allows for agreement among the generals, even in the presence of the treacherous ones, so that the attack can still take place at the same time. As an analogy for distributed systems, the generals can be considered honest nodes, the traitors as Byzantine nodes (that is, nodes with arbitrary behavior), and the messenger can be thought of as a channel of communication among the generals.
This problem was solved in 1999 by Castro and Liskov who presented the Practical Byzantine Fault Tolerance (PBFT) algorithm, which solves the consensus problem in the presence of Byzantine faults in asynchronous networks by utilizing the state machine replication protocol. PBFT goes through a number of rounds to eventually reach an agreement between nodes on the proposed value. PBFT and other consensus protocols will be discussed in greater detail in Chapter 5, Consensus Algorithms.
This type of inconsistent behavior of Byzantine nodes can be intentionally malicious, which is detrimental to the operation of the network. Any unexpected behavior by a node on the network, whether malicious or not, can be categorized as Byzantine.
A small-scale example of a distributed system is shown in the following diagram. This distributed system has six nodes, out of which one (N4) is a Byzantine node leading to possible data inconsistency. L2 is a link that is broken or slow, and this can lead to a partition in the network:
Figure 1.3: Design of a distributed system: N4 is a Byzantine node and L2 is broken or a slow network link
The primary challenge of a distributed system design is the coordination between nodes and fault tolerance. Even if some (a certain threshold dictated by the consensus protocol) of the nodes become faulty or network links break, the distributed system should be able to tolerate this and continue to work to achieve the desired result. This problem has been an active area of distributed system design research for many years, and several algorithms and mechanisms have been proposed to overcome these issues.
Distributed systems are so challenging to design that a theory known as the CAP theorem has been proven, which states that a distributed system cannot have all three of the much-desired properties simultaneously; that is, consistency, availability, and partition tolerance. We will dive into the CAP theorem in more detail later in this chapter.
Even though blockchain can be considered to be both a distributed and decentralized system, there are, however, critical differences between distributed systems and decentralized systems that make both of these systems architecturally different. We will discuss these differences in detail in Chapter 2, Decentralization.
With a better understanding of distributed systems, let's now move on to talking about blockchain itself. We'll begin with a brief rundown of the history of blockchain and Bitcoin.
Blockchain was introduced with the invention of Bitcoin in 2008. Its practical implementation then occurred in 2009. For the purposes of this chapter, it is sufficient to review Bitcoin very briefly, as it will be explored in great depth in Chapter 6, Introducing Bitcoin. However, it is essential to refer to Bitcoin because, without it, the history of blockchain is not complete.
Now we will look at the early history of computing and computer networks and will discuss how these technologies evolved and contributed to the development of Bitcoin in 2008. We can view this in chronological order:
The aforementioned technologies contributed in some way to the development of Bitcoin, even if not directly; the work is relevant to the problem that Bitcoin solved. All previous attempts to create anonymous and decentralized digital currency were successful to some extent, but they could not solve the problem of preventing double spending in a completely trustless or permissionless environment. This problem was finally addressed by the Bitcoin blockchain, which introduced the Bitcoin cryptocurrency.
It should be noted that other concepts such as state machine replication (the SMR problem), introduced in 1978 by Leslie Lamport and formalized in 1980 by Fred Schneider, are also solvable by Bitcoin. Bitcoin solves the SMR problem (probabilistically) by allowing the replication of blocks and ensuring consistency via its PoW consensus mechanism.
The SMR, or state machine replication problem, is a technique used to provide fault-tolerant replication in distributed systems. We will explore this in greater detail in Chapter 5, Consensus Algorithms.
Now we will discuss some of the major milestones in the history of blockchain in more detail.
The concept of electronic cash (e-cash), or digital currency, is not new. Since the 1980s, e-cash protocols have existed that are based on a model proposed by David Chaum.
Just as understanding the concept of distributed systems is necessary to comprehend blockchain technology, the idea of e-cash is also essential in order to appreciate the first, and astonishingly successful, application of blockchain, Bitcoin, and more broadly, cryptocurrencies in general.
Two fundamental e-cash system issues need to be addressed: accountability and anonymity.
Accountability is required to ensure that cash is spendable only once (addressing the double-spending problem) and that it can only be spent by its rightful owner. The double-spending problem arises when the same money can be spent twice. As it is quite easy to make copies of digital data, this becomes a big issue in digital currencies as you can make many copies of the same amount of digital cash.
Anonymity is required to protect users' privacy. With physical cash, it is almost impossible to trace back spending to the individual who actually paid the money, which provides adequate privacy should the consumer choose to hide their identity. In the digital world, however, providing such a level of privacy is difficult due to inherent personalization, tracing, and logging mechanisms in digital payment systems such as credit card payments. This is indeed a required feature for ensuring the security and safety of the financial network, but it is also often seen as a breach of privacy.
This is due to the fact that end users do not have any control over who their data might be shared with, even without their consent. Nevertheless, this is a solvable problem and cryptography is used to address such issues. Especially in blockchain networks, the privacy and anonymity of the participants on the blockchain are sought-after features. We will learn more about this in Chapter 4, Public Key Cryptography.
David Chaum solved both of these problems during his work in the 1980s by using two cryptographic operations, namely, blind signatures and secret sharing. These terminologies and related concepts will be discussed in detail in Chapter 4, Public Key Cryptography. For the moment, it is sufficient to say that blind signatures allow for signing a document without actually seeing it, and secret sharing is a concept that enables the detection of double-spending, that is, using the same e-cash token twice.
In 2009, the first practical implementation of an e-cash system named Bitcoin appeared. The term cryptocurrency emerged later. For the very first time, it solved the problem of distributed consensus in a trustless network. It used public key cryptography with a PoW mechanism to provide a secure, controlled, and decentralized method of minting digital currency. The key innovation was the idea of an ordered list of blocks composed of transactions which is cryptographically secured by the PoW mechanism to prevent double-spending in a trustless environment. This concept will be explained in greater detail in Chapter 6, Introducing Bitcoin.
Other technologies used in Bitcoin, but which existed before its invention, include Merkle trees, hash functions, and hash chains. All these concepts are explained in appropriate depth in Chapter 4, Public Key Cryptography.
Looking at all the technologies mentioned previously and their relevant history, it is easy to see how concepts from e-cash schemes and distributed systems were combined to create Bitcoin and what now is known as blockchain. This concept can also be visualized with the help of the following diagram:
Figure 1.4: The various ideas that supported the invention of Bitcoin and blockchain
With the emergence of e-cash covered, along with the ideas that led to the formation of Bitcoin and blockchain, we can now begin to discuss blockchain itself.
In 2008, a groundbreaking paper, entitled Bitcoin: A Peer-to-Peer Electronic Cash System, was written on the topic of peer-to-peer e-cash under the pseudonym of Satoshi Nakamoto.
This paper is available at https://bitcoin.org/bitcoin.pdf.
It introduced the term chain of blocks. No one knows the actual identity of Satoshi Nakamoto. After introducing Bitcoin in 2009, he remained active in the Bitcoin developer community until 2011. He then handed over Bitcoin development to its core developers and simply disappeared. Since then, there has been no communication from him whatsoever, and his existence and identity are shrouded in mystery. The term "chain of blocks" evolved over the years into the word "blockchain."
As stated previously, blockchain technology incorporates a multitude of applications that can be implemented in various economic sectors. Particularly in the finance sector, significant improvement in the performance of financial transactions and settlements manifests as highly desirable time-and-cost reductions. Additional light will be shed on these aspects of blockchain in Chapter 19, Blockchain – Outside of Currencies, where practical use cases will be discussed in detail for various industries. For now, it is sufficient to say that parts of nearly all economic sectors have already realized the potential and promise of blockchain, and have embarked, or will do so soon, on the journey to capitalize on the benefits of blockchain technology.
A good place to start learning what blockchain is would be to see its definition. There are some different ways that blockchain may be defined; following are two of the most widely accepted definitions:
Layman's definition: Blockchain is an ever-growing, secure, shared recordkeeping system in which each user of the data holds a copy of the records, which can only be updated if all parties involved in a transaction agree to update.
Technical definition: Blockchain is a peer-to-peer, distributed ledger that is cryptographically secure, append-only, immutable (extremely hard to change), and updateable only via consensus or agreement among peers.
Now, let's examine things in some more detail. We will look at the keywords from the technical definition one by one.
The first keyword in the technical definition is peer-to-peer, or P2P This means that there is no central controller in the network, and all participants (nodes) talk to each other directly. This property allows for transactions to be conducted directly among the peers without third-party involvement, such as by a bank.
Dissecting the technical definition further reveals that blockchain is a "distributed ledger," which means that a ledger is spread across the network among all peers in the network, and each peer holds a copy of the complete ledger.
Next, we see that this ledger is "cryptographically secure," which means that cryptography has been used to provide security services that make this ledger secure against tampering and misuse. These services include non-repudiation, data integrity, and data origin authentication. You will see how this is achieved later in Chapter 4, Public Key Cryptography, which introduces the fascinating world of cryptography.
Another property that we encounter is that blockchain is "append-only," which means that data can only be added to the blockchain in time-sequential order. This property implies that once data is added to the blockchain, it is almost impossible to change that data and it can be considered practically immutable. In other words, blocks added to the blockchain cannot be changed, which allows blockchain to become an immutable and tamper-proof ledger of transactions.
However, remember that it can be changed in rare scenarios wherein collusion against the blockchain network by bad actors succeeds in gaining more than 51 percent of the power. Otherwise, the blockchain is practically immutable.
There may be some legitimate reasons to change data in the blockchain once it has been added, such as the "right to be forgotten" or "right to erasure" (also defined in the GDPR ruling: https://gdpr-info.eu/art-17-gdpr/).
However, those are individual cases that need to be handled separately and that require an elegant technical solution. For all practical purposes, blockchain is indeed immutable and cannot be changed.
The most critical attribute of a blockchain is that it is updateable only via consensus. This is what gives it the power of decentralization. In this scenario, no central authority is in control of updating the ledger. Instead, any update made to the blockchain is validated against strict criteria defined by the blockchain protocol and added to the blockchain only after a consensus has been reached among all participating peers/nodes on the network. To achieve consensus, there are various consensus facilitation algorithms that ensure all parties agree on the final state of the data on the blockchain network and resolutely agree upon it to be true. Consensus algorithms are introduced later in this chapter, and then in more detail in Chapter 5, Consensus Algorithms.
Having detailed the primary features of blockchain, we are now in a position to begin to look at its actual architecture. We'll begin by looking at how blockchain acts as a layer within a distributed peer-to-peer network.
Blockchain can be thought of as a layer of a distributed peer-to-peer network running on top of the internet, as can be seen in the following diagram. It is analogous to SMTP, HTTP, or FTP running on top of TCP/IP:
Figure 1.5: The architectural view of a generic blockchain
Now we'll discuss all these elements one by one:
All these concepts will be discussed in detail later in this book in various chapters. Next, we'll look at blockchain from more of a business-oriented perspective.
From a business standpoint, a blockchain can be defined as a platform where peers can exchange value/e-cash using transactions without the need for a centrally trusted arbitrator. For example, for cash transfers, banks act as a trusted third party. In financial trading, a central clearing house acts as a trusted third party between two or more trading parties. This concept is compelling, and, once you absorb it, you will realize the enormous potential of blockchain technology. This disintermediation allows blockchain to be a decentralized consensus mechanism where no single authority is in charge of the database. Immediately, you'll see a significant benefit of decentralization here, because if no banks or central clearing houses are required, then it immediately leads to cost savings, faster transaction speeds, and more trust.
We've now looked at what blockchain is at a fundamental level. Next, we'll go a little deeper and look at some of the elements that comprise a blockchain.
Now, let's walk through the generic elements of a blockchain. You can use this as a handy reference section if you ever need a reminder about the different parts of a blockchain. More precise elements will be discussed in the context of their respective blockchains in later chapters, for example, the Ethereum blockchain. The structure of a generic blockchain can be visualized with the help of the following diagram:
Figure 1.6: Generic structure of a blockchain
Elements of a generic blockchain are described here one by one. These are the elements that you will come across in relation to blockchain:
The following structure is a simple block diagram that depicts a block. Specific block structures relative to their blockchain technologies will be discussed later in the book with greater in-depth technical detail:
Figure 1.7: The generic structure of a block
Generally, however, there are just a few attributes that are essential to the functionality of a block: the block header, which is composed of the hash of the previous block's header, the timestamp, nonce, Merkle root, and the block body that contains the transactions. There are also other attributes in a block, but generally, the components introduced in this section are usually available in a block.
Think of the scripting language as a calculator that only supports standard preprogrammed arithmetic operations. As such, the Bitcoin Script language cannot be called "Turing complete." In simple words, a Turing complete language means that it can perform any computation. It is named after Alan Turing, who developed the idea of a Turing machine that can run any algorithm however complex. Turing complete languages need loops and branching capability to perform complex computations. Therefore, Bitcoin's scripting language is not Turing complete, whereas Ethereum's Solidity language is.
To facilitate arbitrary program development on a blockchain, a Turing complete programming language is needed, and it is now a very desirable feature to have for blockchains. Think of this as a computer that allows the development of any program using programming languages. Nevertheless, the security of such languages is a crucial question and an essential and ongoing research area. We will discuss this in greater detail in Chapter 6, Introducing Bitcoin, Chapter 10, Smart Contracts, and the chapters on Ethereum Development, later in this book.
A high-level diagram of blockchain architecture highlighting the key elements mentioned previously is shown as follows:
Figure 1.8: Generic structure of a blockchain network
The preceding diagram shows a four-node blockchain network (top), each maintaining a chain of blocks, virtual machine, state machine, and address. The blockchain is then further magnified (middle) to show the structure of the chain of blocks, which is again magnified (bottom) to show the structure of a transaction. Note that this is a generic structure of a blockchain; we will see specific blockchains structures in detail in the context of Ethereum and Bitcoin blockchains later in this book.
We have now defined and described blockchain. Now, let's see how a blockchain actually works. Nodes are either miners who create new blocks and mint cryptocurrency (coins) or block signers who validate and digitally sign the transactions. A critical decision that every blockchain network has to make is to figure out which node will append the next block to the blockchain. This decision is made using a consensus mechanism. The consensus mechanism will be described later in this chapter. For now, we will look at how a blockchain validates transactions and creates and adds blocks to grow the blockchain.
We will look at a general scheme for creating blocks. This scheme is presented here to give you a general idea of how blocks are generated and what the relationship is between transactions and blocks:
This process can be visualized in the diagram as follows:
Figure 1.9: How a block is generated
This completes the basic introduction to blockchain. In the next section, you will learn about the benefits and limitations of this technology.
Numerous advantages of blockchain technology have been discussed in many industries and proposed by thought leaders around the world who are participating in the blockchain space. The notable benefits of blockchain technology are as follows:
Smart contracts
Blockchain technology provides a platform for running smart contracts. These are automated, autonomous programs that reside on the blockchain network and encapsulate the business logic and code needed to execute a required function when certain conditions are met. For example, think about an insurance contract where a claim is paid to the traveler if the flight is canceled. In the real world, this process normally takes a significant amount of time to make the claim, verify it, and pay the insurance amount to the claimant (traveler). What if this whole process were automated with cryptographically-enforced trust, transparency, and execution so that as soon as the smart contract received a feed that the flight in question has been canceled, it automatically triggers the insurance payment to the claimant? If the flight is on time, the smart contract pays itself.
This is indeed a revolutionary feature of blockchain, as it provides flexibility, speed, security, and automation for real-world scenarios that can lead to a completely trustworthy system with significant cost reductions. Smart contracts can be programmed to perform any actions that blockchain users need and according to their specific business requirements.
As with any technology, some challenges need to be addressed in order to make a system more robust, useful, and accessible. Blockchain technology is no exception. In fact, much effort is being made in both academia and industry to overcome the challenges posed by blockchain technology. The most sensitive blockchain problems are as follows:
All of these issues and possible solutions will be discussed in detail in Chapter 21, Scalability and Other Challenges.
You now know the basics of blockchain and its benefits and limitations. Now, let's take a look at the various types of blockchain that exist.
Based on the way that blockchain has evolved over the last few years, it can be divided into multiple categories with distinct, though sometimes partially overlapping attributes. You should note that the tiers described earlier in the chapter are a different concept, whereby the logical categorization of blockchain, based upon its evolution and usage, is presented.
In this section, we will examine the different types of blockchains from a technical and business use perspective. These blockchain types can occur on any blockchain tier, as there is no direct relationship between those tiers mentioned earlier and the various types of blockchain.
In this section, we'll examine:
First, I need to clarify an ambiguity. It should be noted that a distributed ledger is a broad term describing shared databases; hence, all blockchains technically fall under the umbrella of shared databases or distributed ledgers. Although all blockchains are fundamentally distributed ledgers, all distributed ledgers are not necessarily blockchains.
A critical difference between a distributed ledger and a blockchain is that a distributed ledger does not necessarily consist of blocks of transactions to keep the ledger growing. Rather, a blockchain is a special type of shared database that is comprised of blocks of transactions. An example of a distributed ledger that does not use blocks of transactions is R3's Corda (https://www.corda.net). Corda is a distributed ledger that is developed to record and manage agreements and is especially focused on the financial services industry. On the other hand, more widely known blockchains like Bitcoin and Ethereum make use of blocks to update the shared database.
As the name suggests, a distributed ledger is distributed among its participants and spread across multiple sites or organizations. This type of ledger can be either private or public. The fundamental idea here is that, unlike many other blockchains, the records are stored contiguously instead of being sorted into blocks. This concept is used in Ripple, which is a blockchain- and cryptocurrency-based global payment network.
It should be noted that over the last few years, the terms distributed ledger or DLT have grown to be commonly used to describe blockchain in the finance industry. Sometimes, blockchain and DLT are used interchangeably. Though this is not entirely accurate, it is how the term has evolved recently, especially in the finance sector. In fact, DLT is now a very active and thriving area of research in the financial sector. From a financial sector point of view, DLTs are permissioned blockchains that are used by consortiums. DLTs usually serve as a shared database, with all participants known and verified. They do not have a cryptocurrency and do not require mining to secure the ledger.
At a broader level, DLT is an umbrella term that represents Distributed Ledger Technology as a whole, comprising of blockchains and distributed ledgers of different types.
As the name suggests, public blockchains are not owned by anyone. They are open to the public, and anyone can participate as a node in the decision-making process. Users may or may not be rewarded for their participation. All users of these "permissionless" or "un-permissioned" ledgers maintain a copy of the ledger on their local nodes and use a distributed consensus mechanism to decide the eventual state of the ledger. Bitcoin and Ethereum are both considered public blockchains.
As the name implies, private blockchains are just that—private. That is, they are open only to a consortium or group of individuals or organizations who have decided to share the ledger among themselves. There are various blockchains now available in this category, such as Kadena and Quorum. Optionally, both of these blockchains can also run in public mode if required, but their primary purpose is to provide a private blockchain.
With semi-private blockchains, part of the blockchain is private and part of it is public. Note that this is still just a concept today, and no real-world proofs of concept have yet been developed. With a semi-private blockchain, the private part is controlled by a group of individuals, while the public part is open for participation by anyone.
This hybrid model can be used in scenarios where the private part of the blockchain remains internal and shared among known participants, while the public part of the blockchain can still be used by anyone, optionally allowing mining to secure the blockchain. This way, the blockchain as a whole can be secured using PoW, thus providing consistency and validity for both the private and public parts. This type of blockchain can also be called a "semi-decentralized" model, where it is controlled by a single entity but still allows for multiple users to join the network by following appropriate procedures.
More precisely known as "pegged sidechains," this is a concept whereby coins can be moved from one blockchain to another and then back again. Typical uses include the creation of new altcoins (alternative cryptocurrencies) whereby coins are burnt as a proof of an adequate stake. "Burnt" or "burning the coins" in this context means that the coins are sent to an address that is un-spendable, and this process makes the "burnt" coins irrecoverable. This mechanism is used to bootstrap a new currency or introduce scarcity, which results in the increased value of the coin.
This mechanism is also called "Proof of Burn" and is used as an alternative method for distributed consensus to PoW and Proof of Stake (PoS). The example provided previously for burning coins applies to a one-way pegged sidechain. The second type is called a two-way pegged sidechain, which allows the movement of coins from the main chain to the sidechain and back to the main chain when required.
This process enables the building of smart contracts for the Bitcoin network. Rootstock is one of the leading examples of a sidechain, which enables smart contract development for Bitcoin using this paradigm. It works by allowing a two-way peg for the Bitcoin blockchain, and this results in much faster throughput.
A permissioned ledger is a blockchain where participants of the network are already known and trusted. Permissioned ledgers do not need to use a distributed consensus mechanism; instead, an agreement protocol is used to maintain a shared version of the truth about the state of the records on the blockchain. In this case, for verification of transactions on the chain, all verifiers are already preselected by a central authority and, typically, there is no need for a mining mechanism.
By definition, there is also no requirement for a permissioned blockchain to be private, as it can be a public blockchain but with regulated access control. For example, Bitcoin can become a permissioned ledger if an access control layer is introduced on top of it that verifies the identity of a user and then allows access to the blockchain.
This is a generic term that is used to describe any application or database that is shared by the public or a consortium. Generally, all blockchains fall into the category of a shared ledger.
There is no mainstream application of these types of blockchains, as they deviate from the core concept of decentralization in blockchain technology. Nonetheless, in specific private settings within an organization, there could be a need to share data and provide some level of guarantee of the authenticity of the data.
An example of this type of blockchain might be to allow for collaboration and the sharing of data between various government departments. In that case, no complex consensus mechanism is required, apart from simple SMR and an agreement protocol with known central validators. Even in private blockchains, tokens are not really required, but they can be used as a means of transferring value or representing some real-world assets.
These blockchains are standard blockchains that generate cryptocurrency as a result of a consensus process via mining or initial distribution. Bitcoin and Ethereum are prime examples of this type of blockchain.
These blockchains are designed in such a way that they do not have the basic unit for the transfer of value. However, they are still valuable in situations where there is no need to transfer value between nodes and only the sharing of data among various trusted parties is required. This is similar to fully private blockchains, the only difference being that the use of tokens is not required. This can also be thought of as a shared distributed ledger used for storing and sharing data between the participants. It does have its benefits when it comes to immutability, tamper proofing, security, and consensus-driven updates but is not used for a common blockchain application of value transfer or cryptocurrency. Most of the permissioned blockchains can be seen as an example of tokenless blockchains, for example, Hyperledger Fabric or Quorum. Tokens can be built on these chains as an application, but intrinsically these blockchains do not have a token associated with them.
All the aforementioned terminologies are used in literature, but fundamentally all these blockchains are distributed ledgers and fall under the top-level category of DLTs. We can view these different types in the simple chart as follows:
Figure 1.10: DLT hierarchy
This ends our examination of the various types of blockchain. We'll now move on to the next section to discuss the concept of consensus.
Consensus is the backbone of a blockchain, as it provides the decentralization of control through an optional process known as mining. The choice of the consensus algorithm to utilize is governed by the type of blockchain in use; that is, not all consensus mechanisms are suitable for all types of blockchains. For example, in public permissionless blockchains, it would make sense to use PoW instead of mechanisms that are more suitable for permissioned blockchains, such as Proof of Authority (PoA) or traditional Byzantine fault-tolerant consensus mechanisms. Therefore, it is essential to choose an appropriate consensus algorithm for a particular blockchain project.
Consensus is a process of achieving agreement between distrusting nodes on the final state of data. To achieve consensus, different algorithms are used. It is easy to reach an agreement between two nodes (in client-server systems, for example), but when multiple nodes are participating in a distributed system and they need to agree on a single value, it becomes quite a challenge to achieve consensus. This process of attaining agreement on a common state or value among multiple nodes despite the failure of some nodes is known as distributed consensus.
A consensus mechanism is a set of steps that are taken by most or all nodes in a blockchain to agree on a proposed state or value. For more than three decades, this concept has been researched by computer scientists in industry and academia. With the advent of blockchain and Bitcoin, consensus mechanisms have come into the limelight again and gained considerable popularity.
There are various requirements for a consensus mechanism. The following describes these requirements:
Having seen these general requirements, we'll now look at the different types of consensus mechanisms.
All consensus mechanisms are developed to deal with faults in a distributed system and to allow distributed systems to reach a final state of agreement. There are two general categories of consensus mechanisms. These categories deal with all types of faults (fail-stop types or arbitrary). These common types of consensus mechanisms are as follows:
To achieve fault tolerance, replication is used. This is a standard and widely used method to achieve fault tolerance. In general, there are two types of faults that a node can experience:
Many other implementations of consensus protocols have been proposed in traditional distributed systems. Paxos is the most famous of these protocols. It was introduced by Leslie Lamport in 1989. With Paxos, nodes are assigned various roles such as Proposer, Acceptor, and Learner. Nodes or processes are named replicas, and consensus is achieved in the presence of faulty nodes by an agreement among a majority of nodes.
An alternative to Paxos is RAFT, which works by assigning any of three states; that is, Follower, Candidate, or Leader to the nodes. A Leader is elected after a Candidate node receives enough votes, and all changes then have to go through the Leader. The Leader commits the proposed changes once replication on the majority of the follower nodes is completed.
We will briefly touch on some aspects of consensus in blockchain now, but more detail on the theory of consensus mechanisms from a distributed system point of view and also from the blockchain perspective will be presented in Chapter 5, Consensus Algorithms.
Consensus is a distributed computing concept that has been used in blockchain in order to provide a means of agreeing to a single version of the truth by all peers on the blockchain network. This concept was previously discussed in the distributed systems section of this chapter. In this section, we will address consensus in the context of blockchain technology. Some concepts presented following are still relevant to the distributed systems theory, but they are explained from a blockchain perspective.
Roughly, the following describes the two main categories of consensus mechanisms:
BFT-based consensus mechanisms perform well when there are a limited number of nodes, but they do not scale well. On the other hand, leader-election lottery-based (PoW) consensus mechanisms scale very well but perform very slowly. As there is significant research being conducted in this area, new types of consensus mechanisms are also emerging, such as the semi-decentralized type, which is used in the Ripple network. The Ripple network will be discussed in detail in this book's online content pages, here: https://static.packt-cdn.com/downloads/Altcoins_Ethereum_Projects_and_More_Bonus_Content.pdf. There are also various other proposals out there, which are trying to find the right balance between scalability and performance. Some notable projects include PBFT, Hybrid BFT, BlockDAG, Tezos, Stellar, and GHOST.
The consensus algorithms available today, or that are being researched in the context of blockchain, are presented as follows. The following is not an exhaustive list, but it includes all notable algorithms:
Some prominent protocols in blockchain will be discussed in detail in Chapter 5, Consensus Algorithms. In this chapter, a light introduction is presented only.
Remember, earlier in this chapter we said that distributed systems are difficult to build and a distributed system cannot have consistency, availability, and partition tolerance at the same time. This is a proven result; however, in blockchain, it seems that this theorem is somehow violated. In the next section, we will introduce the CAP theorem formally and discuss why blockchain appears to achieve all three properties simultaneously.
The CAP theorem, also known as Brewer's theorem, was introduced by Eric Brewer in 1998 as a conjecture. In 2002, it was proven as a theorem by Seth Gilbert and Nancy Lynch. The theorem states that any distributed system cannot have consistency, availability, and partition tolerance simultaneously:
A Venn diagram is commonly used to visualize the CAP theorem:
Figure 1.11: CAP theorem
The preceding diagram shows that only two properties at a time can be achieved. Either AP, CA, or CP.
In summary:
Usually, a network partition cannot be ignored; therefore, the choice mostly becomes either consistency or availability in the case of a network partition.
As shown previously, a distributed system cannot have consistency, availability, and partition tolerance simultaneously. This can be explained with the following example.
Let's imagine that there is a distributed system with two nodes. Now, let's apply the three theorem properties on this smallest of possible distributed systems only with two nodes:
Now think of a scenario where a partition occurs, and nodes can no longer communicate with each other. If new updated data comes in now, it can only be updated on one node only. In that case, if the node accepts the update, then only that one node in the network is updated and therefore consistency is lost. Now, if the update is rejected by the node, that would result in loss of availability. In that case, due to partition tolerance, both availability and consistency are unachievable.
This is strange because somehow blockchain manages to achieve all of these properties—or does it?
It seems that the CAP theorem is violated by blockchain, especially in its most successful implementation, Bitcoin. However, this is not the case. In blockchains, consistency is sacrificed in favor of availability and partition tolerance. In this scenario, Consistency (C) on the blockchain is not achieved simultaneously with Partition tolerance (P) and Availability (A), but it is achieved over time. This is called eventual consistency, where consistency is achieved as a result of validation from multiple nodes over time. It means that there can be a temporary disagreement between nodes on the final state, but it is eventually agreed upon. For example, in Bitcoin, multiple transaction confirmations are required to achieve a good level of confidence that transactions may not be rolled back in the future and eventually a consistent view of transaction history is available to all nodes. Multiple confirmations of a transaction over time provide eventual consistency in Bitcoin. For this purpose, the process of mining was introduced in Bitcoin. Mining is a process that facilitates the achievement of consensus by using the PoW algorithm. At a higher level, mining can be defined as a process that is used to add more blocks to the blockchain. We will cover more on this later in Chapter 6, Introducing Bitcoin.
This chapter introduced blockchain technology at an advanced level. First, we discussed blockchain's progress toward becoming a mature technology, followed by some basic concepts about distributed systems, and then the history of blockchain was reviewed. Concepts such as e-cash were also discussed.
Various definitions of blockchain from different points of view were presented. Some applications of blockchain technology were also introduced. Next, different types of blockchain were explored. Finally, the benefits and limitations of this new technology were also examined. Some topics such as blockchain scalability and adaptability issues were intentionally introduced only lightly, as they will be discussed in depth in later chapters.
In the next chapter, we will introduce the concept of decentralization, which is central to the idea behind blockchains and their vast number of applications.