Hello OPEN Community,
We would like to provide an update on OPEN State, the crucial role it plays in providing interoperability, and an overview of other technical developments that we have been working on.
The number of digital assets and blockchains is growing rapidly. However, blockchains and related solutions remain isolated, operating and developing in their own respective silos. We’re seeing a rise in unique solutions within their own respective environments. They are each providing a unique solution to their approach to decentralized technologies.
Blockchain technology is becoming more appealing amongst the masses. And more and more developers face and will face the problem of interoperability of their application with different blockchains and distributed ledgers.
Interoperability can provide developers with the ability to combine the advantages of optimizations of different consensus mechanisms and virtual machines, in many cases off-loading tasks that are better accomplished on other chains and letting each focus on its core competencies. In the emerging world of blockchains, interoperability can make them more useful, user-friendly, efficient, and scalable.
Nowadays, there are numerous blockchains that are popular and widely used. It is quite a challenge to integrate all of them into an application because developers need to integrate each blockchain separately into their solution. Also, blockchain ecosystems can differ a lot from each other.
Thousands of developers could waste a lot of time and energy searching for or creating solutions to this problem. They often need to integrate a batch of different technologies to achieve their goals.
In some cases, you cannot even integrate blockchains directly and need to use some kind of middleware (i.e. side-chains or state-channels). It makes development complicated, long, and expensive. All these technical roadblocks compel us to develop an approach which provides a simple and convenient way to interoperate with different blockchains.
When we talk about interoperability, we need to be a little more specific about what it means. On high-level, interoperability can be split into two categories:
The first option can be constructed atop blockchains which have fairly simple programming capabilities, as users on both sides can easily produce publicly verifiable signatures for actions that enable atomic transfers or transfers which only complete if both sides do their part.
The second option is the more difficult type of interoperability, as most blockchains are passive systems that can’t produce a signature that can be easily verified by others, and thus needs more complex proofs about their state to be understood by outside parties.
Vitalik Buterin wrote in this paper that there are 3 primary technical approaches to achieving interoperability:
A notary mechanism can be considered to be the technologically simplest way to reach cross-chain interoperability. This approach utilizes trusted intermediaries to attest to chain A that a given event on chain B took place or that a particular claim about chain B is true.
Instead of interaction with intermediaries, relays allow the chains to get information about the target chain by themselves. A relay is a contract on chain A that functions as a light client of chain B, using chain B’s standard verification procedure to verify block headers fed into the contract.
Hash-locking is the most practical technical approach to interoperability and as one might expect, also the most limiting in terms of functionality. A hashed timelock contract (HTLC) is a class of blockchain-based payments that use hashlocks and timelocks to require the receiver of a payment to either acknowledge receipt prior to a deadline or forfeit the ability to claim the payment, returning it to the payer. HTLCs allow for cross-chain atomic swaps and fully funded bi-directional payment channels between assets on certain types of blockchains.
OPEN Platform is not intended to provide cross-chain interoperability. It provides the necessary payment infrastructure such as coin transfers and token transfers. The key feature of Open State is providing transaction tracking options for supported blockchains. There is no need to make complex integrations and no need to use any SDKs. Transaction notifications are implemented via webhook invocation using HTTP protocol, so it provides interoperability between off-chain and on-chain worlds.
Such an infrastructure is a complicated system which consists of several essential parts. These parts should interoperate with each other, and this is a vital point of the whole system. That’s why we need to provide intrinsic interoperability first.
As mentioned above, interoperability between system parts is crucial. Given the system complexity, each part should comply with the single responsibility approach. Each system part should implement a logical atomic set of functionality. It provides efficient scalability and system extensibility. For these purposes the system splits into OPEN API, OPEN Scaffold and OPEN State. Whereas a user interacts with OPEN API and OPEN Scaffold interacts with a blockchain, OPEN State provides general interoperability.
OPEN Platform needs to provide data exchange between the system parts. OPEN State collects all necessary data from OPEN API, blockchains, and smart contracts, storing it in a distributed ledger. Using decentralized storage provides safety and reliability. Also, OPEN State provides data to other system parts.
The way OPEN Platform implements interoperability is described in more detail in the next section.
OPEN Platform provides necessary payment infrastructure for application developers to utilize the blockchain, including cryptocurrency technologies. OPEN Platform is a hybrid technology which harmoniously combines on-chain and off-chain approaches.
At the current moment, OPEN Platform consists of several essential parts:
OPEN API provides a convenient API to integrate with OPEN Platform. OPEN API interacts with a blockchain via OPEN Scaffold.
OPEN Scaffold is a smart contract which provides money flow.
OPEN SDK provides necessary tools for application developers.
OPEN State is intended to track and store a wallet’s state.
OPEN CHAIN serves as multipurpose blockchain technology.
OPEN Scaffold is an abstraction which guarantees the ability to create convenient asset flow. To integrate a blockchain with OPEN Platform, the blockchain should support smart contracts. Then it is necessary to create a smart contract template which implements the OPEN Scaffold approach. The smart contract template provides a developer with a set of adjustments for creating a specific smart contract that satisfies the developer’s needs. Then it can be deployed via OPEN API. After deploying, the developer saves the ability to manage the smart contract.
OPEN API is intended to provide the opportunity to deploy OPEN Scaffold into a target blockchain. Users can interact with OPEN API via its UI. Application developers can integrate with OPEN API through OPEN SDK.
OPEN State is partially based on relay’s approach and built atop a fully managed ledger database which provides a transparent, immutable, and cryptographically verifiable transaction log owned by the central trusted authority. OPEN State tracks each and every wallet state change and maintains the complete and verifiable history of changes over time.
OPEN Chain is a high-performance DPoS-based blockchain network which is going to solve the throughput issue of existing blockchain networks. OPEN Chain will be suggested by default for unspecified transactions for load-balancing purposes.
With OPEN Platform, developers can receive convenient, robust and powerful tools for integrating cryptocurrency payments into their applications. All payment history is reliably stored in the distributed ledger and can be provided at any moment.
The hybrid architecture of OPEN Platform provides several advantages:
Usability – to experience the power of interoperability in an intuitive and easy manner, developers should integrate with OPEN Platform.
Interoperability – at the current moment, OPEN Platform provides partial cross-chain interoperability. However, further development will bring the full power of cross-chain interoperability.
Extensibility – the number of integrated blockchains may be increased at any moment. Also, OPEN Platform’s functionality can be extended by adding on new parts of the platform.
Efficiency – OPEN State may provide the ability to decrease the number of transactions in target blockchains.
If we dive deeper into the OPEN Platform mechanism, we can define five interesting points. To help understand this, please view the diagram below:
OPEN State provides an open source solution for tracking wallets, utilizes the reactive approach, and has been developed with a beautiful architecture. For a developer to accept cryptocurrency, they need to implement many different things. Firstly, there might be many different options, such as Bitcoin and Ethereum. For each currency, the developer needs to implement its integration. With regards to the integration, there are also many options such as native integration or payment gateway integration. Each native integration is usually very complicated and might be vulnerable.
Many backend services these days are REST-ful (i.e. they operate over HTTP), so the underlying protocol is fundamentally blocking and synchronous. Microservice calls or API calls often involve calling other services and then even more services depending on the results from the first calls. With so much IO going on, if you were to wait for one call to complete before sending the next request, your poor client would give up in frustration before you managed to assemble a reply. So external service calls, especially complex orchestrations of dependencies between calls, are a good thing to optimize. In OPEN State, as we call external APIs over HTTP such as getting the state of blocks in Ethereum, we utilize Kotlin coroutines. Additionally, our storage libraries provide reactive drivers.
The architecture of tracking wallets provides an easy way to extend and multiply the number of integrated blockchain networks. And it is not only powerful from the performance perspective, but also extensible. Furthermore, it is already integrated with Ethereum and in the near future, it will support Bitcoin and BNB (Binance blockchain).
The architecture of wallet tracking in OPEN State is implemented in such a way that makes it easier to add new integrations into blockchains. To do this, you just need to create your own class, inherit from a blockchain, and implement its methods. After that, it is imperative to mark it as a bean, so that later your implementation will be noticed by the Spring context.
There are two jobs involved in wallet tracking:
This processor wakes up at a certain time with a fixed delay after each execution. Utilizing a coroutine scope, it receives one random blockchain from the Redis queue that needs to be processed. After, it will immediately try to lock this blockchain for processing by adding a value in Redis. The idea behind this solution is that more than one OPEN State instance will be launched on the servers and to not allow several jobs to do the processing at the same time. This solution allows you to not load servers and guarantees the correct processing of blocks. If the processor fails to take over(lock) for processing, then we would consider that this blockchain is already being processed by another job. Otherwise, we get the number of the last processed block for the current blockchain from Redis. After, through this block number, we get all transactions and find wallets that need to be tracked, saving them in MongoDB. Furthermore, after processing, we increase the number of the currently processed blocks in Redis. At this point, it is worth emphasizing that after each block is processed, we update the lock that we mentioned earlier, since on Redis this is implemented through a timeout. That is, if the processor does not complete the work at the specified time, then the blockchain will be automatically removed from the locks in Redis. Accordingly, another processor will take over. This whole process is performed in a loop until this job processes til the last block(last> current). After iterating these operations, it is logical to release the lock for the current blockchain by deleting the value in Redis.
The structure of storage in Redis is as follows:
It is worth noting that the blockchain used below is the name of the class that is inherited from the parent blockchain class. For example, the already integrated Ethereum – in this case, it will be EthereumBlockchain.
This job wakes up at a certain time with a fixed delay after each execution. This process is needed to periodically check for new unprocessed blocks. Firstly, when launching the Kotlin coroutine, it will go through each blockchain with which OPEN State was integrated. Also, for each blockchain, it will request from the implementation (by calling the getLastBlockNumber method which obliges to implement this method) the last block number in the system. In addition, when receiving information about the last known block from Redis, it compares these block numbers. If these values are not equal, then we update the value of the last block in Redis for a blockchain. Additionally, we add the current blockchain in the queue in Redis so that the Blockchain processor will be processed later. Otherwise, if these values are equal, we simply end the processing of the current blockchain, since we already know the current block number and there is no need to trigger Redis.
A webhook invocation is the URL to which an HTTP request should be sent if this webhook wallet participates in a blockchain transaction (i.e. fund transfer).
This architectural decision was made to provide the system with flexibility, a guarantee of webhook delivery, and even to load distribution to instances.
Single Responsibility principle states that every object should have one responsibility and this responsibility must be completely encapsulated. Following this principle, it was divided into several processors, each one responsible for its specific task. Processors in this context are jobs that wake up at a certain time and perform a task:
A webhook processor makes only one webhook invocation for one wallet at a time. This job wakes up at a certain time with a fixed delay after each execution. This allows for the avoidance of spam and overloading of remote servers. In addition, this starts several instances of OPEN State to make webhook invocation faster. Logically, the architecture of a webhook invocation can be divided into 2 main parts:
The webhook invocation algorithm is presented in the diagram below:
All detected transactions in a blockchain processor are added to a webhook queue. A webhook queue consists of 2 parts:
The webhook invocation processor takes the task that is closest to the current timestamp from the webhook’s queue and tries to execute it. Every task is executed. It retries task execution if the webhook invocation fails. The webhook invocation processor can be configured with the following properties:
The webhook invocation executes as follows:
A webhook payload would be sent via HTTP POST request and include the following data:
A transaction object would include the following data:
Example of payload:
{ “blockchain”: “Ethereum”, “walletAddress”:
“0x9D86b1B2554ec410ecCFfBf111A6994910111340”, “transaction”: { “hash”: “0x9D86b1B2554ec410ecCFfBf111A6994910111340”, “blockHash”:
“0x3aabcf5ca3ec5e459d9aa1314d67c6595beac810”, “blockHeight”: 124414145,
“amount”: 0.314141, “date”: “2020-01-12T00:22:43.511Z”, “to”:
“0x9D86b1B2554ec410ecCFfBf111A6994910111340”, “from”:
“0x3aabcf5ca3ec5e459d9aa1314d67c6595beac810” } }
This processor is needed in order to periodically check that the webhook processor has not exceeded the processing time.
This job works in the following ways:
This processor guarantees delivery if the webhook processor does not finish processing the webhook.
For more details on OPEN State, please refer to our GitHub repository. Stay tuned for more updates!