Problem Description

As a crypto startup company, Futureswap has developed a Decentralized Leverage Trading Exchange on various blockchains (currently Arbitrum and Avalanche).

As a data repository, a Blockchain is nothing more than a kind of database without SQL, where each block has a strong dependency on its predecessor ones. Due to the event-oriented storage schema, ad hoc or aggregated queries aren’t straightforward or cost-efficient, and may even be impossible due to the lack of on-chain data. 

Take for example if someone wants to retrieve the trading frequency per wallet within an exchange. To do so, they must surf the entire blockchain looking for that wallet's operations and then compute the metric. 

Clearly, this would require far too much manual time in order to glean insights. For this reason, Futureswamp was keen on implementing an Analytics Dashboard Solution showing KPIs and insights based on pre-aggregated results and pre-indexed data. 

Wait, what about using DuneAnalytics? 

Naturally, we evaluated DuneAnalytics as an alternative for displaying the desired KPIs for our customer. Everytime we accept a new challenge from our customers, we evaluate de facto solutions and alternative packages before considering developing something from scratch. 

At the moment our solution was designed, DuneAnalytics didn’t support the L2 Arbitrum network nor Avalanche. As of April 2022, Dune only supports Ethereum, Polygon, Binance Smart Chain, Optimism and Gnosis Chain.

Solutions

To start defining Futureswap’s dashboard, we ran a couple of sprint iterations to render our first KPIs from a list of initial agreed metrics. We used AWS since our client had already implemented some web2 features, such as the static website and a group of lambdas functions, among others.

The data pipeline

Let’s start from the beginning by explaining the data sources utilized for our data pipeline.

TheGraph, a.k.a. The "Google of blockchains," is a decentralized blockchain indexer protocol which exposes data via GraphQL. This is the replacement for the REST query language, as it solves the over/under-fetching issues by allowing the client to request only the needed data.

In order to let TheGraph index your protocol, you need to develop your own subgraphs. That is, to define the schema file the indexer will use for scraping the blockchain network your protocol is deployed on. As we mentioned above, our first version uses only Arbitrum blockchain.  

Although our DeFi protocol has their own liquidity pools, we aren’t fully isolated from the cryptocurrencies ecospace. For this reason, we should have knowledge of precise token prices at the moment a user adds liquidity or opens/closes a position, i.e: ETH price. In the same vein, we also had to retrieve data from Oracles. We chose the most used Oracle at the moment, Chainlink, plus a blockchain node provider, Infura. 

However, not every listed token is present in Chainlink. For example, our governance token FST wasn’t there. Also, retrieving historical data from Oracles isn’t a common scenario. That was the reason we added another data provider as a data source: Coingecko via its REST API. 

For gathering data from these data sources, we have implemented a Pulling solution, periodically triggered with EventBridge. A set of lambdas crawls data via GraphQL, REST QL and webjs libraries. 

The storage layer

Sometimes “less is more." For simplicity, we defined a relational schema in RDS. As a common premise on a dashboard backend solution, the storage tables must be designed for fast reads and easy queries. 

The UI

As a Business Intelligence solution, we have selected Apache Superset over AWS Quicksight as it provides us with much more flexibility, richer chart types and cost effectiveness. It also gives us the possibility of exporting charts via iFrame, or even REST API, in order to embed metrics charts within the web user interface in the upcoming upgrades.

Benefits

In order to become an industry leader, Futureswap needed more information on how to boost time-to-market and make client-oriented updates to their core solution. However, they didn’t have the essential insights about their product and the competition to identify these improvements.

Futureswap already had success partnering with Allata to develop their React frontend platform, so they returned to us to create this Analytic Dashboard Solution that enables them to make informed strategic decisions around their DeFI product. 

As an AWS Advanced Partner, Allata integrated AWS WAF best practices when designing the architecture: operational excellence, security, reliability, performance efficiency and cost optimization, plus other benefits like reusability. Thanks to the latest deployed subgraphs in TheGraph, plugging in a new blockchain is now a straightforward task.