How to Evaluate Subgraphs

In this post I am going to use the DODO DEX subgraph to demonstrate how I evaluate subgraphs.

Before I begin, there are a few things to keep in mind.

  • The subgraph may have been changed or updated by the subgraph developer by the time you read this article. That said, the principles remain the same.
  • I assume you understand what The Graph protocol is, what a subgraph is, and how it works.
  • I assume you know what Ethereum is and how to make sense of on-chain data on block explorers like Etherscan.

I created an easy-to-use template for how I rationalize my evaluation of a subgraph. You can duplicate it and modify it to suit your style.

If you create your own version from my template, I’d love to see it. I could improve mine from your implementation.

Curators signal to indexers what subgraphs to index, and they do that by staking the GRT token on a bonding curve. It is important you carefully evaluate a subgraph before staking on it.

Let’s begin.

The first thing I do is try to understand what DODO DEX protocol is and how it functions. A quick google search leads me to the project’s website. I then confirm if the website I’m on is the same as the one in the link from their twitter page. On their website, I see that they are a decentralized exchange where users can swap one token for another, add liquidity to pools and mines the native DODO tokens as a reward for providing liquidity.

When evaluating a subgraph, there are a few key criteria you need to pay attention to. You’ll find these definitions in my template.

  • Subgraph Function: How would you describe what the subgraph does?
  • Production Ready?: Does this subgraph look production ready?
  • Usefulness to Others: Does this subgraph look like it will be useful to others?
  • Improvements: What changes would you make to the schema including additions or modifications to entities, fields, field types, relationships, or any other improvements?
  • Similar Subgraphs & Comparison: Are there any other subgraphs that do similar things? How does this one compare?
  • Status: Identify the degree of completeness, complexity and accuracy of the implementation

I’ll explore how I evaluate each criterion.

Subgraph Function

To find out what the subgraph does, I go to the playground of the subgraph and query it. As you can see in the image below, when you hit the play button, it returns the indexed data.

From seeing the various tokens, including their addresses, symbols, etc., we can conclude that this subgraph provides data for the DODO DEX protocol. I record this information on the template.

I’ll be creating a post on how to use The Graph playground to query data. If you understand GraphQL then this will feel right at home.

The Graph Playground
The Graph playground for the DODO DEX subgraph

Next, I visit the GitHub repository to see who created the subgraph. You do that by clicking on the GitHub link. You can tell this was deployed by a third party because the GitHub repository isn’t from the protocol developers. There’s no information on the DODO website to suggest that they have a GitHub repository for their protocol.

Because Ethereum is permissionless, anyone can spin up a subgraph. A subgraph created by the Protocol team will definitely be more reliable than one created by any subgraph developer. So that’s one thing to keep in mind as you evaluate.

Next thing I check is the subgraph manifest. The subgraph manifest is a file that tells the graph node what events or functions to look for and listen to on Ethereum. You can find it in the GitHub repository. It has the .yaml file extension.

In the subgraph manifest I copy the contract address to my clipboard and search for it on Etherscan to make sure the data being served is coming from the correct DODO DEX protocol.

As you can see in the image above, the contract wasn’t deployed by the DODO DEX team. You’ll find the correct contract that powers the DODO DEX protocol in the image below. It is important that subgraphs serve data from the correct contract.

DODO Dex contract page on Etherscan

Similar Subgraphs & Comparison

A quick search in the graph explorer shows that multiple subgraphs have been deployed for the same project. Upon inspection, I find that they were all created by third parties. The DODO DEX subgraphs in the explorer contain very few entities and are not very robust. I record this in the template.

Dodo DEX subgraphs in the Graph Explorer

Usefulness to Others

No, because it serves incorrect data from the wrong contract.

Production Ready

No

Improvements

Here, I try to figure out how much more data could be served through this subgraph. This is where studying and understanding how the protocol functions come into play.

For DODO DEX, more entities could be added, like:

  • claimedMiningRewards which could show the total number of DODO tokens claimed by liquidity providers
  • tokenSwaps — which could show the traded tokens
  • poolSize — pool size by pairs

Status

I try to determine the degree of completeness, complexity, and accuracy of the implementation. I start by giving it a completion score.

DODO DEX gets a completion score of 10% because there is a lot more data that could be served as mentioned in Improvements above.

I also measure the complexity from the number of entities. You can find this number in the subgraph page. DODO has just 20 entities, which means that there are not many records to query. This will increase significantly if more entities are added to the Schema.

Lastly, I determine accuracy from the data it returns. I observed, upon querying the data in The Graph playground, that the pair count displays 9 token pairs instead of 13 token pairs.

The DODO DEX subgraph pulls data from the wrong contract which makes the data it serves questionable.

Lastly, I like to rate subgraphs I query. My rating determines whether I signal on that subgraph or not.

You’ll find my evaluation of DODO DEX in my template. Perhaps you could use that and this article as a guide when you evaluate subgraphs.

Thanks for reading.

If you have any questions you can email me directly.

Email: jerrycreative@live.co.uk

Primate