Blockchain technology does not only revolutionize the concept of trust, value, and ownership, but also brings about a sea change in data and analytics. For the first time in history, transaction and financial data, which are closely guarded by banks, card networks, and governments, become open and accessible to everyone.
But how can crypto developers and mobile engineers take advantage of this data? Ironically, while the blockchain removes the problem of data scarcity, it creates new challenges in data interpretation and analysis. In our conversations with 80+ web3 developers and mobile engineers, so many have told us how difficult it is to make sense of blockchain transactions, aggregate them, and draw useful conclusions. Luckily, there is a free and powerful tool to do just that: Dune Analytics.
In this post, we explain why it is important for crypto developers to analyze blockchain data, what is Dune Analytics, and how you can master Dune to improve your web3 apps in a 7-step process.
Why is blockchain data so important for crypto developers?
Blockchain data consists of the transaction, account, and metadata information sent by wallet addresses and processed by nodes and validators. For instance, every time an address transfers tokens or invokes a smart contract, the blockchain may record the sender, recipient, transaction amount, timestamp, balance changes, function calls, etc. In other words, blockchain data captures information on all the activities and participants on the network.
It is thus obvious why blockchain data is so essential for web3 developers, as it represents a huge trove of insights across the entire user, validator, and business experience. These insights, in turn, can form actionable hypotheses for nearly every team in an organization, from product and engineering to marketing, sales, and customer success.
Furthermore, blockchain data is objective. While user interviews and surveys can be extremely useful, they can only show the stated preferences of users, not their actual behaviors. As such, crypto developers can view blockchain data as almost a ground truth to uncover clear opportunities for user, product, and business growth.
What makes blockchain data analytics difficult?
Analyzing and understanding blockchain data, however, is notoriously difficult. In our view, there are three main challenges to blockchain data analytics: 1) blockchain data is optimized for processing, not querying; 2) the data structure is complex, incoherent, and fast-changing; and 3) best practices and standards are not mature yet.
We breakdown these there challenges below:
- Blockchain data is optimized for processing, not querying: When blockchain designers build their network architecture, they focus on the blockchain trilemma of decentralization, scalability, and security. Their incentives are to optimize these three dimensions as much as possible. Data querying is an afterthought. As a result, blockchain data is often not human-readable, encoded, and arcane. For example, to understand the smart contract logic in a transaction on Ethereum, a data analyst has to find the ABI of the contract, parse the function signatures, and piece together the function calls from events, logs, and metadata. If the contract owner omits events, the analyst can easily miss critical information and get stuck in their analysis.
- The blockchain data structure is complex, incoherent, and fast-changing: The coherence and accessibility of blockchain data is heavily dependent on the implementation of each smart contract. In some extreme cases, each smart contract requires a completely different data analytics procedure. Additionally, complex smart contracts may rely on outside databases such as IPFS to store critical metadata, resulting in a complicated set of data relations. For instance, to query Bored Ape Yacht Club apes owned by a given address and filter by one of its features, a data analyst needs to process all transfer events, read the metadata from IPFS with the Token ID and IPFS hash, and aggregate the results (source). Even then, once the contract is updated, the data workflow may simply break.
- Best practices and standards are not mature for blockchain data yet: It is clear to everyone that there is no mature data standard within a blockchain network, much less across different chains. As hinted earlier, an Ethereum data analyst might lack access to critical information in function calls, logging, and metadata. At the same time, a Solana data analyst can face the opposite problem, as the Solana network produces too much data and requires heavy storage, computation, and filtering capabilities.
Thankfully, Dune Analytics makes it much easier for crypto developers to overcome these challenges and make sense of crypto data.
Dune Analytics: SQL querying for blockchain data
Dune Analytics is a tool for blockchain research and analytics for crypto engineers, analysts, and investors. It enables users to query, extract, and visualize blockchain data. There are three key product innovations that Dune offers: 1) SQL structure, 2) community dashboards, and 3) data coverage.
The most important innovation that Dune brings about is its conversion of raw blockchain data into SQL data tables. Effectively, Dune Analytics imposes a well-defined, easy-to-query data structure on complex blockchain data. Because SQL is a very familiar framework for data analytics, Dune Analytics enables its users to run extraction, filtering, and join operations to connect, aggregate, and draw insights from disparate blockchain data sources. On top of that, Dune Analytics uses powerful machines to allow for fast execution of heavy computations. To be sure, it is not always perfect, as we do encounter missing data, undecoded information, or data errors in Dune’s table. But by and large, Dune’s data structure is extremely valuable in a maze of intertwined blockchain data.
The second data analytics improvement from Dune is its community dashboards (source). There are tens of thousands of user-generated dashboards on a wide range of topics, from DeFi and NFT to OpenSea and decentralized exchanges. This is not only a powerful source for users to learn and draw inspiration from, but also a place for crypto developers to showcase their apps. We believe that over time, these community dashboards will form the foundation for best practices in interpreting blockchain data.
Finally, Dune covers a wide range of blockchain networks. As of December 2022, the project contains data from 8 blockchains, including Ethereum, Polygon, Avalanche, among others. Dune has also started to allow community tables and user-generated tables, which enable more in-depth and advanced use cases and broaden the range of analysis that one can run.
In the next sections, we offer a 7-step guide for you to learn how to use Dune Analytics as a crypto developer.
Step 1: Master SQL
Learning SQL is an essential skill if you want to master Dune Analytics. Having a strong understanding of SQL will allow you to write more complex and powerful queries to support your analytics needs.
If you are a new to SQL, we have compiled below a list of free SQL resources to get started:
- Watch videos: YouTube videos are a great method for visual learners to familiarize themselves with SQL syntax and vocabulary. We highly recommend Kevin Stratvert’s SQL for Beginners Tutorial, StatisticalProgramming.Net’s Learn Basic SQL in 10 Minutes, and Web Dev Simplified’s Learn SQL in 60 Minutes.
- Online SQL tutorials: If you prefer interactive lessons with examples, exercises, and quizzes, check out SQLCourse.com and W3Schools.
- Online courses: We have seen many crypto developers benefit from courses taught by industry professionals, such as Khan Academy’s Intro to SQL and Udacity’s Intro to Relational Databases.
- Books: Written textbooks on SQL often provide very detailed explanations and examples and cover SQL programming much deeper than other resources. We recommend the SQL for Web Nerds by Philip Greenspun.
- Online communities: For specific questions and debugging issues, Stack Overflow, Stack Exchange, and Reddit are your best bets.
Step 2: Understand blockchain data models and architecture
Although Dune Analytics does a good job of decoding transaction information and organizing blockchain data into human-readable tables, it is important for web3 developers to have a very strong handle on blockchain data structure and data models. This involves two substeps: a) understanding specific blockchain networks’ data models, and b) knowing Dune’s data architecture.
One, you should develop in-depth knowledge of the data models of the blockchains that you are building on. As an example, Ethereum developers need to know Ethereum’s account types, smart contract logs and events, and concepts such as ERC-721, while Solana developers should be familiar with Solana’s programs, different types of transactions, and cross-program invocations. This knowledge will help you build a clear mental model when working with Dune’s data tables. For Ethereum data, we highly recommend the OurNetwork channel.
Two, you can start learning about Dune’s data architecture from its Documentation site. We recommend starting with reading through Dune’s five types of tables: Raw Tables, Decoded Tables, Spells, Community Tables, and User-Generated Tables. In most cases, developers can start building queries with the Raw Tables and Decoded Tables, then move to Spells and Community Tables for more advanced use cases.
Step 3: Know your blockchain explorers
This sounds like an obvious suggestion, but we cannot stress enough how crucial it is to develop a working intuition with blockchain explorers. This means that you should master at least one explorer for each blockchain ecosystem that you work with. If you are new to the crypto space, it is a good idea to use the most popular explorer for a given chain; e.g. Etherscan for Ethereum, or Solscan for Solana.
While building and running queries on Dune Analytics, you should regularly test and check your queries against your blockchain explorer. This includes both simple checks like address verification and more complex tests such as logging and tracing analysis. Over time, you will develop a better grasp of the quirks of different blockchain explorers by navigating through their UIs and experimenting with different features. For instance, to take advantage of all the features that Etherscan offers, you can watch this tutorial
Step 4: Start your first Dune Analytics queries
You are now ready to start your first Dune Analytics queries! The first thing to consider is to choose which queries to work on. Our recommendation is to start with a standard business question on a popular smart contract or token on the Ethereum blockchain. By doing so, you can rely on the project’s user community for data questions and debugging as well as draw inspiration from Dune’s user-generated dashboards. We have found that a web3 project’s Discord community tends to be very generous in helping unblock your Dune Analytics issues. For instance, if you get stuck at a query on SushiSwap, you can try posting your problem on SushiSwap’s Discord or searching for similar queries on Dune’s Browsing page.
We’d also suggest that your first queries should be focused and achievable. Tackling queries that are too broad or ambitious right from the start can easily become overwhelming and result in frustration. Good starting queries include daily transaction volume on Uniswap, the number of active traders on OpenSea, or the monthly lending volume on Lido.
Step 5: Build a dashboard
Once you finish building and running your queries, the next step is to build a dashboard. A Dune Analytics dashboard is a visual display of your analysis and is a great way to highlight your work to the Dune community. Dune also provides drag-and-drop tools and pre-built widgets to help you transform your data results into charts and infographics.
After you are done with your dashboard, you can share your work by publishing it to the user-generated dashboard section on Dune or post your learnings on the Dune Analytics Discord community.
Step 6: Learn from the experts
One of the best strategies to continually learn and grow your blockchain analytics skills is to connect with other developers and experts in the field.
The first action you can take is to join the Dune Analytics Discord server. This is a large and active community of web3 developers, crypto investors, and data analysts, who are always willing to provide new Dune users with advice and guidance. The Dune Analytics team also regularly engages with the community on Discord channels and provides FAQs, feature updates, and news announcements.
Step 7: Apply to your own project
At this point, it is important to step back and think through how you can leverage Dune Analytics to improve the user experience of your apps. In this section, we provide some suggestions on what you can do to create actionable insights from Dune:
- Understand your business needs: The starting point is to really understand what part of the user experience you are trying to move, or what business aspect you are trying to enhance. This in turn depends on target user base, strategic positioning, and product features. For instance, if you are an NFT marketplace, Dune Analytics can help you gauge your trading revenue and volume, number of buyers and sellers, popular NFT collections, and floor prices. If you are a lending protocol, you can use Dune to calculate lending and borrowing volumes, analyze lending pools’ health, and run risk analysis instead.
- Combine with product usage data: Some of the most successful use cases of Dune that we have seen is coupling product usage data (e.g. Google Analytics, Mixpanel, and Amplitude on the front end) with the blockchain analytics from Dune. Projects that combine both sources of data are able to visualize the full user journey, study the engagement and ROI of specific features, and improve retention.
- Run market research and competitive analysis: The great benefit of Dune Analytics in particular and blockchain data in general is how public the activities of smart contracts and users are. A quick search or a simple query on Dune Analytics can easily give you a clear understanding of the strengths and weaknesses, user behaviors, and product strategy of each player in a market. On a macro level, Dune queries can enable trend analysis, industry research, and price prediction.
- Build in public: As hinted earlier, Dune Analytics dashboards can be a useful marketing tool to introduce crypto users to your project. We have also seen projects sponsoring data analytics competitions to attract engineers, data analysts, and investors.
And you’ve got a powerful analytics tool to improve your web3 product
You now have Dune Analytics as a powerful tool that will help you keep track of your business performance, inform your product strategies, and serve your customers.
Be sure to re-evaluate your Dune queries every time you change your core features or smart contract logic. We recommend reviewing your queries once a quarter at least, though some crypto teams like to do it much more often. As you gain more experience in using Dune, you will find that you can brainstorm more business questions and discover new ways to empower your product, engineering, and customer support teams.
Dreamerly is a suite of easy-to-use, compliance-focused, customizable SDKs and APIs powering NFT features in iOS and Android apps. Our mission is to help mobile developers create amazing web3 experiences for their customers.