NoDW #12: Analyzing Reputation Networks to Improve DAO Governance (Part 1)
NoDW #12: Analyzing Reputation Networks to Improve DAO Governance (Part 1)
The Newsletter of Decentralized Work: Data-driven insights on the world of decentralized work
Curated by the talentDAO community. Complied by ItamarGo
In this edition:
Our feature article: Analyzing Reputation Networks to Improve DAO Governance (Part 1)
From the talentDAO community
Elsewhere in Web3
Like what you’re reading? Consider supporting our efforts at talentDAO by becoming a paid subscriber.
Analyzing Reputation Networks to Improve DAO Governance (Part 1)
By: Saulthorin
This article is part one of a two-part series I’m writing to better explain the bull case for reputation research. We are going to be looking at a real dataset from our very own members of the TalentDAO community. I really enjoyed analyzing this data and thinking through the results, and I hope you will too. Let’s get started.
Context
The DAO Activity Drop from talentDAO was an experimental NFT drop we did on Polygon several months ago. We were trying to explore ways to recognize ‘good’ behavior on-chain and treat reputation as a currency for retroactive public goods funding. The reputation model itself factored in things like POAP holdings, as well as voting, and creating Snapshot proposals. Its goal was to recognize the exceptional contributions of individuals within the TalentDAO community and help quantify their impact across the DAO ecosystem as a whole.
Users needed to attain a reputation score in the 66% percentile or above in order to qualify, taking into account the following:
Number of POAPs holdings
Number of proposals ‘created’ across all Snapshot spaces
Number of proposals ‘voted on’ across all Snapshot spaces
Overview
Maybe it’s just me, but I find these numbers really impressive: Within the talentDAO community, members participated in over 800 DAOs and voted on over 8000 proposals. This process consisted of more than 80,000 votes in total being cast! Wowza. Talk about crazy eights. New meme for us?
Jokes aside, the crazy eight trifecta spanned involvement across key DAO ecosystems including Optimism Collective, Uniswap, BanklessDAO, Safe, ENS, and many more. The data explored helps us better understand the voting behavior of our community and what they most care about.
One more caveat, in this article I will use a lot of terms and jargon rather carelessly. After a year of writing about DAO Science, this article assumes you’ve kept up with your readings and done your homework. You are the baddest DAO junkies in the galaxy after all:) If you need a recap on the nuances of on-chain vs off-chain data, I highly recommend reading our discussion on the differences between Verified Credentials and SBTs.
The bull case for research on reputation networks
There are several reasons we all need to start thinking about ways to improve the research being done on identity management and reputation systems. These include reasons such as:
Metagovernance: Sourcing quality delegates for your governance process is incredibly difficult. Procuring potential wallets to delegate to by pre-screening with reputation metrics allows for a more quality selection of potential candidates.
Voter participation: Voter participation is at best in the range of 3-5% across most DAOs. Identifying the most active wallets on-chain and verifying their talent is an incredibly valuable way to improve candidate sourcing. It’s also an easy way to create an allowlist for airdrops to retroactively reward and incentivize voter participation.
Impact Certificates: Identifying novel ways to measure and reward impact through the use of Verified Credentials and Reputation NFTs will drastically improve efforts to accelerate the development of valid methodology for impact certificates. The verified talent becomes a valuable data set and sample to learn from.
Measuring DAO Contribution: There have been dozens and dozens of contributor rewards platforms because of the rising popularity in soul-bound tokens (SBTs) and reputation NFTs. This is really where the TalentDAO expertise in Organizational Science will shine. If we are going to start creating measures of job performance, then you better get someone involved who knows a thing or two about psychometrics.
One could easily write an entire series of articles on these four use cases alone – and maybe we will. The focus for the discussion today will be on the “metagovernance” and “voter participation” use cases.
Which DAOs had the highest voting activity?
This is an important question to answer because to source quality DAO governance participants you want to be able to verify the DAOs they participate in. For example, if I was sourcing delegates for a DeFi protocol, I might want verified talent with experience in DeFi governance and protocol development. Likewise, if I want delegates for a software guild or venture DAO, then I likely would prefer verified talent with experience in those things. By examining Snapshot data filtered to only verified talent, we have a better understanding of the DAOs members of the talentDAO community participate in most.
The Snapshot space that received the most votes across all verified talent wallets was Optimism Collective. Placing it as #1 most active space, the Optimism Collective has received nearly 12,000 votes across all verified talent wallets to date. Other top Snapshot spaces included Aave, Startgate DAO, and Uniswap.
I would be remiss not to have some honorable mentions including Lido, Balancer, Gnosis, and ApeDAO. There was also, of course, a strong showing from the Bankless community which comes as no surprise. We love our bankless friends (bdao strong!).
Based on the existing Snapshot voter information we can determine our verified talent has a diverse set of skills and DAO governance experience. It would appear rather well-rounded and balanced. We have active voters participating in social DAOs like ApeDAO and BanklessDAO, but also DeFi Protocols like Balancer and Aave, as well as infrastructure projects like ENS and Gnosis/Safe. Using this data, one could easily source quality candidates and make a bull case for why verified talent is a preferred approach to delegation. Why trust unverified anons when you can have verified anons with a proven track record? My question exactly.
Which proposals received the most votes?
The most voted DAO proposal came from the ZkSync community concerning their decision to launch SyncSwap on mainnet. Over 1,200 TalentDAO community members voted on this proposal. Other highly voted proposals came from SuperRare and Stargate DAO.
I find this chart harder to interpret, but spot-checking through the data it seemed most of the verified talent voting behavior fell into four categories: a) decisions about budgets, b) decisions about protocol upgrades, c) decisions about token drops and rewards, and d) decisions about infrastructure adoption. Based on this assessment, these are the four areas of metagovernance support we could confidently demonstrate experience in having.
A more nuanced analysis could provide even deeper results to improve delegate sourcing. For example, if a DAO was looking specifically for users with experience navigating protocol upgrades for Aave or budget decisions for ZkSync, that talent could be pre-screened off-chain and verified on-chain. The use cases are endless.
Which DAOs do voters have the most influence on?
To me, this is probably the most interesting question so far. If I’m trying to influence the DAO ecosystem, then it’s really important for us to understand where our community likes to spend its resources and effort. Voting power is a metric calculated by Flipside Crypto’s data API. The amount of voting power one holds is proportional to the amount of the token one holds. Not surprisingly, the TalentDAO community has the most voting power at BanklessDAO! Verified talent holders also have a strong influence at the Optimism Collective, Syncswap, Stargate DAO, PancakeSwap, and ENS DAO. Other notable mentions not pictured below include Safe and Gitcoin DAO coming in at the 14th and 19th spot ranking, respectively.
Voting Network of Most Active Snapshot Spaces with total voter reputation at 75th percentile or greater
With this data, it becomes clear what ecosystems we have a major stake in. Leveraging verified talent we can now approach our partner communities and demonstrate our existing involvement. I’m excited to see where this will go and believe this article is only the start of an ongoing conversation.
What DAOs do the most reputable users vote at?
The title of the article does say analyzing reputation networks but we are only going to cover one in closing. The type of visualization below leverages network analysis and may be less familiar. The network formations you see are a result of a cluster analysis technique that helps identify what subgroups exist in a network structure. The structure can be defined as a hub and spoke. Each center node (hub) represents a reputation percentile score and the surrounding nodes (spoke) are Snapshot spaces. Each color represents a distinct cluster or subgroup of Snapshot spaces based on their voter’s aggregate reputation.
The data were filtered to include reputation scores that fall at or above the 75% percentile. More plainly stated, we are looking at the top 25% of the verified talent and where they spend their time voting. This data helps us understand where overlapping communities exist. It can also help us better understand what DAO characteristics and governance processes attract more reputable voters.
For example, with this data, we can see that Aave (light blue bottom graph) falls into the 88th percentile for voter reputation compared to Harvest Finance (top right yellow graph) which falls into the 99th percentile. Why is this? At this time I am unable to determine the defining characteristics of what attracts more reputable voters to DAOs. I believe this type of analysis will become the future of benchmarking and is essentially the web3 equivalent of employee engagement. What I mean by this is most DAOs will begin measuring voter participation as an outcome and using reputation metrics to benchmark against other DAOs.
Recap
In part one of this article we covered:
The bull case for reputation research
Snapshot voting behavior of verified talent
Voting networks of Snapshot spaces with higher reputation
In part two of this series we will cover:
Additional network visualizations to further explore reputation
Analyze NFT minting behavior
Explore data from the Web3 Commitment Reputation NFT
Answer the question: Does donor behavior drive voter participation or is it the other way around?
Conclusion
It is important to remember the reputation model presented here is overly simplistic and certainly is not an exhaustive list of things that could be included. The best models of reputation will be derived from scores based on peer networks, and validated rigorously by multiple replicable experiments. For a more nuanced overview of our perspective on reputation measurement see parts one and two of our series Quantifying Reputation by Flowscience. The difficulty is going to be figuring out how to manage off-chain data with on-chain verification, while still being privacy-preserving.
A special thanks to charliemarketplace.eth and the Flipside Crypto data science team. Without your help wrangling the datasets, things would have been significantly more difficult. The Flipside Crypto API makes it easy. You can find the code on Github to explore the data yourself. Feel free to email us at talentdao@gmail.com or hop in the Discord if you have questions about our analysis.
From the talentDAO community
Our governance experiments
Calling all DAO junkies, governance gurus, and research lovers to participate in a series of governance experiments being conducted by the talentDAO research team in partnership with Collab.Land and RaidGuild. Check out the link at the top of this section to find out more.
Protocol-first DAO Strategy / 0xJustice
Has a community-first approach to DAOs led us astray? 0xJustice argues that it has. In this article, they present a thoughtful thesis of the challenges of forming DAOs first around a community and offer a thought-provoking alternative - a protocol-first approach: define the core product as a platform and codify it as a protocol and then enable a community to form around it.
Welcome to hyperspace / k3nn.eth
talentDAO contributor k3nn.eth exploring the promising intersection of blockchains and AI and the impact it can have on the future of DAOs.
Elsewhere in Web3
The Future of DAOs is Powered by AI / Samantha Marin
Building up on the previous post, this article offers a more in-depth exploration of the intersection of AI and DAOs, centering around 6 primary use cases: 1) AI bots and assistants: boosting productivity and improving products/services 2) AI at the edge of the DAO: acting as token holders 3)AI in the center of the DAO: interacting with the core contracts 4)AI connectors within or between DAOs: forming a kind of “swarm intelligence” 5) DAOs govern AI as a public good: increasing AI safety 6) AI becomes the DAO: the AI owns a treasury on-chain.
(side note: curious to learn what chatGPT had to say about DAOs? the team at BanklessDAO got you covered)
Zero to Governance / w1nt3r.eth
An easy-to-read parable that illustrates what governance is, and how it can get more complex with scale. A good jumping-off points for the many of us who are just getting our heads around this topic.
Why DAO Contribution Sucks (And How to Fix It) / Finn Lobsien
This ranty title hides a deep and comprehensive overview of the science of organizing and collaboration building on Coase’s 1937 “Theory of the firm”.
It calls attention to some of the core challenges that exist in DAO space today, and offers the Minimum Viable Contribution (MVC) as a concept through which we can start exploring some compelling solutions.
Recontextualizing DAOs: The Case for Developer DAO / Luan
Effective governance is a challenge many DAOs are tackling. IRL not everyone wants to become a politician or a public servant and engage with governance day in and day out. This challenge exists in DAOs too. This article covers the challenges that DeveloperDAO has experienced with its current guild structure, and how its approaching changing it to overcome them.
Decentralized Autonomous Organization Toolkit / Gogel et at
Yes. DAOs made it all the way to the World Economic Forum! This recently published and relatively short (36 pages) report, offers a good primer on what DAOs are, how they operate, governance, and legal structure. It makes recommendations on how to address some of the key challenges around those topics. Even readers more well-versed in DAOs may still find some of the comparison tables and diagnostic tools in the appendices quite illuminating.
Like what you’re reading? Consider supporting our efforts at talentDAO by becoming a paid subscriber.