Hello Web3/blockchain world, great job. You got people to take you seriously, trusting your projects and investing their money. You’ve sold people on your innovations, and people believe in your projects. Mission accomplished. But with great trust comes great responsibility. It’s time to learn valuable lessons from other areas that have gone before you, the most valuable is that security isn’t a task; it’s a process.
With this post, I hope to add some clarity, both for blockchain projects and security professionals who may be new to the space. This is a bit of a quick mental dump and far from being comprehensive, but I hope it’s the start of a conversation between both the blockchain and security communities.
Blockchains and Security
As an outsider looking at the current state of security with blockchains, it seems as though blockchain projects don’t take security seriously. Nothing could be further from the truth. Blockchain projects take security very seriously and understand the impacts of a compromise, and as such, having a security audit has become a blockchain rite of passage. So then, if that’s the case, then why are things the way they are? We’ll get to that in a second, but let’s take a quick detour and talk about security professionals for a moment.
Security Professional Baggage
When experienced security professionals discover the Web3 space, they bring a lot of baggage. They look at recent attacks and assume either the project didn’t have an audit or the auditor didn’t do a good job. This perspective makes an awful lot of assumptions that other processes and procedures were in place. We’ve learned a lot about application security over the past 20 years, but those lessons learned either aren’t applied or don’t directly map to the blockchain space. So, the project may very well have had an audit, but two days after the audit was completed, they pushed vulnerable code to their project. One-shot audits can’t solve that problem.
I also get the feeling from talking with security professionals that they know that blockchain ecosystems are different, but they think they have more in common than they do. So, they may understand that Ethereum, Solana, Algorand, etc., are different, but with minimal tweaking, your expertise on one will apply to the other. This isn’t true, and there’s quite a bit of hidden complexity, especially if you are developing projects on multiple chains or cross-chain projects. Different chains have different value propositions and ways of implementing that value, and it’s easy to make simple mistakes with catastrophic consequences.
Notice I used the term “projects” instead of “companies.” This is very purposeful. Blockchains have unique communities and projects. There’s a culture, much like security communities. They have their own language and views of the world. This can be a challenge for traditional security companies. I mean, try explaining to your accounting department that someone named HODLKing40 would like to pay for an audit.
Many of these projects may have an organization behind them for initial development and launching, but the projects are meant to be owned by the community. It may also be the case that these organizations are just three people. This is an entirely different perspective than what we are used to in the enterprise security space, but it’s essential to keep in mind as you work with the community.
How Did We Get Here?
If I summed up the current state of blockchain security, it would be projects operating with low security maturity. Their view of security is performing a single security audit before launch. Given that these projects are being developed in full public view and used as though they were finished products, this lack of maturity is on full public display.
There was also the early perspective of, “since it uses cryptography, it must be secure.” This view fueled some of the early lack of focus on security.
Hackathons and Side Hustles
Many projects are created during hackathons or as people’s side hustles. Some blockchain developers are new to development altogether and working on their very first project. It’s part of what’s exciting about the space, but these aren’t conditions ripe for security success. As a developer working for a traditional company, there are typically guardrails in place, and (hopefully) you’d be exposed to some structure, standards, and ongoing audit activities. With no previous experience, developers are left to fail in full public view.
More Than Just Blockchain Security
It gets more complicated because Web3 developers need to get both blockchain and traditional security right to succeed. This is because there are traditional applications mixed in as well. Think about a web front-end for an NFT marketplace or a wallet implemented as a browser extension.
Developers may also be writing complex financial products that are quite unlike anything they’ve developed before. There are many ways to mess things up and only one way to do it right. This environment creates an instant high-value target for attackers. Then again, you can also mess things up without an attacker in the loop as well. In the blockchain space, both can have similar outcomes.
We tend to forget that we are seeing technology experiments playing out in public. We think of them as finished products because the user base is high, and there is so much money at stake. This is similar to traditional startups that operate in stealth mode, blitzscaling features into their product. Traditional startups can also exercise a low level of security maturity, but because they are developed in private, with controlled releases, their lack of security maturity isn’t on full display. It also buys them time to fix issues when identified before they are disclosed publicly.
Impacts of Failure
The impacts of hacks in the blockchain space are also higher than many traditional applications. Traditional applications typically have a breadth of features and functionality. Breaches are undoubtedly bad, but most can recover, and there may be layered protections, and resolutions users can take because these traditional systems are centralized.
With blockchain systems, hacks can be irreversible. Blockchain applications and smart contracts are typically very focused on specific functionality, so a violation of that functionality means a complete compromise. Exploiting once basically exploits everywhere without needing to actually go everywhere.
New Ways Of Operating
The experimentation in the space isn’t constrained to the technology. Blockchain ecosystems are also experimenting with new ways to create and run organizations, leaving logistics and critical decisions up to their communities. In some cases, this means even exercising radical transparency. You may find that one of your statements of work ends up on Reddit with the user community voting on whether to go with your company or not.
Transparency is one of the great things about the blockchain space, but you can’t have both radical transparency and security. Sorry. This could only work in a world where nobody acts maliciously—for example, having all of your development and bug reporting open to the world regardless of severity. If someone points out a high severity bug directly on your public GitHub repo, it’s possible an attacker could exploit the issue before you’ve even written a fix. Given the stakes, this is a bad proposition.
What Needs To Happen
In a nutshell, we need greater maturity in the space, both from blockchain and security professionals.
Security professionals can’t pretend blockchains are irrelevant. I know fights with the NFT community are fun, but we’ll have to put that aside. Part of why we are where we are is because the security community has been relatively disengaged. Let’s not continue to be the “There is no cloud, just someone else’s computer” people. That mindset didn’t work out so well for us in the past.
I also get the feeling from some that they have the perspective that if they don’t participate in security conversations on the topic, they are somehow accelerating the demise of the technology. This isn’t the case either.
There are some common themes when an emerging technology comes along. Developers of the new technology don’t implement security lessons from other disciplines, but security professionals want to implement everything we’ve learned. We need to realize that we can’t re-use the exact same approaches we’ve used with traditional enterprises. I mean, there’s no risk mitigation to losing all of your money, and scanning tools won’t solve the most significant challenges.
Treat your initial plunge as an exploratory journey. Look at different security issues that have manifested themselves in the past, be they with smart contracts or core blockchains. These projects are mostly open, so you can look at their Github issues and patches. Review vulnerability write-ups and deconstructions of previous attacks. Projects affected by a compromise will typically post detailed write-ups. It’s a start.
Blockchain developers need to understand that what they are building is laced with landmines, and every line of code is a potential hazard. As of today, it’s impossible to write bug-free software. This thought should be on every developer’s mind from the very start. Blockchain developers need to take a greater security responsibility and not just hope that any security issues are caught during a final audit. An audit should absolutely be part of the security process, but not the only part.
An important consideration is that different ecosystem layers have different threats and concerns. For example, a core blockchain has different security considerations than a developer writing an application to run on top of a chain. A centralized exchange has different concerns than a group participating in a DAO. No quick blog post is going to solve all of these issues. Specifics will have to be outlined by the communities themselves, given the differences between ecosystems, but since this is a conversation starter, here are some of my thoughts.
Security From The Start
Security is a process, not a step, and needs to be considered from the start. One obvious place to start is with the security evaluation of the architecture of a system. An architecture that doesn’t consider security is hard to apply security measures to after the fact. Blockchain ecosystems can be complex, and it’s difficult, if not impossible, to update later.
Developers also need to evaluate threats during their development process. Call it threat modeling, threat assessment, or whatever, having developers think about what could go wrong is necessary for making sure things don’t go wrong. Developers should look at the highest impact areas in their code, such as ownership checks, transfers, minting, etc.
Threat modeling could start simply by using the core questions of the Threat Modeling Manifesto while performing development tasks.
- What are we working on?
- What could go wrong?
- What are we going to do about it?
- Did we do a good job?
The last question is something you have to get right. With the impact of failure so high, you can’t afford to find that out in production.
Tools will help, but they won’t solve all of the issues. This is one point traditional and Web3 applications have in common.
The bottom line is that you’ll need security expertise to get this off the ground. If you don’t have that expertise available, you can engage a partner or consider hiring someone to focus on these issues.
Ongoing Security Processes
Security isn’t something you finish. The entire design and development process should consider questions about risk and security. Make security and ongoing conversation. Recurring audits, either by a trusted partner, pair programming, community representative, etc., should be conducted.
Code additions, be they through dedicated developers or community contributions, should be evaluated for security scrutiny and focus on high-value functions keeping your threat model in mind.
And, of course, continue threat modeling. This should never stop.
Published Security Guidance
Projects and chains should publish clear security guidance for developers on their platforms. This guidance should outline things that are considered unsafe and warn developers of potential landmines. This guidance should be followed up with other awareness activities such as webcasts, workshops, etc. Security guidance should be updated as new attack vectors are discovered. This won’t stop developers from creating vulnerabilities but may reduce the obviously dangerous mistakes.
Process for reporting vulnerabilities
A clear process for reporting potential vulnerabilities should be published. Details of issues, especially for critical vulnerabilities, should not be public. Code fixes should also not be made public until they’ve been applied to the running code. The goal here is to reduce the window for exploitation to a size where, once an attacker finds out, they won’t have time to exploit.
A bug bounty program can also be part of this process to entice people to disclose bugs responsibly. Offering rewards upfront is better than begging attackers to give back what they stole.
I hope this post starts some conversations and explains a bit about how we got where we are. The recommendations made here are only a simple start. There is much more work to be done.
The Web3 space is a challenging place to apply security, something that should get security professionals excited. If we do this right, there may be lessons we can apply back to traditional application security as well.