Skip to content

The CCS Interview

ChainLight CTO on zkSync bug that only a malicious prover could exploit

Andrew Wesie, co-founder and CTO of security firm ChainLight, discusses why smart contract and zero-knowledge security keeps getting harder, including a circuit bug his team found in zkSync Era. The conversation was recorded in February 2024, and everything here reflects what he said at the time.

Interview by Ashton Addison Recorded February 2024 5 min read

Key takeaways

  • ChainLight was founded in 2016 as a traditional cybersecurity firm before pivoting to Web3 and blockchain security, recruiting talent from the CTF competitive hacking community.
  • Smart contract complexity has increased attack surface significantly since Bitcoin's simplicity, with new vulnerabilities like price oracle manipulation and rounding errors emerging over time.
  • Public smart contract code alone does not ensure security; expert audits are necessary since non-technical users cannot assess code safety and attackers exploit undiscovered flaws.
  • Blockchain security threats extend beyond smart contract code to include server compromises and key theft, which audits cannot prevent but remain part of overall attack surface.
  • Exploit dollar amounts are growing partly due to increased total cryptocurrency market value rather than necessarily indicating decreased security effectiveness year-over-year.

Why security gets harder as crypto grows

Ashton Addison: Is staying ahead of attackers an uphill battle?▶ 02:41

Andrew Wesie: There are a couple of reasons it is. The technology itself has become more complex. Bitcoin was a very simple protocol, and there wasn't much an attacker could do with it. Once we got to Ethereum smart contracts, there were suddenly many more ways to steal funds, because people could write complex systems that run in a decentralized way and that anyone can access. These systems have to be bulletproof, because any small chink in their armor an attacker is going to take advantage of. Now we're seeing more zero-knowledge systems, which are even more complex than Ethereum was. We get better each year at protecting against known attacks, but each year new attacks come out and we need to learn how to protect against those too.

Ashton: Smart contract code is public, but you only find out how secure it is when it gets attacked. How can users judge that?▶ 03:58

Andrew: That's one of the real challenges for users of these contracts. Experts like us still have a hard time figuring out whether code is secure or not. It's almost impossible for non-technical users to do that assessment themselves. Yes, the code is public, but that only helps if you have experts looking at it to find the problems.

Ashton: Is an audit mostly a checklist of known issues?▶ 05:41

Andrew: There are several known issues in Ethereum and Solidity smart contracts, but new attacks keep appearing. Price oracle manipulation wasn't even on the radar several years ago. Then someone exploited a contract that way, and now we have to look for it in every contract with an external oracle. Rounding attacks are similar. Computers can't always represent a value precisely, so when you do a division there's a rounding error. That's something else we now have to account for when we go through the code.

Ashton: Are exploits declining as a share of the market, or growing?▶ 06:59

Andrew: We are seeing the numbers grow, but part of that is because the amount of money in crypto is growing too, so attacks are going to get larger. It's not necessarily good, but it's because the market is larger. One of our customers is Blast, and we recently audited them. They were very proactive. They had $1.3 billion of locked value and wanted to make sure it stayed secure, because that's a very juicy target. We can't protect everything, though. We protect customers by looking at their code, but that isn't the entire attack surface. We've seen others hacked not through the smart contract code but because someone got into their servers and got their keys. We can't really protect against that, but it's still part of blockchain security.

Experts like us still have a hard time figuring out whether code is secure or not.

The zkSync bug and the audit gap in ZK

Ashton: ChainLight helped with a vulnerability in zkSync. What happened?▶ 09:14

Andrew: Like many ZK rollups, zkSync uses zero-knowledge circuits, which you can think of as code used to prove that transactions follow the rules of the network. We found a way that the circuit had a bug, so a malicious prover could change the behavior of a transaction. Right now, with all of these ZK rollups, there is generally a centralized prover. Only one party is allowed to put the proofs onto the main chain, Ethereum. So although we found a bug in the circuits, only a malicious prover could exploit it, and at the moment that control is centralized in Matter Labs. Their goal, and the goal of every other L2 rollup, is to decentralize that role so anyone can push out proofs. That's where this bug would have become very dangerous. If anyone can push a proof and there's a bug in the circuit, anyone could manipulate the system and potentially steal all the funds.

Ashton: How much money was at stake?▶ 11:23

Andrew: Roughly $2 billion has been flowing through zkSync Era, before and after we found that bug. It's fixed now, thankfully. zkSync Era has also been very proactive about getting audits and finding more issues. We took part in one of their recent audits and found a couple of similar issues in version two of zkSync Era. They're making sure these vulnerabilities are found and fixed.

Ashton: How does adding ZK technology affect a blockchain's security?▶ 12:38

Andrew: Generally speaking, it doesn't greatly affect security. What it affects is scalability and privacy. You can use zero knowledge to scale more, because you need to send less data to every network participant, or to increase privacy, because you can hide information. Security comes into play because right now fewer people are able to audit ZK circuits and the code that interacts with them. Fewer experts means audits are a lot more expensive for projects, which decreases the chances they actually get done. Every time you push new code or add a feature you want to audit it, and that becomes very difficult if nobody has the expertise.

Ashton: So that's the downside of being first with new technology?▶ 14:24

Andrew: Exactly, and there's another challenge. The zero-knowledge systems are quite distinct. Three or four different ZK rollups are deployed on mainnet, and each has a different code base. They're all based on zero knowledge, but an expert who wants to check that one works correctly has to learn a completely new code base for each. We're talking about hundreds of thousands of lines of code, so it's a substantial time investment.

Attackers, privacy tools and ChainLight's 2024 plans

Ashton: Do privacy-focused ZK chains give hackers an incentive to target them?▶ 16:39

Andrew: Attackers use privacy technologies to hide their tracks whether or not the network is private. On Ethereum, Tornado Cash is used in many attacks because it hides the source of the original funds and lets the attacker get them back out without associating them with the hack. I think you'll see attackers use more private networks as they come out, potentially as a way to mix funds, but not necessarily targeting those networks. Attackers are looking for the juiciest target: where is the most money they can easily get. Then they'll use a bridge or a swapping website, whatever they need, to get it to Tornado Cash or another privacy app.

Ashton: What is ChainLight focused on for 2024?▶ 18:00

Andrew: There are two main ways we're positioning the company. First, a lot of projects and developers are going to need audits, and we're limited by how many people can look at code line by line and write a report. So we're looking at ways to automate parts of that, with static analysis tools and maybe some report generation using generative AI, to make our auditors' time more effective. Second is zero knowledge. It's such a new technology that we're building up our own skills and finding customers who need these audits done. We started last year with zkSync Era, and we're hoping to expand to other ZK rollups as well.

Attackers are looking for the juiciest target: where is the most money they can easily get.

This interview has been edited for length and clarity. Recorded February 2024.

Andrew Wesie
Andrew WesieAndrew Wesie is the co-founder and Chief Technology Officer of Theori, an offensive security company based in Austin, Texas. Theori's Web3 security unit is ChainLight, and its AI-driven application security platform is Xint. Wesie began in competitive hacking at Carnegie Mellon University, where he co-founded… Full profile →
Share this interviewXLinkedIn
▶ Watching the interview