Blog

T'was the day before genesis, when all was prepared,
geth was in sync, my beacon node paired.
Firewalls configured, VLANs galore,
hours of preparation meant nothing ignored.

Then all at once everything went awry,
the SSD in my system decided to die.
My configs were gone, chain data was history,
nothing to do but trust in next day delivery.

I found myself designing backups and redundancies.
Complicated systems consumed my fantasies.
Thinking further I came to realise:
worrying about these kinds of failures was quite unwise.

Events

The beacon chain has several mechanisms to incentivise validator behaviour, all of which are dependant on the current status of the network, so it is vital to consider these failure cases in the greater context of how other validators might fail when deciding what are, and what aren’t, worthwhile ways of securing your node(s).

As an active validator, your balance either increases or decreases, it never goes sideways*. Therefore a pretty reasonable way of maximising your profits, is to minimise your downsides. There are 3 ways your balance can be reduced by the beacon chain:

• Penalties are issued when your validator misses one of their duties (e.g. because they are offline)
• Inactivity Leaks are handed out to validators that miss their duties while the network is failing to finalise (i.e. when your validator being offline is highly correlated with other validators being offline)
• Slashings are given to validators who produce blocks or attestations that are contradictory and therefore could be used in an attack

* On average, a validator’s balance may stay the same, but for any given duty, they are either rewarded or punished.

Correlation

The effect of a single validator being offline or performing slashable behaviour is small in terms of the overall health of the beacon chain. It is therefore not punished heavily. In contrast, if many validators are offline, the balance of offline validators can decrease much more rapidly.

Similarly, if many validators perform slashable actions at the same time, from the beacon chain’s perspective, this is indistinguishable from an attack. It is therefore treated as such, and 100% of the offending validators’ stake is burned.

Because of these “anti-correlation” incentives, validators should worry more about failures that might affect others at the same time rather than isolated, individual issues.

Causes and their probability.

So let’s think through some failure cases and examine them through the lens of how many others would be affected at the same time, and how badly your validators would be punished.

I disagree with @econoar here that these are worst case issues. These are more moderate level issues. Home UPS and Dual WAN address failures aren’t correlated with other users and so should be far down your list of concerns.

🌍 Internet/power failure

If you are validating from home, then it’s highly likely you’ll encounter one of these failures at some point in the future. Residential internet and power connections do not have guaranteed uptime. However, when the internet does go down, or your power is out, the outage is usually limited to your area and even then only for a few hours.

Unless you have very spotty internet/power, it might not be worthwhile paying for fall-over connections. You’ll receive a few hours of penalties, but as the rest of the network is running normally, your penalties will be roughly equal to what your rewards would have been over the same period. In other words, a k hour-long failure sets your validator’s balance back to roughly where it was k hours before the failure, and in k additional hours your validator’s balance will be back to its pre-failure amount.

[Validator #12661 regaining ETH as quickly as it was lost – Beaconcha.in

🛠 Hardware failure

Like internet failure, hardware failure strikes randomly, and when it does, your node might be down for a few days. It is valuable to consider the expected rewards over the lifetime of the validator versus the cost of redundant hardware. Is the expected value of the failure (the offline penalties times the chance of it happening) greater than the cost of the redundant hardware?

Personally, the chance of failure is low enough and the cost of fully redundant hardware high enough, that it almost certainly isn’t worth it. But then again, I am not a whale 🐳 ; as with any failure scenario, you need to evaluate how this applies to your particular situation.

☁️ Cloud services failure

Maybe, to avoid the risks of hardware or internet failure altogether, you decide to go with a cloud provider. With a cloud provider, you have introduced the risk of correlated failures. The question that matters is, how many other validators are using the same cloud provider as you?

A week before genesis, Amazon AWS had a prolonged outage which affected a large portion of the web. If something similar were to happen now, enough validators would go offline at the same time that the inactivity penalties would kick in.

Even worse, if a cloud provider were to duplicate the VM running your node and accidentally leave the old and the new node running at the same time, you could be slashed (the penalties incurred would be especially bad if this accidental duplication affected many other nodes too).

If you are insistent on relying on a cloud provider, consider switching to a smaller provider. It may end up saving you a lot of ETH.

🥩 Staking Services

There are several staking services on mainnet today with varying degrees of decentralisation, but they all contain an increased risk of correlated failures if you trust them with your ETH. These services are necessary components of the eth2 ecosystem, especially for those with less than 32 ETH or without the technical know-how to stake, but they are architected by humans and therefore imperfect.

If staking pools eventually grow to be as large as eth1 mining pools, then it is conceivable that a bug could cause mass slashings or inactivity penalties for their members.

🔗 Infura Failure

Last month Infura went down for 6 hours causing outages across the Ethereum ecosystem; it is easy to see how this is likely to result in correlated failures for eth2 validators.

In addition, 3rd party eth1 API providers necessarily rate-limit calls to their service: In the past this has caused validators to be unable to produce valid blocks (on the Medalla testnet).

The best solution is to run your own eth1 node: you won’t encounter rate-limiting, it will reduce the likelihood of your failures being correlated, and it will improve the decentralisation of the network as a whole.

Eth2 clients have also started adding the possibility of specifying multiple eth1 nodes. This makes it easy to switch to a backup endpoint, in the event your primary endpoint fails (Lighthouse: –eth1-endpoints, Prysm: PR#8062, Nimbus & Teku will likely add support somewhere in the future).

I highly recommend adding backup API options as cheap/free insurance (EthereumNodes.com shows the free and paid API endpoints and their current status). This is useful whether you are running your own eth1 node or not.

🦏 Failure of a particular eth2 client

Despite all the code review, audits, and rockstar work, all of the eth2 clients have bugs hiding somewhere. Most of them are minor and will be caught before they present a major problem in production, but there is always the chance that the client you choose will go offline or cause you to be slashed. If this were to happen, you would not want to be running a client with > 1/3 of the nodes on the network.

You must strike a tradeoff between what you deem to be the best client vs how popular that client is. Consider reading through the documentation of another client so that if something happens to your node, you know what to expect in terms of installing and configuring a different client.

If you have lots of ETH at stake, it is probably worth running multiple clients each with some of your ETH to avoid putting all your eggs in one basket. Otherwise, Vouch is an interesting offering for multi-node staking infrastructure, and Secret Shared Validators are seeing rapid development.

🦢 Black swans

There are of course many unlikely, unpredictable, yet dangerous scenarios that will always present a risk. Scenarios that lie outside the obvious decisions about your staking set-up. Examples such as Spectre and Meltdown at the hardware level, or kernel bugs such as BleedingTooth hint at some of the hazards that exist across the entire hardware stack. By definition, it is not possible to entirely predict and avoid these problems, instead you generally must react after the fact.

What to worry about

Ultimately this comes down to calculating the expected value E(X) of a given failure: how likely an event is to happen, and what the penalties would be if it did. It is vital to consider these failures in the context of the rest of the eth2 network since the correlation greatly affects the penalties at hand. Comparing the expected cost of a failure to the cost of mitigating it will give you the rational answer as to whether it is worth getting in front of.

No one knows all the ways a node can fail, nor how likely each failure is, but by making individual estimates of the chances of each failure type and mitigating the biggest risks, the “wisdom of the crowd” will prevail and on average the network as a whole will make a good estimate. Furthermore, because of the different risks each validator faces, and the differing estimates of those risks, the failures you did not account for will be caught by others and therefore the degree of correlation will be reduced. Yay decentralisation!

📕 DON’T PANIC

Finally, if something does happen to your node, don’t panic! Even during inactivity leaks, penalties are small on short time scales. Take a few moments to think through what happened and why. Then make a plan of action to fix the problem. Then take a deep breath before you proceed. An extra 5 minutes of penalties is preferable to being slashed because you did something ill-advised in a rush.

Most of all: 🚨 Do not run 2 nodes with the same validator keys! 🚨

Thanks Danny Ryan, Joseph Schweitzer, and Sacha Yves Saint-Leger for review

[Slashings because validators ran >1 node – Beaconcha.in]

Friends,

Leaves have fallen for those in the global north, Summer is on the way in the south, and ETH is staked across the globe. The jolliest of holidays has arrived, the birth of the beacon chain. Moreover, incredible progress has been made in recent months by all EF supported teams. But without the ability to interact in person recently, we might’ve missed quite a bit from one another. So as this unique year draws to a close, we’re excited to bring a whole host of detailed updates to you from some of the teams that help to move Ethereum forward in so many ways. It’s a long read, but there is a lot of progress to work through (or feel free to navigate around using the table of contents).

As always, this series of updates focuses on EF-supported teams whose members are working to grow and improve Ethereum as a whole. Included in this edition are updates from many teams highlighted in the previous report, and other new and rotating groups.

Enjoy!

Applied ZKP

Authored by Thore Hildebrandt

The Applied ZKP team works to bridge the gap between cutting-edge research in zero-knowledge proofs, and application development on Ethereum.

Perpetual Powers of Tau

In September 2019, we launched the Perpetual Powers of Tau ceremony (PPOT). PPOT aims to benefit the zero-knowledge ecosystem, particularly zk-SNARK projects built on Ethereum, by partially easing the burden of trusted setup ceremonies. Many zk-SNARK projects require two phases of parameter generation, and PPOT replaces the first phase, which can be shared by all circuits. Individual teams can choose any contribution from the ceremony to branch out and perform their own phase 2 setup.

This ceremony supports circuits up to 2 ^ 28 constraints, which means that each contribution requires a 97G download, a 1-day computation, and a 49G upload. At the time of writing, we collected 63 contributions and all contribution files can be downloaded and independently verified against a public ceremony transcript.

Projects that are planning to use or have used the ceremony include tornado.cash, Semaphore, Hermez, MACI and zkopru. The easiest way to contribute is to reach out to Wei Jie via Telegram @weijiek. Listen to this podcast to hear Wei Jie speak about the ceremony.

Semaphore

Semaphore is a generic privacy gadget which enables use cases such as mixers, anonymous login, and anonymous voting as well as reputation systems.

Semaphore went through an audit and is currently conducting a phase 2 ceremony based on Perpetual Powers of Tau. There are 51 participants so far, please participate to help make Semaphore secure. Join the conversation on the Semaphore Society Telegram chat group.

MACI

Originally proposed by Vitalik Buterin, systems built with MACI make collusion among participants difficult, while retaining the censorship resistance and correct-execution benefits of smart contracts. Although MACI can provide collusion resistance only if the coordinator is honest, a dishonest coordinator can neither censor nor tamper with its execution. See Wei Jie explaining how MACI works on Youtube. You can use the MACI command-line interface to run a demo.

Recently, clr.fund conducted a quadratic funding round in which results were computed using MACI. Bribery could become a major problem for Gitcoin when it reaches scale but MACI can help as it makes it impossible to prove who one voted for. MACI 1.0 is in progress, join the Telegram group to learn more and discuss.

MiMC Bounty

There is an ongoing bounty for collision finding in the MiMC hash function.

Hubble

Optimistic Rollups (OR) allows greater layer 2 scalability with the use of on-chain data availability and fraud proofs. Hubble allows for the creation of optimistic rollup chains with the same interface so that people can enter the rollup space once and then move between chains instantly at negligible costs and remove the need to ever “exit” the low cost rollup world.

Key features include mass migrations and a global account registry. Burn auctions will be used to decentralise the coordinator and to distribute MEV to CLR’s. Transfers to new accounts are possible directly from L2 without having to deposit on L1. With the help of BLS signatures the team was able to achieve 2500 tps on ropsten. The hubble BLS wallet aims to support other OR’s such as Arbitrum, Optimism and Fuel.

Hubble’s code is available on Github. Contracts are frozen and the team is preparing for an upcoming audit.

zkopru

zkopru (zk-optimistic-rollup) is a layer-2 scaling solution for private transactions using zk-SNARK and optimistic rollup. It supports private transfer and private atomic swap within the layer-2 network between ETH, ERC20, ERC721 at a low cost. It also provides instant withdrawal with pay-in-advance features and compliance compatibility using spending key and viewing keys. Wanseob presented the system at zk-summit, the recording will be available on Youtube soon.
zkopru is entering the “Burrito” stage of its roadmap, you can try it out on testnet now. Also, the UI for its trusted setup and burn auction for the decentralized coordination are in pipe. An audit is scheduled to start in January.

Experimental directions

• Blind Find – a p2p network allowing users to search for others without revealing their identity. After a successful search, the user can prove the search path exists in the network with a MPC-based construction, without revealing the path itself. To learn more and discuss, please join the telegram group.
• UniRep – a private and non repudiable reputation system. Users can receive positive and negative reputation from attesters, and voluntarily prove that they have at least a certain amount of reputation without revealing the exact amount. Moreover, users cannot refuse to receive reputation from an attester. Join the telegram channel to learn more and discuss!
• PeekABook – a private order matching system on Ethereum. It allows users to advertise and search for trading orders without leaking the order price. Check out the demo. Join the telegram group for discussion.

Ecosystem Support Program

Authored by ESP Team

Delivering support

The ESP team is always exploring new ways to help and encourage the many amazing builders in the Ethereum ecosystem. We highlighted a few teams that received support other than traditional grants in this post.

Meanwhile, almost $6.3 million in grants were awarded in Q2-Q3; learn more about the 50+ recipients in our Q2 and Q3 allocation updates, and keep an eye out for a Q4 post in the new year!

Outreach and communication

We’ve made an effort to provide more resources for teams looking for support, as well as Ethereum community members who want to keep up with what we’re supporting.

• Events: we’ve been making the (virtual) rounds at meetups and hackathons to connect in real time with builders, spread the word about ESP and help answer any questions or concerns.
• Guide to ESP: this new page of our website goes into detail about ESP inquiry and grant proposal process
• More blogging: We’ve posted regular updates on newly awarded grants throughout the year, but what these teams do with their grant funding is the real story! In November, we published the first post in a new blog series meant to recognize grantees’ ongoing accomplishments.

Eth2 Research

Authored by Danny Ryan

December 1st marked the launch of the eth2 beacon chain. This bootstrapped Ethereum’s proof of stake consensus that will ultimately serve as Ethereum’s new consensus home.

Looking past the launch of the beacon chain, the eth2 research team is concentrating on the next set of upgrades to eth2 — native light client support, the merge of eth1 into eth2, and scalable sharded data.

For the latest from our team, keep posted to the ‘eth2 quick updates’ on the EF blog. A few recent editions can be seen here: #21, #20, #19.

ethereum.org

Authored by Ryan Cordell

The ethereum.org team has spent the last half of 2020 adding yet more content and features to our ever-growing portal of Ethereum goodness.

Onboarding

As we’re quite often the first place users find when searching “Ethereum” for the first time, we’ve spent a lot of time enriching the content that helps you get started.

Eth2

And in the build up to the launch of the deposit contract, Eth2 has filled the headlines and our backlog. It’s clearly been front of mind for our users, as in the last month, visits to our Eth2 pages have surpassed even the homepage.

Developers

Finally our other main initiative in the latter part of 2020 was improving our developer content,.to help users get started with information from a single, trusted source. It’s still a work-in-progress, but so far we’ve built: ]

And after some user testing we’ve got plenty of ideas on how to improve these as we head into the new year.

Translations

It was also a massive year for expanding our reach into other languages. In the last half of 2020 our community of translators has updated content in:

You can find all 33 of the languages we now support at ethereum.org/en/languages and learn how to get involved with translation efforts.

We can’t go into everything, so take a look at ethereum.org and our previous updates to see the other bits we’ve worked on since you last visited.

As always if you want to contribute in any way, stop by our Discord or raise an issue/PR in GitHub.

Ewasm

Written by Alex Beregszaszi, Paweł Bylica, and Sina Mahmoodi

As proposed in the last update, we have continued to work on Eth1, Eth1.x, and Eth2.0 related topics.

Eth1

EVM384

One key motivator for our work on assessing performance bottlenecks of various virtual machine designs (EVM and WebAssembly) and their various implementations is to improve the speed of execution within Ethereum. As reported in a previous update we had success scaling WebAssembly for executing the elliptic curve pairing operation efficiently.

In the past six months we have embarked on the EVM384 project with the goal to reproduce the same scaling on EVM. While it started out as a small proof of concept, we have released a comprehensive explanation and several updates (2, 3, and 4) along the way. In these updates we present:

• an overview of the precompile problem,
• three new EVM opcodes facilitating efficient 384-bit calculations,
• multiple design choices for EVM384,
• a comparison of the languages Yul and Huff,
• and an implementation of the pairing operation over the BLS12-381 curve using the proposed opcodes.

Updates about EVM384 can be followed on the appropriate EthMagicians topic.

EVM subroutines

The team performed analysis and proposed significant changes to EIP-2315 “Simple Subroutines for the EVM”. The resulting discussions highlighted potential risks and opportunities, and led us to conduct research on JUMPDEST analysis techniques. The outcome of this is the baseline interpreter as discussed next.

evmone

Based on our benchmarks, evmone is a really fast EVM interpreter. One of the strengths of evmone is the extensive bytecode analysis it performs, allowing for pre-computation of gas cost and stack requirement checks. However, this can also be a downside or vulnerability.

A new variant of evmone called the “baseline interpreter” has been prototyped in the project. The goal is to evaluate a much simpler interpreter design in order to avoid analysis vulnerabilities and perhaps question the speed benefits of analysis. While this implementation is not yet battle tested, it does pass all of evmone’s unit tests, the Ethereum State Tests, and Solidity’s test suite.

Unexpectedly, this naïve design performs surprisingly well. On “average” contracts the speed difference between the baseline vs. the “advanced” evmone is negligible, however on computation-heavy EVM bytecode the “advanced” evmone would perform up to 40% faster.

The TurboGeth team has bootstrapped the Silkworm project which uses evmone as the EVM implementation. They also proposed a number of interesting changes to evmone and EVMC.

EVMC

Three new versions of EVMC were released: 7.2.0, 7.3.0 and 7.4.0. The releases brought various improvements for the C++, Go, and Java language support, as well as for tooling.

The Java support has received a lot of attention recently as part of its integration into Apache Tuweni. See the list of related changes and discussions.

EVMC support was adopted by a number of projects lately, including TurboGeth, Silkworm, and Second State’s SSVM.

Eth1.x / Stateless Ethereum

Under the Stateless Ethereum umbrella we have been involved with multiple efforts.

Code merkleization

After the initial feasibility experiments on code merkleization, which was mentioned in the last update, we proposed EIP-2926 and implemented the specification in geth and a standalone tool in Python. There have been further experiments, such as using SSZ for merkleization and performance overhead analysis, which can be seen in the discussion thread.

Witness format

The team, with Paul’s lead, has participated in specifying and optimizing the witness format. The format has been implemented in Python as well as Javascript. The Python implementation has been additionally used for generating tests for the format.

Binarification

We have furthermore closely followed the binarification effort and contributed to the design discussion. Our goal is to ensure the design works well with other components of a stateless Ethereum. With this in mind we have aimed to keep the code merkleization proposal aligned with the binarification effort.

ReGenesis

We had previously argued for a state cache to reduce (stateless) witness sizes in Eth2.0 Phase 2. Hence we find ReGenesis to be a promising direction. To help with the discussion, Paul published an early work-in-progress version of a formal specification for a potential variant of ReGenesis. The specification follows the notation introduced by the Yellow Paper.

Light-client sync

Light clients are bottlenecked by the amount of data they “pull” from altruistic servers. We believe techniques first developed for Stateless Ethereum combined with a commitment to chain history can reduce pressure on the servers. As a first step towards this goal we have surveyed approaches that reduce bandwidth requirements for syncing the header chain. Commitment to the chain history might further prove useful in contexts (such as ReGenesis) outside of light client sync.

Eth2.0

Deposit contract

Our team led the Solidity rewrite of the deposit contract, which underwent successful audit and verification, and was adopted as a part of the Eth2.0 specification.

The verification report mentions a benefit of the rewrite:

Despite the additional runtime checks, the Solidity implementation is more gas-efficient (30~40% less gas cost) than the original Vyper implementation, thanks to the advanced code optimization of the Solidity compiler.

Eth1x64

In the previous update we described our work on Eth2 Phase 2 and introduced Eth1x64 as a concept.

The first variant of Eth1x64, named Apostille, specifies a minimally-invasive extension to Eth1 and EVM. This extension allows contracts on the Eth1 chain to communicate with other execution shards on Eth2.

The experiment describes a situation where all 64 shards of Eth2 are homogenous and running the EVM, and allows these shards to communicate with each other. Furthermore the techniques presented can be (re)used in the context of the Eth1-Eth2 merge.

Fizzy

Fizzy aims to be a fast, deterministic, and pedantic WebAssembly interpreter written in C++.

The four main goals of Fizzy are:

1. high code quality and simplicity
2. strict specification conformance
3. efficiency
4. first class support for deterministic applications

Following April significant progress has been made and three out of the four major goals have been already met.

Compared to other interpreters, we have deliberately restricted the scope to WebAssembly 1.0, avoided implementing any pending proposals, and resisted the urge to create an all-encompassing library. Fizzy only supports the canonical binary representation (.wasm) of WebAssembly and does not support the text formats (.wat/.wast). This allowed us to keep the code simple and straightforward.

The pedantic adherence to the specification, combined with exhaustive use of static analyzers and fuzz-testing, allowed us to find missing test coverage in the specification, and upstream issues in wabt and wasm3.

While execution speed was not the first goal, we have been conscious about design decisions affecting it. This allowed Fizzy to become the second fastest interpreter on the market, beating our previous favorite wabt.

Fizzy in numbers:

• 5 major releases
• ~2300 lines of well commented modern C++17 code
• additional ~7300 lines of unit tests
• 99.96% code coverage
• 100% “spectest” pass rate (the official WebAssembly test suite)
• second fastest WebAssembly interpreter

We are preparing the 0.6.0 release which will introduce a public C and Rust API, and provide support for WASI, which allows executing a wide range of applications.

After that our attention will be focused on blockchain specific behaviour (efficient metering) and continued upstream contributions for specification conformance.

Formal Verification

Authored by Leo Alt, Martin Lundfall, David Terry

Act

Act 0.1 is close to being released. It is converging as a language, and a lot of work went into tooling for Act in the past months, as planned:

• SMT backend. Inductive high level properties such as contract invariants and post conditions written in Act can be proven automatically using the Act SMT backend. The properties are encoded as inductive theorems using the required SMT theories, and given to an SMT solver. This feature allows users to prove properties in a simple, easy and standalone way, separated from the bytecode. The SMT backend assumes that the contract’s bytecode is correctly represented by the storage updates in the specification. In order to prove that, you can use the
• Hevm backend. Hevm now has symbolic execution features, which are now fully integrated into Act. Given an Act specification and the compiled bytecode, it will automatically check whether the storage updates described in the specification match what the bytecode actually does.
• Coq backend. Similarly to the SMT backend, this feature is useful to prove inductive high level properties in a standalone way. SMT solvers are automatic and fast, but can’t solve everything. Some problems require more expressiveness, and those are the target of the Coq backend. It generates Coq definitions and theorems so the user can use the Coq proof assistant to prove the high level properties.

Hevm

Hevm’s symbolic execution features were explained in detail earlier in the year, and here’s a summary of the features added since the last update:

• Symbolic execution. As explained in the previous blog post, hevm is now capable of exploring the possible execution paths of a smart contract, keeping one or more variables abstract. The resulting post states can be checked for assertion violations, or against a given specification. State can be fetched from an rpc node, or kept fully abstract. The tree of execution traces can be explored interactively in a debugger view.
• Equivalence checking. hevm equivalence symbolically executes two smart contracts and compares all resulting end states. This is particularly useful for proving safety of compiler optimizations, or comparing different implementations of contracts implementing the same semantics.
• Decompilation of smart contracts. The first initial steps towards a symbolic decompiler have been taken. Running hevm symbolic with the flag –show-tree prints a summary of the reachable end states, their path conditions and the state updates performed. This feature is still in its early stages, but the plan is that this will generate a set of act specifications from EVM bytecode, which could then be used to further reason about the smart contract’s behaviour.
• DSTest integration. hevm’s symbolic execution capabilities have been integrated with the dapp testing framework. This means that you can now write properties and formally verify them in Solidity in the same way you write tests! There is a lot more to be said here and we are working on a longer blog post which goes into detail.

SMTChecker

Many important features have been added to the SMTChecker since the last update, including:

• Much more language support. The SMTChecker is by now almost able to analyze any syntax modulo 1) some language features we don’t see being used that much, such as function pointers and complicated memory/storage aliasing; and 2) things that are too hard to have a reasonably efficient representation in the SMT universe, like ABI encoding/decoding and hash functions.
• External calls to unknown code. This is a big one! The SMTChecker’s Constrained Horn Clauses (CHC) engine abstracts external calls to non-trusted contracts as an unbounded number of nondeterministic calls to the analyzed contract. Using this abstraction the CHC engine can compute precisely what state changes can or cannot happen via this external call, allowing it to keep inductive invariants that were inferred previously and/or compute new ones. This enables the tool to automatically prove and learn invariants about more complex properties, that involve external calls and potential reentrancy. This also reduces greatly the number of false positives caused by external calls.
• Multi-transaction counterexamples. Another big one! The CHC engine now reports the full transaction trace from the contract’s constructor to the function that causes the verification condition to be triggered. It also includes concrete values for the function’s arguments and state variables, if applicable, for value types and arrays. Some upcoming features are reporting concrete values for structs and reporting reentrant calls in the transaction trace.
• Better control and usability. The user can now choose separately which engine should run, BMC (Bounded Model Checker), CHC or both. Besides, a timeout in milliseconds can also be given and is applied to each query. This makes fully deterministic reruns hard, but is useful when trying to prove complex properties that might need more time.

Geth

Authored by Péter Szilágyi

The Geth team’s primary focus in the past months was around laying the groundwork for a number of features we’re gradually rolling out in the next few weeks and also towards Eth1-Eth2 interoperability:

• One feature we’ve run ourselves for the better part of the year but have been holding back on rolling out to everyone is the snapshotter, which allows storing the Ethereum state in a flat format in the database. Although the core functionality worked well, we’ve spent a significant amount of time working on various recovery features so crashes and de-syncs with the state trie can be gracefully handled. You can already run your node with –snapshot, but as it is consensus critical, we’ll only be enabling it by default on our next major release, slated for early January.
• The primary purpose of the snapshotter was to enable snap sync, a successor to fast- and warp sync, having the advantages of both, whilst working around the disadvantages of both. The protocol was fully specced and published in the last few months in the devp2p repo, and it has a full production implementation pending merge on the Geth repo. As a teaser, the stats from our last benchmark: network packets reduced by 5(!) orders of magnitude, serving disk IO and upload by 2 orders of magnitude. 5x faster sync, 2x less download.
• One of the Achilles’ heels of Geth since its inception was that once some data hit the database, we never deleted it. Long term, our disk usage kept growing and bloating with stale states belonging to ancient blocks. The fast iteration capability of the snapshotter provides us with a unique new opportunity to implement a very simple state pruner. We will not be enabling it as a background process within Geth any time soon, but you’ll be able to prune your database offline via geth snapshot prune-state, which should finish in a few hours.
• On a more research-y front, we’ve been investigating the practicality of replacing the current hexary state trie with a binary one, the purpose of which would be to significantly reduce witness sizes (stateless direction) and light client proofs. In addition, binary tries could also help reduce some of the code complexity for new clients that would be willing to join the Ethereum network only from a future block and not care about chain history based on the hexary tries. Landing this work is still far out as this is as much of an open-heart-surgery of Ethereum as we can do.
• Since the Eth2 beacon chain now live, we’ve been working on a possible integration of the current Ethereum chain into a shard of the beacon chain; which would implicitly also enable running arbitrary EVM shards on Eth2. There are still a number of pieces missing to make it production ready (mostly around needing a new sync model, though no new protocol), but a prototype Geth EVM shard on top of Eth2 was already demonstrated.
• We’ve come to the realization that – at this point in time – most clients do not focus on network protocol improvements, even though Geth defined (ACD approved) and shipped quite a few lately. This is a problem as it prevents us from dropping old protocol versions, which makes our code brittle. Implementing these changes is not hard, but validating them is, which is our best guess as to why other clients aren’t keen on doing it. To try to offset some of the burden, we began working on a protocol tester to allow validating an arbitrary client’s implementations of the discovery protocol, devp2p and higher level overlays (eth, snap, etc).
• In a similar vein to the above protocol tester, a lot of effort went into bringing hive up to speed with new clients and new test suites. Our hopes are that other clients will also start contributing some tests to cross validate each others’ code and that soon Eth2 clients would also join a combined effort to keep the network stable.
• Geth implemented and ran 2 testnets for the upcoming Berlin hard fork (YOLO v1 and v2), with the 3rd one currently being prepped once all the proposed EIPs are more or less finalized. This also implicitly means that we’ve spent a lot of time implementing and shipping the various EIPs that will make up the Berlin hard fork. Progress on this front is a big jaggedy as the EIP process is not the leanest and the proposals evolve quite a bit, but we’re hopeful that all the details will get nailed down soon enough to launch YOLOv3 and then Berlin.
• For a list of security improvements in Geth – in order not to duplicate the same things multiple times – please check the Security section a bit further down authored by Martin Holst Swende.

JavaScript Team

Authored by Holger Drewes

The JavaScript team just recently had its big EthereumJS VM v5 release which consists of six major version releases:

VM v5 is the first full-featured JavaScript EVM available with backports for all hardforks down to genesis. It also comes with dedicated support for EIPs, moving away from a hardfork-centric VM. These new library versions complete our TypeScript transition and all now provide a modern JS Promise-based API. Furthermore, necessary refactors have been done on the block and tx libraries to get rid of technical debt. You can read more on the release in the blog post on the EF blog, covering things in a lot more detail.

These new releases provide the ground for us to turn more strongly towards protocol research and development. Our @ethereumjs/client, which we have just integrated into our monorepo, will play a central role. Building upon the new VM, Common and Blockchain release versions we are now able to finish a conceptually simple full-sync implementation on the client and start to join development efforts around developer testnets like the Yolo or EIP-1559 test networks. The client is far from being ready for mainnet but will serve us well in all sorts of development and research scenarios.

There is continued development on Ethers as well, and @ricmoo has done 23 (!) small feature and bug fix releases since the big v5 release in June 2020 (see preceding beta release notes for the full picture). Noteworthy features are the added support for EIP-712 signed typed data in v5.0.18 and new ENS resolver functions for contentHash in v5.0.10, together with many smaller fixes and improvements introduced. A lot of work has also been done on Ethers to ensure it remains a reliable infrastructure component in the large ecosystem it is used within. An example here is the work to ensure more robust Alchemy and Infura provider support: see v5.0.6 and subsequent releases. See this blog post for a more in-depth update on the latest highlights.

Remix

Authored by Yann Levreau & Rob Stupay

In the past 6 months, Remix Project has undergone more than a modicum of developmental loveliness. We’ve updated the functionality, the quality of the codebase, and the ability of the IDE to work with other projects. We have also been preparing for future iterations of the IDE. And yet, the totality of our programmatic handiwork would mean nothing if we didn’t reach out to teach new users, partner with other organizations and grow our community. So we did!

Remix IDE is used by both newcomers to Ethereum and by experienced devs. We have fortified the tool for both types of users. We have also been striving to improve the DevX for those coding plugins for IDE as well as ensuring that their plugins look good and continue to function well.

There are some large changes coming to Remix. Currently the IDE uses the yoyo framework, which is no longer being maintained. So we are moving on and have begun the transition to React and also to Theia – a framework specifically for building IDEs. In addition, we have finished the first iteration of a Remix plugin for VSCode.

See the specifics of the updates on our blog post about the last six months.

Snake Charmers [Python Ecosystem: PyEVM/Trinity/Web3.py/Vyper]

Authored by Piper Merriam

• Fe-Lang, a new smart contract language is under active development.
  
  • The language was originally born as a rewrite of the Vyper compiler (in Rust) but has since moved on to follow its own path. More on the backstory and goals can be read in the official announcement post.
  • We are on track to have an ERC-20 implementation in Fe functional before the end of the year. To be clear, the compiler will in no way be a suitable choice for a production ERC20 by that time, but we look forward to demonstrating the capabilities of Fe with such a well understood working example.
  • More information can be found at https://fe.ethereum.org/ as well as our most recent development update.

• The Trinity team is winding down development on the Trinity Ethereum client in order to re-focus on more impactful problems with the core protocols.
• We are starting work to build out a DHT based network for more effectively serving the chain history, as well as continued research on how to solve on-demand state availability. The project is moving forward under the name “Alexandria”. Read more in our recent development updates.
• The Web3.py team continues to focus on stability of the library which has matured nicely over the past years. Looking forward, we are still working towards implementing native async support. We are also working on exposing the Eth2 beacon chain APIs.

Stateless Ethereum

Authored by Piper Merriam

The “Stateless Ethereum” effort continues. In our latest call earlier in November we reviewed the roadmap and goals. Teams are actively working on conversion of the main Ethereum state trie from its current hexary format to a new binary format, as well as a modification to introduce a merklization layer to the way that contract code is stored. These two changes have the largest impact on reducing witness sizes. Re-genesis is also under active research as it provides major benefits to client developers, allowing them to drop old fork rules from their codebases while also giving us an incremental path towards full statelessness.

Security [Security / Consensus Tests]

Authored by Martin Holst Swende

• We’ve started doing bi-weekly cross-client fuzzing calls among client developers, where we’ve been iterating on the practicalities of evm-based fuzzing, getting the clients to agree on formats and chain definitions. We’ve performed ~100K fuzzed testcases focusing on YOLOv2, specifically EIP-2929, between Geth, OE and Besu.
• Further improvements and additions to Hive (https://hivetests.ethdevops.io), to expand the protocol testing capabilities and cover a larger portion of the ETH/63 protocol. Also improve documentation and API, to make it easier for other eth1 teams to integrate/use for testing, and make it possible to use for eth2 clients later on.
• Go-ethereum is now enrolled in OSS-Fuzz, which resulted in an upstream Go-lang bug being detected. Since the integration, we’ve further improved and expanded the fuzzing targets.
• Work in progress that we hope will land soon, is to implement a(n optional) vulnerability-check into Geth, which fetches information from a vulnerability-feed that can be used to check if the client suffers from any known vulnerability.
• The forkmon (https://forkmon.ethdevops.io) has been created, and 17 public and/or nodes are connected to it. It is a dashboard to help visualize
  
  • if the various clients are in agreement the head block
  • if not, where the disagreement (fork) started
  • whether any particular client has become stuck

• We have also expanded on the number of in-house clients that are used for monitoring the Mainnet, which now includes Geth, Besu, Nethermind and OpenEthereum.

Solidity

Authored by Franziska Heintel and Christian Reitwiessner

End of July, Solidity v0.7.0 was released. You can find a detailed list of all breaking changes here.

With the Solidity 0.8.0 release being just around the corner, we wanted to try out a new format to share the implementation details and changes of breaking releases with you beforehand. Hence, we published a preview release. The preview release post includes 0.8.x test binaries as well as a detailed description of the upcoming “Checked Arithmetic” feature (bye bye SafeMath!) with its three sub-features. We hope you find these kind of preview releases helpful and look forward to hearing your thoughts on the implementation of the breaking changes. And, as per usual, if you are interested in discussing language design with us, make sure to join the solidity-users mailing list!

At the moment, the Solidity team is working on the last items to get into Solidity 0.8. Most notably, adding overflow checks for arithmetic operations and the activation of ABI coder V2 by default. See above for the preview release!

The second big chunk of work, which is also nearing completion, is the use of Yul as an intermediate language in the compiler. For this, we re-implemented most of the code generator. You can try it out using solc –experimental-via-ir or solc –ir and solc –ir-optimized –optimize. The goal of using Yul is that the compiler gets more transparent because you can actually read and understand the code it generates. Furthermore, it allows more opportunities for optimization and you might soon say goodbye to “stack too deep” errors!
With regard to features, we want to change the Solidity language to support more items at the file-level, which allows you to better structure your code and introduce nicely isolated helper functions.
Finally, with the introduction of error codes, you will soon be able to flag problems that are readable to the user and at the same time only consume little gas.
Oh and then the language server…

Solidity Summit

The first Solidity Summit took place online on April 29-30 2020 and featured discussions & talks on Solidity, Yul, language design and tooling. The event encouraged discussions around language design and language improvement and aimed to foster communication between teams working on similar topics. Furthermore, the goal was to identify needs for the Ethereum smart contract ecosystem. With enough content and talks for a packed agenda over two days, it was a great success. The conference featured 23 talks and 13 open discussion rounds. To learn more, read the recap of the event or watch the videos of all talks online in this playlist. We are planning to host more Solidity Summits in future and will keep you updated on when and how.

Solidity Language Portal & Domains

In October, we were excited to announce the launch of the Solidity language portal, a one-stop-shop for all important links and high-level information around Solidity. Alongside with the launch of this portal, we tidied up domains and moved most of the Solidity pages under the new soliditylang.org domain umbrella.
We have also recently changed the hosting of the static builds, which are now available at binaries.soliditylang.org. While the transition should have been seamless for most users, please make sure to switch if you are still using https://ethereum.github.io/solc-bin/.

Solidity Underhanded Contest

Throughout October, we hosted a new edition of the Solidity Underhanded Contest. The contest’s goal is to raise awareness of smart contract security and uncover language design faults by writing innocent-looking Solidity code, which contains malicious behavior or backdoors. Submissions closed on October 31st and we are thrilled to share the winners with you in the coming days!

Sourcify

Besides the core update, the Solidity team has been working on an initiative called Sourcify (formerly known as source-verify) for a while now. Sourcify is a contract verification service and metadata repo, which aims to make interacting with smart contracts on the blockchain safer and more transparent for users. To achieve this goal, Sourcify supports several efforts to foster adoption of open-source source verification, metadata files and NatSpec comments.

At its core, Sourcify currently maintains

Last, but not least, we celebrated the 5th birthday of Solidity version 0.1.0 with a walk down memory lane, reflections on 5+ years of language design and a brief look into the future.

ZKP Research

Authored by Dmitry Khovratovich, Mary Maller and Dankrad Feist

Improving STARK-based VDFs

The idea is to prove a long incompressible iteration using a zkSTARK (Ben-Sasson et al). We suggested a new iterative construction with increased performance and security over the Starkware proposal Veedo. The new one minimizes the arithmetic operations per step and reduces the hardware requirements. We also analyzed:

• How time-memory tradeoffs can affect the construction and suggested secure parameters.
• Quantum attacks on the function and quantum enhancements to tradeoff algorithms.

Designing a more secure hash function for ZK applications

Current ZK-friendly hash functions rely too much on algebraic attacks, but the proof system UltraPLONK (adding the Plookup construction to PLONK) adds a lookup table (like an AES S-box) as a fast operation.
We show how to combine lookups with field operations to be fast both in proofs and in regular hashing.
The security relies much more on regular cryptanalysis than on (not much understood) Groebner basis attacks.

New protocol for Single Secret Leader Election (SSLE) that can be implemented in Eth2

The consensus protocol in Eth2 is leader based. This means that for each shard, a leader is selected who is responsible for proposing blocks and driving progress. Thus a malicious party could theoretically stall the entire network simply by DDOSing the selected leader. We have designed a preventative measure against such attacks based on a single secret leader election (SSLE) put forward by Boneh and others.

The SSLE ensures that the only person who knows who the leader is, is the leader themselves. The leader only reveals their identity upon proposing a block. Thus our malicious party does not know who to attack. The SSLE is achieved by shuffling encryptions in zero-knowledge. We have implemented a shuffle argument inspired by that of Bayer and Groth, but using more modern inner product arguments to improve efficiency. The argument is fast enough to be included in the Eth2 design and does not require a trusted setup. It is still awaiting a formal audit.

ZoKrates

Authored by Jacob Eberhardt

We’ve been hard at work since our last update, and are happy to announce several new features and improvements today.
Throughout the last couple of months, we extended the ZoKrates language and compiler, added a new backend, and improved our documentation to make zkSNARKs even more accessible to Ethereum developers today.

We extended support for unsigned integers with additional operators and efficiency tweaks. This allowed us to rewrite important standard library functions, such as SHA256, fully in our DSL instead of relying on imports of hand-optimised circuits for performance.

To further improve usability and developer experience, we added syntax highlighting to the ZoKrates Remix plugin, extended input validation, and enabled convenient input forms for complex data types.

As groundwork for the integration of new proving schemes and recursive proof composition, we added support for the Arkworks zkSNARK libraries as an additional backend (this library was called ZEXE until very recently). We thank the EY Blockchain R&D team for their valuable contributions in this context!
Building on that, we are currently working towards enabling recursive zkSNARK composition within the ZoKrates toolbox.

Besides the usual documentation updates to describe new features and changes, we added a new tutorial to help onboarding new users.

We’ll close with a little teaser: We’re getting close to finalizing support for generic ZoKrates programs that depend on compile-time constants. This will significantly improve code reusability and make our standard library easier to use. Stay tuned.