Standard format for vulnerability scanners


#1

So, this is an idea I had today at the secureth unconf, which we briefly discussed on the tools-discussion.

Background

The ethereum tests repository contains various types of tests; blockchaintests, vmtests, statetests etc, that are all meant to help client implementors ensure that a client remains in consensus with the other implementations.

One such type of test, is the stateTest (which is somewhat described on readthedocs) contains all the parameters required to showcase a state transition:

  1. Rules
  • (Frontier/Homestead/Byzantium?)
  1. Prestate
  • Which accounts exists, with what code, and what is on the storage slots?
  1. A transaction
  2. An expected post-state
  • needed for verification when doing consensus-testing, not necessarily needed in this context

Use in contract security tooling

Using a statetest, it’s possible to describe any relevant preconditions for an ‘attack’. So any vulnerability-scanner could output a statetest, where the bytecode is placed in the state, the relevant storage slots set, and then a transaction that demonstrates/executes the attack/flaw that was found.

Why

Statetests are today used to discuss and showcase evm quirks between evm-implementors. Similarly, someone could send a smart-contract developer a statetest json file, that they could execute on a VM of their choice, such as geth evm or parity parity-evm. That developer could e.g. run it and get the standardized json trace format (supported by geth, parity and aleth), and step through the execution with e.g. https://github.com/ethereum/evmlab#opviewer .

It would fit well into a developer toolchain, if travis ci tools exported stateTests format for all vulnerability assessment tools, instead of whatever textual or proprietary format that specific tool happens to output.

Making it possible to take the output from a tool, give it to someone else, who investigate it with their own tools of choice would be a good step towards getting security tools into the development pipeline, imo.


#2

This might also make the discussion on the benchmarking a tad easier to implement. The state trace alongside the relevant source code, perhaps with a bound package showing the source code trace for tools that require it.