/VulMat

VulMat: An accurate multi-tool vulnerability <-> detector taxonomy matrix smart contracts

What is this?

Large-scale research and development in smart contracts security demand for robust benchmarks. These benchmarks themselves should be based on correct understanding of different vulnerabilities and their detectors. That is exactly what we offer in VulMat. The matrix could be used to design both research and development pipelines in smart contract security tools.

Vulnerability <-> Detector Relationship Matrix

N General Category Detectors and Applications
Target Applications Slither Smartcheck Semgrep
1 Re-entrancy DeFi N/A N/A compound-borrowfresh-reentrancy
Tokenization N/A N/A erc721-reentrancy
DeFi N/A N/A curve-readonly-reentrancy
Tokenization N/A N/A erc777-reentrancy
Tokenization N/A N/A erc677-reentrancy
N/A reentrancy-benign N/A N/A
N/A Reentrancy-events N/A N/A
N/A reentrancy-no-eth N/A N/A
N/A reentrancy-unlimited-gas N/A N/A
N/A reentrancy-eth N/A N/A
2 Low-level calls N/A N/A N/A Arbitrary-low-level-call
N/A N/A SOLIDITY_CALL_WITHOUT_DATA N/A
3 Access Control DeFi N/A N/A Compound-sweeptoken-not-restricted
Tokenization N/A N/A Erc20-public-burn
N/A N/A N/A Accessible-selfdestruct
DeFi N/A N/A Oracle-price-update-not-restricted
DeFi N/A N/A Uniswap-callback-not-protected
Tokenization arbitrary-send-erc20 N/A N/A
N/A arbitrary-send-eth N/A N/A
N/A suicidal N/A N/A
N/A arbitrary-send-erc20-permit N/A N/A
N/A N/A SOLIDITY_TX_ORIGIN N/A
4 Delegation N/A controlled-delegatecall N/A Delegatecall-to-arbitrary-address
N/A delegatecall-loop N/A N/A
5 Arithmetic N/A N/A N/A Basic-arithmetic-underflow
N/A divide-before-multiply SOLIDITY_DIV_MUL N/A
6 Oracle Manipulation DeFi N/A N/A Keeper-network-oracle-manipulation
7 Input Validation N/A Missing-zero-check N/A N/A
8 Shadowing N/A Shadowing-local N/A N/A
N/A shadowing-state N/A N/A
N/A shadowing-abstract N/A N/A
9 Compliance Tokenization erc20-interface N/A N/A
Tokenization N/A SOLIDITY_ERC20_TRANSFER_SHOULD_THROW N/A
10 Timestamp N/A timestamp N/A N/A
N/A weak-prng N/A N/A
11 Initialization N/A uninitialized-local N/A N/A
N/A uninitialized-state N/A N/A
12 Poor Logic Flaws N/A Incorrect-equality SOLIDITY_EXACT_TIME,
SOLIDITY_BALANCE_EQUALITY		 N/A
N/A boolean-cst N/A N/A
N/A N/A N/A incorrect-use-of-blockhash
13 Denial of Service N/A locked-ether SOLIDITY_LOCKED_MONEY N/A
N/A calls-loop SOLIDITY_TRANSFER_IN_LOOP N/A
N/A msg-value-loop N/A N/A
14 State Corruption N/A controlled-array-length SOLIDITY_ARRAY_LENGTH_MANIPULATION N/A
15 Function Behavior N/A incorrect-modifier N/A N/A
16 External Call Validation N/A unchecked-transfer N/A N/A
N/A unchecked-lowlevel N/A N/A
17 Front-Running Tokenization N/A SOLIDITY_ERC20_APPROVE N/A

General Category Descriptions

Re-entrancy

Any vulnerability directly related to reentrant feature of smart contract execution.

Low-level Calls

This category is concerned with the way low-level calls are made. Especially, it concerns the way data field of the call is assigned (data source, being checked, etc.).

Access Control

Detectors that detect any authentication/authorization vulnerability fall into this category. This includes authorizing the function caller and checking their permissions in callee function or setting up an external call with the correct attributes in the caller function.

Delegation

Delegation is a tricky subject in smart contracts as delegated calls could be exploited by malicious callee to bypass access control mechanism of the contract, and in extreme cases take over the whole contract by setting the state variables pertaining to certain high-permissioned roles.

Arithmetic

Any vulnerability and accordingly detector pertaining to the way arithmetic operations are performed and ordered on blockchain.

Oracle Manipulation

Exploitations regarding changing the integrity of data gathered with the help of oracles; both in the oracle service contract and the user contract.

Input Validation

Instances of this vulnerability class are failing to perform checks on arbitrary user-supplied data not to be uninitialized or with malicious data in a general sense.

Shadowing

Occurs when a variable from a scope has the same identifier from another scope resulting in the unintended value to be used during runtime.

Compliance

Failing to comply to the specification of the standard in the contract implementation. The least severe effect will be the interacting contracts/users (who expect the correct complying implementation) will fail to execute external calls/transactions to the function (reverted transactions if external call validation is implemented).

Timestamp

Usage of unreliable/inaccurate blockchain timestamp primitive in critical scenarios such as random number generation or for checking conditions with high accuracy.

Initialization

Failing to initialize (function-local/state) variables.

Poor Logic Flaws

This class concerns using conditional statements that are poorly designed, and might not result in intended way. Additionally, relying on poorly designed boolean conditions that could be manipulated by an interacting user leads to an exploit.

Denial of Service

Vulnerabilities related to a function in the contract not functioning as intended. Transactions to the vulnerable function might be reverted because of poorly designed usage of loops and msg.* constructs.

State Corruption

The contract state is corrupted or manipulated in unintended ways.

Function Behavior

This category encompasses vulnerabilties related to the behavior and flow of execution within functions, including the use of modifiers.

External Call Validation

The core issue here is ensuring that transactions or external function calls (especially token transfers) are executed as intended and their outcomes are validated. Proper handling of external calls (transfer, transferFrom, etc.) and checking their return values, which is a key aspect of transaction handling in smart contract development.

Front-Running

Vulnerability to manipulation of final transaction interleaving in blockchain.

Notes

  • N/A for target application: This means the vulnerability does not pertain to a specific high-level application category.
  • Vulnerability detection results that should be used in conjunction: If the vulneraiblity detected by two tools refers to the same bug localization point (same function, same line, same code snippet depending on different detectors) and same type of vulnerability, they should be considered together (for comparison/benchmarking).

References

The function SHOULD throw if the message caller’s account balance does not have enough tokens to spend.