EthProv: Ethereum Compiler Provenance Dataset from Bytecode Metadata

About EthProv is a compiler provenance dataset for Ethereum smart contracts. It employs a source code-independent pipeline to extract compiler version labels for over 1 million unique smart contracts. Each sample in this dataset consists of: bytecode_hash: SHA256 hash of each contract that acts as the unique identifier address: The Ethereum address where this contract is deployed compiler: Denotes the compiler family extracted from the metadata (solc or vyper) compiler_version: The compiler version extracted from the metadata. A value of "000510" means 0.5.16. How to use The dataset (ethereum_compiler_provenance_bytecode_metadata_dataset.csv) can be downloaded from Zenodo. Example 1: Count samples for each compiler family >>> import pandas as pd >>> df = pd.read_csv('ethereum_compiler_provenance_bytecode_metadata_dataset.csv') >>> df.head() bytecode_hash address compiler compiler_version 0 5c24585e91454b61ee4dcfbe99d76a6f0b589581c8da0e... 0xb9fbabcab792d23cb7920701a7ab1a240336663a NaN NaN 1 bad0292161a6829cfaafb6b3cf36ba893185303d0b5bb0... 0x068a35fb41ca166bd566b0cf37d94a4e725d9ab6 NaN NaN 2 264627a51fcf1ac37b8d211bf6173d07623ac39d6171ce... 0x52d1d1c87eb75a5d695bffa343cac71222cd518e NaN NaN 3 2f4a823d3324963c9865a1a4735471a2fe2438b3716059... 0x3a9f29bae47670ad9755bb226a155015d922c305 solc 000510 4 a1ede7d8ed4ee59036c245b6c481f2fa7373149293cc0b... 0xdb151ca31e799e0c3ffe66529594ea10926195f4 solc 000602 # Sample count for compiler >>> print(df['compiler'].value_counts()) compiler solc 1015879 vyper 967 Name: count, dtype: int64 EXAMPLE 2: Fetch and verify EtherScan compiler details >>> import pandas as pd >>> import re >>> import requests >>> >>> ETHERSCAN_API_URL = "https://api.etherscan.io/v2/api" >>> ETHERSCAN_API_KEY = "<ETHERSCAN API KEY>" >>> TEST_ADDRESS = "0x3a9f29bae47670ad9755bb226a155015d922c305" >>> >>> def query_etherscan(address: str, api_key: str) -> dict: >>> # Query Etherscan API for contract info. >>> params = { >>> "chainid": "1", >>> "module": "contract", >>> "action": "getsourcecode", >>> "address": address, >>> "apikey": api_key, >>> } >>> try: >>> resp = requests.get(ETHERSCAN_API_URL, params=params, timeout=15) >>> data = resp.json() >>> if data.get("status") == "1" and data.get("result"): >>> return data["result"][0] >>> except Exception as e: >>> print(f" API error for {address}: {e}") >>> return {} >>> >>> # Fetch compiler details from EthProv dataset. >>> df = pd.read_csv('ethereum_compiler_provenance_bytecode_metadata_dataset.csv') >>> ethprov_compiler_version = df.loc[df['address'] == TEST_ADDRESS, 'compiler_version'].values[0] >>> base = int(ethprov_compiler_version[0:2],16) >>> major = int(ethprov_compiler_version[2:4],16) >>> minor = int(ethprov_compiler_version[4:6],16) >>> >>> # Fetch compiler details using EtherScan API. >>> etherscan_compiler_version = query_etherscan(TEST_ADDRESS, ETHERSCAN_API_KEY).get("CompilerVersion", "") >>> >>> print(f"EthProv compiler label: {base}.{major}.{minor}") >>> print(f"EtherScan compiler label: {etherscan_compiler_version}") EthProv compiler label: 0.5.16 EtherScan compiler label: v0.5.16+commit.9c3226ce