Merge pull request #1 from iamdefinitelyahuman/v1

v1.0.0

.gitignore

@@ -7,3 +7,5 @@ build/
 dist/
 *.pyc
 .coverage
+.mypy_cache/
+.tox/

.travis.yml

@@ -0,0 +1,14 @@
+language: python
+dist: xenial
+sudo: true
+install:
+  - sudo apt-get update
+  - pip install -r requirements-dev.txt
+matrix:
+  include:
+  - name: '3.6'
+    python: 3.6
+  - name: '3.7'
+    python: 3.7
+    after_success: coveralls
+script: tox

CHANGELOG

@@ -1,3 +1,10 @@
+1.0.0
+-----
+
+ - Streamlined code using metaclasses
+ - Attributes more consistent with original AST
+ - Require Python 3.6 or greater
+
 0.1.4
 -----
 

README.md

@@ -1,6 +1,6 @@
 # solc-ast
 
-[![Pypi Status](https://img.shields.io/pypi/v/py-solc-ast.svg)](https://pypi.org/project/py-solc-ast/)
+[![Pypi Status](https://img.shields.io/pypi/v/py-solc-ast.svg)](https://pypi.org/project/py-solc-ast/) [![Build Status](https://img.shields.io/travis/com/iamdefinitelyahuman/py-solc-ast.svg)](https://travis-ci.com/iamdefinitelyahuman/py-solc-ast) [![Coverage Status](https://coveralls.io/repos/github/iamdefinitelyahuman/py-solc-ast/badge.svg?branch=master)](https://coveralls.io/github/iamdefinitelyahuman/py-solc-ast?branch=master)
 
 A tool for exploring the Solidity abstrax syntrax tree as generated by the [solc](https://github.com/ethereum/solidity) compiler.
 
@@ -29,79 +29,125 @@ First, use [py-solc-x](https://github.com/iamdefinitelyahuman/py-solc-x) to comp
 >>> output_json = solcx.compile_standard(input_json)
 ```
 
-Next, import ``solcast`` and initialize using ``from_standard_output_json`` or ``from_standard_output``. This returns a list of ``SourceUnit`` objects, which each represent the base node in a Solidity AST.
+Next, import ``solcast`` and initialize using ``from_standard_output_json`` or ``from_standard_output``. This returns a list of ``SourceUnit`` objects, which each represent the base AST node in a Solidity source file.
 
 ```python
 >>> import solcast
 >>> nodes = solcast.from_standard_output(output_json)
+>>> nodes
+[<SourceUnit iterable 'contracts/Token.sol'>, <SourceUnit iterable 'contracts/SafeMath.sol'>]
 ```
 
-From the initial objects, you can explore the AST:
+### Interacting with Nodes
+
+Each node has the following attributes:
 
 ```python
->>> nodes
-[<SourceUnit iterable object 'contracts/Token.sol'>]
->>> s = nodes[0]
->>> s
-<SourceUnit iterable object 'contracts/Token.sol'>
+>>> node
+<FunctionDefinition iterable 'mul'>
+
+>>> node.depth  # Number of nodes between this node and the SourceUnit
+2
 
->>> s.keys()
-['children', 'contract_id', 'contracts', 'depth', 'keys', 'name', 'node_type', 'offset', 'parent', 'path', 'value']
+>>> node.offset  # Absolute source offsets as a (start, stop) tuple
+(1693, 2151)
 
->>> s.contracts
-[<ContractDefinition iterable 'Token'>]
+>>> node.contract_id  # Contract ID as given by the standard compiler JSON
+2
 
->>> s[0]
-<ContractDefinition iterable 'Token'>
+>>> node.fields  # List of fields for this node
+['baseNodeType', 'documentation', 'id', 'implemented', 'kind', 'modifiers', 'name', 'nodeType', 'nodes', 'parameters', 'returnParameters', 'scope', 'src', 'stateMutability', 'superFunction', 'visibility']
 
->>> s['Token']
-<ContractDefinition iterable 'Token'>
+```
 
->>> s['Token'].keys()
-['children', 'contract_id', 'depth', 'functions', 'keys', 'name', 'node_class', 'node_type', 'offset', 'parent', 'value']
+Fields mostly follow the expected [AST grammar](https://solidity.readthedocs.io/en/latest/miscellaneous.html#language-grammar), with the following differences:
 
->>> s['Token'].functions
-[<FunctionDefinition iterable '<constructor>'>, <FunctionDefinition iterable '<fallback>'>, <FunctionDefinition iterable 'balanceOf'>, <FunctionDefinition iterable 'allowance'>, <FunctionDefinition iterable 'approve'>, <FunctionDefinition iterable 'transfer'>, <FunctionDefinition iterable 'transferFrom'>]
+* `Block` nodes are omitted, the body of each `Block` is available within it's parent as `nodes`.
+* `ExpressionStatement` nodes are replaced with their underlying `Expression`
 
->>> s['Token']['transfer']
-<FunctionDefinition iterable 'transfer'>
+The following additional fields are also available:
 
->>> s['Token']['transfer'].statements
-[<ExpressionStatement.FunctionCall 'require(balances[msg.sender] >= _value, Insufficient Balance)'>, <ExpressionStatement.Assignment iterable uint256 'balances[msg.sender] = balances[msg.sender].sub(_value)'>, <ExpressionStatement.Assignment iterable uint256 'balances[_to] = balances[_to].add(_value)'>, <EmitStatement.FunctionCall 'Transfer'>, <Return.Literal bool 'true'>]
-```
+* Most nodes have a `baseNodeType` field as defined in [grammar.py](solcast/grammar.py)
+* `ContractDefinition` nodes have `dependencies` and `libraries` fields that point to related `ContractDefition` nodes
 
-Use the ``Node.children`` and ``Node.parents`` methods to access and filter related nodes:
+When a node has a `nodes` field, it is iterable and can be accessed with list-like syntax. Additionally, any child node with a `name` field is accessible using dict-like syntax.
+
+Some Examples:
 
 ```python
->>> node = s['Token']['transfer']
+>>> source_node
+<SourceUnit iterable 'contracts/math/SafeMath.sol'>
 
->>> node.children(depth=1)
-[<ExpressionStatement.FunctionCall 'require(balances[msg.sender] >= _value, Insufficient Balance)'>, <ExpressionStatement.Assignment iterable uint256 'balances[msg.sender] = balances[msg.sender].sub(_value)'>, <ExpressionStatement.Assignment iterable uint256 'balances[_to] = balances[_to].add(_value)'>, <EmitStatement.FunctionCall 'Transfer'>, <Return.Literal bool 'true'>]
+>>> source_node.keys()
+['absolutePath', 'children', 'contract_id', 'depth', 'exportedSymbols', 'id', 'is_child_of', 'is_parent_of', 'keys', 'nodeType', 'nodes', 'offset', 'parent', 'parents', 'src']
 
->>> node.children(include_children=False, filters={'node_type': "FunctionCall", 'name': "require"})
-[<ExpressionStatement.FunctionCall 'require(balances[msg.sender] >= _value, Insufficient Balance)'>]
+>>> source_node.nodes
+[<PragmaDirective object>, <ContractDefinition iterable 'SafeMath'>]
 
->>> node.parents()
-[<ContractDefinition iterable 'Token'>, <SourceUnit iterable object 'contracts/Token.sol'>]
+>>> source_node[1]
+<ContractDefinition iterable 'SafeMath'>
+
+>>> source_node['SafeMath']
+<ContractDefinition iterable 'SafeMath'>
+
+>>> source_node['SafeMath'].keys()
+['baseContracts', 'children', 'contractDependencies', 'contractKind', 'contract_id', 'dependencies', 'depth', 'documentation', 'fullyImplemented', 'id', 'is_child_of', 'is_parent_of', 'keys', 'libraries', 'linearizedBaseContracts', 'name', 'nodeType', 'nodes', 'offset', 'parent', 'parents', 'scope', 'src']
+
+>>> source_node['SafeMath'].nodes
+[<FunctionDefinition iterable 'add'>, <FunctionDefinition iterable 'sub'>, <FunctionDefinition iterable 'mul'>, <FunctionDefinition iterable 'div'>, <FunctionDefinition iterable 'mod'>]
+
+>>> source_node['SafeMath']['mul']
+<FunctionDefinition iterable 'mul'>
+
+>>> source_node['SafeMath']['mul']
+[<IfStatement object>, <VariableDeclarationStatement object>, <FunctionCall object>, <Return object>]
 ```
 
-Calling ``help`` on either of these methods provides a more detailed explanation of their functionality.
+### Exploring the Tree
 
-## Development
+The `Node.children()` method is used to search and filter through child nodes of a given node. It takes any of the following keyword arguments:
+
+* `depth`: Number of levels of children to traverse. `0` returns only this node.
+* `include_self`: Includes this node in the results.
+* `include_parents`: Includes nodes that match in the results, when they also have child nodes that match.
+* `include_children`: If True, as soon as a match is found it's children will not be included in the search.
+* `required_offset`: Only match nodes with a source offset that contains this offset.
+* `offset_limits`: Only match nodes when their source offset is contained inside this source offset.
+* `filters`: Dictionary of `{'attribute': "value"}` that children must match. Can also be given as a list of dicts, children that match any of the dicts will be returned.
+* `exclude_filter`: Dictionary of `{'attribute': "value"}` that children cannot match.
+
+```python
+>>> node = s['Token']['transfer']
+>>> node.children(
+    include_children=False,
+    filters={'nodeType': "FunctionCall", "expression.name": "require"}
+)
+[<FunctionCall>]
+```
 
-This project is still in development and should be considered an early alpha. All feedback and contributions are welcomed!
+`Node.parent()` and `Node.parents()` are used to travel back up the tree. They take the following arguments:
 
-Not all nodes have been implemented yet. From any object, you can use the ``Node._unimplemented`` method to get a list of keys that contain AST nodes that have not yet been included. The raw json data is stored at ``Node._node``.
+* `depth`: Depth limit. If given as a negative value, it will be subtracted from this object's depth.
+* `filters`: Dictionary of `{'attribute': "value"}` that parents must match.
+
+`Node.parent()` returns one result, `Node.parents()` returns a list of matches.
 
 ```python
->>> s['Token']['transfer']._unimplemented()
-['parameters', 'returnParameters']
+>>> node.parents()
+[<ContractDefinition iterable 'Token'>, <SourceUnit iterable object 'contracts/Token.sol'>]
+```
+
+## Tests
+
+To run the test suite:
 
->>> s['Token']['transfer']._node['returnParameters']
-{'id': 328, 'nodeType': 'ParameterList', 'parameters': [{'constant': False, 'id': 327, 'name': '', 'nodeType': 'VariableDeclaration', 'scope': 373, 'src': '1573:4:2', 'stateVariable': False, 'storageLocation': 'default', 'typeDescriptions': {'typeIdentifier': 't_bool', 'typeString': 'bool'}, 'typeName': {'id': 326, 'name': 'bool', 'nodeType': 'ElementaryTypeName', 'src': '1573:4:2', 'typeDescriptions': {'typeIdentifier': 't_bool', 'typeString': 'bool'}}, 'value': None, 'visibility': 'internal'}], 'src': '1572:6:2'}
+```bash
+$ tox
 ```
 
-See the Solidity documentation for information about the [AST grammar](https://solidity.readthedocs.io/en/latest/miscellaneous.html#language-grammar).
+## Development
+
+Comments, questions, criticisms and pull requests are welcomed! Feel free to open an issue if you encounter a problem or would like to suggest a new feature.
 
 ## License
 

pyproject.toml

@@ -0,0 +1,27 @@
+	# Brownie configuration for Black.
+
+# NOTE: you have to use single-quoted strings in TOML for regular expressions.
+# It's the equivalent of r-strings in Python.  Multiline strings are treated as
+# verbose regular expressions by Black.  Use [ ] to denote a significant space
+# character.
+
+[tool.black]
+line-length = 100
+target-version = ['py36', 'py37', 'py38']
+include = '\.pyi?$'
+exclude = '''
+/(
+    \.eggs
+  | \.git
+  | \.hg
+  | \.mypy_cache
+  | \.tox
+  | \.venv
+  | _build
+  | buck-out
+  | build
+  | dist
+  | env
+  | venv
+)/
+'''

requirements-dev.txt

@@ -0,0 +1,8 @@
+black==19.3b0
+coveralls==1.7.0
+isort==4.3.21
+pytest>=5.0.0
+pytest-cov>=2.7.1
+tox-travis==0.12
+twine==1.13.0
+wheel==0.33.4

setup.cfg

@@ -0,0 +1,6 @@
+[tool:isort]
+force_grid_wrap = 0
+include_trailing_comma = True
+line_length = 100
+multi_line_output = 3
+use_parentheses = True

setup.py

@@ -1,40 +1,35 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
-from setuptools import (
-    setup,
-    find_packages,
-)
+from setuptools import setup, find_packages
 
 with open("README.md", "r") as fh:
     long_description = fh.read()
 
 setup(
-    name='py-solc-ast',
-    version='0.1.4',
+    name="py-solc-ast",
+    version="1.0.0",
     description="""A tool for exploring the abstrax syntrax tree generated by solc.""",
     long_description=long_description,
     long_description_content_type="text/markdown",
-    author='Ben Hauser',
-    author_email='[email protected]',
-    url='https://github.com/iamdefinitelyahuman/py-solc-ast',
+    author="Ben Hauser",
+    author_email="[email protected]",
+    url="https://github.com/iamdefinitelyahuman/py-solc-ast",
     include_package_data=True,
-    py_modules=['solcast'],
+    py_modules=["solcast"],
     setup_requires=[],
-    python_requires='>=3.5, <4',
-    install_requires=[
-    ],
+    python_requires=">=3.6, <4",
+    install_requires=[],
     license="MIT",
     zip_safe=False,
-    keywords='ethereum solidity solc',
+    keywords=["ethereum", "solidity", "solc", "ast"],
     packages=find_packages(exclude=["tests", "tests.*"]),
     classifiers=[
-        'Development Status :: 3 - Alpha',
-        'Intended Audience :: Developers',
-        'License :: OSI Approved :: MIT License',
-        'Natural Language :: English',
-        'Programming Language :: Python :: 3',
-        'Programming Language :: Python :: 3.5',
-        'Programming Language :: Python :: 3.6',
-        'Programming Language :: Python :: 3.7'
+        "Development Status :: 3 - Alpha",
+        "Intended Audience :: Developers",
+        "License :: OSI Approved :: MIT License",
+        "Natural Language :: English",
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.7",
     ],
 )

solcast/__init__.py

@@ -1,6 +1,3 @@
 #!/usr/bin/python3
 
-from .main import (  # noqa 401
-    from_standard_output,
-    from_standard_output_json
-)
+from .main import from_standard_output, from_standard_output_json  # noqa 401