CloudMapper helps you analyze your Amazon Web Services (AWS) environments. The original purpose was to generate network diagrams and display them in your browser. It now contains more functionality.
Demo: https://duo-labs.github.io/cloudmapper/
Intro post: https://duo.com/blog/introducing-cloudmapper-an-aws-visualization-tool
Post to show usage in spotting misconfigurations: https://duo.com/blog/spotting-misconfigurations-with-cloudmapper
Requirements:
pipandvirtualenv- You will also need
jq(https://stedolan.github.io/jq/) and the librarypyjq(https://github.com/doloopwhile/pyjq), which require some additional tools installed that will be shown.
On macOS:
# clone the repo
git clone [email protected]:duo-labs/cloudmapper.git
# Install pre-reqs for pyjq
brew install autoconf automake libtool jq awscli
cd cloudmapper/
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
On Linux:
# clone the repo
git clone [email protected]:duo-labs/cloudmapper.git
# (AmazonLinux2):
# sudo yum install autoconf automake libtool python-devel python-virtualenv jq awscli
# (Centos, Fedora, RedHat etc.):
# sudo yum install autoconf automake libtool python-devel jq awscli
# (Debian, Ubuntu etc.):
# You may additionally need "build-essential"
sudo apt-get install autoconf automake libtool python-dev jq awscli
cd cloudmapper/
virtualenv venv
source venv/bin/activate
pip install -U pip setuptools
pip install -r requirements.txt
With Docker (not well supported):
# Clone the repo
git clone [email protected]:duo-labs/cloudmapper.git
# Edit config.json
vi config.json
# Build the docker container
docker-compose build
# Set the accountname and run the container (assuming aws_* variables are set)
accountname="testaccount" docker-compose up
A small set of demo data is provided. This will display the same environment as the demo site https://duo-labs.github.io/cloudmapper/
python cloudmapper.py prepare --config config.json.demo --account demo
python cloudmapper.py webserver
This will run a local webserver at http://127.0.0.1:8000/
Alternatively using docker:
docker-compose build && accountname="demo" docker-compose up
- Configure information about your account.
- Collect information about an AWS account.
Copy the config.json.demo to config.json and edit it to include your account ID and name (ex. "prod"), along with any external CIDR names. A CIDR is an IP range such as 1.2.3.4/32 which means only the IP 1.2.3.4.
CloudMapper has commands to configure your account:
python cloudmapper.py configure {add-account|remove-account} --config-file CONFIG_FILE --name NAME --id ID [--default DEFAULT]
python cloudmapper.py configure {add-cidr|remove-cidr} --config-file CONFIG_FILE --cidr CIDR --name NAME
This will allow you to define the different AWS accounts you use in your environment and the known CIDR IPs.
This step uses the CLI to make describe and list calls and records the json in the folder specified by the account name under account-data.
Locally, AWS CLI must be configured with proper access key and region information. Generate new access keys in AWS Console and input the generated keys to aws configure if you have not done so yet.
You must have AWS credentials configured that can be used by the CLI with read permissions for the different metadata to collect. This can be granted via the SecurityAudit policy, or can be reduced to an even more minimal set of permissions if desired for network visualization. The minimal policy needed is:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Resource": "*",
"Action": [
"ec2:DescribeRegions",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeVpcs",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups",
"ec2:DescribeVpcPeeringConnections",
"ec2:DescribeInstances",
"ec2:DescribeNetworkInterfaces",
"rds:DescribeDBInstances",
"elasticloadbalancing:DescribeLoadBalancers"
]
}
]
}
Collecting the data can be performed with a bash script or via the python code base. Both options support a --profile-name to specify the AWS account profile to use.
Using the script is helpful if you need someone else to get this data for you without fiddling with setting up the python environment.
NOTE The script will collect a small subset of available data. It is preferable to use Option 2 below whenever possible.
./collect_data.sh --account my_account
my_account is just a name for your account (ex. "prod"). You can also pass a --profile option if you have multiple AWS profiles configured. You should now have a directory with .json files describing your account in a directory named after account name.
python cloudmapper.py collect --account-name my_account
collect: Collect metadata about an account. More details here.find_admins: Look at IAM policies to identify admin users and roles and spot potential IAM issues. More details here.prepare/webserver: See Network Visualizationspublic: Find public hosts and port ranges. More details here.sg_ips: Get geoip info on CIDRs trusted in Security Groups. More details here.stats: Show counts of resources for accounts. More details here.wot: Show Web Of Trust. More details here.
- cytoscape.js: MIT https://github.com/cytoscape/cytoscape.js/blob/master/LICENSE
- cytoscape.js-qtip: MIT https://github.com/cytoscape/cytoscape.js-qtip/blob/master/LICENSE
- cytoscape.js-grid-guide: MIT https://github.com/iVis-at-Bilkent/cytoscape.js-grid-guide
- cytoscape.js-panzoom: MIT https://github.com/cytoscape/cytoscape.js-panzoom/blob/master/LICENSE
- jquery: JS Foundation https://github.com/jquery/jquery/blob/master/LICENSE.txt
- jquery.qtip: MIT https://github.com/qTip2/qTip2/blob/master/LICENSE
- cytoscape-navigator: MIT https://github.com/cytoscape/cytoscape.js-navigator/blob/c249bd1551c8948613573b470b30a471def401c5/bower.json#L24
- cytoscape.js-autopan-on-drag: MIT https://github.com/iVis-at-Bilkent/cytoscape.js-autopan-on-drag
- font-awesome: MIT http://fontawesome.io/
- FileSave.js: MIT https://github.com/eligrey/FileSaver.js/blob/master/LICENSE.md
- circular-json: MIT https://github.com/WebReflection/circular-json/blob/master/LICENSE.txt
- rstacruz/nprogress: MIT https://github.com/rstacruz/nprogress/blob/master/License.md
- mousetrap: Apache https://github.com/ccampbell/mousetrap/blob/master/LICENSE
- akkordion MIT https://github.com/TrySound/akkordion/blob/master/LICENSE