Skip to content

Local Development

Jump to bottom
thebleucheese edited this page May 4, 2022 · 1 revision

How to Locally Develop atomic-operator with external logger modules

WARNING: Given the difficulty involved in setup, this may not be the right way of developing different module types together. However, Poetry / requirements.txt cross-module local development still seems to be an unsettled science so this is the best I could come up with.

atomic-operator uses setup.py and requirements.txt. To develop locally with projects that use Poetry we had to make some modifications.

Setup Local atomic-operator Dev Environment

There are a few steps to getting this working properly. I’ll break them down into major groups:

Prerequisites:

  • When following the directions below, make sure your atomic-operator and atomic-operator-attire-logger repositories are in the same directory, it will make things easier.

Get a local atomic-operator checkout

  1. Clone the project: git clone <YOUR_ATOMIC_OPERATOR_FORK_REPO>
  2. Checkout your working dev branch. git checkout feature/<YOUR_FEATURE_BRANCH>
  3. Open atomic-operator project in your IDE of choice. If pyCharm, allow it to create a virtualenv(venv). You may need to run poetry install in the venv after venv creation. Check the terminal in pyCharm to make sure this is working properly. It should look something like the following:
(venv) devuser@mycomputer ~/dev/python/atomic-operator $
  1. For the rest of this document we’ll refer to this particular (atomic-operator’s) virtualenv as (AOVENV) to try for clarity.

Get a local atomic-operator-attire-logger checkout

  1. Clone the project: git clone [email protected]:SecurityRiskAdvisors/atomic-operator-attire-logger.git
  2. Checkout your working branch… similar to above: git checkout feature/<YOUR_FEATURE_BRANCH>
  3. Open atomic-operator-attire-logger project in your IDE of choice. If pyCharm, allow it to create a virtualenv(venv). In the pyCharm terminal you should see something like:
(atomic-operator-attire-logger-t2Q5zBJ0-py3.8) devuser@mycomputer ~/dev/python/atomic-operator-attire-logger $
  1. For the rest of this document we’ll refer to this particular (atomic-operator-attire-logger’s) virtualenv as (LOGGERVENV).
  2. Add two new files to the atomic-operator-attire-logger project root.
  3. Add setup.py
# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['atomic_operator_attire_logger']

package_data = \
{'': ['*']}

setup_kwargs = {
    'name': 'atomic-operator-attire-logger',
    'version': '1.1.0',
    'description': '',
    'long_description': None,
    'author': 'Your Name',
    'author_email': '[email protected]',
    'maintainer': None,
    'maintainer_email': None,
    'url': None,
    'packages': packages,
    'package_data': package_data,
    'python_requires': '>=3.6,<4.0',
}

setup(**setup_kwargs)
  1. Add requirements.txt
setuptools~=57.0.0
simplejson~=3.17.6
  1. Comment out the build system section of pyproject.toml

AFTER THIS POINT SOME OF THE STEPS MAY NOT BE NEEDED. DOCUMENTING THEM HERE IN CASE THEY NEED TO BE DONE AGAIN.

Connect Dependencies

Note: dependencies are imported here. atomic-operator-attire-logger depends on atomic-operator since it uses some enum’s (log parameter values). atomic-operator-attire-logger will be installed in atomic-operator’s venv

  1. In the atomic-operator-attire-logger venv (LOGGERVENV) I needed to add a dependency on local changes in atomic-operator. This is only needed if changes to the main project are required. I added the following to pyproject.toml (so it's already in there in the branch):
[tool.poetry.dev-dependencies]
atomic-operator = { path = "../atomic-operator/", develop = true }
  1. (MAY NOT BE NEEDED) I couldn’t get poetry to add this, it complained about a missing dependency, pick, so I ran poetry add -D pick==1.2.0 in (LOGGERVENV). This automatically ran poetry install and setup everything needed. You may need to run poetry install here to get everything setup if changes are made to the pyproject.toml file.
  2. (NOT NEEDED IF NOT CHANGING py-attire-schema) If modifying py-attire-schema, it must be added as a dependency in atomic-operator-attire-logger venv (LOGGERVENV). Clone and checkout py-attire-schema locally. Modifying the atomic-operator-attire-logger to add a local path dependency on py-attire-schema like shown in step 1 for atomic-operator. Add a setup.py and requirements.txt similar to used in atomic-operator-attire-logger in steps above. Then cd ../py-attire-schema to move up a directory and into the py-attire-schema directory. Install this in the (LOGGERVENV) by running pip3 install -e .

Check if atomic-operator works

  1. From your atomic-operator virtual env (AOVENV) pyCharm terminal, exec pip3 install -e .
  2. Get the atomics needed. atomic-operator get_atomics
  3. Now we need to install atomic-operator-attire-logger as a local pip dependency. cd ../atomic-operator-attire-logger to move up a directory and into the atomic-operator-attire-logger directory. Install this in the (AOVENV) by running pip3 install -e .
  4. Change directory back to atomic-operator by running cd ../atomic-operator
  5. Test atomic-operator. On linux an easy test is to run atomic-operator run --techniques T1087.001 --select_tests and hit spacebar and enter on the first interactive option.

If all this worked, you should be able to do local development without publishing modules for every change. You can edit your atomic-operator-attire-logger and atomic-operator projects in separate pyCharm windows and run atomic-operator commands to test changes.

Prior to making a PR remove setup.py and requirements.txt from atomic-operator-attire-logger (and possibly py-attire-schema). Uncomment build system in pyproject.toml

Clone this wiki locally