Skip to content

Commit

Permalink
Move tests into an "ops" subdir (#935)
Browse files Browse the repository at this point in the history
* Create __init__.py

* move all of the test files down one dir into `ops`

* update test paths

* update paths where needed

* update the changelog
  • Loading branch information
ThomasLaPiana authored Jul 27, 2022
1 parent 12d5eb7 commit 031e2b8
Show file tree
Hide file tree
Showing 179 changed files with 53 additions and 48 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ The types of changes are:
## [Unreleased](https://github.com/ethyca/fidesops/compare/1.6.3...main)

### Added

* Erasure support for Salesforce [#888](https://github.com/ethyca/fidesops/pull/888)
* Access and erasure support for Sendgrid contacts endpoint [#883](https://github.com/ethyca/fidesops/pull/883)
* Added saas config base info to connection config responses [#904](https://github.com/ethyca/fidesops/pull/904)
Expand All @@ -33,6 +34,7 @@ The types of changes are:
* Updated the python docker base image from slim-buster to slim-bullseye [928](https://github.com/ethyca/fidesops/pull/928)
* Removed ipython from the docker install [928](https://github.com/ethyca/fidesops/pull/928)
* Serve admin UI by default [#906](https://github.com/ethyca/fidesops/pull/936)
* Move tests into an `ops` subdir [#935](https://github.com/ethyca/fidesops/pull/935)
* Updated the docs docker base image to be consistent with the fidesops image [949](https://github.com/ethyca/fidesops/pull/949)

### Breaking Changes
Expand Down
8 changes: 4 additions & 4 deletions docs/fidesops/docs/development/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ cd fidesops
pytest
```

2. From regular command-line:
2. From regular command-line:

```bash
make pytest
Expand All @@ -69,11 +69,11 @@ make pytest
To run a subset of tests, provide a filename or directory; to match a specific test name, use the `-k` flag:

```bash
# run all tests in the tests/integration directory that contain the word "api" in their title
pytest tests/integration/ -k api
# run all tests in the tests/ops/integration directory that contain the word "api" in their title
pytest tests/ops/integration/ -k api
```

Other commands you may need are listed below. The full documentation can be found at: https://docs.pytest.org/en/6.2.x/.
Other commands you may need are listed below. The full documentation can be found at: <https://docs.pytest.org/en/6.2.x/>.

- Run all tests in a directory: `make pytestpath=path/to/dir/ pytest`
- Run all tests in a file: `make pytestpath=path/to/file.py pytest`
Expand Down
4 changes: 3 additions & 1 deletion scripts/run_infrastructure.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,9 @@ def seed_initial_data(
_run_cmd_or_err('echo "Seeding initial data for all datastores..."')
for datastore in datastores:
if datastore in DOCKERFILE_DATASTORES:
setup_path = f"tests/integration_tests/setup_scripts/{datastore}_setup.py"
setup_path = (
f"tests/ops/integration_tests/setup_scripts/{datastore}_setup.py"
)
_run_cmd_or_err(
f'echo "Attempting to create schema and seed initial data for {datastore} from {setup_path}..."'
)
Expand Down
File renamed without changes.
1 change: 1 addition & 0 deletions tests/ops/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
)
from fidesops.models.connectionconfig import ConnectionConfig
from fidesops.models.policy import PolicyPostWebhook, PolicyPreWebhook
from tests.api.v1.endpoints.test_privacy_request_endpoints import stringify_date
from tests.ops.api.v1.endpoints.test_privacy_request_endpoints import stringify_date


def embedded_http_connection_config(connection_config: ConnectionConfig) -> Dict:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
ConnectionConfig,
ConnectionType,
)
from tests.api.v1.endpoints.test_dataset_endpoints import _reject_key
from tests.ops.api.v1.endpoints.test_dataset_endpoints import _reject_key


@pytest.mark.unit_saas
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
V1_URL_PREFIX,
)
from fidesops.core.config import config
from tests.conftest import generate_auth_header_for_user
from tests.ops.conftest import generate_auth_header_for_user

page_size = Params().size

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
)
from fidesops.models.datasetconfig import DatasetConfig
from fidesops.util.saas_util import load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("adobe_campaign")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@
from fidesops.schemas.saas.shared_schemas import HTTPMethod, SaaSRequestParams
from fidesops.service.connectors import SaaSConnector
from fidesops.util.saas_util import format_body, load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.test_helpers.saas_test_utils import poll_for_existence
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.test_helpers.saas_test_utils import poll_for_existence
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("hubspot")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@
from fidesops.schemas.saas.shared_schemas import HTTPMethod, SaaSRequestParams
from fidesops.service.connectors.saas_connector import SaaSConnector
from fidesops.util.saas_util import load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("mailchimp")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
)
from fidesops.models.datasetconfig import DatasetConfig
from fidesops.util.saas_util import load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("outreach")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@
)
from fidesops.models.datasetconfig import DatasetConfig
from fidesops.util.saas_util import load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("salesforce")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@
)
from fidesops.models.datasetconfig import DatasetConfig
from fidesops.util.saas_util import load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.test_helpers.saas_test_utils import poll_for_existence
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.test_helpers.saas_test_utils import poll_for_existence
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("segment")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@
ConnectionType,
)
from fidesops.models.datasetconfig import DatasetConfig
from tests.fixtures.application_fixtures import load_dataset
from tests.fixtures.saas_example_fixtures import load_config
from tests.test_helpers.saas_test_utils import poll_for_existence
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.fixtures.saas_example_fixtures import load_config
from tests.ops.test_helpers.saas_test_utils import poll_for_existence
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("sendgrid")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@
)
from fidesops.models.datasetconfig import DatasetConfig
from fidesops.util.saas_util import load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("sentry")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
)
from fidesops.models.datasetconfig import DatasetConfig
from fidesops.util.saas_util import load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("stripe")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
)
from fidesops.models.datasetconfig import DatasetConfig
from fidesops.util.saas_util import load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.test_helpers.vault_client import get_secrets
from tests.ops.fixtures.application_fixtures import load_dataset
from tests.ops.test_helpers.vault_client import get_secrets

saas_config = load_toml(["saas_config.toml"])
secrets = get_secrets("zendesk")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
OAuth2AuthenticationConfiguration,
)
from fidesops.util.saas_util import load_config
from tests.fixtures.application_fixtures import load_dataset
from tests.ops.fixtures.application_fixtures import load_dataset

saas_config = load_toml(["saas_config.toml"])

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from fidesops.models.policy import ActionType
from fidesops.task.graph_task import retry
from fidesops.task.task_resources import TaskResources
from tests.task.traversal_data import integration_db_graph
from tests.ops.task.traversal_data import integration_db_graph

t1 = Collection(
name="t1",
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from fidesops.service.connectors import get_connector
from fidesops.task import graph_task
from fidesops.task.graph_task import get_cached_data_for_erasures
from tests.graph.graph_test_util import assert_rows_match
from tests.ops.graph.graph_test_util import assert_rows_match


@pytest.mark.skip(reason="Only staging credentials available")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
from fidesops.task.filter_results import filter_data_categories
from fidesops.task.graph_task import get_cached_data_for_erasures
from fidesops.util.saas_util import format_body
from tests.graph.graph_test_util import assert_rows_match, records_matching_fields
from tests.ops.graph.graph_test_util import assert_rows_match, records_matching_fields


@pytest.mark.integration_saas
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from fidesops.schemas.redis_cache import PrivacyRequestIdentity
from fidesops.task import graph_task
from fidesops.task.graph_task import get_cached_data_for_erasures
from tests.graph.graph_test_util import assert_rows_match, records_matching_fields
from tests.ops.graph.graph_test_util import assert_rows_match, records_matching_fields


@pytest.mark.integration_saas
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from fidesops.task import graph_task
from fidesops.task.filter_results import filter_data_categories
from fidesops.task.graph_task import get_cached_data_for_erasures
from tests.graph.graph_test_util import assert_rows_match
from tests.ops.graph.graph_test_util import assert_rows_match


@pytest.mark.skip(reason="Currently unable to test OAuth2 connectors")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from fidesops.service.connectors import get_connector
from fidesops.task import graph_task
from fidesops.task.graph_task import get_cached_data_for_erasures
from tests.graph.graph_test_util import assert_rows_match
from tests.ops.graph.graph_test_util import assert_rows_match


@pytest.mark.integration_saas
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from fidesops.task import graph_task
from fidesops.task.filter_results import filter_data_categories
from fidesops.task.graph_task import get_cached_data_for_erasures
from tests.graph.graph_test_util import assert_rows_match
from tests.ops.graph.graph_test_util import assert_rows_match


@pytest.mark.skip(reason="Pending account resolution")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@
from fidesops.schemas.redis_cache import PrivacyRequestIdentity
from fidesops.task import graph_task
from fidesops.task.graph_task import get_cached_data_for_erasures
from tests.fixtures.saas.sendgrid_fixtures import contact_exists
from tests.graph.graph_test_util import assert_rows_match
from tests.test_helpers.saas_test_utils import poll_for_existence
from tests.ops.fixtures.saas.sendgrid_fixtures import contact_exists
from tests.ops.graph.graph_test_util import assert_rows_match
from tests.ops.test_helpers.saas_test_utils import poll_for_existence


@pytest.mark.integration_saas
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
from fidesops.task import graph_task
from fidesops.task.filter_results import filter_data_categories
from fidesops.task.graph_task import get_cached_data_for_erasures
from tests.graph.graph_test_util import assert_rows_match
from tests.test_helpers.saas_test_utils import poll_for_existence
from tests.ops.graph.graph_test_util import assert_rows_match
from tests.ops.test_helpers.saas_test_utils import poll_for_existence


@pytest.mark.skip(reason="Pending account resolution")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from fidesops.task import graph_task
from fidesops.task.filter_results import filter_data_categories
from fidesops.task.graph_task import get_cached_data_for_erasures
from tests.graph.graph_test_util import assert_rows_match
from tests.ops.graph.graph_test_util import assert_rows_match


@pytest.mark.integration_saas
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from fidesops.schemas.redis_cache import PrivacyRequestIdentity
from fidesops.task import graph_task
from fidesops.task.graph_task import get_cached_data_for_erasures
from tests.graph.graph_test_util import assert_rows_match
from tests.ops.graph.graph_test_util import assert_rows_match


@pytest.mark.integration_saas
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from tests.test_helpers.dataset_utils import generate_collections, generate_dataset
from tests.ops.test_helpers.dataset_utils import generate_collections, generate_dataset


class TestGenerateCollections:
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 comments on commit 031e2b8

Please sign in to comment.