Adds --staging flag create-dev-data.py for tests #3672

kushaldas · 2018-07-31T17:03:49Z

We now have --staging flag to the create-dev-data.py script so
that we can easily create an user in the staging or prodcution test
and then use the functional tests to test the instance.

Status

Ready for review.

Description of Changes

Fixes #3671

Changes proposed in this pull request:

Testing

On a staging or prod vm.

sudo -u www-data bash
cd /var/wwww/securedrop/
./manage.py reset # This will clean the DB for testing
./create-dev-data.py --staging

Deployment

Any special considerations for deployment? Consider both:

Upgrading existing production instances.
New installs.

Checklist

If you made changes to the server application code:

Linting (make ci-lint) and tests (make -C securedrop test) pass in the development container

If you made changes to `securedrop-admin`:

Linting and tests (make -C admin test) pass in the admin development container

If you made changes to the system configuration:

Configuration tests pass

If you made non-trivial code changes:

I have written a test plan and validated it for this PR

If you made changes to documentation:

Doc linting (make docs-lint) passed locally

redshiftzero · 2018-07-31T23:12:13Z

the diff looks good here, @zenmonkeykstop can you run through the test plan when you get a chance?

redshiftzero · 2018-07-31T23:29:01Z

Actually @kushaldas @zenmonkeykstop check out #3678 which I hit during provisioning of staging on this PR, let me know if you reproduce that (presumably the var there should be tbb_directory)

conorsch · 2018-08-01T19:53:59Z

Rebased on top of latest tbb-0.9.0 (which was just rebased on top of latest develop).

We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance.

msheiny · 2018-08-09T19:03:50Z

Rebased on tbb-0.9.0 (a4c0252be752fe48224cd080559a1b66e051b761)

msheiny

👍

One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job)

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality Merge from develop

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality Merge from develop

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality Merge from develop

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality Merge from develop

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality Merge from develop

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality Merge from develop

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality Merge from develop

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality Merge from develop

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality Merge from develop

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: freedomofpress#3691 freedomofpress#3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In freedomofpress#3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in freedomofpress#3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect freedomofpress#3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality

@msheiny

stupid utility to create an test admin user fast Working tbselineum tests for most part. fixes two typos in readme of tor based tests Fixes directory path in README Hardcoded onion addresse work Test for tor broser asking high security in slider Reads instance information from a json file Uses clean variable name Configure the sleep amount between clicks Just tests admin/journalist login and cookies Tests working once again. Use firefox to login for downloads in selenium tests Updates README Matches with 0.6-rc2 functional tests We are using TBB 7.5 We need more time and click to test logout Fixes the assert statement Updates based on current upstream TB functional tests: Gitignore instance config This prevents the accidental commit of private information. TB functional tests: Merge tbselenium dir Updates the ansible files for tbb Updates to the Dockerfile for tbb Updates as suggested in the PR review for ansible and dockerfile Missing tor key Removes duplicated 'when' line from app-test logic The 'when' conditional detecting a grsec kernel, used for running the paxctl commands on the TBB binary, was needlessly duplicated on the relevant task. Fortunately that didn't cause breakage, because the 'when' lines were identical, but only one was active. Updates functional test container image Now installs Firefox 52 ESR, rather than Firefox 46, for use inside the test container. Includes changes to run-test shell script: TOR_FORCE_NET_CONFIG=0 is required to directly connect to Tor network, otherwise it will wait for userinput to either connect or to configure The `run_xvfb` invocation is no longer necessary, since the test suite code bootstraps the headless server now. Creates local test server inside the container Bootstrapping the application services within the functional test suite. Includes some cleanup, culling unused debugging code, and also cleans up the various print statements. Ignore functional test firefox logs (thanks, @msheiny!) Adds retries for tor network connection failure, using the pre-existing logic. Uses nc rather than torsocks in functional tests The version of torsocks in the Trusty repos isn't recent enough to support custom ports. Rather than install from other sources, which requires manual package verification (or configuring non-trusty repos, which could break other packages), let's fall back to good ol' nc. Creates proper orbot specific project to test We need to create a new firefox profile to test the orbot specific warning. This works for both locally and over Tor. Updates test_make_account_changes for Tor Now we can safely execute the account changes in the tests running on the Tor browser. The logic update makes sure to create different user for this test than any other test. Reduces sleep durations in functional tests We don't have to sleep for too long if we are running against local instance. The ultimate goal remains to remove hardcoded sleeps altogether, but we'll circle back and eliminate those calls once the test suite is passing reliably. Increases CircleCI timeout to 20m The functional tests can take a long time, so let's instruct CircleCI to continue waiting, to give the test suite a chance to finish successfully. Updates README for the functional tests Mostly correcting a typo in the `instance_information.json` config filename, but also updated some of the example commands. The notes regarding potentially failing tests also seemed out of date, as several members of the team have confirmed working functional tests under the new TB Selenium logic recently. Removes temporary testing related directories We need to clean up any temporary test directory before running any test, as the previous can create a bad state in the database. For example, one of our pages-layout test adds 123456 as hotp value to the test account, and it will never be able to login again. Adds missing steps for pages-layout tests over tor We can now generate thousands of random journalist names using the generator. Only the first 3 names are used in the functional tests, the rest are being used in the pages-layout. We also added back _source_delete_key function for one the test. Rest of the updates are to add sleep function calls or to have better error message. Adds comment about the user generator and lint fixes We have get_journalist_usernames generator which can generate unlimited number of users for many tests. Moves around all driver creation functions into one place We now have the functional/functional_test.py to handle all driver creation logic. For the pages-layout tests, we are creating only a Firefox driver to connect to the local container itself. This logic does not work against any external server as of now. Updates test user creation logic inside of container Removing old method calls from user creation logic, this is only used inside of the container for the functional testing. Downloads data from server using requests over Tor The test requirements now have requests[socks] as dependency. Using the same we are now directly downloading the files/messages from the .onion address for functional tests. The old external command file also got removed this committ. We are creating the gpg object for both container based local testing and external testing (in functional tests). Fixes: #3691 #3687 Removed xvfb, tor browser, and firefox installs from app-staging - functional tests now run remotely fix to pass make ci-lint Cleans up flake8 errors in functional tests Resolves some lingering flake8 formatting violations that were causing lint checks to fail. These changes are unrelated to the current PR, but better late than never. Written by @msheiny, committed by @conorsch during branch collab. Signed-off-by: Conor Schaefer <[email protected]> Adds --staging flag create-dev-data.py for tests We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance. Add fact gatherer for extracting tor app onion details This is really for functional testing in staging environments, but the script doesnt hurt to be installed in prod. It doesn't elevate permissions or expose any sensitive details - you need to run as root in order to gain useful data. Logic to dump app tor onion data to func config This commit adds logic to the test runner so that a functional test json config gets dumped for usage in the tbb selenium test tools against staging. Wire-up current app-test role to our upgrade scenario Without this change, the upgrade scenario would utilize the app-test logic from the old repo. Which is not what we want. This is of course kind of "hacky". I welcome the opportunity to improve this with feedback :) Move tor fact logic from tor role --> app-test role I'm not sure we are ready to shove this on prod instances AND I was running into weird old/new role logic under the molecule scenario. Ideally the fact should be in both roles but I dunno.. this seemed like an easier short-term fix. ansible spacing' and tag nits Testinfra: Update test dependencies for app-staging In #3697, we removed the application testing pip dependencies. This commit updates the testinfra test variables accordingly. Fix linting failures One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job) Use absolute pathing in i18n testing I'm hoping this shakes out some really weird test failures we were seeing only under CircleCI only under the functional testing branch at a certain point in time. Really wild behavior. *fingers crossed* Set selenium webdriver test output to WARNING Originally was on DEBUG and was sending out mountains of output into the pytest process which made it difficult to assess anything. (cherry picked from commit a1f0134) removed duplicate entry in test requirements Dockerfile: Update Tor Browser to 8.0 Docker development environment: Update Tor signing key Dockerfile: Update geckodriver and firefox-esr Swap x11vnc with tightvncserver Basically installed this because it can be used with pyvirtualdisplay as a backend AND because it brings in the Xvnc tooling which will start an X11 server as well as a VNC server. Wire-up VNC server and helper command for func tests Had to remove x11 display logic inside test scaffolding (initially tried to integrate it there but it kept building and destroying the VNC server per test). Made a VNC helper command with support for GNOME desktop and macOS (havent tested it on mac yet). Updated the docs Bump functional test sleep time 10 seconds is way too short.. 160 seconds.. maybe too long? Fingers crossed I can work with the team to get the wait_for logic running Add functionality to prepare boxes for functional testing Typically these actions were done manually but lets get our good old friend ansible to run them for us (at least under the upgrade env). Added auth to VNC in test container, for OS X compatibility. Bump TBB/ESR to 8.0.1 and 60.2.0esr combo https://blog.torproject.org/new-release-tor-browser-801 Updated geckodriver to 0.22.0 added ini file to get around remote-viewer password prompt Functional tests: xfail test_warning_appears_if_tor_browser_not_in_use Due to defect #3793, when using Firefox Quantum, the incorrect message is displayed on the source interface. This test will not pass until that is resolved. Functional tests: Fix firefox path Testinfra: Fix test failure due to non-DRY variables files staging.yml is a concatenation of multiple other variables files, one was updated during rebase, one was not. Testinfra: Update Flask version to 1.0.2 fixed VNC port being defined twice when running 'make dev' Tests: Modify viewport size for parity between dev and CI env In CI we are getting MoveTargetOutOfBoundsException, but not locally. We have had errors in the past due to different viewport sizes in CI and locally, so setting this to a standard size for the pages layout tests (where the exception is occurring). Functional tests: Resolve NoAlertPresentException We were getting a NoAlertPresentException due to new behavior in geckodriver [0] where interacting with the driver closes the modal. Thus, we do not need to explicitly accept the modal here. [0] mozilla/geckodriver#1171 Replaced parameterized time.sleep()s with self.waitfor()s Stability fixes layout tests, added new functests from develop, updated TBB fixed flaky page layout tests that were broken by long fr_FR strings causing UI elements to wrap. fixed flake8, added explicit scroll to elements before click, stability fixes Dockerfile: get key from Mozilla keyserver For whatever reason, this Firefox signing key was not available on the keyserver in the prior diff, but was available on Mozilla's keyserver. deps: Update requests due to CVE-2018-18074 functional tests: Add wait_for prior to clicking submit functional tests: Remove sleeps and reduce flakiness around modal functional tests: Remove sleeps around js alerts functional tests: use sleep_time as default timeout functional tests: Remove remainder of time.sleeps in source steps Don't clobber existing custom logo Minimal changes to remove external server testing functionality

kushaldas requested review from heartsucker and redshiftzero as code owners July 31, 2018 17:03

conorsch force-pushed the tbb-0.9.0 branch from 4dec807 to c320cd6 Compare August 1, 2018 19:11

conorsch force-pushed the create_stage_user branch from 04b5b85 to e7de6d2 Compare August 1, 2018 19:53

This was referenced Aug 7, 2018

[functional testing] Support tor-browser-selenium tests against app-staging in CI #3661

Open

Using pure python code to download and verify gpg encrypted files/messages #3692

Merged

conorsch force-pushed the tbb-0.9.0 branch from 46aebc0 to 99f3004 Compare August 8, 2018 22:26

Adds --staging flag create-dev-data.py for tests

b08a0d7

We now have --staging flag to the create-dev-data.py script so that we can easily create an user in the staging or prodcution test and then use the functional tests to test the instance.

msheiny force-pushed the create_stage_user branch from e7de6d2 to b08a0d7 Compare August 9, 2018 19:02

msheiny approved these changes Aug 13, 2018

View reviewed changes

msheiny merged commit 1eb2895 into freedomofpress:tbb-0.9.0 Aug 13, 2018

msheiny deleted the create_stage_user branch August 13, 2018 15:14

eloquence mentioned this pull request Aug 14, 2018

[functional testing] create-dev-data.py should get one user for stage based testing #3671

Closed

redshiftzero added a commit that referenced this pull request Aug 15, 2018

Fix linting failures

8ff5d5a

One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job)

redshiftzero added a commit that referenced this pull request Aug 15, 2018

Fix linting failures

f719716

One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job)

conorsch pushed a commit that referenced this pull request Sep 12, 2018

Fix linting failures

686bf2e

One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job)

msheiny pushed a commit that referenced this pull request Oct 5, 2018

Fix linting failures

ad61b3b

One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job)

redshiftzero added a commit that referenced this pull request Oct 12, 2018

Fix linting failures

330e5e3

One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job)

redshiftzero added a commit that referenced this pull request Nov 30, 2018

Fix linting failures

93e6103

One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job)

redshiftzero added a commit that referenced this pull request Dec 21, 2018

Fix linting failures

c989da4

One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job)

redshiftzero added a commit that referenced this pull request Dec 21, 2018

Fix linting failures

f65df67

One of these was introduced in #3672, but not discovered due to other CI failures (e.g. python not found when running the lint job)

eloquence mentioned this pull request Mar 21, 2019

Port support for creating dev data in staging VMs from tbb-0.9.0 to develop #4286

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds --staging flag create-dev-data.py for tests #3672

Adds --staging flag create-dev-data.py for tests #3672

kushaldas commented Jul 31, 2018

redshiftzero commented Jul 31, 2018

redshiftzero commented Jul 31, 2018

conorsch commented Aug 1, 2018

msheiny commented Aug 9, 2018

msheiny left a comment

Adds --staging flag create-dev-data.py for tests #3672

Adds --staging flag create-dev-data.py for tests #3672

Conversation

kushaldas commented Jul 31, 2018

Status

Description of Changes

Testing

Deployment

Checklist

If you made changes to the server application code:

If you made changes to securedrop-admin:

If you made changes to the system configuration:

If you made non-trivial code changes:

If you made changes to documentation:

redshiftzero commented Jul 31, 2018

redshiftzero commented Jul 31, 2018

conorsch commented Aug 1, 2018

msheiny commented Aug 9, 2018

msheiny left a comment

Choose a reason for hiding this comment

If you made changes to `securedrop-admin`: