Remove separate config for island logger #1151

shreyamalviya · 2021-05-10T11:16:46Z

Partial fix for #1146

PR Checklist

Have you added an explanation of what your changes do and why you'd like to include them?
Is the TravisCI build passing?
Was the CHANGELOG.md updated to reflect the changes?
~~Was the documentation framework updated to reflect the changes?~~

Testing Checklist

Added relevant unit tests?
Have you successfully tested your changes locally? Elaborate:

Ran monkey_island.py with --server-config and without
If applicable, add screenshots or log transcripts of the feature working

…rver_config.json

…gger

…land/cc/server_utils/test_island_logger.py)

codecov · 2021-05-10T11:16:56Z

Codecov Report

Merging #1151 (b13839d) into develop (aa959c3) will increase coverage by 0.04%.
The diff coverage is 70.73%.

@@             Coverage Diff             @@
##           develop    #1151      +/-   ##
===========================================
+ Coverage    28.62%   28.67%   +0.04%     
===========================================
  Files          411      412       +1     
  Lines        12742    12755      +13     
===========================================
+ Hits          3648     3658      +10     
- Misses        9094     9097       +3

Impacted Files	Coverage Δ
monkey/monkey_island.py	`0.00% <0.00%> (ø)`
monkey/monkey_island/cc/arg_parser.py	`0.00% <0.00%> (ø)`
monkey/monkey_island/cc/server_utils/consts.py	`100.00% <ø> (ø)`
...key/monkey_island/cc/server_utils/island_logger.py	`100.00% <100.00%> (+9.09%)`	⬆️
monkey/monkey_island/config_loader.py	`100.00% <100.00%> (ø)`
monkey/tests/conftest.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa959c3...b13839d. Read the comment docs.

mssalvatore · 2021-05-10T12:50:36Z

monkey/monkey_island/cc/server_utils/island_logger.py

+    if not os.path.exists(data_dir_path):
+        os.makedirs(data_dir_path, mode=0o700, exist_ok=True)


This should not be the responsibility of the logger. The logger can assume the directory it's been configured to write to exists.

mssalvatore · 2021-05-10T12:56:14Z

monkey/monkey_island/cc/server_utils/island_logger.py

 from typing import Dict

-from monkey_island.cc.server_utils.consts import DEFAULT_LOGGER_CONFIG_PATH
-
 __author__ = "Maor.Rayzin"


Remove this __author__ field.

mssalvatore · 2021-05-10T12:58:38Z

monkey/monkey_island.py

+        server_config_path = os.path.expanduser(island_args.server_config)
+        if not Path(server_config_path).is_file():
+            server_config_generator.create_default_config_file(server_config_path)


If the user has a typo in their command-line argument, this will use a config other than the one they specified and pretend like everything is OK. Monkey Island will run with the wrong config and the user will waste time trying to figure out why the island isn't behaving as expected.

Just read the file. Trap any OSErrors and print a useful message for the user.

mssalvatore · 2021-05-10T13:02:50Z

monkey/monkey_island/cc/server_utils/consts.py

-DEFAULT_LOGGER_CONFIG_PATH = os.path.join(
-    MONKEY_ISLAND_ABS_PATH, "cc", "island_logger_default_config.json"
-)
+DEFAULT_LOG_LEVEL = "NOTSET"


Why not DEFAULT or INFO? Also, if this is only used in monkey_island.py, it can probably be moved there.

mssalvatore · 2021-05-10T13:09:51Z

monkey/monkey_island.py

+        with open(server_config_path, "r") as f:
+            config_content = f.read()
+        data = json.loads(config_content)
+        data_dir = os.path.abspath(
+            os.path.expanduser(
+                os.path.expandvars(data["data_dir"] if "data_dir" in data else DEFAULT_DATA_DIR)
+            )
+        )
+        log_level = data["log_level"] if "log_level" in data else DEFAULT_LOG_LEVEL


This can probably be a separate function called load_server_config() or something similar.

Make sure to write tests for load_server_config() that test all of the possible behaviors.

mssalvatore · 2021-05-10T13:16:18Z

monkey/tests/monkey_island/cc/server_utils/test_island_logger.py

+def test_expanduser_filename(mock_home_env, tmpdir):
+    DATA_DIR = tmpdir
+    INFO_LOG = os.path.join(DATA_DIR, "monkey_island.log")
+    LOG_LEVEL = "DEBUG"


Since this is testing the call to logger.info(), let's make this log level INFO. Or we could leave this as DEBUG and call logger.debug() instead.

We should also add another test where we set the log level to INFO, call logger.debug(), and verify that the file does not contain the line.

mssalvatore · 2021-05-10T13:19:55Z

monkey/tests/monkey_island/cc/server_utils/test_island_logger.py

@@ -17,11 +12,13 @@ def mock_home_env(monkeypatch, tmpdir):
    monkeypatch.setenv("HOME", str(tmpdir))


I don't think this fixture is needed anymore since we're passing data_dir directly to the logger configuration function..

…key_island.py

ghost · 2021-05-11T13:39:19Z

DeepCode's analysis on #b13839 found:

ℹ️ 1 minor issue. 👇

Top issues

Description	Example fixes
Replace exit with sys.exit. Occurrences: monkey_island.py:27 monkey_island.py:31	🔧 Example fixes

👉 View analysis in DeepCode’s Dashboard | Configure the bot

Because `monkey_island.py` has the same name as the `monkey_island` module, pytest can't import monkey_island.py and run any tests against its code.

As the number of configuration items will increase in the future, return the config dict instead of individual config properties.

shreyamalviya added 5 commits May 10, 2021 12:23

Remove `island_logger_config.json\'

5bcd917

Remove --logger-config command-line argument, add "log_level" to se…

ab89590

…rver_config.json

Replace json_setup_logging() with setup_logging() to configure lo…

785f2ef

…gger

Set log filename in config before expanding its paths

f84e4ae

Fix unit tests and modify code based on failed tests (tests/monkey_is…

6d04e7c

…land/cc/server_utils/test_island_logger.py)

Update CHANGELOG.md (removed island logger config)

0556465

shreyamalviya requested review from mssalvatore and VakarisZ and removed request for mssalvatore May 10, 2021 11:19

mssalvatore requested changes May 10, 2021

View reviewed changes

shreyamalviya added 7 commits May 11, 2021 17:56

Assume configured data directory exists when configuring the logger

8dc84ee

Remove "__author__" field

805ab98

Catch and print errors instead of creating a default server config

3c7687a

Move DEFAULT_LOG_LEVEL and add function load_server_config to mon…

e8c1c81

…key_island.py

Modify/add unit tests (test_island_logger.py)

c5ba48d

Add tests for console logging (test_island_logger.py)

5f8145e

Rename unit test functions in test_island_logger.py

83a235b

mssalvatore added 7 commits May 11, 2021 13:00

island: Move load_server_config() to config_loader.py

f7bef76

Because `monkey_island.py` has the same name as the `monkey_island` module, pytest can't import monkey_island.py and run any tests against its code.

island: Add unit test for load_server_config()

5847674

island: Return config dict from load_server_config()

990244c

As the number of configuration items will increase in the future, return the config dict instead of individual config properties.

island: Add tests for default server config values

de7865a

island: Simplify logic in add_default_values_to_config()

5ea241f

island: Handle lower case log levels in server config

08668f3

island: Add debug log level to server_config.json.develop

b13839d

mssalvatore merged commit 0b21dac into develop May 11, 2021

mssalvatore deleted the untangle-logger-config branch May 11, 2021 18:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove separate config for island logger #1151

Remove separate config for island logger #1151

shreyamalviya commented May 10, 2021

codecov bot commented May 10, 2021 •

edited

Loading

mssalvatore May 10, 2021

mssalvatore May 10, 2021 •

edited

Loading

mssalvatore May 10, 2021

mssalvatore May 10, 2021

mssalvatore May 10, 2021

mssalvatore May 10, 2021

mssalvatore May 10, 2021 •

edited

Loading

ghost commented May 11, 2021 •

edited by ghost

Loading

		if not os.path.exists(data_dir_path):
		os.makedirs(data_dir_path, mode=0o700, exist_ok=True)

		@@ -17,11 +12,13 @@ def mock_home_env(monkeypatch, tmpdir):
		monkeypatch.setenv("HOME", str(tmpdir))

Remove separate config for island logger #1151

Remove separate config for island logger #1151

Conversation

shreyamalviya commented May 10, 2021

PR Checklist

Testing Checklist

codecov bot commented May 10, 2021 • edited Loading

Codecov Report

mssalvatore May 10, 2021

Choose a reason for hiding this comment

mssalvatore May 10, 2021 • edited Loading

Choose a reason for hiding this comment

mssalvatore May 10, 2021

Choose a reason for hiding this comment

mssalvatore May 10, 2021

Choose a reason for hiding this comment

mssalvatore May 10, 2021

Choose a reason for hiding this comment

mssalvatore May 10, 2021

Choose a reason for hiding this comment

mssalvatore May 10, 2021 • edited Loading

Choose a reason for hiding this comment

ghost commented May 11, 2021 • edited by ghost Loading

DeepCode's analysis on #b13839 found:

Top issues

👉 View analysis in DeepCode’s Dashboard | Configure the bot

codecov bot commented May 10, 2021 •

edited

Loading

mssalvatore May 10, 2021 •

edited

Loading

mssalvatore May 10, 2021 •

edited

Loading

ghost commented May 11, 2021 •

edited by ghost

Loading