Use logging module instead of prints

.gitignore

@@ -5,6 +5,7 @@ tm_devices.yaml
 tm_devices.yml
 *.log
 *.dat
+**/logs/**
 
 # Poetry lock file
 poetry.lock

CHANGELOG.md

@@ -26,6 +26,12 @@ Things to be included in the next release go here.
 - Added the `get_errors()` method to the `Device` class to enable easy access to the current error code and messages on any device.
 - Added more details to the Architectural Overview page of the documentation as well as highlighting to the device driver diagram on the page.
 - Added regex matching to the `verify_values()` helper function to allow for more flexible value verification.
+- A main logfile is now created by default (can be disabled if desired) that contains all the logging output of the entire tm_devices package during execution.
+    - Use the `configure_logging()` function to set the logging levels for stdout and file logging.
+    - The default settings will log all messages to the log file and maintain the current printout functionality on stdout.
+- A logfile is now created that contains each command sent to a VISA device.
+    - This file is located next to the main log file and will start with the same name, but have the unique address of the device appended.
+    - This file will only be created if file logging is enabled for the package (which is the default behavior).
 
 ### Changed
 
@@ -46,17 +52,23 @@ However, please read through all changes to be aware of what may potentially imp
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Changed the behavior of the `expect_esr()` method to expect an integer error code input and an optional tuple of error messages to compare against the actual error code and messages returned by the `_get_errors()` private method.
 - _**<span style="color:orange">minor breaking change</span>**_: Converted the `device_type` property into an abstract, cached property to force all children of the `Device` class to specify what type of device they are.
 - Updated the auto-generated command mixin classes to no longer use an `__init__()` method to enable the driver API documentation to render in a more usable way.
+- Switched from using standard `print()` calls to using the `logging` module for all logging in the `tm_devices` package.
+    - A configuration function provides the ability to set different logging levels for stdout and file logging.
+    - The config file and environment variable can also be used to control the logging functionality.
+    - The debug logging from the `pyvisa` package is also included in the log file by default.
 
 ### Removed
 
-- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `TekScopeSW` alias to the `TekScopePC` class
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `TekScopeSW` alias to the `TekScopePC` class.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `write_buffers()` from the `TSPControl` class.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed Internal AFG methods from the `TekScopePC` driver, since they wouldn't have worked due to its lack of an IAFG.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `DEVICE_DRIVER_MODEL_MAPPING` constant.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `DEVICE_TYPE_CLASSES` constant and the `device_type_classes.py` module.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed many hacky implementations of `total_channels` and `all_channel_names_list` properties from drivers that don't need them anymore.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `verify_values()`, `raise_failure()`, and `raise_error()` methods from all device drivers.
     - These methods have been converted to helper functions and can be imported from the `tm_devices.helpers` subpackage now.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `print_with_timestamp()` function since this functionality is now handled by the `logging` module.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `get_timestamp_string()` function since this functionality is now handled by the `logging` module.
 
 ---
 

docs/basic_usage.md

@@ -59,6 +59,23 @@ outside the Python code for ease of automation
 --8<-- "examples/miscellaneous/adding_devices_with_env_var.py"
 ```
 
+## Customize logging and console output
+
+The amount of console output and logging saved to the log file can be customized as needed. This
+configuration can be done in the Python code itself as demonstrated here, or by using the
+[config file](configuration.md#config-options) or
+[environment variable](configuration.md#environment-variable).
+
+!!! important
+    If any configuration is performed in the Python code prior to instantiating the
+    [`DeviceManager`][tm_devices.DeviceManager], all other logging configuration methods
+    (config file, env var) will be ignored.
+
+```python
+# fmt: off
+--8<-- "examples/miscellaneous/customize_logging.py"
+```
+
 ## Disable command checking
 
 This removes an extra query that verifies the property was set to the expected

docs/configuration.md

@@ -215,8 +215,7 @@ devices:
 
 ### Config Options
 
-These options are flags that enable/disable runtime behaviors of the Device
-Manager.
+These options are used to configure runtime behaviors of `tm_devices`.
 
 #### Yaml Options Syntax
 
@@ -230,6 +229,12 @@ options:
   retry_visa_connection: false
   default_visa_timeout: 5000
   check_for_updates: false
+  log_console_level: INFO
+  log_file_level: DEBUG
+  log_file_directory: ./logs
+  log_file_name: tm_devices_<timestamp>.log
+  log_colored_output: false
+  log_pyvisa_messages: false
 ```
 
 These are all `false` by default if not defined, set to `true` to modify the
@@ -259,6 +264,30 @@ runtime behavior configuration.
 - `check_for_updates`
     - This config option will enable a check for any available updates on pypi.org for the
         package when the `DeviceManager` is instantiated.
+- `log_console_level`
+    - This config option is used to set the log level for the console output.
+        The default value of this config option is "INFO". See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_file_level`
+    - This config option is used to set the log level for the file output.
+        The default value of this config option is "DEBUG". See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_file_directory`
+    - This config option is used to set the directory where the log files will be saved.
+        The default value of this config option is "./logs". See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_file_name`
+    - This config option is used to set the name of the log file.
+        The default value of this config option is a timestamped filename with the .log extension. See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_colored_output`
+    - This config option is used to enable or disable colored output in the console.
+        The default value of this config option is false. See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_pyvisa_messages`
+    - This config option is used to enable or disable logging of PyVISA messages within the
+        configured log file. The default value of this config option is false. See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
 
 ### Sample Config File
 
@@ -320,6 +349,12 @@ options:
   retry_visa_connection: false
   default_visa_timeout: 10000  # 10 second default VISA timeout
   check_for_updates: false
+  log_console_level: NONE    # completely disable console output
+  log_file_level: DEBUG
+  log_file_directory: ./logs
+  log_file_name: custom_logfile.log    # customize the log file name
+  log_colored_output: false
+  log_pyvisa_messages: true  # log PyVISA messages in the log file
 ```
 
 #### TOML
@@ -393,8 +428,14 @@ standalone = false
 verbose_mode = false
 verbose_visa = false
 retry_visa_connection = false
-default_visa_timeout = 10000  # 10 second default VISA timeout
+default_visa_timeout = 10000         # 10 second default VISA timeout
 check_for_updates = false
+log_console_level = "NONE"           # completely disable console output
+log_file_level = "DEBUG"
+log_file_directory = "./logs"
+log_file_name = "custom_logfile.log" # customize the log file name
+log_colored_output = false
+log_pyvisa_messages = true           # log PyVISA messages in the log file
 ```
 
 ---

docs/key_features.md

@@ -14,3 +14,4 @@ complex examples.
         IntelliSense.
 - Organize connections to an entire "Test Bench" of devices with one package!
     - Full support for all VISA connection types (some require external drivers).
+- Customizable logging to both the console and a log file.

docs/known_words.txt

@@ -19,11 +19,13 @@ ci
 classdiagram
 codebase
 codecov
+colored
 config
 conftest
 cookiecutter
 cov
 csv
+customizable
 deps
 dev
 disable_command_verification
@@ -33,6 +35,7 @@ docstring
 docstrings
 en
 enum
+env
 executables
 filepath
 generate_function

docs/macros.py

@@ -265,8 +265,8 @@ def define_env(env: MacrosPlugin) -> None:
             used to perform a transformation
     """
     # Read in the current package version number to use in templates and files
-    with open(
-        pathlib.Path(f"{pathlib.Path(__file__).parents[1]}") / "pyproject.toml", "rb"
+    with (pathlib.Path(f"{pathlib.Path(__file__).parents[1]}") / "pyproject.toml").open(
+        "rb"
     ) as file_handle:
         pyproject_data = tomli.load(file_handle)
         package_version = "v" + pyproject_data["tool"]["poetry"]["version"]

examples/miscellaneous/customize_logging.py

@@ -0,0 +1,21 @@
+"""The console output and level of logging outputs in the log file can be configured as needed."""
+
+from tm_devices import configure_logging, DeviceManager, LoggingLevels
+from tm_devices.drivers import MSO6B
+
+# NOTE: This configuration will prevent any logging config options from a config file or
+# environment variable from being used.
+configure_logging(
+    log_console_level=LoggingLevels.NONE,  # completely disable console logging
+    log_file_level=LoggingLevels.DEBUG,  # log everything to the file
+    log_file_directory="./log_files",  # save the log file in the "./log_files" directory
+    log_file_name="custom_log_filename.log",  # customize the filename
+    log_pyvisa_messages=True,  # include all the pyvisa debug messages in the same log file
+)
+
+with DeviceManager(verbose=False) as dm:
+    scope: MSO6B = dm.add_scope("192.168.1.5")
+    scope.curve_query(1)
+    scope.check_port_connection(4000)
+    scope.check_network_connection()
+    scope.check_visa_connection()

examples/scopes/tekscope/basic_curve_query.py

@@ -1,9 +1,11 @@
 """An example showing a basic curve query."""
 
+from pathlib import Path
+
 from tm_devices import DeviceManager
 from tm_devices.drivers import AFG3KC, MSO5
 
-EXAMPLE_CSV_FILE = "example_curve_query.csv"
+EXAMPLE_CSV_FILE = Path("example_curve_query.csv")
 
 with DeviceManager(verbose=True) as dm:
     scope: MSO5 = dm.add_scope("MSO56-100083")
@@ -16,7 +18,7 @@
     curve_returned = scope.curve_query(1, output_csv_file=EXAMPLE_CSV_FILE)
 
 # Read in the curve query from file
-with open(EXAMPLE_CSV_FILE, encoding="utf-8") as csv_content:
+with EXAMPLE_CSV_FILE.open(encoding="utf-8") as csv_content:
     curve_saved = [int(i) for i in csv_content.read().split(",")]
 
 # Verify query saved to csv is the same as the one returned from curve_query function call

pyproject.toml

@@ -76,6 +76,7 @@ repository = "https://github.com/tektronix/tm_devices"
 version = "2.5.0"
 
 [tool.poetry.dependencies]
+colorlog = "^6.9.0"
 gpib-ctypes = "^0.3.0"
 libusb-package = "^1.0.26.0,!=1.0.26.2"  # 1.0.26.2 doesn't work with Python 3.12
 packaging = "^24.0"
@@ -94,6 +95,7 @@ tomli = "^2.0.1"
 tomli-w = "^1.0.0"
 traceback-with-variables = "^2.0.4"
 typing-extensions = "^4.10.0"
+tzlocal = "^5.2"
 urllib3 = "^2.0"
 zeroconf = "^0.136.0"
 
@@ -235,6 +237,7 @@ disable = [
   "too-many-lines",  # not necessary to check for
   "too-many-statements",  # caught by ruff
   "too-many-statements",  # caught by ruff
+  "unexpected-keyword-arg",  # caught by pyright
   "unused-argument",  # caught by ruff
   "unused-import",  # caught by ruff
   "use-implicit-booleaness-not-comparison-to-string",  # caught by ruff
@@ -287,6 +290,7 @@ filterwarnings = [
 ]
 junit_family = "xunit2"
 junit_logging = "all"
+log_format = "[%(asctime)s] [%(levelname)8s] %(message)s"
 markers = [
   'docs',
   'order',
@@ -315,27 +319,19 @@ fixable = ["ALL"]
 flake8-pytest-style = {mark-parentheses = false}
 flake8-quotes = {docstring-quotes = "double"}
 ignore = [
-  "ANN101",  # Missing type annotation for self in method
-  "ANN102",  # Missing type annotation for cls in method
-  "ANN401",  # Dynamically typed expressions (typing.Any) are disallowed in *args and **kwargs
-  "COM812",  # Trailing comma missing
-  "EM102",  # Exception must not use an f-string literal, assign to variable first
-  "FA100",  # Missing `from __future__ import annotations`, but uses ...
-  "FBT",  # flake8-boolean-trap
-  "FIX002",  # Line contains TO DO
+  "ANN401",  # Dynamically typed expressions (typing.Any) are disallowed in *args and **kwargs (allowed in this package)
+  "COM812",  # Trailing comma missing (allowed in this package)
+  "FA100",  # Missing `from __future__ import annotations`, but uses ... (allowed in this package)
+  "FBT",  # flake8-boolean-trap  # TODO: enable this
+  "FIX002",  # Line contains TO DO (allowed in this package)
   "ISC001",  # single-line-implicit-string-concatenation (handled by formatter)
-  "PTH109",  # `os.getcwd()` should be replaced by `Path.cwd()`
-  "PTH123",  # `open()` should be replaced by `Path.open()`
-  "PTH207",  # Replace `iglob` with `Path.glob` or `Path.rglob`
-  "PYI021",  # Docstrings should not be included in stubs
-  "T20",  # flake8-print
-  "TD002",  # Missing author in TO DO
-  "TD003",  # Missing issue link on the line following this TO DO
-  "TRY301",  # Abstract raise to an inner function
-  "UP006",  # Use {to} instead of {from} for type annotation
-  "UP007",  # Use `X | Y` for type annotations
-  "UP024",  # Replace aliased errors with `OSError`
-  "UP037"  # Remove quotes from type annotation
+  "PYI021",  # Docstrings should not be included in stubs (allowed in this package)
+  "TD002",  # Missing author in TO DO (allowed in this package)
+  "TD003",  # Missing issue link on the line following this TO DO (allowed in this package)
+  "UP006",  # Use {to} instead of {from} for type annotation (allowed in this package)
+  "UP007",  # Use `X | Y` for type annotations (allowed in this package)
+  "UP024",  # Replace aliased errors with `OSError` (allowed in this package)
+  "UP037"  # Remove quotes from type annotation (allowed in this package)
 ]
 pydocstyle = {convention = "google"}
 pylint = {max-args = 7}
@@ -357,7 +353,9 @@ order-by-type = false
 
 [tool.ruff.lint.per-file-ignores]
 "examples/**" = [
-  "S101"  # Use of assert detected
+  "FBT",  # flake8-boolean-trap
+  "S101",  # Use of assert detected
+  "T201"  # `print` found
 ]
 "examples/miscellaneous/custom_device_driver_support.py" = [
   "ARG002",  # Unused method argument
@@ -445,6 +443,7 @@ setenv =
 commands_pre =
     python -m poetry install --no-root --without=main
 commands =
+    !tests: python -c "import shutil; shutil.rmtree('dist_{envname}', ignore_errors=True)"
     !tests: poetry build --output=dist_{envname}
     !tests: twine check --strict dist_{envname}/*
     !tests: pre-commit run --all-files

scripts/contributor_setup.py

@@ -24,7 +24,7 @@ def create_virtual_environment(virtual_env_dir: str | os.PathLike[str]) -> None:
     Args:
         virtual_env_dir: The directory where the virtual environment should be created
     """
-    print(f"\nCreating virtualenv located at '{virtual_env_dir}'")
+    print(f"\nCreating virtualenv located at '{virtual_env_dir}'")  # noqa: T201
     _run_cmd_in_subprocess(f'"{sys.executable}" -m venv "{virtual_env_dir}" --clear')
 
 
@@ -35,7 +35,7 @@ def _run_cmd_in_subprocess(command: str) -> None:
         command: The command string to send.
     """
     command = command.replace("\\", "/")
-    print(f"\nExecuting command: {command}")
+    print(f"\nExecuting command: {command}")  # noqa: T201
     subprocess.check_call(shlex.split(command))  # noqa: S603
 
 
@@ -48,7 +48,7 @@ def main() -> None:
     starting_dir = Path.cwd()
     try:
         if RUNNING_IN_VIRTUALENV:
-            raise IndexError
+            raise IndexError  # noqa: TRY301
         # This requires contributors to use newer versions of Python even
         # though the package supports older versions.
         if sys.version_info < (3, 9):
@@ -76,7 +76,7 @@ def main() -> None:
         files = list(
             filter(
                 lambda x: "site-packages" not in x and "pythonw" not in x,
-                glob.iglob(
+                glob.iglob(  # noqa: PTH207
                     f"{virtual_env_dir}/{'bin' if RUNNING_ON_LINUX else 'Scripts'}/**/python*",
                     recursive=True,
                 ),

src/tm_devices/__init__.py

@@ -21,12 +21,16 @@
 )
 from tm_devices.helpers.enums import SupportedModels
 from tm_devices.helpers.functions import register_additional_usbtmc_mapping
+from tm_devices.helpers.logging import configure_logging, LoggingLevels
 
 # Read version from installed package.
 __version__ = version(PACKAGE_NAME)
 
+
 __all__ = [
+    "configure_logging",
     "DeviceManager",
+    "LoggingLevels",
     "print_available_visa_devices",
     "PYVISA_PY_BACKEND",
     "register_additional_usbtmc_mapping",

src/tm_devices/commands/helpers/scpi_commands.py

@@ -76,7 +76,7 @@ def verify(self, value: Union[float, str]) -> Tuple[bool, str]:
 
         Returns:
             A tuple containing a boolean indicating if the values match and a string with the actual
-            return value from the device.
+                return value from the device.
 
         Raises:
             tm_devices.commands.NoDeviceProvidedError: Indicates that no device connection exists.
@@ -136,7 +136,7 @@ def verify(self, argument: str, value: Union[float, str]) -> Tuple[bool, str]:
 
         Returns:
             A tuple containing a boolean indicating if the values match and a string with the actual
-            return value from the device.
+                return value from the device.
 
         Raises:
             tm_devices.commands.NoDeviceProvidedError: Indicates that no device connection exists.