Refactor the internal package structure and change how driver classes…

… inherit functionality (#333)

* ci: Update testing/linting to use official Python 3.13 versions

* refactor: Converted the SignalGenerator class into a mixin class called TekAFGAWG.

Also removed the previously deprecated TekScopeSW alias.

* refactor: Remove previously deprecated write_buffers() method

* refactor: Refactor the TekScope inheritance to remove methods from the TekScopePC driver that it shouldn't have.

Also, some miscellaneous cleanup of outstanding TODO items.

* refactor: Moved the REST API class into a mixin.

Also moved other mixins around into subfolders to organize the mixins by type.

* refactor: Moved the Device, PIDevice, and TSPDevice class into device_control mixins.

Also moved shared code into mixin folders and moved all device type subpackages up to the top level of the drivers subpackage.

* refactor: Remove deprecated `DEVICE_DRIVER_MODEL_MAPPING` constant.

Also removed another constant that wasn't really needed.

* refactor: Updated most device types to no longer inherit from the control mixin, the control mixins are now inherited at the family base class level (in most cases)

* docs: Updated docs with the new references to the refactored object locations

* refactor: Checkpoint commit during the process of converting the PIDevice/TSPDevice/RESTAPIDevice classes into true control mixins (PIControl/TSPControl/RESTAPIControl)

* refactor: Checkpoint commit during the process of converting the PIDevice/TSPDevice/RESTAPIDevice classes into true control mixins (PIControl/TSPControl/RESTAPIControl).

This commit updates tests and ruff formatting/linting to pass.

* refactor: First revision completed of converting the PIDevice/TSPDevice/RESTAPIDevice classes into true control mixins (PIControl/TSPControl/RESTAPIControl).

Plenty of TODO items remain, but tests and linting are all passing.

* refactor: Converted the verify_values(), raise_error(), and raise_failure() methods into separate functions.

Also implemented a super high-level abstract class to enable better type checking of control mixin classes.

* refactor: Updated the ReadOnlyCachedProperty decorator to be able to be used to decorate an abstractmethod while still maintaining the checks that abstractmethods perform to prevent instantiation of an object that doesn't implement all abstract methods.

Updated the TSOVu driver to be able to be instantiated properly by implementing all required abstract methods.

* refactor: Simplified the sharing of common method implementations by creating a mixin class that implements the common system:error PI logic necessary for certain device's expect_esr() implementations

* refactor: Removed the get_eventlog_status() method and replaced it with a new get_errors() method.

All devices are now required to implement a _get_errors() method with the necessary implementation to get the current error code and error messages.

* refactor: Removed the device_type_classes.py file since it was no longer needed due to the added ability to go upwards and downwards when creating an automatic class diagram.

Also added back the namespace to the advanced architecture generated class diagrams to make them look nicer.

* docs: Added more detail to the architecture document page.

Added highlighting of the main device driver diagram.

* test: Fix test for DeviceTypes enum that verifies all device types are represented

* ci: Use python -m poetry when installing the package dependencies during tox runs to avoid issues when poetry needs to be updated

* refactor: Checkpoint commit during the refactor of the expect_esr() method

* refactor: Checkpoint commit during the refactor of the expect_esr() method

* refactor: Converted the expect_esr() method into a final method that uses the outputs of the _get_errors() abstract method to compare the expected error code and messages with the actual error code and messages.

Also added regex comparison to the verification functions.

* test: Refactored the test for proper driver inheritance and abstraction to provide granular results on failures.

This means that if more than one failure is found all failures will be reported, rather than just the first failure.

* docs: Update inheritance diagrams and fix an abstract class that wasn't properly marked as abstract

* fix: Updated the hostname lookup to use concurrent.futures to implement a shorter timeout

* refactor: Update TODO comments and remove an unneeded dependency

* refactor: Made a handful of the new mixins private since they don't really need to show up in the documentation

* refactor: Converted the private class constant `_DEVICE_TYPE` into an abstract property that each device must implement to better enforce all devices to have a type defined

* refactor: checkpoint commit in the process of removing the unneeded init functions from the device driver classes

* docs: Update the two contributing guides based on the new folder structure

* docs: Update signal_generators.md to include links to the API documentation when necessary

* docs: Update custom template with newer code to enable rendering classes that make use of multiple inheritance more accurately

* refactor: Re-generated the commands subpackage to use the new class structure that works better with the documentation

* docs: Update the changelog

* refactor: Fixes after pulling latest main branch

* chore: Update development dependencies

* chore: Update development dependencies

* chore: Fixes from pull request review

* chore: Fixes from pull request review

* chore: Removed some unnecessary init methods

* chore(python-deps): Remove safety-schemas pinned version since the broken version was yanked from PyPI

* docs: Add macro to docs for converting github flavored markdown alerts to admonitions

* docs: Made breaking change callout in changelog bold and italics

* docs: Update changelog

* docs: Update changelog

* docs: Switch to lawngreen for the inheritance tree color

* refactor: Update code and comments after reviewing pull request

* ci: Update testing dependency to remove vulnerability

* fix: Fix example to use proper SCPI syntax

* docs: Change the note by the device inheritance tree to simply highlight in the lawngreen color to make the text easier to read

* docs: Fix changelog after merge conflicts

.pre-commit-config.yaml

@@ -60,11 +60,6 @@ repos:
     rev: 8072181c0f2eab9f2dd8db2eb3b9556d7cd0bd74  # frozen: 1.17.0
     hooks:
       - id: yamlfix
-# TODO: get this working
-#  - repo: https://github.com/motet-a/jinjalint
-#    rev: "0.5"
-#    hooks:
-#      - id: jinjalint
   - repo: https://github.com/thibaudcolas/curlylint
     rev: 71adf4d34c290684fd9f94a4d21ac55bcfe640f0  # frozen: v0.13.1
     hooks:

CHANGELOG.md

@@ -18,6 +18,44 @@ Valid subsections within a version are:
 
 Things to be included in the next release go here.
 
+### Added
+
+- Testing/linting on Python 3.13.
+- Added the `get_errors()` method to the `Device` class to enable easy access to the current error code and messages on any device.
+- Added more details to the Architectural Overview page of the documentation as well as highlighting to the device driver diagram on the page.
+- Added regex matching to the `verify_values()` helper function to allow for more flexible value verification.
+
+### Changed
+
+NOTE: Despite all the officially breaking changes, the actual drivers were only affected in
+very minor ways. The primary impact to the drivers was simply the removal of previously
+deprecated functionality. Almost all changes only impacted the internal workings of `tm_devices`.
+However, please read through all changes to be aware of what may potentially impact your code.
+
+- _**<span style="color:orange">minor breaking change</span>**_: Moved `SignalGenerator` class to the `driver_mixins` submodule and renamed it to `_TektronixPIAFGAWGMixin` (also made it a private mixin).
+- _**<span style="color:orange">minor breaking change</span>**_: Renamed the `PIDevice`, `TSPDevice`, and `RESTAPIDevice` classes to `PIControl`, `TSPControl`, and `RESTAPIControl` respectively.
+- _**<span style="color:orange">minor breaking change</span>**_: Moved the `PIControl`, `TSPControl`, and `RESTAPIControl` classes into a mixin folder so that they can be used as mixins rather than being part of the required inheritance structure.
+    - In order to use these control mixins, they must be inherited at the family base class level in the driver hierarchy, along with the device type class (or any class that inherits from the base `Device` class and defines a `device_type` property and the other required abstract property implementations).
+    - Due to this change, it is recommended that the specific device driver (or at least the family base class) for the device being controlled is used for type hinting.
+- _**<span style="color:orange">minor breaking change</span>**_: Moved all device type subpackages (AWGs, AFGs, Scopes, SMUs, etc.) up to the top level of the `drivers` subpackage.
+- _**<span style="color:orange">minor breaking change</span>**_: Converted all family base classes to inherit from the device control mixins.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Renamed the `get_eventlog_status()` method to `_get_errors()` and made it a required, abstract method for all devices to implement.
+    - To get similar functionality to the previous `get_eventlog_status()` method, switch to using the new `get_errors()` method.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Changed the behavior of the `expect_esr()` method to expect an integer error code input and an optional tuple of error messages to compare against the actual error code and messages returned by the `_get_errors()` private method.
+- _**<span style="color:orange">minor breaking change</span>**_: Converted the `device_type` property into an abstract, cached property to force all children of the `Device` class to specify what type of device they are.
+- Updated the auto-generated command mixin classes to no longer use an `__init__()` method to enable the driver API documentation to render in a more usable way.
+
+### Removed
+
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `TekScopeSW` alias to the `TekScopePC` class
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `write_buffers()` from the `TSPControl` class.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed Internal AFG methods from the `TekScopePC` driver, since they wouldn't have worked due to its lack of an IAFG.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `DEVICE_DRIVER_MODEL_MAPPING` constant.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `DEVICE_TYPE_CLASSES` constant and the `device_type_classes.py` module.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed many hacky implementations of `total_channels` and `all_channel_names_list` properties from drivers that don't need them anymore.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `verify_values()`, `raise_failure()`, and `raise_error()` methods from all device drivers.
+    - These methods have been converted to helper functions and can be imported from the `tm_devices.helpers` subpackage now.
+
 ---
 
 ## v2.5.0 (2024-10-30)
@@ -42,8 +80,8 @@ Things to be included in the next release go here.
 
 ### Added
 
-- `collectgarbage()` is now called during cleanup of `TSPDevice` children.
-- Added USB Support for AFG31K and MDO3 models.
+- `collectgarbage()` is now called during cleanup of `TSPControl` children.
+- Added USBTMC Support for the AFG31K and MDO3 drivers.
 
 ---
 
@@ -203,11 +241,11 @@ Things to be included in the next release go here.
 
 ### Changed
 
-- <span style="color:red">BREAKING CHANGE</span>. Changed the term "signal source" to "signal generator".
+- _**<span style="color:red">BREAKING CHANGE</span>**_. Changed the term "signal source" to "signal generator".
     - All uses of this term are changed. Import paths now use `signal_generator` instead of `signal_source`.
-- <span style="color:red">BREAKING CHANGE</span>. Changed the function name of `generate_waveform()` to `generate_function()`.
+- _**<span style="color:red">BREAKING CHANGE</span>**_. Changed the function name of `generate_waveform()` to `generate_function()`.
     - `generate_waveform()` only exists on AWGs now, however the functionality is entirely changed.
-- <span style="color:red">BREAKING CHANGE</span>. Changed the `generate_function()` function by removing burst functionality.
+- _**<span style="color:red">BREAKING CHANGE</span>**_. Changed the `generate_function()` function by removing burst functionality.
     - Any use of burst now must use `setup_burst()` and `generate_burst()` instead.
 - Updated AWGs such that the `family_base_class` is at the series level.
 

docs/_templates/mkdocstrings/python/readthedocs/children.html.jinja

@@ -5,7 +5,7 @@ This template iterates on members of a given object and renders them.
 It can group members by category (attributes, classes, functions, modules) or render them in a flat list.
 
 Context:
-  obj (griffe.dataclasses.Object): The object to render.
+  obj (griffe.Object): The object to render.
   config (dict): The configuration options.
   root_members (bool): Whether the object is the root object.
   heading_level (int): The HTML heading level to use.
@@ -31,7 +31,7 @@ Context:
     </table>
 {%- endmacro -%}
 
-{% if obj.members %}
+{% if obj.all_members %}
   {% block logs scoped %}
     {#- Logging block.
 
@@ -127,7 +127,7 @@ Context:
                 {% endif %}
                 {% with heading_level = heading_level + extra_level %}
                   {% for attribute in attributes|order_members(config.members_order, members_list) %}
-                    {% if members_list is not none or attribute.is_public %}
+                    {% if members_list is not none or (not attribute.is_imported or attribute.is_public) %}
                       {% include attribute|get_template with context %}
                     {% endif %}
                   {% endfor %}
@@ -147,7 +147,7 @@ Context:
                 {% endif %}
                 {% with heading_level = heading_level + extra_level %}
                   {% for class in classes|order_members(config.members_order, members_list) %}
-                    {% if members_list is not none or class.is_public %}
+                    {% if members_list is not none or (not class.is_imported or class.is_public) %}
                       {% include class|get_template with context %}
                     {% endif %}
                   {% endfor %}
@@ -168,7 +168,7 @@ Context:
                 {% with heading_level = heading_level + extra_level %}
                   {% for function in functions|order_members(config.members_order, members_list) %}
                     {% if not (obj.kind.value == "class" and function.name == "__init__" and config.merge_init_into_class) %}
-                      {% if members_list is not none or function.is_public %}
+                      {% if members_list is not none or (not function.is_imported or function.is_public) %}
                         {% include function|get_template with context %}
                       {% endif %}
                     {% endif %}
@@ -189,8 +189,8 @@ Context:
                     {% filter heading(heading_level, id=html_id ~ "-modules") %}Modules{% endfilter %}
                   {% endif %}
                   {% with heading_level = heading_level + extra_level %}
-                    {% for module in modules|order_members(config.members_order, members_list) %}
-                      {% if members_list is not none or module.is_public %}
+                    {% for module in modules|order_members(config.members_order.alphabetical, members_list) %}
+                      {% if members_list is not none or (not module.is_alias or module.is_public) %}
                         {% include module|get_template with context %}
                       {% endif %}
                     {% endfor %}

docs/advanced/architecture.md

@@ -17,8 +17,16 @@ information about the device, and methods, which provide various functionality.
 
 The [`DeviceManager`][tm_devices.DeviceManager] uses a configuration parser
 ([`DMConfigParser`][tm_devices.components.DMConfigParser]) to read in
-connection information from an optional config file as well as to store
-connection information that is provided directly via python code.
+connection information from an optional [config file](../configuration.md#config-file) or
+[environment variable](../configuration.md#environment-variable) as well as to store
+connection information that is provided directly via [python code](../configuration.md#python-code).
+This information contains the device type (e.g., SCOPE, AFG, SMU) as well as the address
+(e.g., hostname, IP address, or model and serial number for USBTMC). The config parser also reads in
+optional configuration settings that apply globally, such as turning on verbose VISA logging. The
+[`DeviceManager`][tm_devices.DeviceManager] then connects to all the devices that the config parser
+has listed and provides access to the [Python driver](#device-drivers) for each device. The
+Python device driver is the class responsible for actual communication with the physical
+(or virtual) device.
 
 ### Block Diagram
 
@@ -32,35 +40,62 @@ classDiagram
 
 ---
 
-## Main device types
+## Device Types
 
-There are currently 9 different supported device types: **Scopes**, **Arbitrary
-Waveform Generators (AWGs)**, **Arbitrary Function Generators (AFGs)**, **Source
-Measure Units (SMUs)**, **Power Supplies (PSUs)**, **Data Acquisition Systems
-(DAQs)**, **Digital Multimeters (DMMs)**, **Systems Switches (SSs)**, and
-**Margin Testers**.
+These are the currently supported device types:
 
-The driver class structure uses inheritance, which means that common attributes
-and methods can be defined in higher-level classes (sometimes called parent
-classes) and inherited by subclasses (sometimes called child classes), to reduce
-the lines of code needed to create new device drivers.
-
-Every single device driver within a particular device type is guaranteed to have
-the same class signature (attribute and method names). Often times specific
-device drivers will need to implement the functionality for specific methods or
-even overwrite inherited functionality due to the differences that can occur
-between different models of the same device type.
+1. Arbitrary Function Generators (AFGs)
+2. Arbitrary Waveform Generators (AWGs)
+3. Data Acquisition Systems (DAQs)
+4. Digital Multimeters (DMMs)
+5. Margin Testers
+6. Power Supplies (PSUs)
+7. Scopes (Oscilloscopes)
+8. Source Measure Units (SMUs)
+9. Systems Switches
 
 ### Block Diagram
 
-{{ auto_class_diagram('tm_devices.drivers.device_type_classes', full=True, namespace='tm_devices.drivers') }}
+{{ auto_class_diagram('tm_devices.drivers.device.Device', full=False, namespace='tm_devices.drivers', tree_direction='down') }}
 
 ---
 
-## All device drivers
+## Device Drivers
 
-This package supports many devices, zoom in to see them all!
+### Object-Oriented Design Principles
+
+The drivers are the biggest part of this package; each unique series of instrument gets a uniquely
+named Python driver class (see [`tm_devices.drivers`][tm_devices.drivers] for the list of
+available driver classes). These classes are implemented using object-oriented programming
+principles and make extensive use of inheritance. This allows common attributes and methods to be
+defined in higher-level, abstract classes that are then inherited by the individual driver classes.
+
+Mixin classes are also used to provide shared implementations and abstract method signatures for the
+drivers. This allows multiple device types to all inherit from the same abstract class (mixin class)
+without sharing the exact same inheritance tree (since they might not share much else). The mixin
+classes define attributes and methods, some of which the driver will overwrite or implement for itself.
+
+### Family Base Classes
+
+One other technique for enabling code reuse in the drivers is by using a “family base class”. Each
+family base class is an abstract class that defines the signature for all child classes that
+inherit from it. Occasionally a child class will need to overwrite an implementation due to the
+differences that can occur between different models of the same device type, but the class signature
+will not change. This means that all drivers that inherit from a common family base class are
+guaranteed to have the same class signature (attributes, methods, etc.), and this rule is enforced by unit tests.
+
+### Object-Oriented Design Benefits
+
+These object-oriented principles result in individual driver classes that are quite simple to
+read, as they have very few actual lines of code. Code duplication is extremely low, which makes
+drivers easy to create and update.
 
 ### Block Diagram
 
-{{ auto_class_diagram('tm_devices.drivers', full=True, namespace='tm_devices.drivers') }}
+This package supports many devices, zoom in to see them all!
+
+!!! note
+    - Family Base Classes are outlined in **<span style="color: orangered;">orange red</span>**.
+    - Device Drivers are highlighted in **<span style="background-color: lawngreen;">lawn green</span>**.
+
+{{ auto_class_diagram('tm_devices.drivers', full=True, namespace='tm_devices.drivers', highlight_family_base_classes=True, highlight_device_drivers=True, chart_direction='LR') }}