Instrument driver organization and access

[alexcjohnson]

Ported from the discussion here

Several related issues:

• how do we organize our driver files
• how do users access them
• how to handle drivers that work with multiple instrument models

File organization

The options here seem to be:

1. Flat, all files in one directory (downside: we'll soon have hundreds, searching this will be a pain)
2. Company subdirectories (downside: companies change names sometimes)
3. Functional subdirectories (downside: categories blur, hard to know where to put or find things - I think we can already rule this out)

User access

The explicit option will of course always work. Assuming we do company subdirectories:

from qcodes.instrument_drivers.tektronix.AWG5014 import Tektronix_AWG5014
awg = Tektronix_AWG5014(address='...')

But this is awfully verbose, and also doesn't help users find the driver they need, particularly if the company name is ambiguous. We could import drivers in __init__.py files to shorten this to:

from qcodes.instrument_drivers import Tektronix_AWG5014
awg = Tektronix_AWG5014(address='...')

but then we're importing all drivers all the time, which could be a big drag on the system. We could make a custom importer, so users could do:

awg = qcodes.instruments.Tektronix_AWG5014(address='...')

That would let us only import drivers we need, alias company names (so qcodes.instruments.HP_34401A would be the same driver as qcodes.instruments.Keysight_34401A, good for backward compatibility). Users could find drivers with dir(qcodes.instruments) or tab completion (we'd have to implement __dir__) but the correspondence between this and the directory structure would be implicit, particularly for aliased companies.

Drivers that work with multiple instrument models

This should certainly be encouraged! Already, @AdriaanRol 's RS_SGS100A driver works not only for the RS_SGS100A but also for the RS_SMB100A.

Do we name these drivers to indicate this (something like RS_S100A_series)? Do we create wrapper files? Then users wouldn't need to know that they were using the same driver, but maybe they should?

# SMB100A.py
from .SGS100A import RS_SGS100A as RS_SMB100A
# let users know that this is an alias?
print('RS_SMB100A loaded using the compatible RS_SGS100A driver')

[MerlinSmiles]

@alexcjohnson, Related to drivers that work with multiple instrument models, there will also be drivers that share many features but ommit / extend on a base set of features, how would one go about this?
Subclassing a base driver and add features?

[alexcjohnson]

drivers that share many features but omit / extend on a base set of features

Yes, subclassing should serve this purpose well.

class AcmeModel1(AcmeBase):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs):
        self.add_parameter('only_in_model1', ...)
        del self.parameters['not_in_model1']
        # and same for functions

[AdriaanRol]

I guess subclassing is indeed the obvious answer but does this mean they get a separate file? I guess it does not improve readability to have a separate file for an instrument driver that is a subclass and only contains 2 functions and 1 extra parameter. Altough I guess at the end it is more about findability of the drivers themselves.

Maybe this is a crazy idea but perhaps we cannot find the best possible solution at this point in time because we are not actually experiencing the problem yet. Maybe we should just stick to the most simple solution for now, being flat (or company named folders) with no fancy redirecting imports and revisit this issue once we hit ~30+ drivers or so. I guess then we have all the edge cases needed to find the most suitable approach.

[alexcjohnson]

perhaps we cannot find the best possible solution at this point in time because we are not actually experiencing the problem yet

🌟 💯 🌟

Lets keep the company subdirectories, because changing that would be painful later. But beyond that,
the direct reference method works, will always work, and does everything we need including the requirement to only import the drivers we really need. Its only downsides are long, unwieldy import statements and it could do more to help the user find the driver they want.

Thinking further about this, We're probably all going to be totally fine with this as long as qcodes is installed in development mode; there won't be too many drivers, and all the users will know where the code is and how to look around in it for drivers. The problems are likely to come when it stabilizes and new people install it as a regular package: lots of times people don't even know where on disk these packages are, and anyway we'd rather they not hunt around in there but have nice ways to discover drivers from within Python.

So we have some time... but one way to do this without forcing us into any particular structure now (including things like one class per file, directory names matching file / class names) would be to make a driver "registry" - perhaps in the __init__.py for each subdirectory so we still avoid actually loading everything, each driver registers itself re: what instrument models it handles (under all the company aliases it might have), what instrument type it drives, a short description... then we can go crazy with a search system but also there could be several responses to statements like

awg = qcodes.instruments.AWG5014(address='...'):

• success
  Connected to AWG5014 in 0.03 sec
• partial match
  KeyError: no driver found for 'AWG5014' - perhaps you meant 'AWG5014C'?
• multiple match

MultiValueError: 'AWG5014' matches 2 drivers:
  qcodes.instrument_drivers.tektronix.AWG5014C.Tektronix_AWG5014C
  qcodes.instrument_drivers.tektronix.AWG5014X.Tektronix_AWG5014X

Obviously there are a lot of things we could do with that, but the point is, there IS a solution that doesn't force any additional constraints on us right now, so we can keep the structure as it is currently, ie well organized for us as coders, and shelve further discussion for now.