Future of this package

[lassoan]

This package is very useful, thanks a lot for providing it! I would like to use it in a large open-source project (3D Slicer) but I'm not sure if I can rely on it in the long term, due to the followings:

1. It seems that there have been no new tags (and releases on PyPI) for almost two years
2. It is still not a native Python3, but patched using 2to3
3. Large backlog of open issues and 10+ open pull requests

Could somebody provide information about current status and future plans for this package?

I see that there have been some recent commits, so I still have some hope. Thanks in advance!

[cfarrow]

Hi Andras. I know of no long term plans for this package, except perhaps for users keeping it running for their use cases. The package can live on for a while in this self-service mode, but nice-to-haves like native python 3 support are not likely be prioritized.

I do virtually no work on this package, as my job and personal life no longer involve coding. @vasily-v-ryabov does the bulk of reviewing and committing. I'd welcome a discussion about building a critical mass of contributors and admins for this project.

[lassoan]

@cfarrow thanks a lot for the quick answer. @vasily-v-ryabov do you have any comments?

[vasily-v-ryabov]

Hi guys, I have some plans on diving deeper into the comtypes source code with potential improvements. My interest is mostly comtypes' usage in pywinauto as a dependency (we're already using it for few years). I'll try to explain my immediate interests and nice-to-have things which I can support as a reviewer.

• I'm not pursuing better Python 2.x/3.x support, but can review/discuss it.
• Running all unit tests on AppVeyor would be more helpful for potential contributors. It is also nice to have.
• The most critical for now is new wheel versions support which is reported in both projects already. Hope to take this task in nearest two months if nobody takes care of it. It could be a reason for 1.1.8 version update.
• I do want to debug some issues with 32-bit/64-bit Python / COM libraries that we have in pywinauto's backlog. This is medium priority for me, but there are a lot of high priority tasks in pywinauto which is also a hobby project in its turn. :)
• Also I'm interested in more convenient packaging into a single executable (low priority for now, may be raised in the future) so I may think about on-the-fly code generation into a memory without saving to cache file or something like that.

No more global things for now.

[lassoan]

A new release would be important, because if there are no new releases then fixes and improvements will not get to users. It would be also important to merge pull request and review issues. Setting up AppVeyor would be nice because then pull requests could be more confidently merged.

If there is a consensus that Python2 support can be dropped then updating the syntax for Python3 should not be hard, especially if there are automated tests. I can help with this, as we try to avoid depending libraries that are not updated to Python3.

[vasily-v-ryabov]

pywinauto is not planning to drop Py2.7 support. So I vote against it in comtypes as well. Also I'd prefer to keep comtypes a pure Python library (without any dependencies). We may think about six usage only, but I'm not sure it's so necessary.

[lassoan]

pywinauto is not planning to drop Py2.7 support

My understanding is that projects are actively encouraged to drop Python2 support before the end of 2020 to reduce frictions in the Python ecosystem (https://python3statement.org/). Anyway, if you are ready to pull the plug on Python2, then I'm ready to help with the update.

[vasily-v-ryabov]

I think it would be easy to make better Py2/Py3 support with already working AppVeyor tests. So making better CI is a fundamental task for further progress. Currently it is implemented as python setup.py test command which looks a bit old fashioned. And this command is not running on AppVeyor due to some failures in the full test set. I added some installation tests to AppVeyor a while ago. I'd appreciate if this work was continued.

[wkschwartz]

I just wanted to contribute one user's voice to this discussion.

1. I vote for the quote below. My use case is packaging apps using PyOxidizer. The apps depend on comtypes. I haven't gotten far enough down the rabbit hole of getting it to work to know whether comtypes needs to change for me to use PyOxidizer. I just wanted to let you know that's a use case I care about.

Also I'm interested in more convenient packaging into a single executable (low priority for now, may be raised in the future) so I may think about on-the-fly code generation into a memory without saving to cache file or something like that.

2. I care about CI/automated testing because it would give me confidence in comtypes' reliability, which matters a lot for my apps.
3. On Python 2/3 in response to the quote below: Since Python 2 has died, perhaps it would be better to use future than six. Putting future; python_version <= "2.7" in setup.py's install_requires would burden the few remaining Python 2 users rather than the mass of Python 3 users.

We may think about six usage only, but I'm not sure it's so necessary.

[nanonyme]

Even if there's no larger changes to the project done, it would be really great if #172 (or something equivalent) could be integrated and a new release could be done. Currently this project is randomly failing on Windows with new Python 3 versions which is causing issues with pywinauto. (which depends on this project for UIAutomation support) Afaik many other UI automation tools also use Windows UIAutomation through comtypes so they have the same problem.

[snoopyjc]

Please let Python 2 die!

[nanonyme]

IMHO debating killing Python 2 is futile if the person who is more or less maintaining this project intents on keeping the support for now. This should not and must not mean halting improving the quality of this project on Python 3 though.

[vasily-v-ryabov]

comtypes==1.1.8 is out.

[junkmd]

A long time has passed since this issue was posted.

And now,

• comtypes==1.1.11 has been released and is coded Python3 native.
  • while still being backward compatible to 2.7
• AppVeyor CI in each Python version now runs on PRs to the main branch.
  • it is only environment-independent, so Excel, Word, and other env-dependent testings are excluded.

As I planned in #327, I am trying to tie the comtypes type definition system into Python's type hinting system.

• It is possible to write type hints in versions where typing or inline (variable or function) annotations are not supported(see PEP561 and PEP484).

I think it is important to once again discuss plans in the future, and this issue is a good place to discuss it.

• For example, it has been more than three years since Python 3.4 EOL. It is longer than since sunset Python 2.

What are participants thoughts on this now?

[vasily-v-ryabov]

Removing Py3.4 is OK to me. Even if someone has legacy Python 3.4 environment, there is a very low risk something is broken in comparison with 3.5 or even 3.6. But keeping Py2.7 covers more legacy users in my opinion. I would remove Py2.7 after 1 more year of support which shouldn't bother all us so much. For example, we can release comtypes 1.1.13 and 1.1.14 with Py2.7 support and then release comtypes 1.2.0 with Py3.6+ support only. So type hints are natively supported from some moment without any additional efforts.

[junkmd]

@vasily-v-ryabov

I agree with your suggestion to support only Py 3.6+ when dropping Py 2.7.

With Py3.6+, we can use the built-in typing.
And enum module is also supported, which may be able to provide new features related to enumerations.

I understood that it will be about a year later we release the version of Py2.7 dropped.

I will develop this package according to that plan.

[junkmd]

Hello members, collaborators and contributors(including @cfarrow, @jaraco, @vasily-v-ryabov and @cmin764)!

I would like to put more efforts into comtypes, including the introduction of type annotations and supporting enumerations in the future.

I would like to be a collaborator on this project.

Any opinions would be appreciated.

[vasily-v-ryabov]

@cfarrow @jaraco I think merge/maintainer permissions make sense for @junkmd. If there are no objections, I will add him as collaborator in next 2 days.

[cfarrow]

I support this.

…

On Mon, Nov 21, 2022, 10:17 AM Vasily Ryabov ***@***.***> wrote: @cfarrow <https://github.com/cfarrow> @jaraco <https://github.com/jaraco> I think merge/maintainer permissions make sense for @junkmd <https://github.com/junkmd>. If there are no objections, I will add him as collaborator in next 2 days. — Reply to this email directly, view it on GitHub <#216 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGNU7TK7DDC3NB3CNAMRPLWJO4EJANCNFSM4PF6YPBQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[vasily-v-ryabov]

@junkmd invitation is sent. Welcome to the club! :)

[junkmd]

@vasily-v-ryabov
@cfarrow

Thank you very much!:smile:

[junkmd]

I added the good first issue tag as my first work as a collaborator.

I hope to see more newcomers in this community.

[junkmd]

I triaged some issues in my free time this weekend.
I closed issues that are currently resolved and issues that have not seen movement for several years.

But it is very voluminous, so It is taking time to do "issue inventory".

I plan to make some progress on this little by little.

[junkmd]

Can we release 033d6e8 (and adding changelogs) as 1.2.1?

[vasily-v-ryabov]

1.2.1 has been released.

[junkmd]

I am planning to observe movements after the release of 1.2.1 over the next few weeks (for regression and bug reports).

1.3.0 will be released after that observation.

In light of this, I have reviewed the current milestones for 1.3.0.

The following items, which have been present since before the drop_py2 plan was initiated, have shown little recent activity and seems unlikely to be resolved before the release. Therefore, they has been removed from the 1.3.0 milestone.

• Can't create VARIANT with typecode VT_ARRAY | BT_BSTR #80
• have 64-bit client check 32-bit registry hive #89
• CreateObject fails and then suceeds #193
• array of BSTR is not supported #347

The following items, originally included in the drop_py2 plan, were also removed from the 1.3.0 milestone. This decision was made due to the significant impact of the changes required for inclusion in 1.3.0 and the necessity to announce these changes to comtypes users prior to implementation.

• Define enumerations in generated modules. #345
• Add inline type annotations to dynamically generated modules #400
• Add enumeration definitions in the friendly modules #475
• Add method annotations (all args and return-values are Any) #490

[manju1847]

We are currently using PyWinAuto with Python 3.8. Here is our installed package list

C:\Users\Administrator>pip freeze
comtypes==1.1.11
Pillow==9.3.0
pywin32==303
pywinauto==0.6.8
six==1.16.0

Now we are planning to move to latest Python 3.12. Looking at this thread I think we cannot use PyWinAuto with Python 3.12. What is the maximum Python version that we can move to provided we still use PyWinAuto

[junkmd]

1.2.1 supports Python3.12.

https://github.com/enthought/comtypes/releases/tag/1.2.1

If you have a problem with the pywinauto installation itself, please submit an issue at pywinauto.

[manju1847]

Even with comtypes 1.2.1, PyWinAuto installation fails with Python 3.12.

Submitted the issue in PyWinAuto
pywinauto/pywinauto#1368

[junkmd]

I released 1.3.0 this Monday, I am about to embrace first weekend.

I am relieved that there have been no regression reports immediately following the release.

A few weeks later, I am planning to release 1.3.1, which #490 was merged.

If there are pull requests from contributors addressing some of the remaining "good first issue", I would like to prioritize reviewing them.

[junkmd]

I created a 'typing' label for the classification of features related to static typing.

[junkmd]

While summarizing my thoughts on numpy for #551, I noticed a few things during my research.

As mentioned in #220, the version of numpy where numpy.ctypeslib._typecodes was removed is 1.16.
numpy==1.16.6 has been EOL for over 5 years according to https://endoflife.date/numpy.
However, according to oldest-supported-numpy, the oldest numpy that runs on Python3.7 (the oldest Python this package currently supports) is numpy==1.14.5.
Therefore, we may need to continue maintenance considering that users using comtypes with Python3.7 may also be using numpy==1.14.x or numpy==1.15.x.

On the other hand, numpy supporting Python3.8 is 1.17 and above. This means that when this project drops support for Python3.7 (the specific timing is still undecided), there will be no need for production code to attempt to import numpy.ctypeslib._typecodes.

Furthermore, there is also room to consider dropping to support numpy<1.16 while continuing to support Python3.7.

[jaraco]

I noticed that Python 3.7 is now EOL. https://devguide.python.org/versions/#unsupported-versions

Even so, I still think it would be better to keep the minimum Python version supported by 1.3.0 at 3.7, because the community might get confused if that is changed.

I've found that dropping support for EOL Pythons is generally uncontroversial, especially if the proper metadata is present. I'd advise to do that as a matter of course with this (or any other) project unless there are specific functional reasons to keep it.

[junkmd]

I've found that dropping support for EOL Pythons is generally uncontroversial, especially if the proper metadata is present. I'd advise to do that as a matter of course with this (or any other) project unless there are specific functional reasons to keep it.

I certainly agree with what you’re saying. Providing a long grace period when we dropped support for Python 2.7 was a special case.

When Python 2 support was dropped, the version increment was made from 1.2.1 to 1.3.0. When Python 2.6 was dropped in the past, the increment was made from 1.1.10 to 1.1.11.

In this project, when releasing the version that dropping support for Python 3.7 (or any Python 3.Y.Z that reaches EOL), which digit of comtypes==x.y.z should be incremented?

[jaraco]

Even if this project honors the well-specified semver standard, the choice for what qualifies as a "bugfix" (patch bump) and not (minor bump) is far from deterministic. Here's the methodology I follow:

1. By default, any change is a minor bump.
2. If all changes are corrections to previously-released behaviors (bugfix) or are not affecting user behavior (docs changes, metadata updates, etc.), then a patch bump may be used (at the release manager's discretion). A minor bump may still be elected if the changes were substantial or there's reason to make room for bugfixes on the previous minor release.
3. If any change to behavior is likely to break a user's expectations, then it's a breaking change and should be a major bump.

By that methodology, since the ecosystem prevents installation of incompatible versions, it's not a breaking change. Since it's only a metadata change not affecting behavior, it could be a patch bump. On the other hand, because it's not a correction and because one may want to release bugfixes for versions supporting older Pythons, I almost always use a minor bump when dropping support for a Python.

It's this last point that I think is the most crucial. Using the 1.1.11 release as an example, if a bug was discovered that affected both 1.1.10 and 1.1.11 and we wished to fix for Python 2.6 users, it would not be possible without a post-release tag (e.g. 1.1.10.post1) and I'm not even sure those are allowed on PyPI. That's why I would prefer to use a minor bump for such changes.

[junkmd]

• By default, any change is a minor bump.
• If all changes are corrections to previously-released behaviors (bugfix) or are not affecting user behavior (docs changes, metadata updates, etc.), then a patch bump may be used (at the release manager's discretion). A minor bump may still be elected if the changes were substantial or there's reason to make room for bugfixes on the previous minor release.
• If any change to behavior is likely to break a user's expectations, then it's a breaking change and should be a major bump.

since the ecosystem prevents installation of incompatible versions, it's not a breaking change.

Thank you for sharing your methodology.

With reference to that, I have the following very rough release schedule in mind:

• Within the next few months, releasing the non-breaking-change comtypes==x.y.z with “this next release will (or might) be the last version to support Python3.7” declaration.
• After the release of non-breaking-change comtypes==x.y.z+1, starting accepting codebase proposals that can only be run on Python3.8 or higher.
• During "the next few months", it is of course conceivable that contributors will propose changes that are useful for this package but may break backward compatibility. In such cases, to distinguish it, by merging the breaking changes after the release of the support drop, or releasing the support drop version after releasing comtypes==x.y+1.0 in which the breaking changes have been merged.

[junkmd]

Since it's only a metadata change not affecting behavior, it could be a patch bump.

Is "metadata" the classifiers in setup.py for this package?

When we dropped support for Python2, 'Programming Language :: Python :: 2.7' was removed from classifiers, and the distribution format was no longer a universal wheel, so I think this qualifies as a "change in metadata".
However, the current classifiers include 'Programming Language :: Python :: 3' but not 'Programming Language :: Python :: 3.Y'.
So, I realized that it might be not possible to determine the supported Python3.Y version from the PyPI metadata for this package. Is that correct?

P.S. I am also aware that it is not recommended to have a large amount of code written in setup.py as it is now. I confess that I do not know much how to properly replace (or abolish) things like class test(Command) and class post_install(install) when adopting more modern ways such as pyproject.toml or setup.cfg.

[jaraco]

Is "metadata" the classifiers in setup.py for this package?

Actually, I'm speaking of the Requires-Python metadata.

And now I see this project isn't declaring that metadata right now. That's not good. That means that a user installing comtypes on Python 2.7 or 3.6 will still be offered the latest release (and all prior releases):

 ~ [1] @ pip download --python-version 2.7 --no-deps comtypes
Collecting comtypes
  Downloading comtypes-1.4.3.zip (229 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 229.6/229.6 kB 2.7 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Saved ./comtypes-1.4.3.zip
Successfully downloaded comtypes

We'll want to get that fixed ASAP and get a release out. PR incoming.

So, I realized that it might be not possible to determine the supported Python3.Y version from the PyPI metadata for this package. Is that correct?

That is correct. The classifiers are essentially useless and I recommend against using them except maybe to declare the major Python version, which today is just "3" and there's no likelihood of "4", so I just omit it.

[junkmd]

I released comtypes==1.4.4, including packaging metadata fix.
https://pypi.org/project/comtypes/1.4.4/

[nanonyme]

If there are versions out there that claim Python 2 support but don't really support it, consider yanking them.

[junkmd]

I plan to release comtypes==1.4.6, the final version to support Python 3.7, in early August.

[junkmd]

I believe that yanking should be done for more critical situations, such as when a vulnerability is discovered or when the package becomes unusable at all.

If we yank versions solely because python_requires was not specified and might allow installation on unsupported Python versions, it could unnecessarily confuse users who are currently using those versions.

[junkmd]

An idea that might effectively clarify which versions of Python this package supports:

As previously discussed, "classifiers are essentially useless", adding or removing the 'Programming Language :: Python :: 3.x' classifier every time this package drops support for older versions or starts supporting newer ones isn't ideal from a maintainability perspective.

I came across this comment.
In that discussion, it was pointed out that, although it's not as simple as setting up other badges, we can use a query to obtain the Requires-Python metadata and display it as a badge.

A badge like this might be useful:

Any opinions would be appreciated.

[jaraco]

+1 to using Requires-Python and reflecting that in a badge.

[nanonyme]

That is correct. The classifiers are essentially useless and I recommend against using them except maybe to declare the major Python version, which today is just "3" and there's no likelihood of "4", so I just omit it.

There is one point of classifiers which is basically "project is has CI testing against these Python versions". Then again, people typically add versions even if there's coverage gaps for interpreter versions... But the main point is that it may work on newer, it's just not been tested. Requires-Python is semantically different and just means you must not install on older than given version.

[junkmd]

A Requires-Python badge would also be effective, but what we might additionally need is a badge that shows the Python versions for which tests have passed in the CI pipeline.

While researching a way to generate such a badge, I came across https://github.com/40ants/github-actions.
However, it’s not as simple as the badge based on 'classifiers', like .

rendered badge image

[junkmd]

We have identified cases where comtypes may not work in Python 3.13, potentially due to changes in the implementation of ctypes, without modifying codebases or fixing bugs in CPython.
Let's discuss this in #618.

[junkmd]

To support Python 3.13, version 1.4.8 includes changes to metaclass initialization.

To prevent comtypes from breaking due to future ctypes changes, tests to ensure the functionality of the metaclasses used by this project were submitted as a PR (python/cpython#125881) on the cpython side and have been merged.