How to typecheck methods and attributes generated by metaprogramming

[akaariai]

I'm wondering how to add type information for a Django project. For example, consider a typical Django model:

class Foo(models.Model):
    field1 = models.TextField(choices=(('a', 'Choice A'), ('b', 'Choice B')))
    parent = models.ForeignKey('self', null=True, blank=True, related_name='children')

The metaprogramming in Django will generate the following attributes and methods:

1. Foo.objects, a QuerySet for Foo objects. For example Foo.objects.get(pk=1) will return a Foo object, and Foo.objects.filter(pk__gte=1) is an iterable of Foo objects.
2. There will be an autogenerated method get_field1_display() for Foo. It returns the human readable value for foo field1's value (that is, it returns an str).
3. foo.parent should be a Foo instance, foo.children should be a QuerySet of Foo instances.

The request itself for this issue is to document a way to add type information for meta-programming generated attributes and methods. Optimally this would be done as part of the meta-programming itself.

[JukkaL]

(1) and (3) might be solvable via "self types" (#1212), though we haven't decided whether this is going to be implemented, and the details would have to be worked out.

(2) could be possible via a plugin architecture (#1240). We'd have a Django-specific plugin that would be run during type checking that would populate the symbol tables to include runtime generated definitions. This could also be useful for inferring precise types for foreign keys and similar.

For the time being, a potential approach would be to not type check model files at all but just to auto-generate stubs that would have all the right definitions. This is clearly not optimal, as there would be an extra build step before type checking (generation of stubs) and it would likely be impossible to type check the actual model code. An even fancier approach would be to transform the source code for models to include all definitions generated via metaprogramming, but that would likely be quite a lot of work to implement, and it would have additional usability issues such as line numbers not matching source code line numbers because the generated files would have extra stuff in them.

The easiest approach would be to use a lot of Any types. The models.Model class could have an Any baseclass so that mypy wouldn't complain about dynamically generated definitions. On the other hand, mypy couldn't check if you access an undefined definition. Using Any types might make it possible to have mypy not complain about Django programs, but it couldn't do very complete type checking.

[akaariai]

Is there any idea yet what the plugin architecture might look like?

The auto-generated stubs approach would be fine if meta-programming generated attrs would be typed in the stub file, but type definitions for user defined methods would be added directly in to the model. If there is no possibility to have both stubs and typing in model definitions mixed then I don't see this working for Django.

[JukkaL]

We haven't thought about the plugin architecture in much detail yet. Perhaps you could define a hook that gets called by mypy when analyzing classes/functions matching certain criteria (such as model subclasses) and you can query and modify the AST in your plugin somehow. The result of a plugin could be a sequence of AST modifications, for example. Many details need to be worked out before we can even start implementing this.

Another idea is to call the plugin as a subprocess, and it would generate a set of stubs that would be merged to the program being type checked. In the latter case the plugin could actually import the module and perform introspection, but that would be problematic for large programs since the imports could pull in a lot of dependencies and slow down type checking significantly.

A stub generator would likely have to parse the model modules and reproduce any annotations in the code that it can find. It would also have to include imports and other stuff, so actually writing a reliable stub generator would be pretty hard. Basically you'd probably have to import mypy to process the code first or something. This is probably not very practical to implement. The source-to-source transformation might be easier to implement, though probably it's still quite tricky to get it right. But it might be useful for some early prototyping at least, and it could be eventually rewritten as a mypy plugin.

[akaariai]

For Django the plugin system would have to import other modules in any case. External modules can have models with relations to models in the module currently under checking.

Maybe mypy could have the stub generator inbuilt. Basically, you would feed the generator a module and the generator would create a stub instance for that module. The stub instance could then be edited programmatically. Finally, the edited stub could be saved into a file.

Would some sort of programmatic stub generator be something you could consider for mypy?

[gvanrossum]

Unclear if you're aware of mypy/stubgen.py?

[JukkaL]

Stubgen may not be quite good enough by itself because it doesn't preserve any type annotations in the code. But plausibly we could provide a mode where it first runs best-effort type checking on the code and spits out an annotated stub. The problem here is that we can't type check code properly until we have the extra stuff provided by the Django plugin.

The right place to run the plugin seems to be some time before type checking. So perhaps mypy could generate a stub without type annotations and feed it to the generator, and the generator would then populate the stub with some new definitions. Mypy would merge the new definitions to the normal AST and run type checking on the merged AST. Alternatively, mypy would ask the generator to just generate a partial stub by itself (the generator could use stubgen internally, perhaps), and mypy would merge the generated stub to the internal AST before or after semantic analysis.

[akaariai]

Wasn't aware of stubgen. I'll take a look if it is usable for my use case.