Python 3 support

[jaraco]

Can mongs be deployed on Python 3? If would be nice to deploy on Python 3 if for no other reason than it will remove all of those ugly 'u' prefixes on the unicode literals in the UI.

[chadwhitacre]

I haven't tried, but good call.

[jaraco]

I've started work on a Python 3 port. First thing I notice is that virtualenv 1.6.4 and Aspen 0.24.3 are very old, so I've updated the project to use the latest virtualenv and Aspen (e8a1f4c).

When I do, the server now runs, but gets this error on a request for /:

pid-10411 thread-140735256182784 (MainThread) Traceback (most recent call last):
pid-10411 thread-140735256182784 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/algorithm.py", line 288, in run
pid-10411 thread-140735256182784 (MainThread)     new_state = function(**deps.as_kwargs)
pid-10411 thread-140735256182784 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/algorithms/website.py", line 115, in get_response_for_resource
pid-10411 thread-140735256182784 (MainThread)     return {'response': resource.respond(state)}
pid-10411 thread-140735256182784 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/http/resource.py", line 59, in respond
pid-10411 thread-140735256182784 (MainThread)     content_type, body = super(Dynamic, self).respond(accept, state)
pid-10411 thread-140735256182784 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/simplates/__init__.py", line 180, in respond
pid-10411 thread-140735256182784 (MainThread)     body = render(context)
pid-10411 thread-140735256182784 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/simplates/renderers/__init__.py", line 150, in __call__
pid-10411 thread-140735256182784 (MainThread)     return self.render_content(context)
pid-10411 thread-140735256182784 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/simplates/renderers/stdlib_percent.py", line 18, in render_content
pid-10411 thread-140735256182784 (MainThread)     return self.compiled % context
pid-10411 thread-140735256182784 (MainThread) TypeError: a float is required
pid-10411 thread-140735256182784 (MainThread) Failed to import pygments. Tracebacks won't be highlighted.
pid-10411 thread-140735256182784 (MainThread) 500 Internal Server Error            /                        ...s/aspen/www/error.spt

Can you point me in the right direction as to what needs to be done to account for this error?

[chadwhitacre]

I've started work on a Python 3 port.

Nice! :-)

First thing I notice is that virtualenv 1.6.4 and Aspen 0.24.3 are very old, so I've updated the project to use the latest virtualenv and Aspen (e8a1f4c).

Our efforts to port Aspen itself to Python 3 (AspenWeb/pando.py#524) have resulted in a significant change of direction for that project. We are working on reducing Aspen's scope from a web framework, to a simplates router to be used with other frameworks: AspenWeb/pando.py#526. We're planning to rename the project to Simplates to reflect the more modest scope.

TypeError: a float is required

Off the top of my head: it's probably using (rightly or wrongly) the stdlib_format renderer, which needs to have % characters escaped.

[chadwhitacre]

Fwiw, I've started contracting for an organization that uses Mongo (actually, TokuMX), so I may get a chance to work on Mongs after all. :-)

[jaraco]

I've made some progress. The issue with the render was that Mongs necessarily uses the Jinja renderer for loops and other logic in the templates.

Now that I've worked through some issues there, including syntax that's changed in Jinja, I'm now up against another Aspen issue - seems the syntax in this simplate is no longer valid. Can you recommend a technique that achieves the same need in the latest aspen?

[jaraco]

Error message is this:

pid-15911 thread-140735159017472 (MainThread) LoadError: Traceback (most recent call last):
pid-15911 thread-140735159017472 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/resources.py", line 70, in get
pid-15911 thread-140735159017472 (MainThread)     entry.resource = load(website, fspath, mtime)
pid-15911 thread-140735159017472 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/resources.py", line 119, in load
pid-15911 thread-140735159017472 (MainThread)     return Class(website, fspath, raw, media_type)
pid-15911 thread-140735159017472 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/http/resource.py", line 52, in __init__
pid-15911 thread-140735159017472 (MainThread)     super(Dynamic, self).__init__(defaults, fs, raw, default_media_type)
pid-15911 thread-140735159017472 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/simplates/__init__.py", line 122, in __init__
pid-15911 thread-140735159017472 (MainThread)     pages = self.parse_into_pages(self.decoded)
pid-15911 thread-140735159017472 (MainThread)   File "/Users/jaraco/Dropbox/code/yg/deploy/mongs/env/lib/python2.7/site-packages/aspen/simplates/__init__.py", line 201, in parse_into_pages
pid-15911 thread-140735159017472 (MainThread)     raise SyntaxError(msg)
pid-15911 thread-140735159017472 (MainThread) SyntaxError:  resources must have at least three pages; /Users/jaraco/Dropbox/code/yg/deploy/mongs/www/%server/%database/%collection/%filter/%page.int/index.html.spt has one.

[chadwhitacre]

Interesting. That file is supposed to be a symlink, and the target appears to be well-formed (the error you're seeing usually means one needs to add [----]\n[----]\n at the top of the file; see AspenWeb/pando.py#297 for some context). When are you getting this error? At startup or request time? If at request-time, what is the URL you're hitting?

[jaraco]

It's at request time - http://localhost:29017/localhost/translations/config.languages/%7B%7D/1/ is the URL. I'll experiment with adding sections.

[chadwhitacre]

Okay. If the symlink proves to be too fragile, we might look at a shim in the one simplate to delegate the request to the other simplate.

[jaraco]

The symlink renders as plaintext, so the error is gone, but now the response is simply a rendering of the symlink itself.

[jaraco]

The branch I'm working on is here - https://github.com/yougov/mongs/tree/python3

It's easy to bootstrap, but you need a MongoDB instance (with a document in a collection) to test it.

[jaraco]

Aha! Thanks to the help of Changaco in IRC, I see that the file is actually intended to be a file-system level symlink, and it only appears as a plaintext reference to the file because the code was originally checked out long ago, probably on a Windows system, and synced to this OS X system via Dropbox, so somewhere in that chain the link was broken. Making a clean checkout of the repo results in a working instance.

[jaraco]

Fixed in #27.

[jaraco]

I was mistaken about Python 3 support. Aspen still doesn't support Python 3.

[chadwhitacre]

We're getting there! :-)

Porting Aspen to Python 3 should be much easier after AspenWeb/pando.py#527 lands.

[jaraco]

@Changaco I'm looking into this again... but I'm getting stuck on a fairly basic hitch - how does one with pando 0.46 (aspen 1.0rc3) configure the WWW_ROOT? I tried PANDO_WWW_ROOT and ASPEN_WWW_ROOT and WWW_ROOT, but none seemed to have any effect (it continues to serve from '.'). Do you know if it's possible to set the root? Can you advise on how to do that?

Also, as an aside - I notice that aspen 1.0rc4 is quite a bit newer and includes a major refactor. Are there plans for pando to follow those updates?

[Changaco]

@jaraco Pando and Aspen have been moving away from using environment variables for configuration (AspenWeb/pando.py#568), instead you should use keyword arguments when creating the Website object, e.g.:

website = Website(
    changes_reload=env.aspen_changes_reload,
    project_root=env.aspen_project_root,
    show_tracebacks=env.aspen_show_tracebacks,
    www_root=env.aspen_www_root,
)

However it seems I haven't completed the transition yet and the boolean arguments like changes_reload still expect a string like 'yes' or 'no' instead of True or False.

Sorry it's still a bit of a mess.

Also, as an aside - I notice that aspen 1.0rc4 is quite a bit newer and includes a major refactor. Are there plans for pando to follow those updates?

Yes, but I don't know when I'll get around to it.

[Changaco]

On the bright side, I can confirm that Pando and Aspen do work under Python 3 now (Liberapay has been running under Python 3.6 in production for a month).

[jaraco]

I've made some progress in this branch. The home page renders, but on the first page with a parameter, the page render fails with this error:

  File "/Users/jaraco/code/yg/deploy/mongs/www/%server/index.html.spt", line 4, in <module>
    server = request.line.uri.path['server']
TypeError: byte indices must be integers or slices, not str

[jaraco]

I think I fixed that issue in yougov/mongs@e91fce5

[jaraco]

Next I've got an issue with from aspen import Response.

[jaraco]

Can someone help with how to write this line on aspen 1.0rc3?

[Changaco]

Response is in pando now, not in aspen, so from pando import Response.

[jaraco]

Thank you @Changaco. That got me over the hump, and this issue is fixed by #36.