join 'em

[chadwhitacre]

If you can't beat 'em, join 'em. This ticket started (per the original description below) by thinking that we'd kill the Aspen project/brand and reemerge as Simplates, a plugin for Django and Flask. We took a step back from the brink when we realized that Aspen is dispatch plus simplates, and it's worth keeping Aspen for the amalgamation. The basic idea is the same, though: Aspen is becoming a "parasitic web framework," a plugin for Django and Flask.

---

Okay! Wow! Over on the November 2015 call, @pjz and I began talking about the Python 3 port (#524), which led back to the discussion about switching to Werkzeug (see #222 and mention on #357), and then expanded even further: What is the essence of Aspen? Answer: dispatch and simplates. What if we cut out everything non-essential? What if we killed Aspen as a brand and instead folded the dispatcher into Simplates, offering shims for Django and Flask?

Django and Flask are the Python web frameworks of record, with Pyramid/Pylons leading the also-rans:

Framework	PyPI Downloads	PyPI Rank
Django	16,128,764	26
Flask	10,290,820	45
Pyramid + Pylons	2,055,805†	212†
Bottle	1,513,060	260
Aspen	918,591	362
Zope2 + zope.publisher	884,100†	368†
CherryPy	749,732	410
web.py	437,894	604

^† combined

The proposal on this ticket is to throw in the towel. Give up. Call it a day. Admit defeat. Stop writing a web framework.

Instead, we would take the most interesting parts of what we've built—simplates and the filesystem dispatcher—and repackage them under the Simplates brand. Simplates and dispatch are interesting and unique. We've already got a shim for Django. We should be able to adapt that for Flask without too much trouble, so the simplates library would look something like this:

simplates
simplates.dispatch
simplates.renderers
simplates.resources
simplates.shims.django
simplates.shims.flask

This would allow us to invest our time and effort where we're really adding value, and collaborate with others where we're currently reinventing the wheel.

[Changaco]

So algorithm.py is non-essential?

[chadwhitacre]

@Changaco Yeah, that's on the edge. I like algorithm.py, and it's unique to Aspen, but I'm not sure I like it enough to keep Aspen going just for its sake. Perhaps there's a future in which the Flask project finds algorithm.py interesting enough to adopt?

[chadwhitacre]

@Changaco Do you think it's essential? Do you like it enough to keep Aspen going for its sake?

[Changaco]

I think the question is: can we easily do with other frameworks what we do with Aspen's algorithm, for example i18n of responses? If yes then algorithm.py is non-essential.

[chadwhitacre]

• https://pythonhosted.org/Flask-Babel/ (Armin wrote Jinja2 and maintains Babel in addition to writing Flask)
• https://docs.djangoproject.com/en/1.8/topics/i18n/

[chadwhitacre]

Armin wrote Jinja2 and maintains Babel in addition to writing Flask

Which is to say, i18n almost certainly works under Flask at least as well as it works under Aspen/Gratipay. I recall noticing somewhere their solution to injecting appropriate gettext functions into template context, though I don't recall where at the moment.

[Changaco]

I'm not so sure Flask-Babel is better than Liberapay's i18n, but they look similar so it should be possible to port ours to Flask.

i18n was only an example though, the underlying question remains: how hackable is Flask? Are there things we can do by modifying Aspen's algorithm that can't be done by extending Flask?

In any case from Aspen's point of view it makes sense to fold, but for everyone who uses Aspen it'll mean "losing" time learning another framework and migrating.

[chadwhitacre]

@Changaco Do you want to take over maintenance of Aspen? We could fork out Simplates rather than killing Aspen outright. As discussed, the plan is to ship framework shims with Simplates. We could include a shim for Aspen.

[chadwhitacre]

I've started a PR in #527.

[Changaco]

I think the strip down of the code should happen in another repo in any case.

[chadwhitacre]

I think the strip down of the code should happen in another repo in any case.

Okay. To inform this decision, I went and reviewed the history of Simplates and Aspen in Zeta's old public svn repo. Here's the story:

2004-08-20—First commit on Simplates, which was a Zope plugin that provided an object similar to Zope Page Templates, but with Python string replacement instead of ZPT's syntax.

2005-01-04—Last commit on Simplates. At some point in there @beren1hand proclaimed, "What if we had a Zope object that was a Script (Python) [Zope's goofy realization of a Python script that was accessible through the web] plus a ZPT in the same file!" From what I can tell, however, we never actually realized this vision in Zope.

2005-08-21—First commit on httpy, which was later subsumed under lib537. Looks like this included a notion of filesystem dispatch.

2006-07-13—First commit on httpyd, a daemon that used httpy.

2006-09-08—Last commit on httpyd; last real commit on httpy.

2006-09-19—First commit on aspen: "initial import forked from httpyd r1259."

2006-10-11—Commit to aspen with message "ported simplate handler." The port is from the Simplates library. It looks like I was using the email library to parse files: the headers of the message defined the context for Template replacement on the body. Still no arbitrary Python page at this point.

2006-11-13—Last commit on httpy: "this has been moved to Google Code."

2006-11-16—First commit on aspen in Google Code (whence we eventually ported to GitHub).

2006-11-17—Last commit on aspen in the old repo. Migration to Google Code.

[chadwhitacre]

The current Aspen repo, in other words, is a confluence of three other repos:

• Simplates—file format combining both context and templating
• httpy—request/response objects, filesystem dispatch
• httpyd—server

We dropped the server in #324. The talk here is to drop the request/response objects. If we were also dropping filesystem dispatch, then it'd be easy to say that we should fork back out a Simplates repo. Since we're keeping filesystem dispatch, I think we should only fork a new repo if someone is going to keep Aspen going as a web framework. If no-one is going to keep Aspen going, then I think we should stick with the same repo, and simply rebrand it around Simplates.

[Changaco]

Okay, though I think it may be a good idea to keep the master branch in a working state until the new stripped down Aspen is fully functional.

[chadwhitacre]

I think it may be a good idea to keep the master branch in a working state until the new stripped down Aspen is fully functional.

I'm fine with that. If #527 gets too unwieldy then maybe we can start making smaller pull requests against the just-simplates branch to keep things moving along.

[chadwhitacre]

If no-one is going to keep Aspen going, then I think we should stick with the same repo, and simply rebrand it around Simplates.

The way things are going on #527 is that we're planning to stick with the same repo, and the Aspen brand, because it includes filesystem dispatch in addition to simplates. We're talking over there about a "parasitic" web framework that depends on a host framework for most things. Aspen is becoming a plugin for Django and Flask that does filesystem dispatch and simplates.

[pjz]

Ideally there will be an easy migration path from old-aspen to (probably) flask-aspen.

[Changaco]

I had a thought last night: if aspen becomes a framework that can integrate into others, then it should be easy to integrate it with a nano-framework based on algorithm.py, which for lack of a better name (AspenWeb/state_chain.py#3) could be called aspen-standalone, and eventually aspen-legacy if we all migrate away from it.

I think we should use aspen as an umbrella name, split it into separate libraries (aspen-{fs-routing,simplates} for the core and aspen-{django,flask,standalone} for the shells), and move all of those into separate repositories under a new aspen organization. The work in #527 wouldn't be lost, it would be the base for everything except the standalone shell.

I'm willing to help with all this once the rush of launching Liberapay is over.

[chadwhitacre]

if aspen becomes a framework that can integrate into others, then it should be easy to integrate it with a nano-framework

@pjz may have been suggesting something similar at #527 (comment):

When we switched to pure WSGI, we could at least doc how to use it and even provide a minimal server (python -m aspen). We should do the same now.

In other words, #527 should preserve some sort of python -m aspen even in the absence of Django or Flask or another proper web framework.

based on algorithm.py

So far #527 is still using algorithm.py, and I'd expect python -m aspen to make use of that to insert some bare-bones WSGI logic (i.e., a WSGI environ in simplates context instead of a request object).

I think we should use aspen as an umbrella name, split it into separate libraries (aspen-{fs-routing,simplates} for the core

If we're factoring out simplates (#341), it shouldn't be tied to web programming, and therefore shouldn't be aspen-simplates, but just simplates.

aspen-{django,flask,standalone} for the shells

I think the inverse will make more sense for the shells: Flask-Aspen would be the normal name for a Flask extension, and django-aspen for a Django package. I believe the plan on #527 is to land that PR with the shims inline in Aspen, to be factored out into separate libraries later on.

@Changaco Up above at #526 (comment) I offered to let you take over maintenance of Aspen-the-web-framework, because at first it looked like we were letting go of Aspen and moving to Simplates as the primary project/brand. Then we realized that Aspen is really dispatch plus simplates, and we may as well keep the Aspen project/brand for the amalgamation. I agree with @pjz that we should have a barest-of-bones python -m aspen that uses environ and start_response instead of Request and Response objects. I think at this point that if you want to build a fuller-featured web framework on top of Aspen that it should be a new project of your own. :)

[chadwhitacre]

I am having a moment of weakness. I looked at the Gratipay algorithm (#527 (comment)), and realized that we won't have that anymore. Instead we'll have to ... use signals, probably? Then I typed raise Response(400) and I realized I won't be able to do that anymore. What instead?

More context: kennethreitz/records#41. The purpose of Aspen, Postgres.py, etc. has never been to be the most popular ________ in the Python community. The purpose is to be the in-house ________ for Gratipay, so that we can have a higher level of control over our stack.

If we were a closed company, those projects would be proprietary. The fact that they're open-source is a side-effect of our openness. We're not primarily concerned to build large communities around those products themselves.

[pjz]

You'll use whatever Flask or Django or your webframework uses. If you really don't like it, you can wrap your webframework's whatever in whatever you want instead.

As far as having a higher level of control over the stack... I disagree. The point of Aspen is to add features that simply aren't present in any other web framework. In the course of this, we've discovered how much grunt work there is in maintaining the rest of the web framework (request objects, authentication, cookies, session management, etc), and that's what we're trying to move away from by turning Aspen into a plugin of other frameworks. Much like Postgres.py/Records' core value-add is the API (so making it work with as many databases as possible is a good thing), Aspen's core value-add is simplates+filesystem-dispatch, so making it work with as many web frameworks as possible is a good thing. And just as it'd be the edge of insanity to try to write your own database to base Postgres.py/Records' API on, I think it's the edge of insanity to try to write and maintain our own web framework.

[Changaco]

Okay, the Liberapay launch rush is pretty much over, I can spend some time on this.

@whit537 I think being "Gratipay's in-house thing" is one of Aspen's problems. I invite you to reread my previous comment, in which I suggested moving the project into its own org, and splitting it. Your answer was mostly about the package names I used, but the names are details, what's important is that splitting the code could make it cleaner and easier to hack and document.

If you want Aspen to stay "Gratipay's thing" then I guess my only option will be to fork it, but that would be a duplication of effort, a waste of resources. :-/

@pjz The way I see it, it could be more work to move Liberapay to Flask or Django than to clean up and maintain Aspen.

[chadwhitacre]

And just as it'd be the edge of insanity to try to write your own database to base Postgres.py/Records' API on, I think it's the edge of insanity to try to write and maintain our own web framework.

A better parallel here would be writing our own database driver—replacing psycopg2, in other words.

Web	Data
Gratipay, Liberapay	Gratipay, Liberapay
Flask, Django, Aspen 0.x	Postgres.py, Records
gunicorn, uwsgi	psycopg2
nginx, Apache	PostgreSQL

[pjz]

I'm all for helping provide a migration path, but I don't think it's in Aspen's best interest to try and be a standalone webframework.

[Changaco]

That's one thing I don't understand, what exactly is so difficult/problematic about keeping Aspen standalone?

[pjz]

1. Request/Reponse objects/API are complicated. Strings or unicode? calls on them or just data? raise responses or return them? or something else? Lots of decisions that have very little to do with aspen core functionality.
2. web servers are evolving: HTTP2, websockets, etc... Again, very little of which have much to do with aspen core functionality.
3. Less is more: by integrating with other frameworks, we become part of that framework's ecosystem so we don't have to independently implement: session handling, authentication, compression support, 3rd party caching, etc.

[chadwhitacre]

moving the project into its own org

Alright, @Changaco. I'm game. :-)

Reticketed as #547.

[chadwhitacre]

@pjz I think that if @Changaco is willing to put effort into maintaining Aspen as a standalone web framework, then we should take that into account (i.e., the burden doesn't fall on just you and me). I think he's right that shoring up Aspen could prove to be less work than porting Gratipay and Liberapay to Flask or Django.

[pjz]

See discussion of #547... on #547.

[chadwhitacre]

@pjz Sure, ... but did we also decide to keep Aspen as a standalone web framework as @Changaco suggests? Splitting out a new org is orthogonal to that, strictly speaking, and this ticket is about whether to be a standalone web framework.

@pjz You okay to keep Aspen as a full web framework, given @Changaco's intention to contribute effort to that?

[pjz]

I don't care about org, though I'll admit I'm getting tired of retargeting my repos: whit537, gittip, gratipay... it would be nice to have an 'aspen' org that would stay put.

I'm okay with keeping a full web framework as long as it's spun out of the main repo; I don't think churn in the web framework should cause churn in the aspen repo.

[chadwhitacre]

I don't care about org, though I'll admit I'm getting tired of retargeting my repos: whit537, gittip, gratipay... it would be nice to have an 'aspen' org that would stay put.

Blah blah blah, cry me a river ... you weren't even around for the jump from Subversion to Git. ;-)

I'm okay with keeping a full web framework as long as it's spun out of the main repo; I don't think churn in the web framework should cause churn in the aspen repo.

Cool. Sounds like that's the direction @Changaco is planning on taking it.

And in that case, I guess we go ahead and land #527, and then if we want to salvage the http submodule currently on master we do that in a new separate repo? Sound right, @Changaco?

[Changaco]

Request/Reponse objects/API are complicated. Strings or unicode? calls on them or just data? raise responses or return them? or something else? Lots of decisions that have very little to do with aspen core functionality.

I don't see the problem here, if you don't like/want to design APIs then leave it to someone else, use Flask's or Django's API, or even as much of both as possible to the extent that they don't conflict, so that it's easier for people to adopt Aspen.

web servers are evolving: HTTP2, websockets, etc... Again, very little of which have much to do with aspen core functionality.

Liberapay is using HTTP2 right now, it's not something that requires changes to Aspen. I'm not familiar enough with websockets.

Less is more: by integrating with other frameworks, we become part of that framework's ecosystem so we don't have to independently implement: session handling, authentication, compression support, 3rd party caching, etc.

It's too late, we've already implemented all that in Gratipay/Liberapay (though I'm not sure what "compression" you're talking about). Plus, my proposal above was to go both ways: allow Aspen to integrate into other frameworks, but also keep a standalone version.

[pjz]

I don't see the problem here, if you don't like/want to design APIs then leave it to someone else, use Flask's or Django's API, or even as much of both as possible to the extent that they don't conflict, so that it's easier for people to adopt Aspen.

EXACTLY. That's what we're trying to do. Do you realize how much of the standalone server depends on the request object?

[Changaco]

Sound right, @Changaco?

I think we're on the same page, my proposal was to have several separate repos in the new org, that's what you're saying too, right?

[chadwhitacre]

@Changaco Yes, and I hear @pjz reiterating that aspen-core should be separate from the standalone Aspen framework.

[Changaco]

The discussion is continuing in AspenWeb/salon#2.