Upgrading Aspen to 0.37+ #263

techtonik · 2015-06-18T06:33:49Z

https://github.com/gratipay/aspen-python/blob/master/Changelog.md

Scenario:

add startapp.py script to run website
- make it WSGI compatible for production
- make it run as a standalone web app
~~[ ] find a way how aspen can discover it automatically (python -m aspen doesn't work correctly on windows AspenWeb/pando.py#440)~~

Current issues:

when startapp.py is run, it doesn't show host and port

The startapp.py can be patched to show it, but that would complicate every Aspen app with such custom code. It is better to expose Website.run() method that can also get the host/port setting from site configuration.

~~[ ] when python -m aspen is run, it shows port, but not host~~
both ways can't find the site

Reading configuration from defaults, environment, and command line.
  changes_reload         False                          default                 
  charset_dynamic        UTF-8                          default                 
  charset_static         None                           default                 
  configuration_scripts  []                             default                 
  indices                [u'index.html', u'index.json', u'index', u'index.html.spt', u'index.json.spt', u'index.spt'] default                 
  list_directories       False                          default                 
  logging_threshold      0                              default                 
  media_type_default     text/plain                     default                 
  media_type_json        application/json               default                 
  project_root           None                           default                 
  renderer_default       stdlib_percent                 default                 
  show_tracebacks        False                          default                 
  www_root               None                           default                 
project_root not configured (no template bases, etc.).
...
IOError: [Errno 2] No such file or directory: u'C:\\gratipay\\inside\\'

Failed to import pygments. Tracebacks won't be highlighted.

Also here - AspenWeb/pando.py#440

This doesn't show the URL and the port when run (AspenWeb/pando.py#440)

chadwhitacre · 2015-06-18T12:33:24Z

find a way how aspen can discover it automatically

There is no aspen and there won't be.

I don't use localhost, because I am running this on remote node

techtonik · 2015-06-18T12:40:10Z

python startapp.py fails with a strange error:

pid-6419 thread-140672088307520 (MainThread) project_root not configured (no template bases, etc.).
Traceback (most recent call last):
  File "startapp.py", line 11, in <module>
    application = Website([])
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/website.py", line 
35, in __init__
    self.configure(argv)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/configuration/__in
it__.py", line 278, in configure
    render_module = entrypoint.load()
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/pkg_resources.py", line 
2047, in load
    if require: self.require(env, installer)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/pkg_resources.py", line 
2060, in require
    working_set.resolve(self.dist.requires(self.extras),env,installer)))
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/pkg_resources.py", line 
592, in resolve
    raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (first 2.0.0 (/home/techtonik/inside.gratipay.com/xxx/lib/python2.7/site-pa
ckages), Requirement.parse('first==2.0.1'))

techtonik · 2015-06-18T20:45:52Z

@whit537, @pjz I stuck. How can I specify www_root from wsgi_script (startapp,py in this PR)?

techtonik · 2015-06-18T21:02:17Z

Looks like Website accepts command line argument as parameters. Very unusual.

chadwhitacre · 2015-06-18T21:03:12Z

@techtonik That's legacy behavior from when we used to provide a server; see AspenWeb/pando.py#374.

chadwhitacre · 2015-06-18T21:05:28Z

How can I specify www_root from wsgi_script (startapp,py in this PR)?

Have you tried application.www_root = 'foo'? Fwiw, on gratipay.com it looks like we're configuring that via the ASPEN_WWW_ROOT environment variable.

default conventions AspenWeb/pando.py#437

techtonik · 2015-06-18T21:20:05Z

This is AspenWeb/pando.py#436 and I need help here, because I don't know how it supposed to work before and how it should look like now.

Traceback (most recent call last):
  File "startapp.py", line 16, in <module>
    application = Website(aspen_config)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/website.py", line 
35, in __init__
    self.configure(argv)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/configuration/__in
it__.py", line 295, in configure
    self.run_config_scripts()
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/configuration/__in
it__.py", line 338, in run_config_scripts
    execfile(filepath, {'website': self})
  File "/home/techtonik/inside.gratipay.com/configure-aspen.py", line 39, in <module>
    add_nav_to_website(website)
  File "/home/techtonik/inside.gratipay.com/configure-aspen.py", line 27, in add_nav_to_website
    website.nav = NavItem(website, None, website.www_root, '')
  File "/home/techtonik/inside.gratipay.com/nav.py", line 49, in __init__
    context = get_simplate_context(website, simplate)
  File "/home/techtonik/inside.gratipay.com/nav.py", line 17, in get_simplate_context
    resource = resources.get(request)
TypeError: get() takes exactly 2 arguments (1 given)

chadwhitacre · 2015-06-18T21:25:03Z

Per the signatures you linked at AspenWeb/pando.py#436 (comment), try changing:

def get_simplate_context(website, fs):
    request = Request()
    request.fs = fs
    request.website = website
    resource = resources.get(request)
    return {} if isinstance(resource, StaticResource) else resource.pages[0]

to:

def get_simplate_context(website, fs):
    resource = resources.get(website, fs)
    return {} if isinstance(resource, StaticResource) else resource.pages[0]

techtonik · 2015-06-18T21:25:34Z

Have you tried application.www_root = 'foo'?

I tried application.set(), but it looks like it is too late to set it when application object was already created. Might be a mistake on my side.

Fwiw, on gratipay.com it looks like we're configuring that via the ASPEN_WWW_ROOT environment variable.

Yes, I saw. Tried that with os.environ. Didn't work. Anyway, that's a solved problem. The stacktrace about get() is more important (as well as canonizer.py content and configure-aspen.py, which are likely outdated).

chadwhitacre · 2015-06-18T21:25:45Z

Actually:

def get_simplate_context(website, fs):
    resource = resources.get(website, fs)
    return {} if isinstance(resource, StaticResource) else resource.pages[0]

(No need to instantiate a Request anymore.)

techtonik · 2015-06-18T21:31:58Z

What does this function do? Without a docstring I don't really get it. For standard template (like Jinja2), the context is a dict of template variables - http://jinja.pocoo.org/docs/dev/api/#jinja2.runtime.Context And what is the context here?

@whit537

@whit537 in #263 (comment)

chadwhitacre · 2015-06-18T21:34:55Z

It gets a simplate context, which is an empty dictionary in the case of a static file, or the zeroth page within the simplate otherwise.

chadwhitacre · 2015-06-18T21:36:52Z

"All simplate pages are cached in a compiled state," and for pages[0] this compiled state is a dictionary, which is used as the template context for rendering the content pages.

techtonik · 2015-06-18T21:37:02Z

Next traceback:

Traceback (most recent call last):
  File "startapp.py", line 16, in <module>
    application = Website(aspen_config)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/website.py", line 
35, in __init__
    self.configure(argv)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/configuration/__in
it__.py", line 295, in configure
    self.run_config_scripts()
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/configuration/__in
it__.py", line 338, in run_config_scripts
    execfile(filepath, {'website': self})
  File "/home/techtonik/inside.gratipay.com/configure-aspen.py", line 39, in <module>
    add_nav_to_website(website)
  File "/home/techtonik/inside.gratipay.com/configure-aspen.py", line 27, in add_nav_to_website
    website.nav = NavItem(website, None, website.www_root, '')
  File "/home/techtonik/inside.gratipay.com/nav.py", line 52, in __init__
    child = NavItem(website, self, new_fs, slug)
  File "/home/techtonik/inside.gratipay.com/nav.py", line 46, in __init__
    context = get_simplate_context(website, simplate)
  File "/home/techtonik/inside.gratipay.com/nav.py", line 14, in get_simplate_context
    resource = resources.get(website, fs)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/resources/__init__
.py", line 176, in get
    raise entry.exc[0]  # TODO Why [0] here, and not above?
aspen.exceptions.LoadError: Traceback (most recent call last):
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/resources/__init__
.py", line 168, in get
    entry.resource = load(website, fspath, mtime)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/resources/__init__
.py", line 137, in load
    resource = Class(website, fspath, raw, media_type, is_bound, mtime)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/resources/simplate
.py", line 47, in __init__
    self.pages = self.compile_pages(pages)
  File "/home/techtonik/inside.gratipay.com/xxx/local/lib/python2.7/site-packages/aspen/resources/simplate
.py", line 127, in compile_pages
    exec one in context    # mutate context
  File "/home/techtonik/inside.gratipay.com/www/howto/index.spt", line 4, in <module>
    nav_children = [ name.rsplit('.', 1)[0] for name in os.listdir('howto')
OSError: [Errno 2] No such file or directory: 'howto'

chadwhitacre · 2015-06-19T01:27:21Z

What's os.getcwd() in howto/index.spt:4?

techtonik · 2015-06-19T06:18:53Z

  File "/home/techtonik/inside.gratipay.com/www/howto/index.spt", line 3
    print os.getcwd()
           ^
SyntaxError: invalid syntax

Python is 2.7.6. Is Aspen based on from future?

techtonik · 2015-06-19T06:20:08Z

What's os.getcwd() in howto/index.spt:4?

/home/techtonik/inside.gratipay.com

techtonik · 2015-06-19T06:23:10Z

See, project_root and www_root are confusing AspenWeb/pando.py#438 (comment) site_aspen is a better name for junk drawer (c) http://aspen.io/project_root/ =) (moved to AspenWeb/pando.py#447)

chadwhitacre · 2015-06-19T11:52:16Z

Is Aspen based on from future?

Yes! :-)

We use __future__ imports across Aspen, which means that these apply to simplates as well.

techtonik · 2015-06-19T11:58:56Z

Although this doesn't explain what directory should be CWD for simplates.

chadwhitacre · 2015-06-19T12:05:50Z

@techtonik Well, it's looking for howto in CWD, so presumably it expects it to be www_root. Aaaaaand that's a change that's definitely missing from the Changelog: Aspen no longer changes the working directory for you. For the present case I suggest doing:

os.listdir(os.path.join(website.www_root, 'howto'))

techtonik · 2015-06-19T12:18:26Z

@whit537 this looks like a place for some API to get files from the www/ filesystem. Or it is just the perfectionist in me complaining..

techtonik · 2015-07-01T18:00:01Z

@whit537 this is ready for review. I am trying to add Travis tests, but these are not critical.

techtonik · 2015-07-01T18:17:05Z

@whit537 take a look.

chadwhitacre · 2015-07-06T15:48:27Z

README.md

@@ -2,6 +2,16 @@

 [![Build Status](https://travis-ci.org/gratipay/inside.gratipay.com.svg)](https://travis-ci.org/gratipay/inside.gratipay.com)

+**NOTE**: The quickstart may not work until vendorized
+`virtualenv` + `pip` are not upgraded to latest versions


s/not//, no?

Thanks. Fixed.

Review comments

techtonik · 2015-07-09T23:30:53Z

@whit537 addressed review comments and cleaned up. Please take a look.

chadwhitacre · 2015-07-09T23:33:07Z

Makefile

-	./$(env_bin)/honcho -e defaults.env,local.env run ./env/bin/aspen \
-		--network_address :8536 --www_root=./www/ --project_root=./
+	./$(env_bin)/honcho -e defaults.env,local.env run ./env/bin/python \
+		./startapp.py --port=8537


This changes the port from 8536 to 8537. Was that intentional? I had chosen 8536 so it could be run simultaneously with gratipay.com, which runs on 8537 by default.

Not intentional. Thought that 8537 is like default for all GP sites. Fixed now.

techtonik · 2015-07-09T23:38:20Z

Force pushed last commit to fix Travis test as well.

techtonik · 2015-07-09T23:52:38Z

@whit537 looks like it can finally be merged.

chadwhitacre · 2015-07-10T00:21:18Z

Here goes nothin'! :-)

Upgrading Aspen to 0.37+

chadwhitacre · 2015-07-10T00:37:26Z

Appears to be working, @techtonik. Great work! 💃

!m @techtonik

techtonik · 2015-07-10T04:19:06Z

Awesome!

Thanks for review. =)

See #263 (comment)

techtonik added 2 commits June 18, 2015 08:56

Upgrading Aspen - add startup script for Aspen 0.37

d4fcd44

This doesn't show the URL and the port when run (AspenWeb/pando.py#440)

Bring Aspen 0.37 into dependencies and add state of branch to README.md

e986fef

Show the host:port for application access

8449bd2

I don't use localhost, because I am running this on remote node

techtonik added 2 commits June 18, 2015 22:57

Aspen 0.37 needs newer first

6b57f4a

It also depends on filesystem_tree

a5fc4b6

Set www root and project root, because Aspen 0.37 comes without

981fefc

default conventions AspenWeb/pando.py#437

Trying to fix stacktrace from updated Aspen as proposed by

6dd9bcc

@whit537 in #263 (comment)

techtonik mentioned this pull request Jun 19, 2015

aspen.resources API break not documented AspenWeb/pando.py#436

Closed

techtonik added 2 commits July 1, 2015 21:01

Try to run server in background

cc0f226

Add sanity test for curl

64acc5b

Make HOST, PORT dynamic

27c80ce

chadwhitacre reviewed Jul 6, 2015
View reviewed changes

chadwhitacre mentioned this pull request Jul 6, 2015

Radar 14 #283

Closed

techtonik added 5 commits July 10, 2015 02:27

Makefile - new command to run website (no aspen server anymore)

b776cfb

Update Procfile

9b6c8ac

Update README.md

fd28562

Review comments

README.md describe complete instruction for make-less setup

d62f81e

Review comments about algorithm.py

e7e00ce

techtonik force-pushed the startapp branch from b33df93 to e7e00ce Compare July 9, 2015 23:29

chadwhitacre reviewed Jul 9, 2015
View reviewed changes

Fix the port back to 8536

fbb9fab

techtonik force-pushed the startapp branch from 959d9fc to fbb9fab Compare July 9, 2015 23:37

chadwhitacre added a commit that referenced this pull request Jul 10, 2015

Merge pull request #263 from gratipay/startapp

cb0042e

Upgrading Aspen to 0.37+

chadwhitacre merged commit cb0042e into master Jul 10, 2015

chadwhitacre deleted the startapp branch July 10, 2015 00:33

techtonik mentioned this pull request Jul 10, 2015

Upgrading Aspen #133

Closed

techtonik added a commit that referenced this pull request Jul 10, 2015

Default port for inside is 8536

1ef7ebd

See #263 (comment)

andrew mentioned this pull request Dec 23, 2016

Make listing match GitHub listing octobox/octobox#147

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrading Aspen to 0.37+ #263

Upgrading Aspen to 0.37+ #263

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

techtonik commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 19, 2015

techtonik commented Jun 19, 2015

techtonik commented Jun 19, 2015

techtonik commented Jun 19, 2015

chadwhitacre commented Jun 19, 2015

techtonik commented Jun 19, 2015

chadwhitacre commented Jun 19, 2015

techtonik commented Jun 19, 2015

techtonik commented Jul 1, 2015

techtonik commented Jul 1, 2015

chadwhitacre Jul 6, 2015

techtonik Jul 6, 2015

techtonik commented Jul 9, 2015

chadwhitacre Jul 9, 2015

techtonik Jul 9, 2015

techtonik commented Jul 9, 2015

techtonik commented Jul 9, 2015

chadwhitacre commented Jul 10, 2015

chadwhitacre commented Jul 10, 2015

techtonik commented Jul 10, 2015

Upgrading Aspen to 0.37+ #263

Upgrading Aspen to 0.37+ #263

Conversation

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

techtonik commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

chadwhitacre commented Jun 18, 2015

techtonik commented Jun 18, 2015

chadwhitacre commented Jun 19, 2015

techtonik commented Jun 19, 2015

techtonik commented Jun 19, 2015

techtonik commented Jun 19, 2015

chadwhitacre commented Jun 19, 2015

techtonik commented Jun 19, 2015

chadwhitacre commented Jun 19, 2015

techtonik commented Jun 19, 2015

techtonik commented Jul 1, 2015

techtonik commented Jul 1, 2015

chadwhitacre Jul 6, 2015

Choose a reason for hiding this comment

techtonik Jul 6, 2015

Choose a reason for hiding this comment

techtonik commented Jul 9, 2015

chadwhitacre Jul 9, 2015

Choose a reason for hiding this comment

techtonik Jul 9, 2015

Choose a reason for hiding this comment

techtonik commented Jul 9, 2015

techtonik commented Jul 9, 2015

chadwhitacre commented Jul 10, 2015

chadwhitacre commented Jul 10, 2015

techtonik commented Jul 10, 2015