ckanext-showcase breaks search for non-ASCII tags #34

torfsen · 2016-04-26T09:15:36Z

If I enable ckanext-showcase in my CKAN 2.5.2 then searching (using the standard package search) for a tag that contains non-ASCII characters results in an HTTP 500 Server Error:

Error - <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode character u'\\xf6' in position 9: ordinal not in range(128)
URL: http://172.16.16.17:9000/data/dataset?tags=bev%C3%B6lkerung
File '/usr/lib/ckan/default/lib/python2.7/site-packages/weberror/errormiddleware.py', line 171 in __call__
  app_iter = self.application(environ, sr_checker)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
  resp = self.call_func(req, *args, **self.kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
  return self.func(req, *args, **kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/fanstatic/publisher.py', line 234 in __call__
  return request.get_response(self.app)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
  application, catch_exc_info=False)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
  app_iter = application(self.environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
  resp = self.call_func(req, *args, **self.kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
  return self.func(req, *args, **kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/fanstatic/injector.py', line 54 in __call__
  response = request.get_response(self.app)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
  application, catch_exc_info=False)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
  app_iter = application(self.environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/config/middleware.py', line 389 in inner
  result = application(environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/beaker/middleware.py', line 73 in __call__
  return self.app(environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/beaker/middleware.py', line 155 in __call__
  return self.wrap_app(environ, session_start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/routes/middleware.py', line 131 in __call__
  response = self.app(environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/wsgiapp.py', line 125 in __call__
  response = self.dispatch(controller, environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/wsgiapp.py', line 324 in dispatch
  return controller(environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 338 in __call__
  res = WSGIController.__call__(self, environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 221 in __call__
  response = self._dispatch_call()
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 172 in _dispatch_call
  response = self._inspect_call(func)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 107 in _inspect_call
  result = self._perform_call(func, args)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 60 in _perform_call
  return func(**args)
File '/usr/lib/ckan/default/src/ckan/ckan/controllers/package.py', line 268 in search
  query = get_action('package_search')(context, data_dict)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py', line 416 in wrapped
  result = _action(context, data_dict, **kw)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/action/get.py', line 1895 in package_search
  data_dict = item.before_search(data_dict)
File '/usr/lib/ckan/default/src/ckanext-showcase/ckanext/showcase/plugin.py', line 253 in before_search
  DATASET_TYPE_NAME)
UnicodeEncodeError: 'ascii' codec can't encode character u'\\xf6' in position 9: ordinal not in range(128)


CGI Variables
-------------
  AUTH_TYPE: 'cookie'
  CKAN_CURRENT_URL: '/dataset?tags%3Dbev%25C3%25B6lkerung'
  CKAN_LANG: 'de'
  CKAN_LANG_IS_DEFAULT: True
  CONTENT_TYPE: '; charset=utf-8'
  CONTEXT_DOCUMENT_ROOT: '**********'
  DOCUMENT_ROOT: '**********'
  GATEWAY_INTERFACE: 'CGI/1.1'
  HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5'
  HTTP_CONNECTION: 'keep-alive'
  HTTP_COOKIE: '**********'
  HTTP_HOST: '172.16.16.17:9000'
  HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:45.0) Gecko/20100101 Firefox/45.0'
  PATH_INFO: '/dataset'
  PATH_TRANSLATED: '**********'
  QUERY_STRING: 'tags=bev%C3%B6lkerung'
  REMOTE_ADDR: '172.16.16.1'
  REMOTE_PORT: '51321'
  REMOTE_USER: u'admin'
  REMOTE_USER_DATA: 'userid_type:unicode'
  REMOTE_USER_TOKENS: ['']
  REQUEST_METHOD: 'GET'
  REQUEST_SCHEME: 'http'
  REQUEST_URI: '/data/dataset?tags=bev%C3%B6lkerung'
  SCRIPT_FILENAME: '/etc/ckan/default/apache.wsgi'
  SCRIPT_NAME: '/data'
  SERVER_ADDR: '172.16.16.17'
  SERVER_ADMIN: '[no address given]'
  SERVER_NAME: '172.16.16.17'
  SERVER_PORT: '9000'
  SERVER_PROTOCOL: 'HTTP/1.1'
  SERVER_SIGNATURE: '<address>Apache/2.4.7 (Ubuntu) Server at 172.16.16.17 Port 9000</address>\\n'
  SERVER_SOFTWARE: 'Apache/2.4.7 (Ubuntu)'


WSGI Variables
--------------
  application: <fanstatic.publisher.Delegator object at 0xa6b9042c>
  beaker.cache: <beaker.cache.CacheManager object at 0xa6b90e0c>
  beaker.get_session: <bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0xa6bf42ac>>
  beaker.session: {'_accessed_time': 1461661626.80025, '_creation_time': 1461661626.80025}
  fanstatic.needed: <fanstatic.core.NeededResources object at 0xa67dd06c>
  mod_wsgi.application_group: 'localhost:9000|/data'
  mod_wsgi.callable_object: 'application'
  mod_wsgi.enable_sendfile: '0'
  mod_wsgi.handler_script: ''
  mod_wsgi.input_chunked: '0'
  mod_wsgi.listener_host: ''
  mod_wsgi.listener_port: '9000'
  mod_wsgi.process_group: 'ckan_default'
  mod_wsgi.queue_start: '1461661614370370'
  mod_wsgi.request_handler: 'wsgi-script'
  mod_wsgi.script_reloading: '1'
  mod_wsgi.version: (3, 4)
  paste.cookies: (<SimpleCookie: auth_tkt='**********'>, 'auth_tkt="**********"')
  paste.parsed_dict_querystring: (MultiDict([('tags', 'bev\\xc3\\xb6lkerung')]), 'tags=bev%C3%B6lkerung')
  paste.parsed_querystring: ([('tags', 'bev\\xc3\\xb6lkerung')], 'tags=bev%C3%B6lkerung')
  paste.registry: <paste.registry.Registry object at 0xa6b9050c>
  paste.throw_errors: True
  pylons.action_method: <bound method PackageController.search of <ckan.controllers.package.PackageController object at 0xa68d418c>>
  pylons.controller: <ckan.controllers.package.PackageController object at 0xa68d418c>
  pylons.environ_config: {'session': 'beaker.session', 'cache': 'beaker.cache'}
  pylons.pylons: <pylons.util.PylonsContext object at 0xa67e8eec>
  pylons.routes_dict: {'action': u'search', 'controller': u'package'}
  repoze.who.api: <repoze.who.api.API object at 0xa67c1d4c>
  repoze.who.identity: <repoze.who identity (hidden, dict-like) at 2828288620>
  repoze.who.logger: <logging.Logger object at 0xa6b90e4c>
  repoze.who.plugins: {'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator object at 0xa6b86acc>, 'friendlyform': <FriendlyFormPlugin 2797144044>, 'auth_tkt': <CkanAuthTktCookiePlugin 2793149548>}
  routes.route: <routes.route.Route object at 0xa680f06c>
  routes.url: <routes.util.URLGenerator object at 0xa6b60f6c>
  webob._parsed_query_vars: (GET([('tags', 'bev\\xc3\\xb6lkerung')]), 'tags=bev%C3%B6lkerung')
  webob.adhoc_attrs: {'response': <Response at 0xa67dd0cc 200 OK>, 'language': 'en-us'}
  wsgi process: 'Multi process AND threads (?)'
  wsgi.file_wrapper: <built-in method file_wrapper of mod_wsgi.Adapter object at 0xb16cfd10>
  wsgiorg.routing_args: (<routes.util.URLGenerator object at 0xa6b60f6c>, {'action': u'search', 'controller': u'package'})

If I disable ckanext-showcase then the search works fine.

Interestingly, searching for a query string that contains non-ASCII characters works fine.

The text was updated successfully, but these errors were encountered:

ckanext-showcase automatically hides showcases from package searches when they are not explicitly requested. The code assumes that the filter queries of the search are pure ASCII. Using non-ASCII filter queries therefore breaks CKAN's package search. This commit makes non-ASCII filter queries work and adds a corresponding test case.

ckanext-showcase automatically hides showcases from package searches when they are not explicitly requested. The code assumes that the filter queries of the search are pure ASCII. Using non-ASCII filter queries therefore breaks CKAN's package search. This commit makes non-ASCII filter queries work and adds a corresponding test case. Signed-off-by: blagoja <[email protected]>

torfsen mentioned this issue Apr 26, 2016

Fix #34: search with nonascii filter queries #35

Closed

brew closed this as completed in 1800b0e Jun 22, 2016

torfsen mentioned this issue Oct 15, 2016

Showcase plugin makes Error 500 searches on tags containing special characters #46

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ckanext-showcase breaks search for non-ASCII tags #34

ckanext-showcase breaks search for non-ASCII tags #34

torfsen commented Apr 26, 2016 •

edited

Loading

ckanext-showcase breaks search for non-ASCII tags #34

ckanext-showcase breaks search for non-ASCII tags #34

Comments

torfsen commented Apr 26, 2016 • edited Loading

torfsen commented Apr 26, 2016 •

edited

Loading