Wrong handling of urls encoded in UTF8

[GoogleCodeExporter]

What steps will reproduce the problem?
 - enter an url in UTF8 (non-ASCII)

What is the expected output? What do you see instead?
triggers "SubscriptOutOfBounds", or "Error: Improper store into indexable 
object"
(near ZnZincServerAdaptor)

What version of the product are you using? On what operating system?
Pharo 1.4, Seaside 3.0.7

Please provide any additional information below.

Live BUG example: 
 http://book.seaside.st/book/%C3%A1%C4%8D
Seaside on localhost: 
 http://localhost:8080/%C3%A1%C4%8D
 http://localhost:8080/browser/%C3%A1%C4%8D

Original issue reported on code.google.com by [email protected] on 11 Mar 2013 at 10:01

[GoogleCodeExporter]

Zinc has its own URL implementation, ZnUrl, that I think deals correctly with 
the example URL.

Could you try inspecting the result of

  'http://book.seaside.st/book/%C3%A1%C4%8D' asZnUrl 

You will see that the second path segment is equal to 'áč' (internal, 
undecoded format, given a capable Unicode font).

I also thought that Seaside uses its own URL object, WAUrl.

We'll have to focus how exactly the adaptor converts between all these objects.

Could you please provide a stack trace ?

Sven

Original comment by [email protected] on 13 Mar 2013 at 9:02

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

inspect works for ZnURl, the first segment is ByteString, the second is 
WideSring
for WAUrl absolute: 'http://book.seaside.st/book/%C3%A1%C4%8D', both ByteString

--- The full stack ---
Array(Object)>>errorSubscriptBounds:
Array(Object)>>at:
WAUrlEncoder(WAEncoder)>>nextPutAll:
WAUrlEncoder>>nextPutAll:
GRNullCodecStream>>nextPutAll:
[:index :each | 
aDocument nextPut: $/.
    aDocument urlEncoder nextPutAll: each.
    pathParameters isNil
        ifFalse: [self encodePathParametersAt: index on: aDocument]] in WAUrl>>encodePathOn:
OrderedCollection(SequenceableCollection)>>keysAndValuesDo:
WAUrl>>encodePathOn:
[:stream | 
document := WAHtmlDocument on: stream.
    self encodePathOn: document] in WAUrl>>pathString
String class(SequenceableCollection class)>>new:streamContents:
String class(SequenceableCollection class)>>streamContents:
WAUrl>>pathString
WAResponseGenerator>>notFound
[aRequestContext responseGenerator notFound; respond] in 
WADispatcher>>handleFiltered:named:
[^ aBlock value] in [:dispatcher :token | dispatcher handlers
        at: token
        ifAbsent: [^ aBlock value]] in WADispatcher>>handlerAt:ifAbsent:
Dictionary>>at:ifAbsent:
[:dispatcher :token | dispatcher handlers
        at: token
        ifAbsent: [^ aBlock value]] in WADispatcher>>handlerAt:ifAbsent:
[:each | nextValue := binaryBlock value: nextValue value: each] in 
Array(Collection)>>inject:into:
Array(SequenceableCollection)>>do:
Array(Collection)>>inject:into:
WADispatcher>>handlerAt:with:
WADispatcher>>handlerAt:ifAbsent:
WADispatcher>>handleFiltered:named:
WADispatcher>>handleFiltered:
[self filter handleFiltered: aRequestContext] in 
WADispatcher(WARequestHandler)>>handle:
BlockClosure>>on:do:
WACurrentRequestContext class(WADynamicVariable class)>>use:during:
[WACurrentRequestContext use: self during: aBlock] in 
WARequestContext>>push:during:
BlockClosure>>ensure:
WARequestContext>>push:during:
WADispatcher(WARequestHandler)>>handle:
[self requestHandler handle: aRequestContext] in 
ZnZincServerAdaptor(WAServerAdaptor)>>handleRequest:
BlockClosure>>on:do:
ZnZincServerAdaptor(WAServerAdaptor)>>handleRequest:
ZnZincServerAdaptor(WAServerAdaptor)>>handle:
[self handle: context;
         responseFrom: context] in ZnZincServerAdaptor(WAServerAdaptor)>>process:
BlockClosure>>ensure:
ZnZincServerAdaptor(WAServerAdaptor)>>process:
ZnSeasideServerAdaptorDelegate>>handleRequest:
 - - - - - - - - - - - - - - -  
            - - - - - - - - - - - - - - - - - -
[self delegate isNil
        ifTrue: [ZnResponse notFound: request uri]
        ifFalse: [self delegate handleRequest: request]] in ZnManagingMultiThreadedServer(ZnSingleThreadedServer)>>handleRequest:
ZnManagingMultiThreadedServer(ZnSingleThreadedServer)>>authenticateRequest:do:
ZnManagingMultiThreadedServer(ZnSingleThreadedServer)>>handleRequest:
ZnManagingMultiThreadedServer(ZnMultiThreadedServer)>>executeRequestResponseLoop
On:
[self executeRequestResponseLoopOn: stream] in [[self 
executeRequestResponseLoopOn: stream]
        ensure: [self log debug: 'Closing stream'.
            self closeSocketStream: stream]] in [[[self executeRequestResponseLoopOn: stream]
        ensure: [self log debug: 'Closing stream'.
            self closeSocketStream: stream]]
        ifCurtailed: [self log debug: 'Destroying socket'.
            socket destroy]] in ZnManagingMultiThreadedServer(ZnMultiThreadedServer)>>serveConnectionsOn:
BlockClosure>>ensure:
[[self executeRequestResponseLoopOn: stream]
        ensure: [self log debug: 'Closing stream'.
            self closeSocketStream: stream]] in [[[self executeRequestResponseLoopOn: stream]
        ensure: [self log debug: 'Closing stream'.
            self closeSocketStream: stream]]
        ifCurtailed: [self log debug: 'Destroying socket'.
            socket destroy]] in ZnManagingMultiThreadedServer(ZnMultiThreadedServer)>>serveConnectionsOn:
BlockClosure>>ifCurtailed:
[[[self executeRequestResponseLoopOn: stream]
        ensure: [self log debug: 'Closing stream'.
            self closeSocketStream: stream]]
        ifCurtailed: [self log debug: 'Destroying socket'.
            socket destroy]] in ZnManagingMultiThreadedServer(ZnMultiThreadedServer)>>serveConnectionsOn:
[self value.
    Processor terminateActive] in BlockClosure>>newProcess

Original comment by [email protected] on 13 Mar 2013 at 10:15

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

>> for WAUrl absolute: 'http://book.seaside.st/book/%C3%A1%C4%8D', both 
ByteString

is that correct ?

That would mean that WAUrl stores UTF-8 encoded data as a plain ByteString 
internally - weird and not very Smalltalk like IMHO.

Also, running Seaside 3.1 on Pharo 2.0 I get no stack trace , just the 
exception printString in the reply - while 'Allow debugging of errors' is true. 
How do I get the stack trace / debugger up ?

Original comment by [email protected] on 13 Mar 2013 at 10:33

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

It seems that WAUrl can exist in two forms, encoded and decoded. The 'correct' 
conversion, IMO, is then

(WAUrl absolute: 'http://book.seaside.st/book/%C3%A1%C4%8D')
    decodedWith: (GRCodec forEncoding: 'utf-8').

which is what 

ZnZincSeasideAdaptor>>requestUrlFor: aZincRequest
    ^ (WAUrl absolute: aZincRequest uri greaseString)
        decodedWith: self codec.

does.

The reverse, the rendering, fails with the normal #printString because a Null 
Codec is used by default.

That is also what is happening in your stacktrace: Seaside tries to print a not 
found error but can't print the URL it cannot find because of an encoding 
problem.

Original comment by [email protected] on 13 Mar 2013 at 10:44

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

To get the debugger or stack trace, I'm simply opening the url in web browser.
I you try the one from seaside book, the page is broken (pier showed me "not 
found error" only once)

Original comment by [email protected] on 13 Mar 2013 at 11:03

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

It is the same on Pharo 2.0, with Seaside 3.1.

dispatcher /
with default WAErrorHandler, it shows properly "Not Found /%C3%A1%C4%8D" 
with WAExceptionHandler, SubscriptOutOfBounds: 270
with WADebugErrorHandler raises debugger

Original comment by [email protected] on 13 Mar 2013 at 1:02

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

This isn't a Zinc issue, it's a general issue. WAUrl >> #pathString does URL 
and HTML escaping. However it forgets to do encoding before url escaping.

Right know I'm not sure whether we should really do URL and HTML escaping since 
we're just printing a plain text string.

Grml Google Code, the quality I come to expect from Google.

Original comment by [email protected] on 13 Mar 2013 at 1:24

• Changed title: Wrong handling of urls encoded in UTF8
• Added labels: Encoding
• Removed labels: ****

[GoogleCodeExporter]

Here is a quick patch/hack to get a proper not found showing the encoded URL

WAUrl>>pathString
    "Answer the path converted to a string."

    ^ String streamContents: [ :stream | 
        | document |
        document := WAHtmlDocument on: stream codec: (GRCodec forEncoding: 'utf-8').
        self encodePathOn: document ]

Original comment by [email protected] on 13 Mar 2013 at 8:09

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by [email protected] on 30 Mar 2013 at 4:44

• Changed state: Started
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by [email protected] on 8 Sep 2013 at 2:56

• Changed state: Fixed
• Added labels: Version-Seaside3.1
• Removed labels: ****

[GoogleCodeExporter]

reopen, unit tests are failing
WAServerAdaptor >> #requestCookiesFor: will fail

Original comment by [email protected] on 8 Sep 2013 at 3:08

• Changed state: Started
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Name: Seaside-Adaptors-Comanche-pmm.79
Author: pmm
Time: 11 September 2013, 12:45:22 pm
UUID: d44a1b46-59c1-4d52-b617-53a0a535b40f
Ancestors: Seaside-Adaptors-Comanche-pmm.78

Issue 749:  Wrong handling of urls encoded in UTF8
http://code.google.com/p/seaside/issues/detail?id=749

Name: Seaside-Core-pmm.796
Author: pmm
Time: 11 September 2013, 12:45:55 pm
UUID: 2ed02514-a37b-4fc5-b3a3-084971e1ebea
Ancestors: Seaside-Core-pmm.795

Issue 749:  Wrong handling of urls encoded in UTF8
http://code.google.com/p/seaside/issues/detail?id=749

Name: Seaside-Tests-Core-pmm.270
Author: pmm
Time: 11 September 2013, 12:46:15 pm
UUID: da11311a-d91f-4481-9f5d-1df9f5cf5abe
Ancestors: Seaside-Tests-Core-pmm.269

Issue 749:  Wrong handling of urls encoded in UTF8
http://code.google.com/p/seaside/issues/detail?id=749

fuck you Google Code

Original comment by [email protected] on 11 Sep 2013 at 10:47

• Changed state: Fixed
• Added labels: ****
• Removed labels: ****