Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request streaming with Zinc #1196

Merged
merged 42 commits into from
Dec 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
aa6da59
Moved WAExternalFile to Seaside-Core as I need it for streamed uploads
theseion Jul 23, 2019
4337401
Added option extention to ZnServer for configuring streamed uploads
theseion Jul 23, 2019
b393421
Modified ZnZincServerAdaptor>>configureServerForBinaryReading.
theseion Jul 23, 2019
b793b49
Refactored ZnZincServerAdaptor to work with streamed uploads
theseion Jul 23, 2019
b5c74b3
Improved separation between Seaside and Zinc
theseion Jul 23, 2019
326f965
Unified WAFile and WAExternalFile in common hierarchy
theseion Apr 15, 2020
0db210f
Adapted WAUploadCallback>>valueForField: to accept all kinds of WAAbs…
theseion Apr 15, 2020
405b359
When decoding file parts in multipart streams, treat parts with no co…
theseion Apr 15, 2020
b6d9641
Added #size, #exists, and #fileReference methods to WAExternalFile
theseion Apr 15, 2020
9ac2e0f
When the Content-Type header isn't set correctly ZnZincServerAdaptor>…
theseion Apr 20, 2020
708869e
Moved ZnRingBuffer and ZnStreamingMultiPartFormDataEntity to Zinc-Sea…
theseion May 20, 2020
7b1a375
Added Zinc overrides to fix default encoding
theseion May 20, 2020
08c1b7e
Renamed Zinc-Seaside to Seaside-Zinc and split the request streaming …
theseion May 21, 2020
2ea7b2a
Renamed Zinc-Seaside to Seaside-Zinc for GemStone too, as there's no …
theseion May 21, 2020
89e0459
Reverted renaming of Zinc-Seaside to Seaside-Zinc. We'll do that later.
theseion May 21, 2020
74dbe2b
Fixed ZnRingBufferTest>>testEquals for older Pharo versions
theseion May 23, 2020
24ceb75
Removed GRPharoPlatform>>newTemporaryFileReference and replaced it wi…
theseion May 27, 2020
7cc9e84
fix(streaming): properly writhe space and crlf
theseion Jun 4, 2022
569b76b
Use #writeFileStreamOn:do:binary: instead of inventing a new method i…
theseion Jun 5, 2022
91565ff
Fix ZnZincStreamServerAdaptor to work on Pharo 11+
jbrichau Dec 17, 2023
02cf3ec
revert change
jbrichau Dec 17, 2023
7935159
remove Seaside-Zinc packages from branch
jbrichau Dec 17, 2023
0adf392
remove Seaside-Zinc packages from the branch
jbrichau Dec 17, 2023
31d3db5
Moved all classes from Zinc-Seaside-Pharo to Zinc-Seaside
jbrichau Dec 17, 2023
2421d69
Fix GemStone incompatibilities
jbrichau Dec 17, 2023
2025af2
Remove over-writes from Zinc classes (breaks on GemStone?)
jbrichau Dec 17, 2023
1fbbf24
revert addition of Swazoo adaptor to the baseline
jbrichau Dec 17, 2023
b38d7a2
re-add Swazoo for older GemStone versions
jbrichau Dec 17, 2023
e9c28c2
Move Pharo-specific ZnRingBuffer code to Pharo-specific package and s…
jbrichau Dec 17, 2023
48bc2f7
renamed the Zinc-Seaside, Zinc-Seaside-Pharo packages to Seaside-Zinc…
jbrichau Dec 17, 2023
77c6c95
removed Zinc-Seaside* packages
jbrichau Dec 17, 2023
bf2e363
Renamed packages in BaselineOfSeaside3
jbrichau Dec 17, 2023
652b1fb
Fix merge conflict (reverting change on master...)
jbrichau Dec 17, 2023
6508746
Merge branch 'master' into streaming-uploads
jbrichau Dec 17, 2023
2959e99
Fix missing method
jbrichau Dec 17, 2023
029ed34
Default method for ringBufferClass on all platforms
jbrichau Dec 17, 2023
52f1165
Fix unoptimized version of ZnRingBuffer
jbrichau Dec 17, 2023
7f9ebff
Fix wrappedPosition calculation in unoptimised ZnRingBuffer
jbrichau Dec 18, 2023
e4f4164
Use GRPlatform>>sizeOfFile:
jbrichau Dec 23, 2023
cbfd208
Code style and comment fixes, as suggested by @theseion
jbrichau Dec 24, 2023
245d8b7
Replace String new with literal string
jbrichau Dec 24, 2023
51d6654
Some more code style improvements
jbrichau Dec 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ baselineadaptors: spec
repository: 'http://ss3.gemtalksystems.com/ss/WebClient' ].
spec
package: 'WebClient-Seaside-Adaptor'
with: [
spec
with: [
spec
repository: 'http://ss3.gemtalksystems.com/ss/WebClient';
requires: #('WebClient Project' 'Seaside-Core') ].
spec group: 'WebClient' with: #('WebClient-Seaside-Adaptor') ].
Expand All @@ -38,14 +38,18 @@ baselineadaptors: spec

spec
for: #pharo
do: [ spec
do: [
spec
package: 'Seaside-Adaptors-Comanche' with: [ spec requires: #('KomHttpServerLight' 'Seaside-Core') ];
package: 'KomHttpServerLight' with: [ spec repository: 'http://www.smalltalkhub.com/mc/Seaside/KomHttpServer/main' ];
package: 'Seaside-Tests-Adaptors-Comanche' with: [ spec requires: #('Seaside-Adaptors-Comanche') ];
package: 'Zinc-Seaside' with: [ spec requires: #('Seaside-Core') ].
package: 'Seaside-Zinc-Core' with: [ spec requires: #('Seaside-Core') ];
package: 'Seaside-Zinc-Pharo' with: [ spec requires: #('Seaside-Zinc-Core') ];
package: 'Seaside-Zinc-Tests' with: [ spec requires: #('Seaside-Zinc-Pharo') ].
spec
group: 'Kom' with: #('Seaside-Adaptors-Comanche');
group: 'Zinc' with: #('Zinc-Seaside');
group: 'Zinc' with: #('Seaside-Zinc-Pharo');
group: 'Tests' with: #('Seaside-Zinc-Tests');
group: 'default' with: #('Zinc');
group: 'CI' with: #('Zinc') ].

Expand All @@ -69,22 +73,24 @@ baselineadaptors: spec
repository: 'github://GsDevKit/zinc:gs_master/repository' ].
spec
package: 'Seaside-GemStone-GemServer-Core' with: [ spec requires: #('GsApplicationTools' 'Seaside-Core') ];
package: 'Seaside-GemStone-GemServer-Tests' with: [ spec requires: #('Seaside-GemStone-GemServer-Core' 'Seaside-GemStone-Development' 'Seaside-GemStone-Adaptors-Zinc' 'Seaside-Adaptors-FastCGI') ];
package: 'Zinc-Seaside'
package: 'Seaside-GemStone-GemServer-Tests' with: [ spec requires: #('Seaside-GemStone-GemServer-Core' 'Seaside-GemStone-Development' 'Seaside-GemStone-Adaptors-Zinc' 'Seaside-Adaptors-FastCGI') ];
package: 'Seaside-Zinc-Core'
with: [ spec
requires: #('Zinc Project' 'Seaside-Core');
includes: #('Seaside-GemStone-Adaptors-Zinc') ];
package: 'Seaside-Adaptors-FastCGI' with: [ spec requires: #('Seaside-Core' 'FastCGI Project' 'Seaside-GemStone-GemServer-Core') ];
package: 'Seaside-GemStone-Adaptors-Zinc' with: [ spec requires: #('Zinc-Seaside' 'Seaside-GemStone-GemServer-Core') ].
package: 'Seaside-GemStone-Adaptors-Zinc' with: [ spec requires: #('Seaside-Zinc-Core' 'Seaside-GemStone-GemServer-Core') ];
package: 'Seaside-Zinc-Tests' with: [ spec requires: #('Seaside-Zinc-Core') ].
spec
group: 'FastCGI' with: #('Seaside-Adaptors-FastCGI');
group: 'Tests' with: #('Seaside-GemStone-GemServer-Tests');
group: 'Zinc' with: #('Zinc-Seaside') ].
group: 'Tests' with: #('Seaside-GemStone-GemServer-Tests' 'Seaside-Zinc-Tests');
group: 'Zinc' with: #('Seaside-Zinc-Core') ].

spec
for: #(#'gs3.0.x' #'gs3.1.x' #'gs3.2.x' #'gs3.3.x' #'gs3.4.x' #'gs3.5.x' )
do: [
spec
package: 'Seaside-GemStone-GemServer-Tests' with: [ spec includes: #( 'Seaside-GemStone-GemServer-Swazoo-Tests') ];
package: 'Seaside-GemStone-GemServer-Swazoo-Tests' with: [ spec requires: #('Seaside-GemStone-Adaptors-Swazoo') ];
package: 'Seaside-Adaptors-Swazoo' with: [ spec includes: #('Seaside-GemStone-Adaptors-Swazoo') ];
package: 'Seaside-GemStone-Adaptors-Swazoo' with: [ spec requires: #('Seaside-Adaptors-Swazoo' 'Seaside-GemStone-GemServer-Core') ] ].
package: 'Seaside-GemStone-Adaptors-Swazoo' with: [ spec requires: #('Seaside-Adaptors-Swazoo' 'Seaside-GemStone-GemServer-Core') ] ]
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
accessing
fileName
^ fileName

Check warning on line 3 in repository/Seaside-Core.package/WAAbstractFile.class/instance/fileName.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Core.package/WAAbstractFile.class/instance/fileName.st#L1-L3

Added lines #L1 - L3 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
accessing
rawContents
^ self subclassResponsibility
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
accessing
size
^ self subclassResponsibility
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"commentStamp" : "",
"super" : "WAObject",
"category" : "Seaside-Core-HTTP",
"classinstvars" : [ ],
"pools" : [ ],
"classvars" : [ ],
"instvars" : [
"contentType",
"fileName"
],
"name" : "WAAbstractFile",
"type" : "normal"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
I represent a file on a file system.

I know the name, the location and the MIME type of the file. I am mainly used for representing temporary files from streamed multipart uploads.

You will probably want to move me from my initial directory (which is probably a temporary directory) to my final destination. E.g.

externalFile fullyQualifiedFilePath asFileReference moveTo: FileLocator imageDirectory / 'final-destination'
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
testing
exists

^ GRPlatform current fileExists: self fullyQualifiedFilePath

Check warning on line 4 in repository/Seaside-Core.package/WAExternalFile.class/instance/exists.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Core.package/WAExternalFile.class/instance/exists.st#L1-L4

Added lines #L1 - L4 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
accessing
filePath: aString
filePath := aString

Check warning on line 3 in repository/Seaside-Core.package/WAExternalFile.class/instance/filePath..st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Core.package/WAExternalFile.class/instance/filePath..st#L1-L3

Added lines #L1 - L3 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
accessing
fullyQualifiedFilePath
^ filePath

Check warning on line 3 in repository/Seaside-Core.package/WAExternalFile.class/instance/fullyQualifiedFilePath.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Core.package/WAExternalFile.class/instance/fullyQualifiedFilePath.st#L1-L3

Added lines #L1 - L3 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
accessing
rawContents
^ GRPlatform current contentsOfFile: filePath binary: true

Check warning on line 3 in repository/Seaside-Core.package/WAExternalFile.class/instance/rawContents.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Core.package/WAExternalFile.class/instance/rawContents.st#L1-L3

Added lines #L1 - L3 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
accessing
size
^ self exists
ifTrue: [ GRPlatform current sizeOfFile: self fullyQualifiedFilePath ]
ifFalse: [ 0 ]

Check warning on line 5 in repository/Seaside-Core.package/WAExternalFile.class/instance/size.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Core.package/WAExternalFile.class/instance/size.st#L1-L5

Added lines #L1 - L5 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"commentStamp" : "mml 07/23/2019 09:57",
"super" : "WAAbstractFile",
"category" : "Seaside-Core-HTTP",
"classinstvars" : [ ],
"pools" : [ ],
"classvars" : [ ],
"instvars" : [
"filePath"
],
"name" : "WAExternalFile",
"type" : "normal"
}
6 changes: 2 additions & 4 deletions repository/Seaside-Core.package/WAFile.class/properties.json
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
{
"commentStamp" : "pmm 5/31/2008 23:54",
"super" : "WAObject",
"super" : "WAAbstractFile",
"category" : "Seaside-Core-HTTP",
"classinstvars" : [ ],
"pools" : [ ],
"classvars" : [ ],
"instvars" : [
"contents",
"contentType",
"fileName"
"contents"
],
"name" : "WAFile",
"type" : "normal"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ writing
writeStatusOn: aStream
aStream nextPutAll: self httpVersion.
aStream
nextPut: $ ;
space;
nextPutAll: self status greaseString.
self message isNil ifFalse: [
aStream
nextPut: $ ;
space;
nextPutAll: self message ].
aStream crlf
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
converting
valueForField: anObject
^ (anObject isKindOf: WAFile) ifTrue: [ anObject ]
^ (anObject isKindOf: WAAbstractFile) ifTrue: [ anObject ]
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*Seaside-Pharo-Core
fileReference
^ self fullyQualifiedFilePath asFileReference

Check warning on line 3 in repository/Seaside-Pharo-Core.package/WAExternalFile.extension/instance/fileReference.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Pharo-Core.package/WAExternalFile.extension/instance/fileReference.st#L1-L3

Added lines #L1 - L3 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"name" : "WAExternalFile"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
*Seaside-Zinc-Core
ringBufferClass

^ ZnRingBuffer

Check warning on line 4 in repository/Seaside-Zinc-Core.package/GRPlatform.extension/instance/ringBufferClass.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Zinc-Core.package/GRPlatform.extension/instance/ringBufferClass.st#L1-L4

Added lines #L1 - L4 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"name" : "GRPlatform"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*Seaside-Zinc-Core
crlf
self nextPutAll: String crlf

Check warning on line 3 in repository/Seaside-Zinc-Core.package/ZdcAbstractSocketStream.extension/instance/crlf.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Zinc-Core.package/ZdcAbstractSocketStream.extension/instance/crlf.st#L1-L3

Added lines #L1 - L3 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*Seaside-Zinc-Core
space
self nextPut: Character space asciiValue

Check warning on line 3 in repository/Seaside-Zinc-Core.package/ZdcAbstractSocketStream.extension/instance/space.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Zinc-Core.package/ZdcAbstractSocketStream.extension/instance/space.st#L1-L3

Added lines #L1 - L3 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"name" : "ZdcAbstractSocketStream"
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"commentStamp" : "<historical>",
"super" : "ZnResponse",
"category" : "Zinc-Seaside",
"category" : "Seaside-Zinc-Core",
"classinstvars" : [ ],
"pools" : [ ],
"classvars" : [ ],
Expand Down
15 changes: 15 additions & 0 deletions repository/Seaside-Zinc-Core.package/ZnRingBuffer.class/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
I am an implementation of a ring buffer, i.e. a buffer where the start index can be moved along a virtual ring.

I buffer a fixed amount of data and provide array like access to it.

Users use me mostly like an array. My only interesting method is #moveStartTo:, which moves the start index of the
buffer to the specified position. Example:

buffer
at: 1 put: 1;
at: 2 put: 2;
at: 1. "----> 1"

buffer
moveStartTo: 2;
at: 1 "----> 2"
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
instance creation
new: aInteger
^ self basicNew
initialize: aInteger;
initialize;
yourself
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
instance creation
new
ZnRingBufferIllegalUse signal: 'Use #new and specify an explicit size'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
instance creation
on: aCollection
^ self basicNew
initializeWithCollection: aCollection;
initialize;
yourself
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
comparing
= aCollection
1
to: self size
do: [ :index |
(self at: index) ~= (aCollection at: index) ifTrue: [ ^ false ] ].

^ true
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
accessing
at: anInteger

| wrappedPosition |
self checkBoundsFor: anInteger.
wrappedPosition := ((anInteger + offset - 1) rem: buffer size) + 1.
^ buffer at: wrappedPosition

Check warning on line 7 in repository/Seaside-Zinc-Core.package/ZnRingBuffer.class/instance/at..st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Zinc-Core.package/ZnRingBuffer.class/instance/at..st#L1-L7

Added lines #L1 - L7 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
accessing
at: anInteger put: anObject

| wrappedPosition |
self checkBoundsFor: anInteger.
wrappedPosition := ((anInteger + offset - 1) rem: buffer size) + 1.
buffer
at: wrappedPosition
put: anObject

Check warning on line 9 in repository/Seaside-Zinc-Core.package/ZnRingBuffer.class/instance/at.put..st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Zinc-Core.package/ZnRingBuffer.class/instance/at.put..st#L1-L9

Added lines #L1 - L9 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
private
checkBoundsFor: anInteger
(anInteger < 1 or: [
anInteger > self size ]) ifFalse: [ ^ self ].

ZnRingBufferIllegalAccess signal
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
accessing
copyFrom: start to: stop
| copy endIndex copySize |
copySize := stop - start + 1.
copy := buffer species new: copySize.
endIndex := copySize min: self size - start - offset + 1.
copy
replaceFrom: 1
to: endIndex
with: buffer
startingAt: start + offset.

stop + offset > self size ifFalse: [ ^ copy ].

^ copy
replaceFrom: endIndex + 1
to: copySize
with: buffer
startingAt: 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
enumerating
do: aBlock
1
to: self size
do: [ :index |
aBlock value: (self at: index) ]
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
comparing
hash
"Copy of SequenceableCollection>>hash"
| hash |
hash := self species hash.
1
to: self size
do: [ :i | hash := (hash + (self at: i) hash) hashMultiply ].
^ hash

Check warning on line 9 in repository/Seaside-Zinc-Core.package/ZnRingBuffer.class/instance/hash.st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Zinc-Core.package/ZnRingBuffer.class/instance/hash.st#L1-L9

Added lines #L1 - L9 were not covered by tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
accessing
indexOf: anElement ifAbsent: exceptionBlock
| index |
index := buffer
indexOf: anElement
ifAbsent: [ ^ exceptionBlock value ].

^ index > offset
ifTrue: [ index - offset ]
ifFalse: [ self size - offset + index ]
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
initialization
initialize: anInteger
self initializeWithCollection: (Array new: anInteger)
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
initialization
initialize
super initialize.

offset := 0
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
initialization
initializeWithCollection: aCollection
buffer := aCollection.
offset := 0
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
operations
moveStartTo: anInteger
self checkBoundsFor: anInteger.

offset := (offset + anInteger - 1) \\ self size
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
printing
printOn: aStream
super printOn: aStream.

aStream nextPut: $(.
(self
copyFrom: 1
to: self size) printOn: aStream.
aStream nextPut: $)

Check warning on line 9 in repository/Seaside-Zinc-Core.package/ZnRingBuffer.class/instance/printOn..st

View check run for this annotation

Codecov / codecov/patch

repository/Seaside-Zinc-Core.package/ZnRingBuffer.class/instance/printOn..st#L1-L9

Added lines #L1 - L9 were not covered by tests
Loading
Loading