Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for compiling and running NIO on Raspberry Pi 32-bit #383

Merged
merged 8 commits into from
May 4, 2018
Merged

Add support for compiling and running NIO on Raspberry Pi 32-bit #383

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
905826c
Use UInt, not Int as storage for AtomicBox
helje5 May 2, 2018
145ecf5
Make NIOHTTP1 compile on 32-bit platforms
helje5 May 2, 2018
095e668
Disable ByteBuffer size assert on 32-bit
helje5 May 2, 2018
80e1f1b
Use Int64 for time values
helje5 May 2, 2018
124e532
Fix indent
helje5 May 2, 2018
65537e5
Use `Int` to store nanosecs on 64b and `Int64` on 32b
helje5 May 4, 2018
7a94933
Simplify deadbeef, as request
helje5 May 4, 2018
864a9f6
Merge branch 'master' into feature/raspi-nio-1
normanmaurer May 4, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions Sources/NIO/ByteBuffer-core.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,10 @@ public struct ByteBufferAllocator {
hookedRealloc: @escaping @convention(c) (UnsafeMutableRawPointer?, Int) -> UnsafeMutableRawPointer?,
hookedFree: @escaping @convention(c) (UnsafeMutableRawPointer?) -> Void,
hookedMemcpy: @escaping @convention(c) (UnsafeMutableRawPointer, UnsafeRawPointer, Int) -> Void) {
assert(MemoryLayout<ByteBuffer>.size <= 3 * MemoryLayout<Int>.size,
"ByteBuffer has size \(MemoryLayout<ByteBuffer>.size) which is larger than the built-in storage of the existential containers.")
#if !arch(arm) // only complain on 64-bit, this is unfortunate reality on 32-bit
assert(MemoryLayout<ByteBuffer>.size <= 3 * MemoryLayout<Int>.size,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just kill this assertion. We now have much better tests (allocation counters) which will blow up if this assertion would fail

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume there is no chance to make this work better on 32-bit?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@helje5 the existential buffer size is 3 words. On 32 bit that means 12 bytes. Given that byte buffer needs one pointer, we’re left with 8 bytes for reader index, writer index, slice begin and end. I don’t think it’s feasible. So unfortunately for 32bit we’ll need to be larger than the builtin storage of an existential. But we still have the byte buffer special case in NIOAny so not all too bad...

"ByteBuffer has size \(MemoryLayout<ByteBuffer>.size) which is larger than the built-in storage of the existential containers.")
#endif
self.malloc = hookedMalloc
self.realloc = hookedRealloc
self.free = hookedFree
Expand Down
14 changes: 7 additions & 7 deletions Sources/NIO/ChannelHandlers.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ public final class AcceptBackoffHandler: ChannelDuplexHandler {
public typealias InboundIn = Channel
public typealias OutboundIn = Channel

private var nextReadDeadlineNS: Int?
private var nextReadDeadlineNS: TimeAmount.Value?
private let backoffProvider: (IOError) -> TimeAmount?
private var scheduledRead: Scheduled<Void>?

Expand All @@ -46,7 +46,7 @@ public final class AcceptBackoffHandler: ChannelDuplexHandler {
guard scheduledRead == nil else { return }

if let deadline = self.nextReadDeadlineNS {
let now = Int(DispatchTime.now().uptimeNanoseconds)
let now = TimeAmount.Value(DispatchTime.now().uptimeNanoseconds)
if now >= deadline {
// The backoff already expired, just do a read.
doRead(ctx)
Expand All @@ -62,7 +62,7 @@ public final class AcceptBackoffHandler: ChannelDuplexHandler {
public func errorCaught(ctx: ChannelHandlerContext, error: Error) {
if let ioError = error as? IOError {
if let amount = backoffProvider(ioError) {
self.nextReadDeadlineNS = Int(DispatchTime.now().uptimeNanoseconds) + amount.nanoseconds
self.nextReadDeadlineNS = TimeAmount.Value(DispatchTime.now().uptimeNanoseconds) + amount.nanoseconds
if let scheduled = self.scheduledRead {
scheduled.cancel()
scheduleRead(in: amount, ctx: ctx)
Expand Down Expand Up @@ -246,7 +246,7 @@ public class IdleStateHandler: ChannelDuplexHandler {
return
}

let diff = Int(DispatchTime.now().uptimeNanoseconds) - Int(self.lastReadTime.uptimeNanoseconds)
let diff = TimeAmount.Value(DispatchTime.now().uptimeNanoseconds) - TimeAmount.Value(self.lastReadTime.uptimeNanoseconds)
if diff >= timeout.nanoseconds {
// Reader is idle - set a new timeout and trigger an event through the pipeline
self.scheduledReaderTask = ctx.eventLoop.scheduleTask(in: timeout, self.newReadTimeoutTask(ctx, timeout))
Expand Down Expand Up @@ -275,7 +275,7 @@ public class IdleStateHandler: ChannelDuplexHandler {
ctx.fireUserInboundEventTriggered(IdleStateEvent.write)
} else {
// Write occurred before the timeout - set a new timeout with shorter delay.
self.scheduledWriterTask = ctx.eventLoop.scheduleTask(in: .nanoseconds(Int(timeout.nanoseconds) - Int(diff)), self.newWriteTimeoutTask(ctx, timeout))
self.scheduledWriterTask = ctx.eventLoop.scheduleTask(in: .nanoseconds(TimeAmount.Value(timeout.nanoseconds) - TimeAmount.Value(diff)), self.newWriteTimeoutTask(ctx, timeout))
}
}
}
Expand All @@ -293,15 +293,15 @@ public class IdleStateHandler: ChannelDuplexHandler {
let lastRead = self.lastReadTime
let lastWrite = self.lastWriteCompleteTime

let diff = Int(DispatchTime.now().uptimeNanoseconds) - Int((lastRead > lastWrite ? lastRead : lastWrite).uptimeNanoseconds)
let diff = TimeAmount.Value(DispatchTime.now().uptimeNanoseconds) - TimeAmount.Value((lastRead > lastWrite ? lastRead : lastWrite).uptimeNanoseconds)
if diff >= timeout.nanoseconds {
// Reader is idle - set a new timeout and trigger an event through the pipeline
self.scheduledReaderTask = ctx.eventLoop.scheduleTask(in: timeout, self.newAllTimeoutTask(ctx, timeout))

ctx.fireUserInboundEventTriggered(IdleStateEvent.all)
} else {
// Read occurred before the timeout - set a new timeout with shorter delay.
self.scheduledReaderTask = ctx.eventLoop.scheduleTask(in: .nanoseconds(Int(timeout.nanoseconds) - diff), self.newAllTimeoutTask(ctx, timeout))
self.scheduledReaderTask = ctx.eventLoop.scheduleTask(in: .nanoseconds(TimeAmount.Value(timeout.nanoseconds) - diff), self.newAllTimeoutTask(ctx, timeout))
}
}
}
Expand Down
29 changes: 18 additions & 11 deletions Sources/NIO/EventLoop.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,17 @@ public protocol EventLoop: EventLoopGroup {
///
/// - note: `TimeAmount` should not be used to represent a point in time.
public struct TimeAmount {

#if arch(arm) // 32-bit, Raspi/AppleWatch/etc
public typealias Value = Int64
#else // 64-bit, keeping that at Int for SemVer in the 1.x line.
public typealias Value = Int
#endif

/// The nanoseconds representation of the `TimeAmount`.
public let nanoseconds: Int
public let nanoseconds: Value

private init(_ nanoseconds: Int) {
private init(_ nanoseconds: Value) {
self.nanoseconds = nanoseconds
}

Expand All @@ -110,7 +117,7 @@ public struct TimeAmount {
/// - parameters:
/// - amount: the amount of nanoseconds this `TimeAmount` represents.
/// - returns: the `TimeAmount` for the given amount.
public static func nanoseconds(_ amount: Int) -> TimeAmount {
public static func nanoseconds(_ amount: Value) -> TimeAmount {
return TimeAmount(amount)
}

Expand All @@ -119,7 +126,7 @@ public struct TimeAmount {
/// - parameters:
/// - amount: the amount of microseconds this `TimeAmount` represents.
/// - returns: the `TimeAmount` for the given amount.
public static func microseconds(_ amount: Int) -> TimeAmount {
public static func microseconds(_ amount: Value) -> TimeAmount {
return TimeAmount(amount * 1000)
}

Expand All @@ -128,7 +135,7 @@ public struct TimeAmount {
/// - parameters:
/// - amount: the amount of milliseconds this `TimeAmount` represents.
/// - returns: the `TimeAmount` for the given amount.
public static func milliseconds(_ amount: Int) -> TimeAmount {
public static func milliseconds(_ amount: Value) -> TimeAmount {
return TimeAmount(amount * 1000 * 1000)
}

Expand All @@ -137,7 +144,7 @@ public struct TimeAmount {
/// - parameters:
/// - amount: the amount of seconds this `TimeAmount` represents.
/// - returns: the `TimeAmount` for the given amount.
public static func seconds(_ amount: Int) -> TimeAmount {
public static func seconds(_ amount: Value) -> TimeAmount {
return TimeAmount(amount * 1000 * 1000 * 1000)
}

Expand All @@ -146,7 +153,7 @@ public struct TimeAmount {
/// - parameters:
/// - amount: the amount of minutes this `TimeAmount` represents.
/// - returns: the `TimeAmount` for the given amount.
public static func minutes(_ amount: Int) -> TimeAmount {
public static func minutes(_ amount: Value) -> TimeAmount {
return TimeAmount(amount * 1000 * 1000 * 1000 * 60)
}

Expand All @@ -155,7 +162,7 @@ public struct TimeAmount {
/// - parameters:
/// - amount: the amount of hours this `TimeAmount` represents.
/// - returns: the `TimeAmount` for the given amount.
public static func hours(_ amount: Int) -> TimeAmount {
public static func hours(_ amount: Value) -> TimeAmount {
return TimeAmount(amount * 1000 * 1000 * 1000 * 60 * 60)
}
}
Expand Down Expand Up @@ -755,19 +762,19 @@ final public class MultiThreadedEventLoopGroup: EventLoopGroup {
private final class ScheduledTask {
let task: () -> Void
private let failFn: (Error) ->()
private let readyTime: Int
private let readyTime: TimeAmount.Value

init(_ task: @escaping () -> Void, _ failFn: @escaping (Error) -> Void, _ time: TimeAmount) {
self.task = task
self.failFn = failFn
self.readyTime = time.nanoseconds + Int(DispatchTime.now().uptimeNanoseconds)
self.readyTime = time.nanoseconds + TimeAmount.Value(DispatchTime.now().uptimeNanoseconds)
}

func readyIn(_ t: DispatchTime) -> TimeAmount {
if readyTime < t.uptimeNanoseconds {
return .nanoseconds(0)
}
return .nanoseconds(readyTime - Int(t.uptimeNanoseconds))
return .nanoseconds(readyTime - TimeAmount.Value(t.uptimeNanoseconds))
}

func fail(error: Error) {
Expand Down
4 changes: 2 additions & 2 deletions Sources/NIO/Selector.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ private enum SelectorLifecycleState {

private extension timespec {
init(timeAmount amount: TimeAmount) {
let nsecPerSec: Int = 1_000_000_000
let nsecPerSec: TimeAmount.Value = 1_000_000_000
let ns = amount.nanoseconds
let sec = ns / nsecPerSec
self = timespec(tv_sec: sec, tv_nsec: ns - sec * nsecPerSec)
self = timespec(tv_sec: Int(sec), tv_nsec: Int(ns - sec * nsecPerSec))
}
}

Expand Down
10 changes: 5 additions & 5 deletions Sources/NIOConcurrencyHelpers/atomics.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -474,11 +474,11 @@ extension UInt: AtomicPrimitive {
///
/// It behaves very much like `Atomic<T>` but for objects, maintaining the correct retain counts.
public class AtomicBox<T: AnyObject> {
private let storage: Atomic<Int>
private let storage: Atomic<UInt>

public init(value: T) {
let ptr = Unmanaged<T>.passRetained(value)
self.storage = Atomic(value: Int(bitPattern: ptr.toOpaque()))
self.storage = Atomic(value: UInt(bitPattern: ptr.toOpaque()))
}

deinit {
Expand Down Expand Up @@ -507,8 +507,8 @@ public class AtomicBox<T: AnyObject> {
let expectedPtr = Unmanaged<T>.passUnretained(expected)
let desiredPtr = Unmanaged<T>.passUnretained(desired)

if self.storage.compareAndExchange(expected: Int(bitPattern: expectedPtr.toOpaque()),
desired: Int(bitPattern: desiredPtr.toOpaque())) {
if self.storage.compareAndExchange(expected: UInt(bitPattern: expectedPtr.toOpaque()),
desired: UInt(bitPattern: desiredPtr.toOpaque())) {
_ = desiredPtr.retain()
expectedPtr.release()
return true
Expand All @@ -528,7 +528,7 @@ public class AtomicBox<T: AnyObject> {
/// - Returns: The value previously held by this object.
public func exchange(with value: T) -> T {
let newPtr = Unmanaged<T>.passRetained(value)
let oldPtrBits = self.storage.exchange(with: Int(bitPattern: newPtr.toOpaque()))
let oldPtrBits = self.storage.exchange(with: UInt(bitPattern: newPtr.toOpaque()))
let oldPtr = Unmanaged<T>.fromOpaque(UnsafeRawPointer(bitPattern: oldPtrBits)!)
return oldPtr.takeRetainedValue()
}
Expand Down
7 changes: 6 additions & 1 deletion Sources/NIOHTTP1/HTTPDecoder.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,12 @@ public class HTTPDecoder<HTTPMessageT>: ByteToMessageDecoder, AnyHTTPDecoder {

public func decoderRemoved(ctx: ChannelHandlerContext) {
// Remove the stored reference to ChannelHandlerContext
parser.data = UnsafeMutableRawPointer(bitPattern: 0x0000deadbeef0000)
#if arch(arm) // 32-bit
let veryDeadBeef : UInt = 0xdeadbeef
parser.data = UnsafeMutableRawPointer(bitPattern: veryDeadBeef)
#else
parser.data = UnsafeMutableRawPointer(bitPattern: 0x0000deadbeef0000)
#endif
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jw Do we need this change? I think we only wanted this pointer value to be easy to spot in debuggers and core dumps: does it need to have the deadbeef portion shifted left 4 bits on 64-bit operating systems or is it ok to leave it in the least significant 32 bits?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can just use a 32 bit value 0xdeadbeef is totally fine by me

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That still needs the UInt32 cast though, because parser.data seems to be Int32.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@helje5 isn’t parser.data a pointer?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right it is. I guess the problem is the Swift type overloading again. It will treat 0xdeadbeef as an Int. At least that's the error w/o the explicit UInt (cannot represent Integer literal as Int or something like that).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but can't we just compress this into one line? parser.data = UnsafeMutableRawPointer(bitPattern: UInt(0xdeadbeef)? That'll work correctly on all platforms.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@helje5 I'm happy with this except for this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should, but you have to remember that we are talking Swift here! ;->

/src/Sources/NIOHTTP1/HTTPDecoder.swift:382:64: error: integer literal '3735928559' overflows when stored into 'Int'
        parser.data = UnsafeMutableRawPointer(bitPattern: UInt(0xdeadbeef))

Same issue: UInt essentially has a UInt.init(Int), hence Swift will treat the literal as such ...


// Set the callbacks to nil as we dont need these anymore
settings.on_body = nil
Expand Down
2 changes: 1 addition & 1 deletion Sources/NIOHTTP1Server/main.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ private final class HTTPHandler: ChannelInboundHandler {
return { ctx, req in
self.handleJustWrite(ctx: ctx,
request: req, string: "Hello World\r\n",
delay: Int(howLong).map { .milliseconds($0) } ?? .seconds(0))
delay: TimeAmount.Value(howLong).map { .milliseconds($0) } ?? .seconds(0))
}
}

Expand Down