Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove rtio #18557

Merged
merged 11 commits into from
Nov 9, 2014
Merged

Remove rtio #18557

merged 11 commits into from
Nov 9, 2014

Conversation

aturon
Copy link
Member

@aturon aturon commented Nov 3, 2014

This PR includes a sequence of commits that gradually dismantles the librustrt rtio system -- the main trait previously used to abstract over green and native io. It also largely dismantles libnative, moving much of its code into libstd and refactoring as it does so.

TL;DR:

  • Before this PR: rustc hello.rs && wc -c hello produces 715,996
  • After this PR: rustc hello.rs && wc -c hello produces 368,100

That is, this PR reduces the footprint of hello world by ~50%.

This is a major step toward #17325 (i.e. toward implementing the runtime removal RFC.) What remains is to pull out the scheduling, synchronization and task infrastructure, and to remove libgreen. These will be done soon in a follow-up PR.

Part of the work here is eliminating the rtio abstraction, which in many cases means bringing the implementation of io closer to the actual API presented in std::io.

Another aspect of this PR is the creation of two new, private modules within std that implement io:

  • The sys module, which represents a platform-specific implementation of a number of low-level abstractions that are used directly within std::io and std::os. These "abstractions" are left largely the same as they were in libnative (except for the removal of Arc in file descriptors), but they are expected to evolve greatly over time. Organizationally, there are sys/unix/ and sys/windows/ directories which both implement the entire sys module hierarchy; this means that nearly all of the platform-specific code is isolated and you can get a handle on each platform in isolation.
  • The sys_common module, which is rooted at sys/common, and provides a few pieces of private, low-level, but cross-platform functionality.

In the long term, the sys modules will provide hooks for exposing high-level platform-specific APIs as part of libstd. The first such API will be access to file descriptors from std::io abstractions, but a bit of design work remains before that step can be taken.

The sys_common module includes some traits (like AsFileDesc) which allow communication of private details between modules in disparate locations in the hierarchy; this helps overcome the relatively simple hierarchical privacy system in Rust.

To emphasize: the organization in sys is very preliminary and the main goal was to migrate away from rtio as quickly and simply as possible. The design will certainly evolve over time, and all of the details are currently private.

Along the way, this PR also entirely removes signal handling, since it was only supported on librustuv which was removed a while ago.

Because of the removal of APIs from libnative and librustrt, and the removal of signal handling, this is a:

[breaking-change]

Some of these APIs will return in public from from std over time.

r? @alexcrichton

@rust-highfive
Copy link
Collaborator

warning Warning warning

  • These commits modify unsafe code. Please review it carefully!

@aturon aturon force-pushed the io-removal branch 2 times, most recently from 30c8273 to 05fdad8 Compare November 3, 2014 00:26
@alexcrichton
Copy link
Member

This may also want to note as being a [breaking-change] in the sense that native::io was gobbled up into the standard library and is no longer being exposed at this time. I expect that will be a more painful change than losing signal handling, sadly (I know I've got a lib or two that uses it!).

In general I definitely feel like this is the right direction to go in, and I'd definitely be willing to help out in making some of these details public over time.

@aturon
Copy link
Member Author

aturon commented Nov 3, 2014

@alexcrichton Ah yes, I've been so used to thinking of things like libnative as "use at your own risk" that I didn't think to warn about that. I'll adjust the commit messages.

@alexcrichton
Copy link
Member

Ah and I meant to say: other than that, r=me, amazing work!

@brson
Copy link
Contributor

brson commented Nov 4, 2014

🎈

@aturon
Copy link
Member Author

aturon commented Nov 4, 2014

Status update: going to make sure things are passing try before passing this along to @bors.

@aturon aturon force-pushed the io-removal branch 5 times, most recently from c31ea90 to 1b212df Compare November 6, 2014 00:42
@mcpherrinm
Copy link
Contributor

P=1000000000

@nikomatsakis
Copy link
Contributor

🎯

c::fd_set(&mut set, fd);
max = cmp::max(max, fd + 1);
}
max = fds.len() as sock_t;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out the test timed out because the acceptor never actually accepted (it never returned from this function). I think that this was an erroneous copy/paste from

for &fd in fds.iter() {
c::fd_set(&mut set, fd);
max = cmp::max(max, fd + 1);
}
if cfg!(windows) {
max = fds.len() as net::sock_t;
}
which just needs an if cfg!(windows) around this max = fds.len() as sock_t

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also turn tcp-stress parameters back up to what they were?

@aturon aturon force-pushed the io-removal branch 2 times, most recently from eb8d439 to 2c9bb3f Compare November 8, 2014 18:03
bors added a commit that referenced this pull request Nov 8, 2014
This PR includes a sequence of commits that gradually dismantles the `librustrt` `rtio` system -- the main trait previously used to abstract over green and native io. It also largely dismantles `libnative`, moving much of its code into `libstd` and refactoring as it does so.

TL;DR:

* Before this PR: `rustc hello.rs && wc -c hello` produces 715,996
* After this PR:  `rustc hello.rs && wc -c hello` produces 368,100

That is, this PR reduces the footprint of hello world by ~50%.

This is a major step toward #17325 (i.e. toward implementing the [runtime removal RFC](rust-lang/rfcs#230).) What remains is to pull out the scheduling, synchronization and task infrastructure, and to remove `libgreen`. These will be done soon in a follow-up PR.

Part of the work here is eliminating the `rtio` abstraction, which in many cases means bringing the implementation of io closer to the actual API presented in `std::io`.

Another aspect of this PR is the creation of two new, *private* modules within `std` that implement io:

* The `sys` module, which represents a platform-specific implementation of a number of low-level abstractions that are used directly within `std::io` and `std::os`. These "abstractions" are left largely the same as they were in `libnative` (except for the removal of `Arc` in file descriptors), but they are expected to evolve greatly over time. Organizationally, there are `sys/unix/` and `sys/windows/` directories which both implement the entire `sys` module hierarchy; this means that nearly all of the platform-specific code is isolated and you can get a handle on each platform in isolation.

* The `sys_common` module, which is rooted at `sys/common`, and provides a few pieces of private, low-level, but cross-platform functionality.

In the long term, the `sys` modules will provide hooks for exposing high-level platform-specific APIs as part of `libstd`. The first such API will be access to file descriptors from `std::io` abstractions, but a bit of design work remains before that step can be taken. 

The `sys_common` module includes some traits (like `AsFileDesc`) which allow communication of private details between modules in disparate locations in the hierarchy; this helps overcome the relatively simple hierarchical privacy system in Rust.

To emphasize: the organization in `sys` is *very preliminary* and the main goal was to migrate away from `rtio` as quickly and simply as possible. The design will certainly evolve over time, and all of the details are currently private.

Along the way, this PR also entirely removes signal handling, since it was only supported on `librustuv` which was removed a while ago. 

Because of the removal of APIs from `libnative` and `librustrt`, and the removal of signal handling, this is a:

[breaking-change]

Some of these APIs will return in public from from `std` over time.

r? @alexcrichton
@aturon
Copy link
Member Author

aturon commented Nov 9, 2014

@alexcrichton Is this latest failure known to be spurious? run-pass/sepcomp-lib-lto.rs on android. seems unrelated to the changes here.

These modules will house the code that used to be part of the runtime system
in libnative. The `sys_common` module contains a few low-level but
cross-platform details. The `sys` module is set up using `#[cfg()]` to
include either a unix or windows implementation of a common API
surface. This API surface is *not* exported directly in `libstd`, but is
instead used to bulid `std::os` and `std::io`.

Ultimately, the low-level details in `sys` will be exposed in a
controlled way through a separate platform-specific surface, but that
setup is not part of this patch.
Since signal handling was only implemented through librustuv, which is
now gone, there's no reason to even provide the API.

[breaking-change]
This moves the filesystem implementation from libnative into the new
`sys` modules, refactoring along the way and hooking into `std::io::fs`.

Because this eliminates APIs in `libnative` and `librustrt`, it is a:

[breaking-change]

This functionality is likely to be available publicly, in some form,
from `std` in the future.
This patch continues the runtime removal by moving pipe and
networking-related code into `sys`.

Because this eliminates APIs in `libnative` and `librustrt`, it is a:

[breaking-change]

This functionality is likely to be available publicly, in some form,
from `std` in the future.
This patch continues the runtime removal by moving
libnative::io::helper_thread into sys::helper_signal and
sys_common::helper_thread

Because this eliminates APIs in `libnative` and `librustrt`, it is a:

[breaking-change]

This functionality is likely to be available publicly, in some form,
from `std` in the future.
This patch continues the runtime removal by moving and refactoring the
process implementation into the new `sys` module.

Because this eliminates APIs in `libnative` and `librustrt`, it is a:

[breaking-change]

This functionality is likely to be available publicly, in some form,
from `std` in the future.
This patch continues runtime removal by moving out timer-related code
into `sys`.

Because this eliminates APIs in `libnative` and `librustrt`, it is a:

[breaking-change]

This functionality is likely to be available publicly, in some form,
from `std` in the future.
This patch continues runtime removal by moving the tty implementations
into `sys`.

Because this eliminates APIs in `libnative` and `librustrt`, it is a:

[breaking-change]

This functionality is likely to be available publicly, in some form,
from `std` in the future.
This patch cleans up the remnants of the runtime IO interface.

Because this eliminates APIs in `libnative` and `librustrt`, it is a:

[breaking-change]

This functionality is likely to be available publicly, in some form,
from `std` in the future.
bors added a commit that referenced this pull request Nov 9, 2014
This PR includes a sequence of commits that gradually dismantles the `librustrt` `rtio` system -- the main trait previously used to abstract over green and native io. It also largely dismantles `libnative`, moving much of its code into `libstd` and refactoring as it does so.

TL;DR:

* Before this PR: `rustc hello.rs && wc -c hello` produces 715,996
* After this PR:  `rustc hello.rs && wc -c hello` produces 368,100

That is, this PR reduces the footprint of hello world by ~50%.

This is a major step toward #17325 (i.e. toward implementing the [runtime removal RFC](rust-lang/rfcs#230).) What remains is to pull out the scheduling, synchronization and task infrastructure, and to remove `libgreen`. These will be done soon in a follow-up PR.

Part of the work here is eliminating the `rtio` abstraction, which in many cases means bringing the implementation of io closer to the actual API presented in `std::io`.

Another aspect of this PR is the creation of two new, *private* modules within `std` that implement io:

* The `sys` module, which represents a platform-specific implementation of a number of low-level abstractions that are used directly within `std::io` and `std::os`. These "abstractions" are left largely the same as they were in `libnative` (except for the removal of `Arc` in file descriptors), but they are expected to evolve greatly over time. Organizationally, there are `sys/unix/` and `sys/windows/` directories which both implement the entire `sys` module hierarchy; this means that nearly all of the platform-specific code is isolated and you can get a handle on each platform in isolation.

* The `sys_common` module, which is rooted at `sys/common`, and provides a few pieces of private, low-level, but cross-platform functionality.

In the long term, the `sys` modules will provide hooks for exposing high-level platform-specific APIs as part of `libstd`. The first such API will be access to file descriptors from `std::io` abstractions, but a bit of design work remains before that step can be taken. 

The `sys_common` module includes some traits (like `AsFileDesc`) which allow communication of private details between modules in disparate locations in the hierarchy; this helps overcome the relatively simple hierarchical privacy system in Rust.

To emphasize: the organization in `sys` is *very preliminary* and the main goal was to migrate away from `rtio` as quickly and simply as possible. The design will certainly evolve over time, and all of the details are currently private.

Along the way, this PR also entirely removes signal handling, since it was only supported on `librustuv` which was removed a while ago. 

Because of the removal of APIs from `libnative` and `librustrt`, and the removal of signal handling, this is a:

[breaking-change]

Some of these APIs will return in public from from `std` over time.

r? @alexcrichton
@bors bors closed this Nov 9, 2014
@bors bors merged commit 5ea09e6 into rust-lang:master Nov 9, 2014
@l0kod l0kod mentioned this pull request Nov 19, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants