Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LibOS] Clean up select and poll implementations #1067

Merged
merged 3 commits into from
Jan 6, 2023
Merged

Conversation

boryspoplawski
Copy link
Contributor

@boryspoplawski boryspoplawski commented Nov 30, 2022

Description of the changes

Fixed some bugs, hopefully the code easier to manage too now.


This change is Reviewable

Copy link
Contributor Author

@boryspoplawski boryspoplawski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 15 files reviewed, 1 unresolved discussion, not enough approvals from maintainers (2 more required), not enough approvals from different teams (1 more required, approved so far: ITL)


libos/src/sys/libos_poll.c line 113 at r2 (raw file):

            continue;

            dev_tty_hack:;

Alternative version (I dislike both):

Suggestion:

            if (!(ret == -EAGAIN && handle->uri && !strcmp(handle->uri, "dev:tty"))) {
                if (ret < 0) {
                    unlock(&map->lock);
                    goto out;
                }

                fds[i].revents = events;
                if (events) {
                    ret_events_count++;
                }

                continue;
            }

Copy link

@dimakuv dimakuv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 10 of 12 files at r1, 5 of 5 files at r2, all commit messages.
Reviewable status: all files reviewed, 9 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (1 more required, approved so far: ITL) (waiting on @boryspoplawski)


libos/src/fs/libos_fs_mem.c line 117 at r2 (raw file):

int mem_file_poll(struct libos_mem_file* mem, file_off_t pos, int events, int* out_events) {
    *out_events = events & (POLLOUT | POLLWRNORM);
    if ((events & (POLLIN | POLLRDNORM)) && (pos < mem->size))

Why do you need the first condition? This will be cleared anyway in the below assignment if false.


libos/src/fs/libos_fs_pseudo.c line 487 at r2 (raw file):

            *out_events = 0;
            if ((events & POLLIN) && node->dev.dev_ops.read)
                *out_events |= POLLIN;

Why no POLLRDNORM?


libos/src/fs/libos_fs_pseudo.c line 489 at r2 (raw file):

                *out_events |= POLLIN;
            if ((events & POLLOUT) && node->dev.dev_ops.write)
                *out_events |= POLLOUT;

Why no POLLWRNORM?


libos/src/sys/libos_poll.c line 95 at r2 (raw file):

             * Initial 0,1,2 fds in Gramine are represented by "/dev/tty" (whatever that means)
             * and have `generic_inode_poll` set as poll callback, which returns `-EAGAIN` on
             * non-regular-file handles. In such case we let PAL do the actuall polling.

actual


libos/src/sys/libos_poll.c line 113 at r2 (raw file):

Previously, boryspoplawski (Borys Popławski) wrote…

Alternative version (I dislike both):

Yeah, both versions are ugly. Let's keep the current one.


libos/src/sys/libos_poll.c line 120 at r2 (raw file):

            pal_handle = __atomic_load_n(&handle->info.sock.pal_handle, __ATOMIC_ACQUIRE);
            if (!pal_handle) {
                /* UNIX sockets that are still not connected have no `pal_handle`. */

Why only UNIX sockets? Can't other kinds of sockets (INET) have the same problem?


libos/src/sys/libos_poll.c line 239 at r2 (raw file):

    if (tsp) {
        if (is_user_memory_writable_no_skip(tsp, sizeof(*tsp))) {

Actually, why do we care about skip vs no_skip versions? Isn't libos.check_invalid_pointers = false already gives you no guarantees what happens at runtime, so the user is on his own?


libos/src/sys/libos_poll.c line 344 at r2 (raw file):

    long ret = do_select(nfds, read_set, write_set, except_set, tv ? &timeout_us : NULL);

    if (tv && is_user_memory_writable_no_skip(tv, sizeof(*tv))) {

ditto


libos/src/sys/libos_poll.c line 382 at r2 (raw file):

    long ret = do_select(nfds, read_set, write_set, except_set, tsp ? &timeout_us : NULL);

    if (tsp && is_user_memory_writable_no_skip(tsp, sizeof(*tsp))) {

ditto


libos/test/ltp/ltp.cfg line 1364 at r2 (raw file):

[personality02]
skip = yes

Could you add a comment why you added this?

Copy link
Contributor Author

@boryspoplawski boryspoplawski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 9 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (1 more required, approved so far: ITL) (waiting on @dimakuv)


libos/src/fs/libos_fs_mem.c line 117 at r2 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Why do you need the first condition? This will be cleared anyway in the below assignment if false.

Done.


libos/src/fs/libos_fs_pseudo.c line 487 at r2 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Why no POLLRDNORM?

Done.
(leftover from an intermediate version, which handled those in actual poll syscall code


libos/src/fs/libos_fs_pseudo.c line 489 at r2 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Why no POLLWRNORM?

Done.


libos/src/sys/libos_poll.c line 95 at r2 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

actual

Done.


libos/src/sys/libos_poll.c line 120 at r2 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Why only UNIX sockets? Can't other kinds of sockets (INET) have the same problem?

No, only UNIX sockets. IP sockets have pal handles right from beginning.
(also this code was already here before this PR)


libos/src/sys/libos_poll.c line 239 at r2 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Actually, why do we care about skip vs no_skip versions? Isn't libos.check_invalid_pointers = false already gives you no guarantees what happens at runtime, so the user is on his own?

But this can be a valid pointer, just not writable (RO memory). Added a comment.


libos/src/sys/libos_poll.c line 344 at r2 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

ditto

ditto


libos/src/sys/libos_poll.c line 382 at r2 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

ditto

ditto


libos/test/ltp/ltp.cfg line 1364 at r2 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Could you add a comment why you added this?

Because we do not implement personality syscall? This test is just badly written and does not even check return value. It was "working" before, because it tests something we did not do, but should have done in select.
Anyway, do we want these comments that a syscall is not implemented in cases where the syscall and test name are the same?

Copy link

@yamahata yamahata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 12 of 15 files reviewed, 12 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (1 more required, approved so far: ITL), "fixup! " found in commit messages' one-liners (waiting on @boryspoplawski and @dimakuv)


libos/src/sys/libos_poll.c line 237 at r3 (raw file):

    ret = do_poll(fds, nfds, tsp ? &timeout_us : NULL);

signal mask isn't restored to the orignal state.


libos/src/sys/libos_poll.c line 239 at r3 (raw file):

    /* If `tsp` is in read-only memory, skip the update. */
    if (tsp && is_user_memory_writable_no_skip(tsp, sizeof(*tsp))) {

If g_check_invalid_ptrs is true, -EFAULT should be returned.


libos/src/sys/libos_poll.c line 381 at r3 (raw file):

    long ret = do_select(nfds, read_set, write_set, except_set, tsp ? &timeout_us : NULL);

Signal mask isn't restored to the orignal state.

Copy link

@yamahata yamahata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 3 files at r3.
Reviewable status: 14 of 15 files reviewed, 13 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (1 more required, approved so far: ITL), "fixup! " found in commit messages' one-liners (waiting on @boryspoplawski and @dimakuv)


-- commits line 19 at r3:
Please mention that FS_POLL_* is dumped and POLL_* is adapted.

Copy link
Contributor Author

@boryspoplawski boryspoplawski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 14 of 15 files reviewed, 13 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (1 more required, approved so far: ITL), "fixup! " found in commit messages' one-liners (waiting on @dimakuv and @yamahata)


-- commits line 19 at r3:

Previously, yamahata wrote…

Please mention that FS_POLL_* is dumped and POLL_* is adapted.

IMO there's no point - this is only a minor part of the PR and doesn't seem important at all


libos/src/sys/libos_poll.c line 237 at r3 (raw file):

Previously, yamahata wrote…

signal mask isn't restored to the orignal state.

It restored while returning to userapp in libos_emulate_syscall


libos/src/sys/libos_poll.c line 239 at r3 (raw file):

Previously, yamahata wrote…

If g_check_invalid_ptrs is true, -EFAULT should be returned.

No, see the comment above this if


libos/src/sys/libos_poll.c line 381 at r3 (raw file):

Previously, yamahata wrote…

Signal mask isn't restored to the orignal state.

ditto

dimakuv
dimakuv previously approved these changes Dec 5, 2022
Copy link

@dimakuv dimakuv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 3 files at r3, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions, "fixup! " found in commit messages' one-liners (waiting on @yamahata)


libos/src/sys/libos_poll.c line 239 at r2 (raw file):

Previously, boryspoplawski (Borys Popławski) wrote…

But this can be a valid pointer, just not writable (RO memory). Added a comment.

I got it now.


libos/test/ltp/ltp.cfg line 1364 at r2 (raw file):

It was "working" before, because it tests something we did not do, but should have done ...

Ah, got it. No need for any comments then.

Copy link
Member

@mkow mkow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 8 of 12 files at r1, 4 of 5 files at r2, 3 of 3 files at r3, all commit messages.
Reviewable status: all files reviewed, 7 unresolved discussions, "fixup! " found in commit messages' one-liners (waiting on @boryspoplawski and @yamahata)


libos/include/libos_fs.h line 949 at r3 (raw file):

int generic_inode_hstat(struct libos_handle* hdl, struct stat* buf);
file_off_t generic_inode_seek(struct libos_handle* hdl, file_off_t offset, int origin);
int generic_inode_poll(struct libos_handle* hdl, int events, int* out_events);

In the function above you named it in_events. Are they different or you just didn't update all the names when changing them at some point?

Code quote:

int events

libos/src/sys/libos_poll.c line 2 at r3 (raw file):

/* Copyright (C) 2014 Stony Brook University

This should be dropped, I think you rewrote this whole implementation, the only not-rewritten part is in pselect6, but it's also authored by you in some older commit.


libos/src/sys/libos_poll.c line 141 at r3 (raw file):

        libos_handles[i] = handle;
        get_handle(handle);
        pal_handles[i] = pal_handle;

The invariants of this loop were pretty hard for me to analyze without any comments/asserts. I see that for each i either fds[i].revents is set or libos_handles[i] + pal_handles[i] are.
Could you document this at the top of the loop? It will be easier to reason about this code with such a hint.

Copy link
Contributor Author

@boryspoplawski boryspoplawski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 7 unresolved discussions, not enough approvals from maintainers (1 more required), not enough approvals from different teams (1 more required, approved so far: ITL), "fixup! " found in commit messages' one-liners (waiting on @mkow and @yamahata)


libos/include/libos_fs.h line 949 at r3 (raw file):

Previously, mkow (Michał Kowalczyk) wrote…

In the function above you named it in_events. Are they different or you just didn't update all the names when changing them at some point?

Done.


libos/src/sys/libos_poll.c line 2 at r3 (raw file):

Previously, mkow (Michał Kowalczyk) wrote…

/* Copyright (C) 2014 Stony Brook University

This should be dropped, I think you rewrote this whole implementation, the only not-rewritten part is in pselect6, but it's also authored by you in some older commit.

Done.


libos/src/sys/libos_poll.c line 141 at r3 (raw file):

Previously, mkow (Michał Kowalczyk) wrote…

The invariants of this loop were pretty hard for me to analyze without any comments/asserts. I see that for each i either fds[i].revents is set or libos_handles[i] + pal_handles[i] are.
Could you document this at the top of the loop? It will be easier to reason about this code with such a hint.

Done.

dimakuv
dimakuv previously approved these changes Jan 3, 2023
Copy link

@dimakuv dimakuv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 3 files at r3, 3 of 3 files at r4, all commit messages.
Reviewable status: all files reviewed, 7 unresolved discussions, "fixup! " found in commit messages' one-liners (waiting on @mkow and @yamahata)

mkow
mkow previously approved these changes Jan 4, 2023
Copy link
Member

@mkow mkow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r4, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions, "fixup! " found in commit messages' one-liners (waiting on @yamahata)

yamahata
yamahata previously approved these changes Jan 5, 2023
Copy link

@yamahata yamahata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 6 of 12 files at r1, 4 of 5 files at r2, 3 of 3 files at r4, all commit messages.
Reviewable status: all files reviewed, all discussions resolved, "fixup! " found in commit messages' one-liners

`is_user_memory_writable` can be skipped via manifest option, which
can have some performance benefits. This commit adds a version of this
function which always performs the checks.

Signed-off-by: Borys Popławski <[email protected]>
Useful when we want to ignore some handles without reallocating handles
array.

Signed-off-by: Borys Popławski <[email protected]>
@dimakuv dimakuv dismissed stale reviews from yamahata, mkow, and themself via cc9175d January 6, 2023 08:31
Copy link

@dimakuv dimakuv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants