Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce DiagnosticContext, an 'error document' like type. #1323

Merged
merged 42 commits into from
Dec 7, 2024

Conversation

BillyONeal
Copy link
Member

@BillyONeal BillyONeal commented Jan 10, 2024

In several PRs, such as #908 and #1210 , it has become clear that we need a thread safe channel through which errors and warnings can both be reasonably reported. Now that #1279 is landed and functionally everything in the codebase uses ExpectedL, we can look at what the new thing that fixes issues is.

Consider the following:

    ExpectedL<T> example_api(int a);

    ExpectedL<std::unique_ptr<SourceControlFile>> try_load_port_manifest_text(StringView text,
                                                                              StringView control_path,
                                                                              MessageSink& warning_sink);

The reason this can't return the warnings through the ExpectedL channel is that we don't want the 'error' state to be engaged when there are merely warnings. Moreover, that these channels are different channels means that situations that might want to return errors and warnings together, as happens when parsing files, means that order relationships between errors and warnings is lost. It is probably a good idea in general to put warnings and errors about the same location next to each other in the output, but that's hard to do with this interface.

Rather than multiplexing everything through the return value, this proposal is to multiplex only the success or failure through the return value, and report any specific error information through an out parameter.

  1. Distinguish whether an overall operation succeeded or failed in the return value, but
  2. record any errors or warnings via an out parameter.

Applying this to the above gives:

    Optional<T> example_api(DiagnosticContext& context, int a);

    // unique_ptr is already 'optional'
    std::unique_ptr<SourceControlFile> try_load_port_manifest_text(DiagnosticContext& context,
                                                                   StringView text,
                                                                   StringView control_path);

Issues this new mechanism fixes:

  • Errors and warnings can share the same channel and thus be printed together
  • The interface between code wanting to report events and the code wanting to consume them is a natural thread synchronization boundary. Other attempts to fix this have been incorrect by synchronizing individual print calls ( Make individual print calls thread safe #1290 ) or complex enough that we are not sure they are correct by trying to recover boundaries by reparsing our own error output ( Binary cache: async push_success #908 )
  • This shuts down the "error: error:" and similar bugs where it isn't clear who is formatting the overall error message vs. talking about individual components

Known issues that are not fixed by this change:

  • This still doesn't make it easy for callers to programmatically handle specific types of errors. Currently, we have some APIs that still use explicit std::error_code because they want to do different things for 'file does not exist' vs. 'there was an I/O error'. Given that this condition isn't well served by the ExpectedL mechanism I don't want to wait until we have a better solution to it to proceed.
  • Because we aren't making the context parameter the 'success carrier' it's more complex to implement 'warnings as errors' or similar functionality where the caller decides how 'important' something is. I would be in favor of moving all success tests to the context parameter but I'm not proposing that because the other vcpkg maintainers do not like it.
  • Contextual information / stack problems aren't solved. However, the context parameter might be extended in the future to help with this.

Unblocks:

@BillyONeal BillyONeal added the depends:different-pr This PR depends on a different PR which has been filed label Jan 10, 2024
@BillyONeal BillyONeal marked this pull request as draft January 10, 2024 19:59
@BillyONeal BillyONeal removed the depends:different-pr This PR depends on a different PR which has been filed label Jan 26, 2024
…ome clear that we need a thread safe channel through which errors and warnings can both be reasonably reported. Now that microsoft#1279 is landed and functionally everything in the codebase uses ExpectedL, we can look at what the new thing that fixes issues is.

Consider the following:

```c++
    ExpectedL<T> example_api(int a);

    ExpectedL<std::unique_ptr<SourceControlFile>> try_load_port_manifest_text(StringView text,
                                                                              StringView control_path,
                                                                              MessageSink& warning_sink);
```

The reason this can't return the warnings through the ExpectedL channel is that we don't want the 'error' state to be engaged when there are merely warnings. Moreover, that these channels are different channels means that situations that might want to return errors and warnings together, as happens when parsing files, means that order relationships between errors and warnings is lost. It is probably a good idea in general to put warnings and errors about the same location next to each other in the output, but that's hard to do with this interface.

Rather than multiplexing everything through the return value, this proposal is to multiplex only the success or failure through the return value, and report any specific error information through an out parameter.

1. Distinguish whether an overall operation succeeded or failed in the return value, but
2. record any errors or warnings via an out parameter.

Applying this to the above gives:

```c++
    Optional<T> example_api(MessageContext& context, int a);

    // unique_ptr is already 'optional'
    std::unique_ptr<SourceControlFile> try_load_port_manifest_text(MessageContext& context,
                                                                   StringView text,
                                                                   StringView control_path);
```

Issues this new mechanism fixes:

* Errors and warnings can share the same channel and thus be printed together
* The interface between code wanting to report events and the code wanting to consume them is a natural thread synchronization boundary. Other attempts to fix this have been incorrect by synchronizing individual print calls ( microsoft#1290 ) or complex enough that we are not sure they are correct by trying to recover boundaries by reparsing our own error output ( microsoft#908 )
* This shuts down the "error: error:" and similar bugs where it isn't clear who is formatting the overall error message vs. talking about individual components

Known issues that are not fixed by this change:

* This still doesn't make it easy for callers to programmatically handle specific types of errors. Currently, we have some APIs that still use explicit `std::error_code` because they want to do different things for 'file does not exist' vs. 'there was an I/O error'. Given that this condition isn't well served by the ExpectedL mechanism I don't want to wait until we have a better solution to it to proceed.
* Because we aren't making the context parameter the 'success carrier' it's more complex to implement 'warnings as errors' or similar functionality where the caller decides how 'important' something is. I would be in favor of moving all success tests to the context parameter but I'm not proposing that because the other vcpkg maintainers do not like it.
* Contextual information / stack problems aren't solved. However, the context parameter might be extended in the future to help with this.
@BillyONeal BillyONeal marked this pull request as ready for review January 26, 2024 23:50
@BillyONeal
Copy link
Member Author

@autoantwort What do you think of this direction?

@autoantwort
Copy link
Contributor

LGTM.

Optional<T> example_api(MessageContext& context, int a);

I don't know if this can get annoying if you have to declare a MessageContext object before each function call (Don't know if this would happen in real code).
If this gets annoying maybe something like OptionalWithMessages<T> example_api(int a); could work (the message part is always there even if there is an object)

@autoantwort
Copy link
Contributor

Ah MessageContext/DiagnosticContext is like a MessageSink and would directly print to stdout instead of ExpectedL where the caller has to do that.

@BillyONeal
Copy link
Member Author

Ah MessageContext/DiagnosticContext is like a MessageSink and would directly print to stdout instead of ExpectedL where the caller has to do that.

That's right, it's like MessageSink but the granularity is 'whole diagnostic' rather than 'individual prints' meaning it works for the synchronization boundary several PRs are looking for here.

@ras0219-msft
Copy link
Contributor

ras0219-msft commented Feb 14, 2024

This still doesn't make it easy for callers to programmatically handle specific types of errors. Currently, we have some APIs that still use explicit std::error_code because they want to do different things for 'file does not exist' vs. 'there was an I/O error'. Given that this condition isn't well served by the ExpectedL mechanism I don't want to wait until we have a better solution to it to proceed.

I think this proposal could still handle this well by viewing the DiagnosticContext as the localized, user-facing content and then using an ExpectedT<T, std::error_code> return (or other such machine-relevant error type).

Because we aren't making the context parameter the 'success carrier' it's more complex to implement 'warnings as errors' or similar functionality where the caller decides how 'important' something is. I would be in favor of moving all success tests to the context parameter but I'm not proposing that because the other vcpkg maintainers do not like it.

I think this interface could handle this via layering. Just as some top of mind thoughts:

Optional<int> foo(Ctx& ctx) {
   WarningsAsErrorsCtx inner_ctx(ctx);
   auto maybe_bar = bar(inner_ctx, 123);
   if (auto b = maybe_bar.get()) { return *b + 1; }
   return nullopt;
}

Or, when alternatives should be attempted:

Optional<int> foo(Ctx& ctx) {
   BufferCtx buffer_ctx(ctx);
   auto maybe_bar = bar(buffer_ctx, 123);
   if (auto b = maybe_bar.get()) { return *b + 1; }
   else if (/* something else */) {
     /* something else worked, drop the messages */
     buffer_ctx.drop();
     /* or perhaps convert them to warnings? */
     buffer_ctx.errors_to_warnings();
     return 5;
   }
   /* something else failed; buffer_ctx will release messages "normally" */
   return nullopt;
}

Comment on lines +68 to +69
Optional<std::string> m_origin;
TextRowCol m_position;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to bundle these (origin + position) into a single Optional? Alternatively: it's a bit unclear to me what the intended states are.

  1. No origin, No position
  2. Empty-string origin, No Position
  3. Origin, No Position
  4. Empty-string origin, Position
  5. Origin, Position

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string origin is never intended. Position without origin is never intended either. I'll look to clarify.


struct DiagnosticLine
{
template<class MessageLike, std::enable_if_t<std::is_convertible_v<MessageLike, LocalizedString>, int> = 0>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these constructors all have different arities, is this SFINAE needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's needed to make is_constructible_v<DiagnosticLine, DiagKind, int> be false.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Perfect forwarding is psycho greedy so it basically always needs SFINAE)

Copy link
Contributor

@ras0219-msft ras0219-msft May 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the concrete consequence of is_constructible_v being true in this case? Issues with wrapping this type inside Expected<>?

Conversely, what value/perf is provided by making this a template versus just taking a LocalizedString?

include/vcpkg/base/diagnostics.h Show resolved Hide resolved
}

template<class MessageLike, std::enable_if_t<std::is_convertible_v<MessageLike, LocalizedString>, int> = 0>
DiagnosticLine(DiagKind kind, StringView origin, TextRowCol position, MessageLike&& message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a thought, not yet a suggestion, based on wild prediction of what this will look like in practice:

This design requires origin and position to be separately plumbed through every function call layer down to where the DiagnosticLine is created -- typically very close to the leaf function that has the appropriate context. I'm afraid many, many functions will end up looking like the following even though they themselves never create a DiagnosticLine -- they just want to call other functions!

Optional<std::string> do_the_thing(
    DiagnosticContext& ctx,
    Optional<StringView> origin,
    TextRowCol position,
    /* Now I can start talking about _what I actually care about_ */
    int a,
    int b);

To me, this sounds like a lot of boilerplate that (gut feeling) most layers don't care about. Would it make sense to have a source position "stack-y" side channel as part of the context convention?

Optional<std::string> do_the_thing_from_the_file(DiagnosticContext& ctx, Path p, const Filesystem& fs) {
    if (auto content = fs.read_contents(ctx, p)) {
        DiagnosticFrame with_pos(ctx, p, TextRowCol{});
        /* Look ma, short calling conventions! */
        return do_the_thing(ctx, content);
    }
    return nullopt;
}

This would necessitate 3 different "reporting" overloads:

  1. I know this DiagnosticLine is attached to this specific source position
  2. I know this DiagnosticLine is not attached to any source position
  3. I want to inherit the contextual source position

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This design requires origin and position to be separately plumbed through every function call layer down to where the DiagnosticLine is created -- typically very close to the leaf function that has the appropriate context. I'm afraid many, many functions will end up looking like the following even though they themselves never create a DiagnosticLine -- they just want to call other functions!
[...]
/* Now I can start talking about _what I actually care about_ */

I argue that for functions that are talking about file / position information like this, position information is not 'something they don't care about', it's essential in implementing what that function does.

I can see an argument for a type that contains:

  1. A DiagnosticContext&
  2. The origin
  3. Tracks a cursor of where the currently being parsed position is

and today the type trying to do that is ParserBase. I'm not a fan of how ParserBase wants and encourages deriving from and making things members when I think a struct containing these 3 members being passed around is a more composable design. I didn't go all the way there here though because

  1. That starts to approach the design where callers are expected to provide additional wrapping context I asked for but you have said over time IRL that you do not like, and
  2. That would require making this PR bigger since it would change more about how ParserBase is used.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing that ParserBase tracks state incorrectly since it accepted a column but set m_start_of_line(m_it) (so the only valid col was 0 or 1) I've become convinced that 2+3 should go together but 1 should not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to add at least a comment outlining the different valid states of this object for future readers. Presumably this should be derivable by looking at all the constructor overloads, but that's much more complex than a single comment block that stands alone.

include/vcpkg/packagespec.h Outdated Show resolved Hide resolved
include/vcpkg/paragraphparser.h Outdated Show resolved Hide resolved
@@ -72,7 +72,7 @@ TEST_CASE ("parse proc/pid/stat file", "[cgroup-parser]")
std::string contents =
R"(4281 (cpptools-srv) S 4099 1676 1676 0 -1 1077936384 51165 303 472 0 81 25 0 0 20 0 10 0 829158 4924583936 39830 18446744073709551615 4194304 14147733 140725993620736 0 0 0 0 16781312 16386 0 0 0 17 1 0 0 5 0 0 16247120 16519160 29999104 140725993622792 140725993622920 140725993622920 140725993627556 0)";

auto maybe_stat = try_parse_process_stat_file({contents, "test"});
auto maybe_stat = try_parse_process_stat_file(console_diagnostic_context, {contents, "test"});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it should be possible to create a context emitter that interoperates with Catch2 to only display messages on failure.

(I know there's an INFO() macro, but I believe it's stack based)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know of a good way of doing this. The problem with INFO() et al. is that they destroy contextual information about why something happened.

I could see changing these to null_diagnostic_context if you wish?

src/vcpkg-test/manifests.cpp Outdated Show resolved Hide resolved
src/vcpkg/base/diagnostics.cpp Show resolved Hide resolved
src/vcpkg/base/diagnostics.cpp Outdated Show resolved Hide resolved
# Conflicts:
#	azure-pipelines/end-to-end-tests-dir/cli.ps1
#	include/vcpkg/base/parse.h
#	include/vcpkg/input.h
#	include/vcpkg/packagespec.h
#	include/vcpkg/paragraphparser.h
#	include/vcpkg/sourceparagraph.h
#	src/vcpkg/base/parse.cpp
#	src/vcpkg/binarycaching.cpp
#	src/vcpkg/binaryparagraph.cpp
#	src/vcpkg/commands.build-external.cpp
#	src/vcpkg/commands.build.cpp
#	src/vcpkg/commands.check-support.cpp
#	src/vcpkg/commands.depend-info.cpp
#	src/vcpkg/commands.export.cpp
#	src/vcpkg/commands.install.cpp
#	src/vcpkg/commands.package-info.cpp
#	src/vcpkg/commands.remove.cpp
#	src/vcpkg/commands.set-installed.cpp
#	src/vcpkg/commands.upgrade.cpp
#	src/vcpkg/input.cpp
#	src/vcpkg/packagespec.cpp
#	src/vcpkg/paragraphs.cpp
#	src/vcpkg/sourceparagraph.cpp
@autoantwort
Copy link
Contributor

After in person discussion with @ras0219-msft , he is unhappy with binary caching or similar displaying anything while port builds are happening, so this isn't as natural a synchronization barrier as I thought. I still think this will be helpful in implementing what those other PRs wanted to do but it won't do it entirely.

I thought this was the reason for this PR. Because of robert's concerns BGMessageSink was introduced to prevent interleaved output. But we don't know when a message begins and ends with MessageSinks which would be resolved by the "error document" wrapping the message in an object

@BillyONeal
Copy link
Member Author

I thought this was the reason for this PR. Because of robert's concerns BGMessageSink was introduced to prevent interleaved output. But we don't know when a message begins and ends with MessageSinks which would be resolved by the "error document" wrapping the message in an object

I think that's still true. The effect of the discussion there was that implementers of the interface
don't need to make it safe to submit from arbitrary threads. In your case, a context gets made for the BG thread and only the BG thread ever writes to it, so the interface doesn't need to allow interleaved access from threads.

@Thomas1664
Copy link
Contributor

  • The interface between code wanting to report events and the code wanting to consume them is a natural thread synchronization boundary. Other attempts to fix this have been incorrect by synchronizing individual print calls ( Make individual print calls thread safe #1290 )

@BillyONeal I don't think the approach in #1290 is wrong because that is exactly what's needed. Without that PR, you can't even print single messages from multiple threads. We need both PRs!

If I understand this PR correctly, the approach here is to collect all errors and print them when the operation is finished. I think this is more complicated than it needs to be for simple cases like #1256.

Copy link
Contributor

@autoantwort autoantwort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thing we need documentation when one should use msg::println_error, MessageSink::println_error or DiagnosticContext::report_error

std::vector<DiagnosticLine> lines;

// Prints all diagnostics to the terminal.
void print(MessageSink& sink) const;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
void print(MessageSink& sink) const;
void print_to(MessageSink& sink) const;

or print_diagnostics_to makes it more clear what this does.


buf.append(m_message.data());
buf.push_back('\n');
sink.print(Color::none, buf);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific reason why only the error is printed in red and not the whole error message as it is now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've gotten feedback in the past that users find our error messages annoying and/or difficult to read because their terminal is bleeding. Coloring only the 'error' or 'warning' part is consistent with what compilers do:

gcc and clang error output showing that not the whole message is red

int column = 0;
};

struct DiagnosticLine
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe DiagnosticMessage or simple Message? Since it is allowed that the diagnostic line contains multiple lines

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The word 'message' is heavily used already by the localization infrastructure and I didn't want to step on such names.

@BillyONeal
Copy link
Member Author

BillyONeal commented May 1, 2024

I thing we need documentation when one should use msg::println_error, MessageSink::println_error or DiagnosticContext::report_error

msg:: vs DiagnosticContext: Unfortunately I think that one will remain at 'it depends' for a long time. Ideally over time I expect more things will choose the latter but we don't want to set a standard requiring people to upgrade swaths of code in order to do anything.

MessageSink -> DiagnosticContext SoonTM

BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Jun 7, 2024
# Conflicts:
#	include/vcpkg/base/message-data.inc.h
#	locales/messages.json
#	src/vcpkg/base/json.cpp
#	src/vcpkg/commands.ci-verify-versions.cpp
#	src/vcpkg/commands.format-manifest.cpp
#	src/vcpkg/paragraphs.cpp
}

template<class MessageLike, std::enable_if_t<std::is_convertible_v<MessageLike, LocalizedString>, int> = 0>
DiagnosticLine(DiagKind kind, StringView origin, TextRowCol position, MessageLike&& message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to add at least a comment outlining the different valid states of this object for future readers. Presumably this should be derivable by looking at all the constructor overloads, but that's much more complex than a single comment block that stands alone.

Comment on lines 61 to 62
// Prints this diagnostic to the terminal.
// Not thread safe: The console DiagnosticContext must apply its own synchronization.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused by this comment since MessageSink might not be a terminal.

Why is console DiagnosticContext mentioned in this comment, when the function takes a MessageSink?

"https://github.com/microsoft/vcpkg, "
"with the following output:\n{error_msg}\nSerialized Binary Paragraph:")
DECLARE_MESSAGE(
FailedToParseSerializedBinParagraphSuffix,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message seems substantially more generic now than just failing to parse a serialized binary paragraph. Could this get a more descriptive name?

ParserBase(StringView text, Optional<StringView> origin, TextRowCol init_rowcol = {});
// When parsing an in memory entity or similar not-on-disk entity, init_row should be set to 0
// When parsing a file, init_rowcol should be set to 1 if starting from the top of the file
ParserBase(DiagnosticContext& context, StringView text, Optional<StringView> origin, int init_row);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it valid to pass nullopt, 1? If not, then should init_row be moved into the Optional<> via a pair or such?

Suggested change
ParserBase(DiagnosticContext& context, StringView text, Optional<StringView> origin, int init_row);
ParserBase(DiagnosticContext& context, StringView text, Optional<std::pair<StringView, int>> origin);

Or, should this just be overloads?

Suggested change
ParserBase(DiagnosticContext& context, StringView text, Optional<StringView> origin, int init_row);
ParserBase(DiagnosticContext& context, StringView text);
// Triggers fatal error on origin.empty()
ParserBase(DiagnosticContext& context, StringView text, StringView origin, int init_row);

@@ -53,7 +53,9 @@ namespace vcpkg
bool operator()(const PackageSpec& spec) const;
};

std::vector<CiBaselineLine> parse_ci_baseline(StringView text, StringView origin, ParseMessages& messages);
Optional<std::vector<CiBaselineLine>> parse_ci_baseline(DiagnosticContext& context,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Should this include base/diagnostics.h?

Suggested change
Optional<std::vector<CiBaselineLine>> parse_ci_baseline(DiagnosticContext& context,
Optional<std::vector<CiBaselineLine>> parse_ci_baseline(ParserBase parse);

@@ -43,10 +43,18 @@ namespace vcpkg
std::vector<LocalizedString> errors;
};

Optional<std::vector<std::string>> parse_default_features_list(DiagnosticContext& context,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Optional<std::vector<std::string>> parse_default_features_list(DiagnosticContext& context,
Optional<std::vector<std::string>> parse_default_features_list(ParserBase& context);

@BillyONeal BillyONeal enabled auto-merge (squash) December 7, 2024 00:09
@BillyONeal BillyONeal merged commit b4b372c into microsoft:main Dec 7, 2024
6 checks passed
@BillyONeal BillyONeal deleted the error-document branch December 7, 2024 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants