Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce DiagnosticContext, an 'error document' like type. #1323

Merged
merged 42 commits into from
Dec 7, 2024
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
3138143
In several PRs, such as https://github.com/microsoft/vcpkg-tool/pull/…
BillyONeal Dec 21, 2023
0223881
Add color printing.
BillyONeal Dec 29, 2023
94722bb
Add unique_ptr handling.
BillyONeal Dec 30, 2023
ba58259
Forward in more places.
BillyONeal Dec 31, 2023
bbd39f2
More tests and bugfixes.
BillyONeal Dec 31, 2023
1549d60
More testing!
BillyONeal Dec 31, 2023
164a8dc
Tests, bugfixes, comments. Simplify the unique_ptr adapter to use ref…
BillyONeal Jan 2, 2024
fbcf1e7
First crack at applying DiagnosticContext to ParserBase.
BillyONeal Jan 10, 2024
9396f8a
Fix unit test failures.
BillyONeal Jan 10, 2024
6e3f487
Revert the TextRowCol rename
BillyONeal Jan 26, 2024
9fdf0e2
Minor self CR feedback.
BillyONeal Jan 26, 2024
04c46ec
Remove unused WarningsTreatedAsErrors
BillyONeal Jan 26, 2024
e0287e6
Avoid -Wunneeded-internal-declaration
BillyONeal Jan 26, 2024
ddb26ce
Avoid -Wmissing-prototypes
BillyONeal Jan 26, 2024
58eb07e
"<unknown>" is not an origin
BillyONeal Feb 17, 2024
0d3005b
Merge branch 'no-unknown' into error-document
BillyONeal Feb 17, 2024
bc2ac20
Merge remote-tracking branch 'origin/main' into no-unknown
BillyONeal Feb 21, 2024
bd20e7a
<command> is not an origin either.
BillyONeal Feb 21, 2024
a02e4d4
Merge branch 'no-unknown' into error-document
BillyONeal Feb 21, 2024
9c22b23
Add more comments for adapt_context_to_expected.
BillyONeal Feb 21, 2024
8d49cf0
Drop the _context convention.
BillyONeal Feb 21, 2024
062c7f8
Delete the bogus 'example text' from the full package spec parsing.
BillyONeal Feb 22, 2024
2c43b40
Merge branch 'no-unknown' into error-document
BillyONeal Feb 22, 2024
b49ad4f
Block empty origins.
BillyONeal Feb 22, 2024
0a92366
Remove implication that report() is locked.
BillyONeal Feb 22, 2024
9a11f96
Other code review fixes.
BillyONeal Feb 22, 2024
de35a73
clang-format
BillyONeal Feb 22, 2024
b28aadc
Merge remote-tracking branch 'origin/main' into error-document
BillyONeal Apr 20, 2024
24ac5bf
Merge remote-tracking branch 'origin/main' into error-document
BillyONeal Apr 22, 2024
f7c13ab
Correctly track row information and avoid emitting diagnostics for sp…
BillyONeal Apr 22, 2024
87a6c0e
Dedupe CLI e2e tests.
BillyONeal Apr 22, 2024
1616bbd
Fix posix build
BillyONeal Apr 22, 2024
3159407
Downgrade clang-format to 16
BillyONeal Apr 22, 2024
bab1477
Merge remote-tracking branch 'origin/main' into error-document
BillyONeal Jul 25, 2024
159f9f3
clang-format
BillyONeal Jul 26, 2024
2224689
clang-format again?
BillyONeal Jul 26, 2024
5cad35c
Rename print to print_to as suggested by @autoantwort
BillyONeal Jul 26, 2024
2a2bcf5
Merge remote-tracking branch 'origin/main' into error-document
BillyONeal Dec 6, 2024
a817cf7
Revert most everything related to ParserBase.
BillyONeal Dec 6, 2024
75f997b
Format
BillyONeal Dec 6, 2024
c1239d3
Fix test failures oops
BillyONeal Dec 7, 2024
bfed436
format
BillyONeal Dec 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
273 changes: 273 additions & 0 deletions include/vcpkg/base/diagnostics.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,273 @@
#pragma once

#include <vcpkg/base/expected.h>
#include <vcpkg/base/messages.h>
#include <vcpkg/base/optional.h>

#include <memory>
#include <string>
#include <type_traits>
#include <vector>

namespace vcpkg
{
enum class DiagKind
{
None, // foo.h: localized
Message, // foo.h: message: localized
Error, // foo.h: error: localized
Warning, // foo.h: warning: localized
Note, // foo.h: note: localized
COUNT
};

struct TextRowCol
{
// '0' indicates uninitialized; '1' is the first row/column
int row = 0;
int column = 0;
};

struct DiagnosticLine
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe DiagnosticMessage or simple Message? Since it is allowed that the diagnostic line contains multiple lines

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The word 'message' is heavily used already by the localization infrastructure and I didn't want to step on such names.

{
template<class MessageLike, std::enable_if_t<std::is_convertible_v<MessageLike, LocalizedString>, int> = 0>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these constructors all have different arities, is this SFINAE needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's needed to make is_constructible_v<DiagnosticLine, DiagKind, int> be false.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Perfect forwarding is psycho greedy so it basically always needs SFINAE)

Copy link
Contributor

@ras0219-msft ras0219-msft May 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the concrete consequence of is_constructible_v being true in this case? Issues with wrapping this type inside Expected<>?

Conversely, what value/perf is provided by making this a template versus just taking a LocalizedString?

DiagnosticLine(DiagKind kind, MessageLike&& message)
: m_kind(kind), m_origin(), m_position(), m_message(std::forward<MessageLike>(message))
{
}

template<class MessageLike, std::enable_if_t<std::is_convertible_v<MessageLike, LocalizedString>, int> = 0>
DiagnosticLine(DiagKind kind, StringView origin, MessageLike&& message)
: m_kind(kind), m_origin(origin.to_string()), m_position(), m_message(std::forward<MessageLike>(message))
{
if (origin.empty())
{
Checks::unreachable(VCPKG_LINE_INFO, "origin must not be empty");
}
}

template<class MessageLike, std::enable_if_t<std::is_convertible_v<MessageLike, LocalizedString>, int> = 0>
DiagnosticLine(DiagKind kind, StringView origin, TextRowCol position, MessageLike&& message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a thought, not yet a suggestion, based on wild prediction of what this will look like in practice:

This design requires origin and position to be separately plumbed through every function call layer down to where the DiagnosticLine is created -- typically very close to the leaf function that has the appropriate context. I'm afraid many, many functions will end up looking like the following even though they themselves never create a DiagnosticLine -- they just want to call other functions!

Optional<std::string> do_the_thing(
    DiagnosticContext& ctx,
    Optional<StringView> origin,
    TextRowCol position,
    /* Now I can start talking about _what I actually care about_ */
    int a,
    int b);

To me, this sounds like a lot of boilerplate that (gut feeling) most layers don't care about. Would it make sense to have a source position "stack-y" side channel as part of the context convention?

Optional<std::string> do_the_thing_from_the_file(DiagnosticContext& ctx, Path p, const Filesystem& fs) {
    if (auto content = fs.read_contents(ctx, p)) {
        DiagnosticFrame with_pos(ctx, p, TextRowCol{});
        /* Look ma, short calling conventions! */
        return do_the_thing(ctx, content);
    }
    return nullopt;
}

This would necessitate 3 different "reporting" overloads:

  1. I know this DiagnosticLine is attached to this specific source position
  2. I know this DiagnosticLine is not attached to any source position
  3. I want to inherit the contextual source position

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This design requires origin and position to be separately plumbed through every function call layer down to where the DiagnosticLine is created -- typically very close to the leaf function that has the appropriate context. I'm afraid many, many functions will end up looking like the following even though they themselves never create a DiagnosticLine -- they just want to call other functions!
[...]
/* Now I can start talking about _what I actually care about_ */

I argue that for functions that are talking about file / position information like this, position information is not 'something they don't care about', it's essential in implementing what that function does.

I can see an argument for a type that contains:

  1. A DiagnosticContext&
  2. The origin
  3. Tracks a cursor of where the currently being parsed position is

and today the type trying to do that is ParserBase. I'm not a fan of how ParserBase wants and encourages deriving from and making things members when I think a struct containing these 3 members being passed around is a more composable design. I didn't go all the way there here though because

  1. That starts to approach the design where callers are expected to provide additional wrapping context I asked for but you have said over time IRL that you do not like, and
  2. That would require making this PR bigger since it would change more about how ParserBase is used.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing that ParserBase tracks state incorrectly since it accepted a column but set m_start_of_line(m_it) (so the only valid col was 0 or 1) I've become convinced that 2+3 should go together but 1 should not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to add at least a comment outlining the different valid states of this object for future readers. Presumably this should be derivable by looking at all the constructor overloads, but that's much more complex than a single comment block that stands alone.

: m_kind(kind)
, m_origin(origin.to_string())
, m_position(position)
, m_message(std::forward<MessageLike>(message))
{
if (origin.empty())
{
Checks::unreachable(VCPKG_LINE_INFO, "origin must not be empty");
}
}

// Prints this diagnostic to the supplied sink.
void print_to(MessageSink& sink) const;
// Converts this message into a string
// Prefer print() if possible because it applies color
std::string to_string() const;
void to_string(std::string& target) const;

LocalizedString to_json_reader_string(const std::string& path, const LocalizedString& type) const;

DiagKind kind() const noexcept { return m_kind; }

private:
DiagKind m_kind;
Optional<std::string> m_origin;
TextRowCol m_position;
Comment on lines +75 to +76
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to bundle these (origin + position) into a single Optional? Alternatively: it's a bit unclear to me what the intended states are.

  1. No origin, No position
  2. Empty-string origin, No Position
  3. Origin, No Position
  4. Empty-string origin, Position
  5. Origin, Position

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string origin is never intended. Position without origin is never intended either. I'll look to clarify.

LocalizedString m_message;
};

struct DiagnosticContext
{
virtual void report(const DiagnosticLine& line) = 0;
virtual void report(DiagnosticLine&& line) { report(line); }

void report_error(const LocalizedString& message) { report(DiagnosticLine{DiagKind::Error, message}); }
void report_error(LocalizedString&& message) { report(DiagnosticLine{DiagKind::Error, std::move(message)}); }
template<VCPKG_DECL_MSG_TEMPLATE>
void report_error(VCPKG_DECL_MSG_ARGS)
{
LocalizedString message;
msg::format_to(message, VCPKG_EXPAND_MSG_ARGS);
this->report_error(std::move(message));
}

protected:
~DiagnosticContext() = default;
};

struct BufferedDiagnosticContext final : DiagnosticContext
{
virtual void report(const DiagnosticLine& line) override;
virtual void report(DiagnosticLine&& line) override;

std::vector<DiagnosticLine> lines;

// Prints all diagnostics to the supplied sink.
void print_to(MessageSink& sink) const;
// Converts this message into a string
// Prefer print() if possible because it applies color
std::string to_string() const;
void to_string(std::string& target) const;

bool any_errors() const noexcept;
};

extern DiagnosticContext& console_diagnostic_context;
extern DiagnosticContext& null_diagnostic_context;

// The following overloads are implementing
// adapt_context_to_expected(Fn functor, Args&&... args)
//
// Given:
// Optional<T> functor(DiagnosticContext&, args...), adapts functor to return ExpectedL<T>
// Optional<T>& functor(DiagnosticContext&, args...), adapts functor to return ExpectedL<T&>
// Optional<T>&& functor(DiagnosticContext&, args...), adapts functor to return ExpectedL<T>
// std::unique_ptr<T> functor(DiagnosticContext&, args...), adapts functor to return ExpectedL<std::unique_ptr<T>>

// If Ty is an Optional<U>, typename AdaptContextUnwrapOptional<Ty>::type is the type necessary to return U, and fwd
// is the type necessary to forward U. Otherwise, there are no members ::type or ::fwd
template<class Ty>
struct AdaptContextUnwrapOptional
{
// no member ::type, SFINAEs out when the input type is:
// * not Optional
// * volatile
};

template<class Wrapped>
struct AdaptContextUnwrapOptional<Optional<Wrapped>>
{
// prvalue, move from the optional into the expected
using type = Wrapped;
using fwd = Wrapped&&;
};

template<class Wrapped>
struct AdaptContextUnwrapOptional<const Optional<Wrapped>>
{
// const prvalue
using type = Wrapped;
using fwd = const Wrapped&&;
};

template<class Wrapped>
struct AdaptContextUnwrapOptional<Optional<Wrapped>&>
{
// lvalue, return an expected referencing the Wrapped inside the optional
using type = Wrapped&;
using fwd = Wrapped&;
};

template<class Wrapped>
struct AdaptContextUnwrapOptional<const Optional<Wrapped>&>
{
// const lvalue
using type = const Wrapped&;
using fwd = const Wrapped&;
};

template<class Wrapped>
struct AdaptContextUnwrapOptional<Optional<Wrapped>&&>
{
// xvalue, move from the optional into the expected
using type = Wrapped;
using fwd = Wrapped&&;
};

template<class Wrapped>
struct AdaptContextUnwrapOptional<const Optional<Wrapped>&&>
{
// const xvalue
using type = Wrapped;
using fwd = const Wrapped&&;
};

// The overload for functors that return Optional<T>
template<class Fn, class... Args>
auto adapt_context_to_expected(Fn functor, Args&&... args)
-> ExpectedL<
typename AdaptContextUnwrapOptional<std::invoke_result_t<Fn, BufferedDiagnosticContext&, Args...>>::type>
{
using Unwrapper = AdaptContextUnwrapOptional<std::invoke_result_t<Fn, BufferedDiagnosticContext&, Args...>>;
using ReturnType = ExpectedL<typename Unwrapper::type>;
BufferedDiagnosticContext bdc;
decltype(auto) maybe_result = functor(bdc, std::forward<Args>(args)...);
if (auto result = maybe_result.get())
{
// N.B.: This may be a move
return ReturnType{static_cast<typename Unwrapper::fwd>(*result), expected_left_tag};
}

return ReturnType{LocalizedString::from_raw(bdc.to_string()), expected_right_tag};
}

// If Ty is a std::unique_ptr<U>, typename AdaptContextDetectUniquePtr<Ty>::type is the type necessary to return
template<class Ty>
struct AdaptContextDetectUniquePtr
{
// no member ::type, SFINAEs out when the input type is:
// * not unique_ptr
// * volatile
};

template<class Wrapped, class Deleter>
struct AdaptContextDetectUniquePtr<std::unique_ptr<Wrapped, Deleter>>
{
// prvalue, move into the Expected
using type = std::unique_ptr<Wrapped, Deleter>;
};

template<class Wrapped, class Deleter>
struct AdaptContextDetectUniquePtr<const std::unique_ptr<Wrapped, Deleter>>
{
// const prvalue (not valid, can't be moved from)
// no members
};

template<class Wrapped, class Deleter>
struct AdaptContextDetectUniquePtr<std::unique_ptr<Wrapped, Deleter>&>
{
// lvalue, reference the unique_ptr itself
using type = std::unique_ptr<Wrapped, Deleter>&;
};

template<class Wrapped, class Deleter>
struct AdaptContextDetectUniquePtr<const std::unique_ptr<Wrapped, Deleter>&>
{
// const lvalue
using type = const std::unique_ptr<Wrapped, Deleter>&;
};

template<class Wrapped, class Deleter>
struct AdaptContextDetectUniquePtr<std::unique_ptr<Wrapped, Deleter>&&>
{
// xvalue, move into the Expected
using type = std::unique_ptr<Wrapped, Deleter>;
};

template<class Wrapped, class Deleter>
struct AdaptContextDetectUniquePtr<const std::unique_ptr<Wrapped, Deleter>&&>
{
// const xvalue (not valid, can't be moved from)
// no members
};

// The overload for functors that return std::unique_ptr<T>
template<class Fn, class... Args>
auto adapt_context_to_expected(Fn functor, Args&&... args)
-> ExpectedL<
typename AdaptContextDetectUniquePtr<std::invoke_result_t<Fn, BufferedDiagnosticContext&, Args...>>::type>
{
using ReturnType = ExpectedL<
typename AdaptContextDetectUniquePtr<std::invoke_result_t<Fn, BufferedDiagnosticContext&, Args...>>::type>;
BufferedDiagnosticContext bdc;
decltype(auto) maybe_result = functor(bdc, std::forward<Args>(args)...);
if (maybe_result)
{
return ReturnType{static_cast<decltype(maybe_result)&&>(maybe_result), expected_left_tag};
}

return ReturnType{LocalizedString::from_raw(bdc.to_string()), expected_right_tag};
BillyONeal marked this conversation as resolved.
Show resolved Hide resolved
}
}
1 change: 0 additions & 1 deletion include/vcpkg/base/fwd/parse.h
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,5 @@ namespace vcpkg
};

struct ParseMessage;
struct ParseMessages;
struct ParserBase;
}
4 changes: 4 additions & 0 deletions include/vcpkg/base/git.h
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

#include <vcpkg/base/fwd/git.h>

#include <vcpkg/base/diagnostics.h>
#include <vcpkg/base/expected.h>
#include <vcpkg/base/path.h>
#include <vcpkg/base/stringview.h>
Expand Down Expand Up @@ -47,6 +48,9 @@ namespace vcpkg
std::string try_extract_port_name_from_path(StringView path);

// Attempts to parse the git status output returns a parsing error message on failure
Optional<std::vector<GitStatusLine>> parse_git_status_output(DiagnosticContext& context,
StringView git_status_output,
StringView git_command_line);
ExpectedL<std::vector<GitStatusLine>> parse_git_status_output(StringView git_status_output,
StringView git_command_line);

Expand Down
4 changes: 3 additions & 1 deletion include/vcpkg/base/jsonreader.h
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,10 @@ namespace vcpkg::Json
void add_expected_type_error(const LocalizedString& expected_type);
void add_extra_field_error(const LocalizedString& type, StringView fields, StringView suggestion = {});
void add_generic_error(const LocalizedString& type, StringView message);
void add_diagnostic_error(const LocalizedString& type, const DiagnosticLine& line);

void add_warning(LocalizedString type, StringView msg);
void add_warning(const LocalizedString& type, StringView msg);
void add_diagnostic_warning(const LocalizedString& type, const DiagnosticLine& warning);

const std::vector<LocalizedString>& warnings() const { return m_warnings; }

Expand Down
13 changes: 5 additions & 8 deletions include/vcpkg/base/message-data.inc.h
Original file line number Diff line number Diff line change
Expand Up @@ -1252,12 +1252,11 @@ DECLARE_MESSAGE(FailedToParseConfig, (msg::path), "", "Failed to parse configura
DECLARE_MESSAGE(FailedToParseVersionFile, (msg::path), "", "Failed to parse version file: {path}")
DECLARE_MESSAGE(FailedToParseNoTopLevelObj, (msg::path), "", "Failed to parse {path}, expected a top-level object.")
DECLARE_MESSAGE(FailedToParseNoVersionsArray, (msg::path), "", "Failed to parse {path}, expected a 'versions' array.")
DECLARE_MESSAGE(FailedToParseSerializedBinParagraph,
(msg::error_msg),
"'{error_msg}' is the error message for failing to parse the Binary Paragraph.",
"[sanity check] Failed to parse a serialized binary paragraph.\nPlease open an issue at "
"https://github.com/microsoft/vcpkg, "
"with the following output:\n{error_msg}\nSerialized Binary Paragraph:")
DECLARE_MESSAGE(
FailedToParseSerializedBinParagraphSuffix,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message seems substantially more generic now than just failing to parse a serialized binary paragraph. Could this get a more descriptive name?

(),
"",
"This is a bug in vcpkg. Please open an issue at https://github.com/microsoft/vcpkg, including the following:")
DECLARE_MESSAGE(FailedToParseVersionXML,
(msg::tool_name, msg::version),
"",
Expand Down Expand Up @@ -1843,7 +1842,6 @@ DECLARE_MESSAGE(InvalidArgumentRequiresZeroOrOneArgument,
(msg::binary_source),
"",
"invalid argument: binary config '{binary_source}' requires 0 or 1 argument")
DECLARE_MESSAGE(InvalidBuildInfo, (msg::error_msg), "", "Invalid BUILD_INFO file for package: {error_msg}")
DECLARE_MESSAGE(
InvalidBuiltInBaseline,
(msg::value),
Expand Down Expand Up @@ -3133,7 +3131,6 @@ DECLARE_MESSAGE(VSExaminedPaths, (), "", "The following paths were examined for
DECLARE_MESSAGE(VSNoInstances, (), "", "Could not locate a complete Visual Studio instance")
DECLARE_MESSAGE(WaitingForChildrenToExit, (), "", "Waiting for child processes to exit...")
DECLARE_MESSAGE(WaitingToTakeFilesystemLock, (msg::path), "", "waiting to take filesystem lock on {path}...")
DECLARE_MESSAGE(WarningsTreatedAsErrors, (), "", "previous warnings being interpreted as errors")
DECLARE_MESSAGE(WarnOnParseConfig, (msg::path), "", "Found the following warnings in configuration {path}:")
DECLARE_MESSAGE(WhileCheckingOutBaseline, (msg::commit_sha), "", "while checking out baseline {commit_sha}")
DECLARE_MESSAGE(WhileCheckingOutPortTreeIsh,
Expand Down
Loading
Loading