Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix some image attachments in comments not loading in game #30866

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

frenzibyte
Copy link
Member

@frenzibyte frenzibyte commented Nov 25, 2024

End result can be tested by viewing comments such as this one, where previously the game will not load such images, but now it should.

I was initially going to implement this locally in OsuMarkdownImages (more or less similar to how osu-web did it), but I figured it would be a more solid approach to cover the entire OnlineStore layer instead, since the end goal is to avoid looking up resources from third-party links.

Comment on lines 23 to 26
if (Uri.TryCreate(url, UriKind.Absolute, out Uri? uri) && uri.Host.EndsWith(@".ppy.sh", StringComparison.OrdinalIgnoreCase))
return url;

return $@"{apiEndpointUrl}/beatmapsets/discussions/media-url?url={url}";
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This conditional may technically be pointless, but I felt proxying .ppy.sh links is too much proxying, and also makes network logs look ugly.

if (Uri.TryCreate(url, UriKind.Absolute, out Uri? uri) && uri.Host.EndsWith(@".ppy.sh", StringComparison.OrdinalIgnoreCase))
return url;

return $@"{apiEndpointUrl}/beatmapsets/discussions/media-url?url={url}";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if this is a big deal, but it feels wrong to me for lazer to be explicitly depending on a non-api route, even if it will still function properly in this case. and it should probably be updated to not have a uri specific to discussions 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure what the concern is with a non-API route. As for the definition of the endpoint looking like it's specific to discussions, that's probably something to bring to @ppy/team-web, if action is deemed necessary and feasible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I dunno about this.

I believe that endpoint should only be used for discussion links (are we somehow limiting this? could an abusive user be using this endpoint to create arbitrary proxied links?). If we're using it for more than this it should be something more generalised.

Can the store be used local to the discussions/comments system? And the default osu!-side online store just blocks all non-ppy resources?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds feasible, will update as such.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that endpoint should only be used for discussion links (are we somehow limiting this? could an abusive user be using this endpoint to create arbitrary proxied links?)

https://i.ppy.sh/b5810b2de4431c96218200725bcdb795376c780f/68747470733a2f2f7570322e636c6179746f6e2e63632f6970707973682e706e67

I guess this is more of a web issue then...

Copy link
Collaborator

@bdach bdach Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am so baffled as to how things got as far as this PR without talking about any of the concerns with using the route in question in such an undocumented manner first.

To me this either should have not been ever PR'd, or should have at least been preempted with a "is this even ok to do" sanity check.

Wholehearted agreement with @cl8n, and if anything I think that remark was too subtle. This 100% feels too broken to live and is susceptible to breakage in the future because no sane person would assume the discussions route would be used by client in contexts it is clearly not supposed to be used in.

The route either needs to be somehow generalised, or the entire burden of doing this should be offloaded to web by doing something like replacing the broken URLs with the proxied URLs on the server side so that the proxying is transparent to the client.

Copy link

@nanaya nanaya Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm yeah, I'm thinking of changing the url to something more generic (ppy/osu-web#11686). Dunno if api will need its own specific url under /api but I suppose it can be added...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bdach I think you've extremely raised the tone up there, I will say I have chose to open this PR so that if it's alright then it's alright and if it's not then the points of why and whatever are discussed appropriately in here with easier access to the diff, period.

Copy link

@notbakaneko notbakaneko Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll point out that the media-url proxying on discussions is mainly so someone can't just drop in an image for link tracking or fingerprinting users and only partly for caching if the host falls over. If there's a way to do that without proxying, that would be preferable.

Copy link

@nanaya nanaya Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the proxying happens on all user posted images...

media-url thing is specifically because the markdown parsing happens on frontend so it doesn't have the camo key and can't just generate the actual proxied link like in backend.

it's currently only beatmapset discussions it happens on frontend (on web) so that's why it's been sitting there.

@pull-request-size pull-request-size bot added size/M and removed size/L labels Nov 25, 2024
@frenzibyte
Copy link
Member Author

As per #30866 (comment), resource lookups from external websites are now blocked (with reasonably loud warning logs), and an exception is given to osu! markdown images which are used by comments, wiki, and future overlays to come such as beatmap discussions and news.

The warning logs can turn into errors to grab our attention via sentry, but perhaps that's an overkill.

@bdach
Copy link
Collaborator

bdach commented Nov 25, 2024

resource lookups from external websites are now blocked (with reasonably loud warning logs)

Did you test changelogs? Specifically on production? I have a suspicion they would be broken by this.

@frenzibyte
Copy link
Member Author

Huh, I thought changelogs have been using markdown already. Thankfully it doesn't do any kind of online link lookups so it's not broken by this.

var message = new TextFlowContainer
{
AutoSizeAxes = Axes.Y,
RelativeSizeAxes = Axes.X,
};
// todo: use markdown parsing once API returns markdown
message.AddText(WebUtility.HtmlDecode(Regex.Replace(entry.MessageHtml, @"<(.|\n)*?>", string.Empty)), t =>
{
t.Font = fontMedium;
t.Colour = colourProvider.Foreground1;
});

@bdach
Copy link
Collaborator

bdach commented Nov 25, 2024

I meant images embedded in changelogs...? Are you saying those just don't display?

@frenzibyte
Copy link
Member Author

frenzibyte commented Nov 25, 2024

Yes, the code above shows that everything in the changelogs is being treated as text and nothing but that.

@peppy
Copy link
Member

peppy commented Nov 25, 2024

Am I hallucinating or did images used to be a thing on the changelog overlay?

@frenzibyte
Copy link
Member Author

Am I hallucinating or did images used to be a thing on the changelog overlay?

Well there's #12465, which is probably a definite proof that we never had support for it in the first place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Image attachments hosted in s-ul.eu do not show in game
6 participants