[RFC] Allow post-selection-box tools to operate on non-screenshot images

[MyriaCore]

The post-selection-box tools (e.g. the image editing tools that pop up on the sides of the selection box) are easily the best feature of flameshot. They allow quick and easy editing of screenshots in an intuitive way.

It would be amazing if the user could open non-screenshot images with these tools. Users could open up images on their computer in flameshot's tools, do a bit of tinkering, and then make use of flameshot's built in actions (save, copy, upload, etc).

For the purposes of this issue, there are 4 main types of 'non-screenshot-image', in the order of importance:

1. Local Images
2. Image URLs
3. Images in the clipboard
4. Image URLs in the clipboard

A potential implementation of this proposal should support types 1 and 2 at minumum, as this would enable the infrastructure needed to meet all the UX Goals without the user needing to do anything too hacky. However, supporting image types 3 and 4 would really round the whole UX out.

UX Goals

Here, I just wanna outline some examples of new ways this proposal could allow the user to interact with the software. Nothing comprehensive, just something to get a feel for new things this would allow.

The user can now:

• Pin images at an imgur link
• take a screenshot → upload to imgur → realize they forgot to edit the screenie → open the imgur link to make that edit → re-upload to imgur
• Copy an image from somewhere¹ → Add a few numbered circles for reference later → Upload the image

UX Challenges

The main design challenge here would be how to visually handle non-screenshot-image modification. I do have a few design notes to give a rough idea of what could be done here:

Problem: What if the image is smaller than the display size?

• We'll just put the image itself at the center of the screen, and let it grow out to wherever it stops
• Maybe Ctrl + M1 Drag will move the image around as it's being edited, so it doesn't feel like the image is fixed in place
• Disallow the user from dragging the corners of the selection box past the image's actual size. This prevents whatever was in the background from leaking into the resulting image by accident.

Problem: What should we put in the background?

• We could put a UI-Colored background in, like how we handle the Tool Settings flyout menu.
• We could have a white-and-gray checkerboard background, or just a solid-color one of the user's preference.
• We could treat it like it's a screenshot, and have a grayed-out freeze-frame of the desktop.
  • It would be useful to consider a darker gray than what we usually use for the desktop freeze frame, so the normal gray can be used for unselected image space. This would make the difference between the selection bounds and the image space bounds more clear:
    

Implementation Challenges

The other gotcha that I can think of rn is if the image being edited is a higher resolution than the monitor flameshot is being run on. The way I see it, there are a few approaches we could take to tackle this problem:

Approach 1: Display a downscaled image

1. when displayed, the image could be downscaled to fit the screen
2. Edits with flameshot's tools would take place over the image's "real" resolution in memory
   • edits to the "real image" would have to be upscaled at the same rate as the image was downscaled
   • this would make what the user sees match what the tools do. Otherwise the scales would desync

Approach 2: Actually downscale the image

This kinda undermines the utility¹ of copying images, and reduces it to just convenience, but if this is a road we wanna go down, we can just actually downscale the image to fit the user's screen. Make sense, it's easy and quick. And, the convenience aspect is still maintained.

Approach 3: Don't let the user open images that are too big

This one's probably the lamest approach, but it's also a road we could go down, so I figured I'd include it. It's cheap and easy, and requires no further dev time.

Footnotes

1. Useful for maintaining the image's resolution. Users don't always wanna take a screenshot of an image that's copyable ↩ ↩²

[mmahmoudian]

This is an interesting idea. I myself sometimes feel the need to open the image again and add few more annotations (esp. arrow and highlight), so I somehow feel the usefulness of such feature. On the other hand, this feature is at the edge of the definition of a screenshot capture application. I have mixed feeling here, on one hand I really sometimes need this feature and on the other hand I feel like it is off-topic. So I vote neutral for the time being as I can live with or without this feature, plus I don't have a solid understanding of how hard it is to implement this (something that @borgmanJeremy, @ZetaoYang, @hosiet, @AlexP11223, @Martin-Eckleben and others can assess better).

Regarding the text, there is one point that is missing: What if the picture is larger than the monitor's resolution? (URL or Clipboard can be from any source) Logically, one way would be to have the same zooming functionality as the pin tool and show the user the percentage of zoom in the bubble in the upper left corner (similar to the thickness/size indicator).

Also, about the UX, instead of freezing user's monitor, it can be opened in a window and have all those tools as buttons (either floating like Gimp or as toolbars like inkscape). This way user can switch window and do other tasks as well.

[AlexP11223]

I feel like it is off-topic

Then the current editing after capture is off-topic too, can just launch GIMP :D

On Windows it is included in ShareX (via Image History window) and Monosnap (tray -> Open image).

it can be opened in a window

Yeah, I think it would be better.

how hard it is to implement this

I think most of the things are already implemented, the capture window has the normal window mode (used for debugging). Only zoom is missing, but it can be added later, for now just scrollbars.

May need to adjust some options like "Save after copy" (does it make sense, and should it save to a new file or to the opened one?).

should support types 1 and 2 at minumum

I think 2 is much less important, it's easy to save the image locally.

[MyriaCore]

how hard it is to implement this

I think most of the things are already implemented, the capture window has the normal window mode (used for debugging). Only zoom is missing, but it can be added later, for now just scrollbars.

@AlexP11223 didn't know there was already window infrastructure, but using windows would definitely be better. How does the UI/UX for them work currently? I feel like it'd be cool for them to have a completely transparent buffer space around the image or something so the tool icons and right-click color picker could still be displayed as floating UI elements.

Regardless, at worst we could just provide enough buffer space for the tool icons to float, and then arrange the color picker's colors into quarter-pies and half-pies if the mouse is too close to the corner or edge of the screen. The only other thing I can think of with this is that it might be weird bringing up the spacebar tool settings menu, but I guess we could always settle for simple window position / size requests to open that window up from the left or right side of the screen.

May need to adjust some options like "Save after copy" (does it make sense, and should it save to a new file or to the opened one?)

I think we should treat non-screenshot images like they're just newly-taken screenshots as far as export options go. I personally think having an option to overwrite a re-opened screenshot would be cool, but we can always implement that in a later issue/PR, so I'd say it's out of scope. Also, the implementation for a feature like this would be affected by the potential resolution of #360, which could better inform whether an opened image was actually a re-opened screenshot.

Waiting on it would also give us space to figure out if we only wanna allow local file imports, or if we actually wanna support clipboard and URL.

Regarding the text, there is one point that is missing: What if the picture is larger than the monitor's resolution? (URL or Clipboard can be from any source) Logically, one way would be to have the same zooming functionality as the pin tool and show the user the percentage of zoom in the bubble in the upper left corner (similar to the thickness/size indicator).

@mmahmoudian I did talk about that in the Implementation Challenges section. This is probably the most difficult one to get right, since it'd require some sacrifices somewhere to get a quick and easy implementation. My problem is that I want next to full feature parity. I'd really prefer to avoid having functionality specific to non-screenshots unless it's absolutely necessary.

While a zoom feature would fix the problem, I worry that it's too nuclear a solution. I feel like downscaling the image (approach 2) is the better for a number of reasons:

• Solves the problem without any relative-scale issues. zoom & display-only-downscale (approach 1) could be a bit problematic here
• Doesn't add or change the current editing UX, which means it'll be more familiar to users
• My hunch is that it'd fast to prototype & easy to implement. I'm not sure abt this though, since im not familiar with the tech stack being used.

It'd be interesting to hear what you two think about my notes on the zoom vs downscaling, since you both were more on board with the zoom solution than I was. You both have more experience with the codebase than I do though, so you could easily be seeing something that I'm not.

should support types 1 and 2 at minumum

I think 2 is much less important, it's easy to save the image locally.

I agree that it's less important than the first one, but it's still necessary IMO. The big upshot of this feature is that it's a convenience boost that doesn't sacrifice flameshot's speedy editing workflows. Saving files to disk is tedious and slow compared to link and clipboard-based solutions. Forcing users to go that route would undermine some of the benefits we'd get here.

I know that if users care so much, they could just write a shell script to keep things snappy. But, flameshot is pretty widely used, so we can't assume that users will be technical enough to be able to bang out a shell script in an afternoon.

[borgmanJeremy]

Thanks for all the detailed feedback and ideas. Overall I want to integrate the ability to open images from a users hard drive. I really appreciate you taking the time to think through many of the issues (I for one would have not thought of the resolution issue). Let me address some key points and ideas.

The main design challenge here would be how to visually handle non-screenshot-image modification. I do have a few design notes to give a rough idea of what could be done here:

I would like to screenshot the desktop and set it grey like normal. Then the image would open centered in the screen as it would for any normal screenshot. I want to do it this way because it is intuitive, and then it integrates with the rest of the software with minimal change.

It'd be interesting to hear what you two think about my notes on the zoom vs downscaling, since you both were more on board with the zoom solution than I was. You both have more experience with the codebase than I do though, so you could easily be seeing something that I'm not.

I would like to check if the image is larger than the screen, if it is, ask the user if we should downscale or cancel. No special code is required after this, we simply draw the tools the same way we would for a real screenshot.

Waiting on it would also give us space to figure out if we only wanna allow local file imports, or if we actually wanna support clipboard and URL.

I strongly do not want to import images from URL's. Flameshot is at its core a screenshot software, and I think this deviates too far from its goals. It is a good idea, but maintaining that feature takes away my time from other features and it will be tricky to get 100% right. Users can simply save the image to their desktop.

[AlexP11223]

didn't know there was already window infrastructure, but using windows would definitely be better. How does the UI/UX for them work currently?

it's mainly for debugging now, some things don't work correctly, e.g. centering of the initial hint, I just wanted to say that the capture screen is just a fullscreen window, so it should not be too difficult to support window mode

screenshot the desktop and set it grey like normal. Then the image would open centered in the screen as it would for any normal screenshot.

This can be confusing if the area size can be changed to include the desktop.

And sometimes it's nice to be able to switch to other windows when editing.

[MyriaCore]

Waiting on it would also give us space to figure out if we only wanna allow local file imports, or if we actually wanna support clipboard and URL.

I strongly do not want to import images from URL's. Flameshot is at its core a screenshot software, and I think this deviates too far from its goals. It is a good idea, but maintaining that feature takes away my time from other features and it will be tricky to get 100% right. Users can simply save the image to their desktop.

So I did talk about this a bit in one of my earlier posts, but I guess I could try talking about it again. I think that ultimately, flameshot can't meaningfully be thought of as only an screenshot utility, but can reasonably be understood as a screenshot utility with light markup and export functionality.

Adding light import functionality for URLs hardly seems out of place compared to local file imports, or even what's already currently being supported (screenshot editing, export to local disk, copy, upload to imgur, etc). These are features that do more than what a screenshot tool is "supposed to do", but they're unsurprisingly among the most used features of flameshot.

Also, what's different about URL import compared to local file import that makes it uniquely out of scope? We know it can't be the fact that it's a remote resource download, because remote resource upload is supported for export. Why isn't that out of scope as well?

If compromising here is what it takes for the community to reach consensus on this implementation, I'll be glad to leave the conversation about URL import for another issue. I just worry that we'd be throwing away an opportunity to make something cool here.

[mmahmoudian]

I feel like downscaling the image (approach 2) is the better for a number of reasons:

In my use-case, which is adding more annotations to an already annotated screenshot, downscaling would not be useful as then it would be impossible to have the same size arrows and highlights (I always use arrow size 0). Regarding the zoom feature, we already have the zoom functionality in the pinned window which works fine, but also as an alternative we can display the full-size picture and then let the user to move the picture by drag and drop. This is something similar to what all PDF viewers do.

I strongly do not want to import images from URL's.

I personally don't have strong feeling about this, but I can imagine that importing from local file and from clipboard would also solve the URL issue as one can always right click on the image on the website (or open the image in a new tab) and the right click and "copy image" which would copy the image into clipboard. This imho is even faster or at best equal speed compared to getting the URL of an image.

About the UI/UX, I still vote for a separate window as it would give the user (including us 😉) the freedom to use our computer and switch windows if needed.

[MyriaCore]

I strongly do not want to import images from URL's.

I personally don't have strong feeling about this, but I can imagine that importing from local file and from clipboard would also solve the URL issue as one can always right click on the image on the website (or open the image in a new tab) and the right click and "copy image" which would copy the image into clipboard. This imho is even faster or at best equal speed compared to getting the URL of an image.

About the UI/UX, I still vote for a separate window as it would give the user (including us 😉) the freedom to use our computer and switch windows if needed.

Honestly, I kinda see the logic here. I still don't see why we couldn't / shouldn't do both if we're sold on both, but having clipboard would be fantastic - I use the clipboard feature very heavily. I do also think separate windows is the way to go.

In my use-case, which is adding more annotations to an already annotated screenshot, downscaling would not be useful as then it would be impossible to have the same size arrows and highlights (I always use arrow size 0). Regarding the zoom feature, we already have the zoom functionality in the pinned window which works fine, but also as an alternative we can display the full-size picture and then let the user to move the picture by drag and drop. This is something similar to what all PDF viewers do.

Now that you mention it, we do have zoom with pinned images. If the zoom here was implemented similarly - maybe the user could ctrl scroll up to blow the image up, make some edits, and ctrl scroll down to shrink the image back down, I could totally see that.

I do still think that the zooming feature could make it harder to produce more consistent edits, but I didn't imagine it would pair as nice with the window idea. I thought you were proposing that we blow the image up within the boundaries of the window. However, if we're actually scaling the window size itself when the user zooms, I'd be totally on board.

This is a weird thing to be stuck on I know, but I just want to maintain the 'free floating' vibe that flameshot has. Anything that looks too much like a window would kinda ruin the magic for me. Using pin-zoom would also have the added benefit of re-using the user's existing pin-zoom intuition. Operating this zoom feature won't feel weird or different since it'll literally just be a pin zoom.

[gdw2]

This feature would be useful with Chrome OS with Crostini (Linux app support). The native screen capture functionality in Chrome OS will dump the selected region to the filesystem or clipboard but has poor annotating capabilities. I wish I could get the image into something like Flameshot. Due to the nuances of Chrome OS, apps like flameshot can't take Screenshot themselves, but could be used to edit Screenshot from disk or clipboard.

[thiemo-taken]

I would love to see this feature. A screenshot without annotation/markings is sometimes quite pointless, especially after some time viewing at it again. And I for sure am not the one to always get it right at the first time. Greenshot can do this, but it is available only for M$ Windows.

[borgmanJeremy]

@mmahmoudian I've had some recent thoughts on this. Curious what you think:

• Flameshot should have a "windowed" mode

• When taking a screenshot flameshot will still enter its normal full screen mode.

• A new button will be added thats similar to pin, but all it does is hide the greyed out area of the screenshot, all of the tools flow like normal.

• Pinned screenshots shall have the capability to enter "windowed mode" so they can be edited again

• Users can specify that flameshot open an existing image, and it will open this new windowed widget

I think something like this will unify many feature requests and flameshot will keep its core identity.

[paul1149]

Thank you for FS. It is such a great handy editor, I would love to see this.