Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCR to clipboard hook for selections #702

Open
kaihendry opened this issue Mar 19, 2020 · 25 comments · May be fixed by #3074
Open

OCR to clipboard hook for selections #702

kaihendry opened this issue Mar 19, 2020 · 25 comments · May be fixed by #3074
Labels
discussion Enhancement Feature requests and code enhancements Plugin Issues related to plugins and API system

Comments

@kaihendry
Copy link

Sometimes I'm working with screenshots and would be awesome to make a selection around some text and send it off to say https://aws.amazon.com/rekognition/ for it analyse and return the text to set my clipboard.

@MyriaCore
Copy link

OCR would be great honestly!

@Martin-Eckleben Martin-Eckleben added discussion Enhancement Feature requests and code enhancements labels Sep 4, 2020
@nsa
Copy link

nsa commented Sep 12, 2020

There is a powerful OCR engine from Google called https://github.com/tesseract-ocr/tesseract and it's open-sourced. It would be super cool if the maintainers could integrate tesseract to flameshot as the author suggested, by doing so there would no need for external API connections such as Amazon's, because tesseract works offline.

@JScriber
Copy link

JScriber commented Oct 8, 2020

There's a project called dpscreenocr that does the job, yet the selection is not as convenient and user-friendly as flameshot's.
OCR integrated to flameshot with a dpscreenocr inspired configuration would be really great.

@enoy19
Copy link

enoy19 commented Jan 15, 2021

I implemented that feature and created a pull request: #1239

@xeaon
Copy link

xeaon commented Mar 31, 2021

@enoy19 after fixing you typo/c&p error in 8432d12, this compiles in Manjaro and the OCR works really great, thanks!

edit: you have to change the header to baseapi.h in src/tools/ocr/ocrtool.cpp#L22 as suggested in #1239 (comment)

@kaihendry
Copy link
Author

https://news.ycombinator.com/item?id=27242392 discusses the topic in a MacOS context

@Pheggas
Copy link

Pheggas commented Nov 10, 2021

Anything new around this feature request? It would be super cool to me tho. Oftentimes i'm working with text copying and this
feature would be absolute killer!

@xeaon
Copy link

xeaon commented Nov 10, 2021

@Pheggas If you are comfortable with compiling yourself, check my comment above.

I'm using this feature since then and it works great.

@borgmanJeremy
Copy link
Contributor

I am okay to merge this feature if we can:

  • Fix the merge request to it passes CI
  • Explain how to package / distribute the language packs on all platforms.

@Pheggas
Copy link

Pheggas commented Nov 10, 2021

@Pheggas If you are comfortable with compiling yourself, check my comment above.

I'm using this feature since then and it works great.

I'd like to. But it would be my first compilation. I'll try to look at it. In case of failure, I'll write you 🙂

@leoneivaw
Copy link

I am anxious waiting for this feature!
Thanks!

@irfan798
Copy link

irfan798 commented Mar 9, 2022

For anybody waiting for the functionality I found this blog post:
https://slint.github.io/blog/ocr-screenshot.html

If you install tesseract, it is basically a one liner

flameshot gui --raw | tesseract stdin stdout | xclip -in -selection clipboard

@mmahmoudian mmahmoudian added the Plugin Issues related to plugins and API system label Mar 15, 2022
@ek1ng
Copy link

ek1ng commented May 22, 2022

For anybody waiting for the functionality I found this blog post: https://slint.github.io/blog/ocr-screenshot.html

If you install tesseract, it is basically a one liner

flameshot gui --raw | tesseract stdin stdout | xclip -in -selection clipboard

It's an amazing solution! Thanks!

@siva-sub
Copy link

For anybody waiting for the functionality I found this blog post: https://slint.github.io/blog/ocr-screenshot.html

If you install tesseract, it is basically a one liner

flameshot gui --raw | tesseract stdin stdout | xclip -in -selection clipboard

For those using wayland use this command instead:

flameshot gui --raw | tesseract stdin stdout | wl-copy

it requires the installation of wl-clipboard, together with tesseract and the respective tesseract language data package.

For me in arch linux the packages are :
tesseract
tesseract-data-eng
wl-clipboard

in arch linux it can be installed with:
sudo pacman -S tesseract tesseract-data-eng wl-clipboard

@xeaon
Copy link

xeaon commented Jul 20, 2022

Last year, I've spent a view hours to adapt @enoy19's pull request #1239 to the newer codebase but rm -rf'ed the directory by accident.

Yesterday I've done it again. You can find my fork here: xeaon/flameshot. After taking a screenshot, you can choose the OCR tool to copy the tesseract output to your clipboard. If successful there is a notification.

As of now, you need to install the dependency tesseract-ocr and leptonica manually. It won't build in CI but I'm trying to fix it for Linux builds.

As mentioned before, I'm no C++/Qt dev, so this is pretty challenging. Feel free to contribute to my fork, if you have any skills. This feature can be merged, if we can

  • pass CI
  • Explain how to package / distribute the language packs on all platforms

as @borgmanJeremy mentioned earlier.

@pitfiend

This comment was marked as off-topic.

@mmahmoudian
Copy link
Member

@pitfiend Thanks for your comment, but I am going to hide your comment, and here is few points to clarify my reasoning:

  • The software you suggested it platform specific (Windows)
  • If we implement a feature, our emphasis would be to implement it in a cross-platform way. There are some features that are Linux and macOS specific (like support of CLI), but Windows will never be our main focus, unless the maintainers of the project collectively agree otherwise. For now, the best-case scenario for the C# software you suggested is via a Windows-specific plugin.
  • The OCR can easily be implemented with 3 lines of shell script (detailed explanation here: https://mehrad.ai/posts/20210702-extracting-payment-info-in-rasterized-invoice/#ocr ).

rsrdesarrollo pushed a commit to rsrdesarrollo/flameshot that referenced this issue Jan 27, 2023
@rsrdesarrollo rsrdesarrollo linked a pull request Jan 27, 2023 that will close this issue
@Nasreddine
Copy link

Nasreddine commented Dec 15, 2023

This is my version in Ubuntu using custom shortcuts.

I've created a shell script named shot.sh and placed it in /home/me/apps/. [change me to your username]
The script contains the following line:

flameshot gui --raw | tesseract stdin stdout | tr -d '\n' | xclip -in -selection clipboard

The addition of tr -d '\n' is specifically to remove line breaks from the output.

To utilize this script in Ubuntu, you can set up a custom keyboard shortcut as follows:

  1. Open Settings.
  2. Navigate to Keyboard > Keyboard Shortcuts.
  3. Click on View and Customize Shortcuts.
  4. Under Custom Shortcuts, click the + sign to add a new shortcut.
  5. In the new shortcut setup, enter the following details:
    • Name: flameshot_tesseract
    • Command: sh /home/me/apps/shot.sh
    • Shortcut: [Choose your preferred key combination]

This will enable you to use the script conveniently with a keyboard shortcut of your choice.

Linux makes your digital life easier : )

@Sheldonimo
Copy link

Sheldonimo commented Jan 6, 2024

Hi guys, this is my custom version to linux mint.

I have been exploring methods to enhance the OCR output, and my first step was to define the language settings, which can also involve combinations. Through varying the --psm parameters in Tesseract, I discovered that the value of 6, corresponding to "Assume a single uniform block of text," yields the most favorable results. Below is the complete command for your experimentation:

flameshot gui --raw | tesseract stdin stdout -l eng+spa --psm 6 | xclip -in -selection clipboard

It's noteworthy that the optimal performance is achieved using Tesseract version 5. There are various ways to download or even compile it. I'm attaching the simplest method, which is through the PPA:

sudo add-apt-repository -y ppa:alex-p/tesseract-ocr-devel

This PPA was sourced from the following link, which originates from the official Tesseract page:

@erfanium
Copy link

+1 tesseract ocr should be a built-in functionality

@GrabbenD
Copy link

GrabbenD commented Apr 21, 2024

+1 tesseract ocr should be a built-in functionality

Here's a really good PR by @rsrdesarrollo 🎉
#3074

It checks for Tesseract in PATH meaning the feature is opt-in and uses upstream updates rather than a statically linked library (which would inevitably become outdated)


Here's how I'm using it in Sway (Wayland) with @DO-Ui's Tesseract workaround for small letters:

bindsym --no-repeat $MOD+PRINT exec flameshot gui --raw | convert -resize 400% png:- png:- | tesseract -l eng stdin stdout | wl-copy

@X88R88
Copy link

X88R88 commented May 8, 2024

How do i run flameshot with tesseract on windows?

@f-x1-1
Copy link

f-x1-1 commented May 9, 2024

We need built-in OCR. Coming from someone familiar with ShareX and CleanShotX, I can see FlameShot has the potential to get there. Features like OCR, GIF recording, and video capture would make FlameShot a better tool. As someone new to Linux, I haven't found any screen capture tool that even comes close to ShareX and CleanShotX.

@mmahmoudian
Copy link
Member

@f-x1-1

Features like OCR, GIF recording, and video capture would make FlameShot a better tool

As it have been discussed, the dev team unanimously agreed that none of these is in the scope of a screenshot tool. We will only and only allow these via the plugin system.

@X88R88

How do i run flameshot with tesseract on windows?

I have provided some explanation here:
#702 (comment)

@f-x1-1
Copy link

f-x1-1 commented May 9, 2024

@f-x1-1

Features like OCR, GIF recording, and video capture would make FlameShot a better tool

As it have been discussed, the dev team unanimously agreed that none of these is in the scope of a screenshot tool. We will only and only allow these via the plugin system.

@X88R88

How do i run flameshot with tesseract on windows?

I have provided some explanation here:
#702 (comment)

This sounds good. Similar to what Flow Launcher has come up with. Instead of implementing features natively, they have implemented a plugin system. That way, the app itself isn't bloated with unnecessary features. You choose what you want to install. (It's a universal search engine)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion Enhancement Feature requests and code enhancements Plugin Issues related to plugins and API system
Projects
None yet
Development

Successfully merging a pull request may close this issue.