Skip to content

Commit

Permalink
Merge pull request #8 from asimmon/feature/docs
Browse files Browse the repository at this point in the history
Added a multi-image example
  • Loading branch information
asimmon authored Jul 3, 2022
2 parents 5c405da + 7825454 commit 465c337
Show file tree
Hide file tree
Showing 6 changed files with 39 additions and 38 deletions.
61 changes: 31 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,37 +61,9 @@ using (var driver = MarionetteDriver.Create(/* optional DriverOptions */))
}
```

The [sample project](https://github.com/asimmon/askaiser-marionette/tree/master/samples/Askaiser.Marionette.ConsoleApp) will show you the basics of using this library.
The [sample project](https://github.com/asimmon/askaiser-marionette/tree/master/samples/Askaiser.Marionette.ConsoleApp) shows the basics of using this library.


## Creating image and text elements manually

#### Image search

```csharp
// Instead of relying on the source generator that works with image files, you can create an ImageElement manually
var bytes = await File.ReadAllBytesAsync("path/to/your/image.png");
var image = new ImageElement(name: "sidebar-close-button", content: bytes, threshold: 0.95m, grayscale: false);
```

* `ImageElement.Threshold` is a floating number between 0 and 1. It defines the accuracy of the image search process. `0.95` is the default value.
* `ImageElement.Grayscale` defines whether or not the engine will apply grayscaling preprocessing. Image search is faster with grayscaling.

**Image recognition works best with PNG images.**

#### Text search

```csharp
Although many methods accept a simple string as an element, you can manually create a TextElement
var text = new TextElement("Save changes", options: TextOptions.BlackAndWhite | TextOptions.Negative);
```

**Text options** are flags that define the preprocessing behavior of your monitor's screenshots before executing the OCR.
* `TextOptions.None` : do not use preprocessing,
* `TextOptions.Grayscale` : Use grayscaling,
* `TextOptions.BlackAndWhite` : Use grayscaling and binarization (this is the default value),
* `TextOptions.Negative` : Use negative preprocessing, very helpful with white text on dark background.

## Change the source generator behavior

Given the following partial image library class:
Expand Down Expand Up @@ -127,7 +99,36 @@ You can mix these modifiers. Here we will create an single array property `libra
* `header_gs_0.9_1.png` (second item of the array, grayscaled with a 0.9 threshold),
* `header_2.png` (second and last item of the array, keep original colors with and use default threshold).

## Show me the `MarionetteDriver` APIs
## Creating image and text elements manually

#### Image search

```csharp
// Instead of relying on the source generator that works with image files, you can create an ImageElement manually
var bytes = await File.ReadAllBytesAsync("path/to/your/image.png");
var image = new ImageElement(name: "sidebar-close-button", content: bytes, threshold: 0.95m, grayscale: false);
```

* `ImageElement.Threshold` is a floating number between 0 and 1. It defines the accuracy of the image search process. `0.95` is the default value.
* `ImageElement.Grayscale` defines whether or not the engine will apply grayscaling preprocessing. Image search is faster with grayscaling.

**Image recognition works best with PNG images.**

#### Text search

```csharp
Although many methods accept a simple string as an element, you can manually create a TextElement
var text = new TextElement("Save changes", options: TextOptions.BlackAndWhite | TextOptions.Negative);
```

**Text options** are flags that define the preprocessing behavior of your monitor's screenshots before executing the OCR.
* `TextOptions.None` : do not use preprocessing,
* `TextOptions.Grayscale` : Use grayscaling,
* `TextOptions.BlackAndWhite` : Use grayscaling and binarization (this is the default value),
* `TextOptions.Negative` : Use negative preprocessing, very helpful with white text on dark background.


## MarionetteDriver methods

Many parameters are optional. Most methods that look for an element (image or text) expect to find **only one occurrence** of this element. `ElementNotFoundException` and `MultipleElementFoundException` can be thrown.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@
</PropertyGroup>

<ItemGroup>
<ProjectReference Include="..\..\src\Askaiser.Marionette\Askaiser.Marionette.csproj"/>
<ProjectReference Include="..\..\src\Askaiser.Marionette.SourceGenerator\Askaiser.Marionette.SourceGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false"/>
<ProjectReference Include="..\..\src\Askaiser.Marionette\Askaiser.Marionette.csproj" />
<ProjectReference Include="..\..\src\Askaiser.Marionette.SourceGenerator\Askaiser.Marionette.SourceGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
</ItemGroup>

<ItemGroup>
<None Include="images\**" CopyToOutputDirectory="PreserveNewest"/>
<None Include="images\**" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
</Project>
9 changes: 5 additions & 4 deletions samples/Askaiser.Marionette.ConsoleApp/Program.cs
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.IO;
using System.Linq;
using System.Threading.Tasks;

namespace Askaiser.Marionette.ConsoleApp;

[ImageLibrary("images")]
[SuppressMessage("StyleCop.CSharp.MaintainabilityRules", "SA1402:FileMayOnlyContainASingleType", Justification = "It shows how to build an image library in the main Program class.")]
public partial class MyLibrary
{
}
Expand All @@ -29,8 +28,10 @@ private static async Task AutomaticallyGeneratedLibrary()
var monitor = await driver.GetCurrentMonitorAsync();
var ideLogoRect = monitor.FromTopLeft(200, 200);

// RiderLogo and VsLogo properties will be generated from the *.png files
await driver.MoveToAnyAsync(new[] { library.RiderLogo, library.VsLogo }, waitFor: TimeSpan.FromSeconds(2), searchRect: ideLogoRect);
// RiderLogo and VsLogo properties will be generated from the *.png files in the images directory specified in the MyLibrary class definition
// VsLogo is an array because there are multiple images suffixed with "_n" (vs-logo_0.png, vs-logo_1.png)
var ideLogos = new[] { library.RiderLogo }.Concat(library.VsLogo);
await driver.MoveToAnyAsync(ideLogos, waitFor: TimeSpan.FromSeconds(2), searchRect: ideLogoRect);

// Also, in the same area, we expect to find the toolbar item "Edit". Negative preprocessing should be used if the IDE use a dark theme.
await driver.MoveToAsync("Edit", searchRect: ideLogoRect, textOptions: TextOptions.BlackAndWhite | TextOptions.Negative);
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 0 additions & 1 deletion src/Askaiser.Marionette/Rectangle.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

namespace Askaiser.Marionette;

[SuppressMessage("StyleCop.Analyzers.ReadabilityRules", "SA1101:PrefixLocalCallsWithThis", Justification = "StyleCop.Analyzers is not aware of records 'with' syntax.")]
[SuppressMessage("Usage", "CA2225:Operator overloads have named alternates", Justification = "I don't want to")]
public record Rectangle
{
Expand Down

0 comments on commit 465c337

Please sign in to comment.