08/07/21. v 1.2.5

README.md

@@ -25,6 +25,7 @@
 
 ![](doodler-logo.png)
 
+## Website
 Check out the [Doodler website](https://dbuscombe-usgs.github.io/dash_doodler/)
 
 
@@ -89,6 +90,8 @@ This is python software that is designed to be used from within a `conda` enviro
 
 ## <a name="install"></a>Installation
 
+> Check out the installation guide on the [Doodler website](https://dbuscombe-usgs.github.io/dash_doodler/docs/tutorial-basics/deploy-local)
+
 Open a terminal
 
 Clone/download this repository
@@ -122,6 +125,8 @@ pip install -r environment/requirements.txt
 and good luck to you!
 
 ## <a name="use"></a>Use
+> Check out the user guide on the [Doodler website](https://dbuscombe-usgs.github.io/dash_doodler/docs/tutorial-basics/what-to-do)
+
 Move your images into the `assets` folder. For the moment, they must be jpegs with the `.jpg` (or `JPG` or `jpeg`) extension. Support for other image types forthcoming ...
 
 Run the app. An IP address where you can view the app in your browser will be displayed in the terminal. Some browsers will launch automatically, while others you may have to manually type (or copy/paste) the IP address into a browser. Tested so far with Chrome, Firefox, and Edge.
@@ -213,12 +218,12 @@ sudo docker stop www
 sudo docker rm www
 ```
 
-Don't ask me about Docker. That's all I know. Please contribute Docker workflows and suggestions!
+Please don't ask me about Docker - that's all I know. Please contribute Docker workflows and suggestions!
 
 
 ## <a name="ack"></a>Acknowledgements
 
-Based on [this plotly example](https://github.com/plotly/dash-sample-apps/tree/master/apps/dash-image-segmentation) and the previous openCV based implementation [doodle_labeller](https://github.com/dbuscombe-usgs/doodle_labeller), that actually has origins in a USGS CDI-sponsored class I taught in summer of 2018, called [dl-tools](https://github.com/dbuscombe-usgs/dl_tools). So, it's been a 3+ year effort!
+Inspired by [this plotly example](https://github.com/plotly/dash-sample-apps/tree/master/apps/dash-image-segmentation) and the previous openCV based implementation [doodle_labeller](https://github.com/dbuscombe-usgs/doodle_labeller), that actually has origins in a USGS CDI-sponsored class I taught in summer of 2018, called [dl-tools](https://github.com/dbuscombe-usgs/dl_tools). So, it's been a 3+ year effort!
 
 ## <a name="contribute"></a>Contributing
 Contributions are welcome, and they are greatly appreciated! Credit will always be given.
@@ -248,6 +253,8 @@ We could always use more documentation, whether as part of the docs, in docstrin
 
 #### Get Started!
 
+> See the [how to contribute](https://dbuscombe-usgs.github.io/dash_doodler/docs/tutorial-extras/how-to-contribute) section of the Doodler website
+
 Ready to contribute? Here's how to set up for local development.
 
 * Fork the dash_doodler repo on GitHub.
@@ -281,7 +288,29 @@ Submit a pull request through the GitHub website.
 
 ## <a name="developers"></a>Developers notes
 
+### Entrypoint
+* The entrypoint is `doodler.py`, which will first download sample imagery if `DOWNLOAD_SAMPLE=True` in `environment\settings.py`.
+
+* By default, `DOWNLOAD_SAMPLE=False` so imagery is not downloaded.
+
+* The other variables in `environment\settings.py` are found in Dash's `app.run_server()` documentation.
+  * `HOST="127.0.0.1"`` (should be `#"0.0.0.0"` for web deployment)
+  * `PORT="8050"`
+  * `DEBUG=False`
+  * `DEV_TOOLS_PROPS_CHECK=False`
+
+* `doodler.py` basically just calls and serves `app`, from `app.py`
+
+## Application
+
+* Loads classes and files and creates results folders and log file
+* Creates the application layout and links all buttons to callback functions
 
+## Callbacks
+* utility functions are in `app_files\src\app_funcs.py`
+* functions for drawing the imagery on the screen and making label overlays are in `app_files\src\plot_utils.py`
+* functions for converting SVG annotations to raster label annotations and segmentations are in `app_files\src\annotations_to_segmentations.py`
+* image segmentation/ML functions are in `app_files\src\image_segmentation.py`
 
 ## <a name="progress"></a>Progress report
 
@@ -291,7 +320,7 @@ Submit a pull request through the GitHub website.
 
 10/22/20
 * modified layout so image window is larger, and button bank is narrower. Hopefully easier to label, less zooming, etc. see https://github.com/dbuscombe-usgs/dash_doodler/issues/4. Thanks Dan Nowacki
-* added yml installation file, modified requirements.txt to remove gunicorn dependency. see https://github.com/dbuscombe-usgs/dash_doodler/issues/1. Thanks Dan Nowacki, Chris Sherwood, Rich Signell
+* added yml installation file, modified requirements.txt to remove gunicorn dependency. see https://github.com/dbuscombe-usgs/dash_doodler/issues/1.
 * updates to docs, README, videos
 
 10/29/20

website/blog/2021-05-09-blog-post.md

@@ -4,4 +4,67 @@ author: Dan Buscombe
 authorURL: http://twitter.com/magic_walnut
 ---
 
-Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc vehicula nunc sit amet augue vehicula, ut rutrum felis accumsan. Cras a magna porttitor nunc imperdiet ultricies. Donec egestas sed quam vel convallis. Duis magna ipsum, hendrerit molestie risus pretium, sagittis fringilla libero. Maecenas convallis accumsan velit, rhoncus pulvinar ante vulputate non. Vestibulum consectetur interdum purus eu ultricies. Proin finibus, nibh sed consectetur egestas, sapien sem tempus dui, eget cursus ex lacus non massa. Praesent vestibulum justo a risus ultricies, nec feugiat eros ultrices. Ut laoreet velit eu aliquet dignissim. Suspendisse dignissim, tellus quis eleifend rhoncus, ante urna feugiat lorem, nec volutpat sapien odio ac magna. Duis ornare scelerisque metus, at sodales orci pulvinar at. Quisque id nisi a ante congue malesuada. Nulla venenatis nisi sed neque facilisis, ac condimentum quam egestas. Nam eros ex, sagittis ac dignissim nec, lobortis vitae nibh. Suspendisse risus felis, luctus vitae mauris id, placerat consectetur nulla. Proin mauris dui, euismod sed lectus scelerisque, malesuada sagittis mauris.
+Doodler works well with small to medium sized imagery where the features and objects can be labeled without much or any zoom or pan. This depends a lot on the image resolution and content so it is difficult to make general guidelines.
+
+But it's easy enough to chop images into pieces, so you should experiment with a few different image sizes.
+
+Let's start with this image called `big.jpg`:
+
+![](../static/img/cropping/big.jpg)
+
+I recommend the command-line program [imagemagick](https://imagemagick.org/index.php), available for all major platforms. It's an incredibly powerful and useful set of tools for manipulating images. You can use the imagemagick command line [tools](https://imagemagick.org/script/command-line-processing.php) for splitting and merging imagery. We use the `magick` command (`convert` on some Linux distributions)
+
+## Halves
+Split into two lengthways:
+
+```cmd
+magick big.jpg -crop 50%x100% +repage fordoodler_%02d.jpg
+```
+
+![](../static/img/cropping/fordoodler_00.jpg)
+![](../static/img/cropping/fordoodler_01.jpg)
+
+
+## Quarters
+Following the same logic, to chop the image into quarters, use:
+
+```cmd
+magick big.jpg -crop 50%x50% +repage quarters_fordoodler_%02d.jpg
+```
+
+The first two quarters are shown below:
+
+![](../static/img/cropping/quarters_fordoodler_00.jpg)
+![](../static/img/cropping/quarters_fordoodler_01.jpg)
+
+
+## Specific size
+To chop the image into tiles of a specific size, for example 1024x1024 pixels, use:
+
+```cmd
+magick big.jpg -crop 1024x1024 +repage px1024_fordoodler_%02d.jpg
+```
+
+The first three tiles are shown below:
+
+![](../static/img/cropping/px1024_fordoodler_00.jpg)
+![](../static/img/cropping/px1024_fordoodler_01.jpg)
+![](../static/img/cropping/px1024_fordoodler_02.jpg)
+
+
+Easy peasy!
+
+## Recombine
+
+After you've labeled, you may want to recombine your label image. Imagemagick includes the [montage](https://imagemagick.org/script/montage.php) tool that is handy for the task. For example, the image quarters can be recombined like this:
+
+```cmd
+magick montage -mode concatenate -tile 2x2 quarters*.jpg recombined.jpg
+```
+and the equivalent command to combine the two vertical halves is:
+
+```cmd
+magick montage -mode concatenate -tile 2x1 fordoodler*.jpg recombined.jpg
+```
+
+Happy image cropping!

website/blog/2021-05-15-blog-post.md

@@ -1,5 +1,5 @@
 ---
-title: viz_npz.py
+title: How to use viz_npz.py
 author: Dan Buscombe
 authorURL: http://twitter.com/magic_walnut
 ---

website/blog/2021-05-16-blog-post.md

@@ -1,5 +1,5 @@
 ---
-title: What's new in version 1.2.1 (May 10, 2021)
+title: What's new in version 1.2.1? (May 10, 2021)
 author: Dan Buscombe
 authorURL: http://twitter.com/magic_walnut
 ---

website/docs/intro.md

@@ -5,7 +5,7 @@ sidebar_position: 1
 # What is Doodler for?
 
 
-Doodler is a web application for [image segmentation](tutorial-basics/glossary#image-segmentation), which is the process of turning this (for example):
+Doodler is a web application for image segmentation, which is the process of turning this (for example):
 
 ![](/img/ex_image.jpg)
 
@@ -37,7 +37,7 @@ For natural and other scenes, doodler can be a relatively quick (in terms of the
 
 Here's a movie of Doodler in action:
 
-![](https://raw.githubusercontent.com/dbuscombe-usgs/dash_doodler/main/assets/logos/quick-satshoreline-x2c.gif)
+![](https://github.com/dbuscombe-usgs/dash_doodler/releases/download/gifs/quick-satshoreline-x2c.gif)
 
 There are two tabs, one for 'doodling' on images, and one for selecting new images from a 'to-do' list. Doodler will remove image names from the list if a label image has previously been generated. The basic workflow is:
 
@@ -47,8 +47,9 @@ There are two tabs, one for 'doodling' on images, and one for selecting new imag
 4. [if still not happy with the result] Modify the parameters and repeat step 2. above
 
 Here are more examples of Doodler in action
+![](https://github.com/dbuscombe-usgs/dash_doodler/releases/download/gifs/doodler-demo-2-9-21-short-elwha.gif)
 
-![](https://raw.githubusercontent.com/dbuscombe-usgs/dash_doodler/main/assets/logos/quick-satshore2-x2c.gif)
+![](https://github.com/dbuscombe-usgs/dash_doodler/releases/download/gifs/doodler-demo-2-9-21-short-coast2.gif)
 
 You upload your own imagery, and define your own class set (list of class names, each of which will be assigned a different color button like in the examples above), and off you go!
 
@@ -66,7 +67,7 @@ Before we begin, let's state some important factors around Doodler and its use
 ### Credits
 Doodler is written and maintained by Dr Daniel Buscombe, Marda Science, LLC, contracted to the U.S. Geological Survey Pacific Coastal and Marine Science Center in Santa Cruz, CA. Doodler development is funded by the U.S. Geological Survey Coastal Hazards Program, and is for the primary usage of U.S. Geological Survey scientists, researchers and affiliated colleagues working on the Hurricane Florence Supplemental Project and other coastal hazards research.
 
-Thanks to Jon Warrick, Phil Wernette, Chris Sherwood, Jenna Brown, Andy Ritchie, Jin-Si Over, Christine Kranenburg, and the rest of the Florence Supplemental team; to Evan Goldstein and colleagues at University of North Carolina Greensboro; Leslie Hsu at the USGS Community for Data Integration; and LCDR Brodie Wells, formerly of Naval Postgraduate School, Monterey. Doodler was initially based on [this](https://github.com/plotly/dash-sample-apps/tree/master/apps/dash-image-segmentation) plotly example and also the previous openCV based implementation called [doodle_labeller](https://github.com/dbuscombe-usgs/doodle_labeller), also written by Dr Daniel Buscombe.
+Thanks to Jon Warrick, Phil Wernette, Chris Sherwood, Jenna Brown, Andy Ritchie, Jin-Si Over, Christine Kranenburg, and the rest of the Florence Supplemental team; to Evan Goldstein and colleagues at University of North Carolina Greensboro; Leslie Hsu at the USGS Community for Data Integration; and LCDR Brodie Wells, formerly of Naval Postgraduate School, Monterey. Doodler was inspired by [this](https://github.com/plotly/dash-sample-apps/tree/master/apps/dash-image-segmentation) plotly example and also the previous openCV based implementation called [doodle_labeller](https://github.com/dbuscombe-usgs/doodle_labeller), also written by Dr Daniel Buscombe.
 
 ### Sounds good! How do I install?
 

website/docs/tutorial-basics/deploy-local.md

@@ -90,7 +90,7 @@ It's important you know what the various files do. It's also important to stress
 ![](/img/install/install4.PNG)
 
 #### Assets folder
-This is the `assets` folder where you should put the images you want to label. The program comes with a set of default files that look like this:
+This is the `assets` folder where you should put the images you want to label. The program comes with a set of default files that can optionally download by running the `download_data.py` script that look like this:
 
 ![](/img/install/install5.PNG)
 
@@ -106,10 +106,6 @@ This is the 'labeled' folder, where the program copies the images you have label
 
 ![](/img/install/install9.PNG)
 
-#### Sample folder
-The program also comes with a `sample` directory to test the default classifier (water/land). However, your sample images don't need to be located here; they can be anywhere you have read and write access to on your computer
-
-![](/img/install/install7.PNG)
 
 #### Utils folder
 Finally, these python scripts are run from the command line, and are a handy collection of things once you have some data, so we'll talk about them later