Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revamp Bite Selection Visualization #108

Merged
merged 5 commits into from
Dec 23, 2023
Merged

Revamp Bite Selection Visualization #108

merged 5 commits into from
Dec 23, 2023

Conversation

amalnanavati
Copy link
Contributor

@amalnanavati amalnanavati commented Dec 21, 2023

Describe this pull request. Link to relevant GitHub issues, if any.

In our pilot study, we found the following issues with bite selection:

  1. Sometimes SegmentFromPoint segmented two food items together, and it was hard for the user to realize that the fork would skewer the middle.
  2. Sometimes SegmentFromPoint would mask the entire plate as one of the masks. Because we ensured all masks we equal size, this resulted in the other masks being very small and hard to see.

This PR addresses both of these as follows:

  1. Renders a small red dot in the middle of the image to display the center. Note that this is the center of the roi, not the center of mass of the mask. Although these are close, they aren't quite the same, so even this rendered point will be slightly off.
  2. Allows each mask to be scaled differently to fill up the button.

Screenshot 2023-12-21 at 10 40 38 AM
The above image shows what Bite Selection looked like before this PR.

Screenshot 2023-12-21 at 10 35 48 AM
The above image shows what Bite Selection looks like now.

Screenshot 2023-12-21 at 10 52 54 AM
The above image illustrates the benefit of this approach.

Explain how this pull request was tested, including but not limited to the below checkmarks.

  • Setup:
    • ros2 launch feeding_web_app_ros2_test feeding_web_app_dummy_nodes_launch.xml run_food_detection:=false
    • ros2 launch ada_feeding_perception ada_feeding_perception.launch.py
    • cd src/feeding_web_interface/feedingwebapp/; npm start
  • Testing:
    • Go to bite selection.
    • Click a point on the image.
    • Verify that the masks scale to fill the button, and that a red point is rendered in the center of each mask.
    • Repeat for multiple food items.

Before creating a pull request

  • Format React code with npm run format
  • [N/A] Format Python code by running python3 -m black . in the top-level of this repository
  • Thoroughly test your code's functionality, including unintended uses.
  • Fully test the responsiveness of the feature as documented in the Responsiveness Testing Guidelines. If you deviate from those guidelines, document above why you deviated and what you did instead.
  • [N/A] Consider the user flow between states that this feature introduces, consider different situations that might occur for the user, and ensure that there is no way for the user to get stuck in a loop.

Before merging a pull request

  • Squash all your commits into one (or Squash and Merge)

@amalnanavati amalnanavati changed the base branch from main to amaln/web_video_server December 21, 2023 18:51
Copy link
Contributor

@taylorkf taylorkf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is ready to merge. However, it may be helpful to add something on the screen to indicate what the red dot means to users (if it won't clutter up the screen too much). @amalnanavati , if you agree, can you add an issue?

Base automatically changed from amaln/web_video_server to main December 23, 2023 01:37
@amalnanavati amalnanavati merged commit 43b00f4 into main Dec 23, 2023
4 checks passed
@amalnanavati amalnanavati deleted the amaln/mask_button branch December 23, 2023 01:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants