Augmented-Reality-based-Realtime-Sudoku-Solver

An AR program, which reads the Sudoku problem from webcam feed and using computer vision techniques digitize the image through CNN approach , then solves, displays the solution in the same perspective of the input feed , thus implementing Augmented reality

Directions to run the program

• Firstly make sure u have all the required libs installed from the requirements.txt
• Traverse your terminal to this folder location.
• Then , In your terminal type python sudoku_main.py and hit enter

Workflow of prog :

1) Input:

its the input raw_img passes to the system through the cam feed

2) Extract and Thresholding sudoku:

Here the sudoku is extracted , warped and thesholded from the raw_image .this img is then preprocessed and used for grid-extraction

3) grid-extraction and number prediction:

Here ,

• the left image is collection of all extracted number-img-grids collaged together.
• the right image is collection of all predicted numbers of the respective grids by the CNN model
  
  

4) Solving and updating the sudoku:

After solving , the cropped and warped sudoku is updated with solution.

5) Final Output Image:

Here , the solved-cropped sudoku is warped_inversed to the original image in the same perspective

References and Resources:

• Dataset : http://www.ee.surrey.ac.uk/CVSSP/demos/chars74k
• extracting Sudoku Grid: https://maker.pro/raspberry-pi/tutorial/grid-detection-with-opencv-on-raspberry-pi
• Finding all 4 corners from a contour: https://www.programcreek.com/python/example/89417/cv2.arcLength
• article about WarpPerspective : https://www.pyimagesearch.com/2014/05/05/building-pokedex-python-opencv-perspective-warping-step-5-6
• Optimize digit Prediction accuracy : https://medium.com/@o.kroeger/tensorflow-mnist-and-your-own-handwritten-digits-4d1cd32bbab4
• Solving Sudoku : http://norvig.com/sudoku.html