This is a repo that can recognize a continuous frames from video and generate the board state into PDF files in ascending.
python extract_frames.py
python hand_occlusion_CLIP.py
python main.py
read_pic.ipynb
| Idx | Content | State | Note |
|---|---|---|---|
| 1 | Split video into frames | ✔️ | |
| 2 | Recognize frames and generate board, black point and white point | ✔️ | only detect pieces, make a version with predict pieces condition |
| 3 | Select valid frames and remove duplicate, occlusion frames | ||
| 4 | Draw output frame with specific input and output | ✔️ | |
| 5 | Generate PDF file |
- Calculate corner pixel and corresponding R,G,B values
- Achieved perspective homography transformation
- GO board line detection
- TODO: calibration on parameters to obtain a precise detection results
- recognize black and white point
- some defect error
- use CNN model to generate board coordinates and pieces existence
- [TOFIX] our model can only detect pieces existence, add labels to show each position condition probability
- generate board graph with pieces numbers
- extract board images in PDF format
Accomplished
- use conventional CV to remove light reflection effect
- successful detect all pieces with correct outputs
- generate board with pieces numbers and states
Accomplished
- Make a function to handle input array of board state, and draw circles in an standard form
Accomplished
- Consider multiple frames inputs, and update the current board state. (Rewrite function)
- Ensure pieces algo is accurate without hand cases.
Future Works:
- Solve the hand occlusion problem by checking the saved global board to detect whether have any missing pieces
- Consider the eaten pieces case and label
- Output into PDF format
TODO and Discussions:
- Remove hand case by checking pixel RGB values and remove hand frames (Ho)
- Remove redundant frames by adding delay into videocaptured (Zac)
- Output board and string information into PDF (Zac)
Accomplished
- Remove redundant frames by adding delay into videocaptured (Zac)
- Output board and string information into PDF (Zac)
Future Works:
- Solve the hand occlusion problem by checking the saved global board to detect whether have any missing pieces
- Consider the eaten pieces case and label
TODO and Discussions:
- Remove hand case by checking pixel RGB values and remove hand frames
Accomplished
- Consider the eaten pieces case and label
Future Works:
- Solve the hand occlusion problem by checking the saved global board to detect whether have any missing pieces
- My dict need to store peices color and counter number, build a global board to store the pieces coming time.
Accomplished
- Fix global array with eaten pieces case.
Future Works:
- PDF write eaten pieces statement
- Solve the hand occlusion problem by checking the saved global board to detect whether have any missing pieces
Accomplished
- PDF write eaten pieces statement
Future Works:
- Reunion all the components
- Solve the hand occlusion problem by checking the saved global board to detect whether have any missing pieces
- Rearrange code
Accomplished
- Add CLIP to detect hand occlusion
- Select the first frame from each bundle as prediction results
Discussion
- How to decide bundle results: (i) Max of W&B pieces combination bundle (ii) Min of pieces count in bundle (iii) First frame of bundle
Accomplished
- Draw pieces
- Convert read_pic.ipynb into read_pic.py to import global variables into draw_test.py
Available Fonts
Accomplished
- Draw the status of the eaten piece in a PDF file.
Future Works
- Bug: For other video input (GO2.mp4), find some way to solve fixed warp perspective coordinates. (This error may cause perspective bias and make piece detection fail.)