RegiFamily Shiny or NotShiny

I was build a simple CNN to identify the Legendary Titans：Regirock, Registeel, Regice, Regieleki, Regidrago these 5 pokemons in Switch Game Pokemon Sword and Shield.

Then use this model to encounter their shiny style automatically.

Result_NotShiny：

Result_Shiny：

List

Requirements

Environment

	Version & Specification
OS	Windows10
GPU	GeForce GTX 1050Ti
NVIDIA Driver	456.71 - WHQL
Tensorflow-GPU	2.3.1
CUDA & cuDNN	10.1 & v7.6.5

Necessary for Python Packages
- 【tensorflow-gpu 2.3.1】
  
  This Package is must important to use GPU for high performance numerical computation.
  
  pypi : tensorflow-gpu
  - Install Command : pip install tensorflow-gpu
- 【pyserial 3.4】
  
  This Package can control our arduino equipment.
  
  pypi : pyserial
  - Install Command : pip install pyserial
- 【pytesseract 0.3.6】
  
  This Package can let us to Identify the word.
  
  pypi : pytesseract
  - Install Command : pip install pytesseract
  - After pip pytesseract, go to Github and download/install for your PC. → UB-Mannheim/tesseract
  - Go to installed path to find tesseract.exe. Then add this file's path to your Environment Variable.
- 【opencv-python 3.4.11.39】
  
  This Package can let us to video stream from Switch and image process.
  
  pypi : opencv-python
  - Install Command : pip install opencv-python
- 【numpy 1.18.5】
  
  This Package is for image process.
  
  pypi : numpy
  - Install Command : pip install numpy

Make Training Pictures

【 Video Capture Device 】

I use HDMI→USB video capture device as my Switch's video input.
【 Record videos ： Video_save.py 】

I use Video Capture Device as my Switch's video input, and use cv2.VideoWriter_fourcc to save videos.

Because of I saved the video as MP4, openh264-1.7.0-win64.dll this file was necessary.

openh264 Download here
【 Save Pictures from videos ： Pictures_save.py 】

Set how frequency to get pictures and take frames to numpy array from videos.

According to I discovered the cave's light (background) was different in morning and night, I decided to gather all the morning and night pictures to enhance my identity rate.
- Shiny RegiFamily
  
  Because of I never encounter the shiny RegiFamily, I found the shiny RegiFamily from YouTube as my Training pictures.
- NotShiny RegiFamily
  
  I recorded all the RegiFamily normal color with Video Capture Device morning and night the time in the game.
  
  Then the frame was setted 1 when I was saved training pictures.

Training

I build two models and the Dence layers was the same but convolutional layers was different for training.

First was use Transfer Learning (VGG16), second was slef-build convolutional layers. Use these two models to compare witch one is more simple, fast, high accuracy.

According to the result, the VGG16 is the best model in this project.

Why not try more other famous models？ The reason is use VGG16 to identify this project is enough. In actual fact, its identity rate was vary high even closed to 100% when I was used.

Regirock training loss & accuracy ：

Regice training loss & accuracy ：

Registeel training loss & accuracy ：

Regieleki training loss & accuracy ：

Regidrago training loss & accuracy ：

But Regice and Regieleki are different. Other RegiFamily's shiny type has large of difference from normal type, only Regice and Regieleki there two of types are vary close, even human eyes can not identify without any information.

Thus, I use image process to catch definitely appeared the shiny light when encountered shiny Pokemon. shiny_detect.py

I use cv2.inRange to catch the ShinyLight, and use cv2.dilate to make my target more cleary, then use cv2.findContours to find the contour point and count it.

If the ShinyLight contour point is more than 6 even 8, it must be encounterd the shiny Regice certainly.

m1 = cv2.inRange(m1, (240,240,240),(255,255,255))
m1 = cv2.dilate(m1, np.ones((30,23)))
z, a, b = cv2.findContours(m1, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)

Demo Video