21_summer_internship

21년도 하계 인턴십 with_vision

hrnet 학습 결과

모델학습

model: hrnet_w32_256x192 사용

model1 = test loss : 0.0005 Accuracy : 100

테스트 지표 생성:

boundary_size = (top_y – middle_y)/ z 의 값을 기준으로 얼마나 벗어났는가를 판단.

boundary_size is 10, count of overrange : 0, Accuracy: 100.0%

boundary_size is 15, count of overrange : 0, Accuracy: 100.0%

boundary_size is 20, count of overrange : 1, Accuracy: 99.0%

boundary_size is 30, count of overrange : 14, Accuracy: 95%

성능을 개선하기 위해 기존 전체 사진기준 bbox를 좌표의 min,max값으로 bbox를 줄여 재학습을 진행함

model2 = test loss : 0.0001 Accuracy : 99

모델1 + 모델2를 결합하여 테스트를 진행한 결과:

boundary_size is 10, count of overrange : 0, Accuracy: 100.0%

boundary_size is 15, count of overrange : 0, Accuracy: 100.0%

boundary_size is 20, count of overrange : 0, Accuracy: 100.0%

boundary_size is 30, count of overrange : 0, Accuracy: 100.0%

boundary_size is 40, count of overrange : 4, Accuracy: 98.67549668874172%

모델을 통해 생성된 keypoint를 이용한 왼손 정방향 조정 코드:

        
def rotation(image, angleInDegrees):
    h, w = image.shape[:2]
    img_c = (w / 2, h / 2)
    rot = cv2.getRotationMatrix2D(img_c, angleInDegrees, 1)
    rad = math.radians(angleInDegrees)
    sin = math.sin(rad)
    cos = math.cos(rad)
    b_w = int((h * abs(sin)) + (w * abs(cos)))
    b_h = int((h * abs(cos)) + (w * abs(sin)))
    rot[0, 2] += ((b_w / 2) - img_c[0])
    rot[1, 2] += ((b_h / 2) - img_c[1])
    outImg = cv2.warpAffine(image, rot, (b_w, b_h), flags=cv2.INTER_LINEAR)
    return outImg
def position_checker(image_path,top,middle,thumb):
    img = cv2.imread(image_path)
    try:
        angle = math.atan2(top[0]-middle[0],middle[1]-top[1])
        angle = math.degrees(angle)
        print(angle)
        #시계방향 90도
        if 45<= angle<=135:
            img = cv2.rotate(img, cv2.ROTATE_90_COUNTERCLOCKWISE)
            image_height, image_width = img.shape[0], img.shape[1]
            top[0], top[1] = top[1],image_height -  top[0]
            middle[0], middle[1] = middle[1], image_height - middle[0]
            thumb[0], thumb[1] = thumb[1], image_height - thumb[0]
        #반시계방향 90도
        elif -135<= angle<=-45:
            img = cv2.rotate(img, cv2.ROTATE_90_CLOCKWISE)
            image_height, image_width = img.shape[0], img.shape[1]
            top[0], top[1] = image_width - top[1], top[0]
            middle[0], middle[1] = image_width - middle[1], middle[0]
            thumb[0], thumb[1] = image_width - thumb[1], thumb[0]
        #180도
        elif 135<= angle<=225 or -225<=angle<=-135:
            img = cv2.rotate(img, cv2.ROTATE_180)
            image_height, image_width = img.shape[0], img.shape[1]
            top[0],top[1] = image_width - top[0],image_height-top[1]
            middle[0],middle[1] = image_width - middle[0],image_height - middle[1]
            thumb[0],thumb[1] = image_width - thumb[0],image_height - thumb[1]
        #좌우 대칭
        if middle[0] > thumb[0]:
            img = cv2.flip(img, 1)
            image_height, image_width = img.shape[0], img.shape[1]
            top[0] = image_width - top[0]
            middle[0] = image_width - middle[0]
            thumb[0] = image_width - thumb[0]
        #양손 사진일 경우 왼손만 인식하기 위해 사진크기 조정
        image_height, image_width = img.shape[0], img.shape[1]
        if middle[0] < image_width//2 and top[0]< image_width//2 and thumb[0]< image_width//2:
            flip_thumb_x = image_width - thumb[0]
            cutting_x_position = thumb[0]+flip_thumb_x/5
            img = img[0:image_height, 0: int(cutting_x_position)]
        #좌표에 점 찍기
        img = cv2.circle(img, (int(top[0]),int(top[1])), 6, (0, 0, 255), -1)
        img = cv2.circle(img, (int(middle[0]),int(middle[1])), 6, (0, 0, 255), -1)
        img = cv2.circle(img, (int(thumb[0]),int(thumb[1])), 6, (0, 0, 255), -1)
        angle = math.atan2(top[0] - middle[0], middle[1] - top[1])
        angle = math.degrees(angle)
        if -60<=angle<=60:
            img = rotation(img,angle)