measuring 3d coorfdination for each part of face.
galihjozu opened this issue · 5 comments
regarding to this code
P3D_RIGHT_SIDE = numpy.float32([-100.0, -77.5, -5.0]) #0
P3D_GONION_RIGHT = numpy.float32([-110.0, -77.5, -85.0]) #4
P3D_MENTON = numpy.float32([0.0, 0.0, -122.7]) #8
P3D_GONION_LEFT = numpy.float32([-110.0, 77.5, -85.0]) #12
P3D_LEFT_SIDE = numpy.float32([-100.0, 77.5, -5.0]) #16
P3D_FRONTAL_BREADTH_RIGHT = numpy.float32([-20.0, -56.1, 10.0]) #17
P3D_FRONTAL_BREADTH_LEFT = numpy.float32([-20.0, 56.1, 10.0]) #26
P3D_SELLION = numpy.float32([0.0, 0.0, 0.0]) #27
P3D_NOSE = numpy.float32([21.1, 0.0, -48.0]) #30
P3D_SUB_NOSE = numpy.float32([5.0, 0.0, -52.0]) #33
P3D_RIGHT_EYE = numpy.float32([-20.0, -65.5,-5.0]) #36
P3D_RIGHT_TEAR = numpy.float32([-10.0, -40.5,-5.0]) #39
P3D_LEFT_TEAR = numpy.float32([-10.0, 40.5,-5.0]) #42
P3D_LEFT_EYE = numpy.float32([-20.0, 65.5,-5.0]) #45
#P3D_LIP_RIGHT = numpy.float32([-20.0, 65.5,-5.0]) #48
#P3D_LIP_LEFT = numpy.float32([-20.0, 65.5,-5.0]) #54
P3D_STOMION = numpy.float32([10.0, 0.0, -75.0]) #62
#The points to track
#These points are the ones used by PnP
to estimate the 3D pose of the face
TRACKED_POINTS = (0, 4, 8, 12, 16, 17, 26, 27, 30, 33, 36, 39, 42, 45, 62)
ALL_POINTS = list(range(0,68)) #Used for debug only
that code explain that, there are 68 points of the face. every point is represent by 3d coord.
the thing that i confused is how can i determine that ? (let say how can i measure that stomion is 10, 0, -75
i need to measure the gap between eyelid to determine whether the aye is closed or not by measuring both axis, and i have no idea to find 3d coord, could you please help me?
Hi @galihjozu
The coordinates are based on a simple convention. As reference point (0,0,0) there is the sellion (the bridge of the nose):
The sellion is the deepest point of the nasofrontal angle at the intersection of forehead slope and the proximal nasal bridge.
All the other points are estimated based on the distance from the sellion. The three coordinates represent the distance in millimeters from that point and are expressed as X-Y-Z coordinates, with X pointing forward, Y on the left and Z up. All these measures are average distances based on an average human head. The results can be improved if the distances are calibrated on a specific user's head.
Be careful, if you want to find out if the eyes are closed this method is not the proper one, because the point used here must be stable points, whereas the eyelid distance can change. The best thing you can do is to dynamically find the point on the eylids and estimate their distance instead of using an external point as reference. If you can calibrate the system on a single user then everything is easier and an external reference point may be used.
ok then ,, one more question, i red some articles and they only used 5 or 7 point of 3D coord, i tried to modified your code and used only 8 3D coord, around nose, around forehead, and around lips.
but the result kinda weird, the 3d lines suddenly move to center of lip, while i dont change this code
sellion_xy = (landmarks_2D[7][0], landmarks_2D[7][1])
cv2.line(frame, sellion_xy, tuple(imgpts[1].ravel()), (0,255,0), 3) #GREEN
cv2.line(frame, sellion_xy, tuple(imgpts[2].ravel()), (255,0,0), 3) #BLUE
cv2.line(frame, sellion_xy, tuple(imgpts[0].ravel()), (0,0,255), 3) #RED
do you have any idea what is going on?
Yes you can use just a few 3D points but you have to choose points located on planes that are as much as possible non-parallel in order to facilitate the PNP algorithm. For instance, you can use points on the tip of the nose, eyes and close to the ears.
i am trying to use this only,
P3D_MENTON = numpy.float32([0.0, 0.0, -122.7]) #8
P3D_NOSE = numpy.float32([21.1, 0.0, -48.0]) #30
P3D_RIGHT_EYE = numpy.float32([-20.0, -65.5,-5.0]) #36
P3D_LEFT_EYE = numpy.float32([-20.0, 65.5,-5.0]) #45
P3D_LIP_RIGHT = numpy.float32([-20.0, 65.5,-5.0]) #48
P3D_LIP_LEFT = numpy.float32([-20.0, 65.5,-5.0]) #54
and i dont change this
sellion_xy = (landmarks_2D[7][0], landmarks_2D[7][1])
cv2.line(frame, sellion_xy, tuple(imgpts[1].ravel()), (0,255,0), 3) #GREEN ke pinggir
cv2.line(frame, sellion_xy, tuple(imgpts[2].ravel()), (255,0,0), 3) #BLUE yg ke atas
cv2.line(frame, sellion_xy, tuple(imgpts[0].ravel()), (0,0,255), 3) #RED yang lurus
either this..
cv2.putText(frame, "x core =" + str(landmarks_2D[7][0]), ( 200, 20 ) ,font,0.5,(167,119,35),2)
cv2.putText(frame, "y core =" + str(landmarks_2D[7][1]), ( 200, 40 ) ,font,0.5,(167,119,35),2)
but i have an error
sellion_xy = (landmarks_2D[7][0], landmarks_2D[7][1])
IndexError: index 7 is out of bounds for axis 0 with size 6
my questions are
- what is it for ((landmarks_2D[7][0], landmarks_2D[7][1]))
- can i change the coordinate? which coordinate do i supposed to use?
i change it into [0][0], [0][1],
and i get this error
OverflowError: Python int too large to convert to C long
may i know why?
help me please.. i am trying to understand every line of your code
Hi @galihjozu
You have to be more specific in your report. Which code in the Deepgaze library are you using? Which kind of modifications to the original code did you implement?
The error that you get is connected with the shape of the object called landmarks_2D[7][0]`. In particular this object does not have an element in the position that you are trying to access.