Obtain labels for keypoints

[aakaashjois]

I followed the demo.ipynb notebook to obtain the key points. I am trying to label each key point detected in the image. But, I am not able to figure out how to do that from the data in all_peaks.

The all_peaks list is something like,

[[(417, 162, 0.23725834488868713, 0)], [(438, 264, 0.22009609639644623, 1)], [(360, 260, 0.22483864426612854, 2)], [(310, 435, 0.22168569266796112, 3)], [(302, 596, 0.23836150765419006, 4)], [(511, 270, 0.2024109661579132, 5)], [], [], [(388, 580, 0.18829642236232758, 6)], [(381, 824, 0.2216421663761139, 7)], [], [(503, 577, 0.19702039659023285, 8)], [(528, 828, 0.21746164560317993, 9)], [], [(406, 140, 0.2475488781929016, 10)], [(444, 141, 0.23833531141281128, 11)], [], [(487, 159, 0.22596435248851776, 12)]]

From what I have understood, the last element in each item of the list is the label index for the keypoint. But when a certain feature is not detected, the item is empty. So I am not able to figure out how to label the keypoints.

Can anyone help me out with this?

[michalfaber]

Hi @aakaashjois
all_peaks is an 18 element list where each item is a sublist of body parts. The sublist contains information about detected body parts of a single type. So all_peaks[0] contains all detected noses, all_peaks[1] - all detected necks, etc. For example first detected nose is at all_peaks[0][0] - it is a tuple (444, 141, 0.23833531141281128, 11) where the first 2 items are coordinates, the third element is a score and the last element is an identifier which is a sequence number (it can be any unique number)
This id is used in this part of code:

get the body parts of type A and B

candA = all_peaks[limbSeq[k][0]-1]
candB = all_peaks[limbSeq[k][1]-1]

create a connection [id of body part A, id of body part B, …]

connection = np.vstack([connection, [candA[i][3], candB[j][3], s, i, j]])

Those identifiers are used to find valid connections. Take a look at the code for example

if subset[j][indexA] == partAs[i] or subset[j][indexB] == partBs[i]:
    subset_idx[found] = j
    found += 1

Indeed, when there is no detected body part the sublist is empty

[aakaashjois]

Hey @michalfaber
I was able to find the labels for the key points in the config file. I am able to get a better understanding of the code now. Thank you so much for the clarification!