About function "transform" in transforms.py
Closed this issue · 11 comments
Hi,
Thanks for your code. I have one question about some code in the "transform" function.
new_pt = np.array([pt[0] - 1, pt[1] - 1, 1.]).T
new_pt = np.dot(t, new_pt)
return new_pt[:2].astype(int) + 1
According to the above code, you first subtract 1 from the coordinates and then add 1 after the transformation. I don't see the reason of doing this. There are two places calling this "transform" function. The first place is in datasets/mpii.py function,
tpts[i, 0:2] = to_torch(transform(tpts[i, 0:2]+1, c, s, [self.out_res, self.out_res], rot=r))
target[i] = draw_labelmap(target[i], tpts[i]-1, self.sigma, type=self.label_type)
Here you first add 1 and then subtract 1 before and after calling the "transform" function, which just offset what you do inside it. For this case, we could remove the plus 1 and minus 1 for clarity.
Second, function "final_preds" calls function "transform_preds" which then calls "transform" as follows:
coords[p, 0:2] = to_torch(transform(coords[p, 0:2], center, scale, res, 1, 0))
In this case, I also read the original torch code:
https://github.com/anewell/pose-hg-demo/blob/master/util.lua
It seems they don't add 1 and subtract 1 afterwards. I think adding 1 is not equivalent to subtracting 1 after the trasformation. Could please explain your reason?
Thanks,
Hi @bestlin ,
I know that python's index starts from 0 while lua index starts from 1. I don't think this is the reason. Because the index difference should considered in the data preprocessing part, i.e. generating the json files. A more important reason is that minus 1 at the resolution of 64x64 and plus 1 at the resolution of 256x256 is obviously not equivalent not matter in python or lua.
For transform
, please refer to the original lua code.
https://github.com/anewell/pose-hg-train/blob/master/src/util/img.lua#L54-L65
function transform(pt, center, scale, rot, res, invert)
local pt_ = torch.ones(3)
pt_[1],pt_[2] = pt[1]-1,pt[2]-1
local t = getTransform(center, scale, rot, res)
if invert then
t = torch.inverse(t)
end
local new_point = (t*pt_):sub(1,2)
return new_point:int():add(1)
end
The +1
and -1
in the following code is left here for historical reason. To compare with previously trained models, I will leave them unchanged.
tpts[i, 0:2] = to_torch(transform(tpts[i, 0:2]+1, c, s, [self.out_res, self.out_res], rot=r))
target[i] = draw_labelmap(target[i], tpts[i]-1, self.sigma, type=self.label_type)
@bearpaw ,Thanks! I was reading the file in repo pose-hg-demo:
https://github.com/anewell/pose-hg-demo/blob/master/img.lua
function transform(pt, center, scale, rot, res, invert)
-- For managing coordinate transformations between the original image space
-- and the heatmap
local pt_ = torch.ones(3)
pt_[1] = pt[1]
pt_[2] = pt[2]
local t = getTransform(center, scale, rot, res)
if invert then
t = torch.inverse(t)
end
local new_point = (t*pt_):sub(1,2):int()
return new_point
end
Do you know why doesn't this transform function have the +1 and -1? Thanks!
@zhiqiangdon It seems that there are some mismatches between pose-hg-demo
and pose-hg-train
. I think pose-hg-train
is relatively up-to-date.
I do not quite know the reason. But I guess it might due to the int
operation. The trick might improve the computational accuracy (float and int stuff) and the performance a little bit. But it's absolutely not a key point, and I've never verified this either.
@bearpaw Thanks!
I have some questions about transform.py
, can anyone help me out?
def get_transform(center, scale, res, rot=0):
# General image processing matrix
h = 200 * scale # what does 200 means?
t = np.zeros((3, 3)) # this is transform matrix, what is our aim? Any mathematical formula?
t[0, 0] = float(res[1]) / h
t[1, 1] = float(res[0]) / h
t[0, 2] = res[1] * (-float(center[0]) / h + .5)
t[1, 2] = res[0] * (-float(center[1]) / h + .5)
t[2, 2] = 1
Thanks, I just get stuck on them.
Affine transform should be performed on 0-indexed points. So it's totally fine to remove the +1/-1 in transform
and use consistent (0-indexed) coordinates.
@bearpaw There might exist an inconsistency in
pytorch-pose/pose/utils/transforms.py
Lines 132 to 134 in ad2309e
where 0-indexed points are fed to
transform
but it expects 1-indexed points.@sydney0zq This is the matrix for the affine transform. You may learn the relevant knowledge from image processing or computer vision courses.
@xiaoyong Actually I am a little bit confused about this 1 to 0 index from lua to torch (and from the annotation). Would you create a pull request to clarify this if it won't cost much of your time? Thank you very much!