[PR] Saving Inferred 3D Hand Keypoints

[Kaszanas]

As the name suggests.

There were some unused imports in the demo.pl.

As described in the issues below this should calculate the 3D keypoints and save them with np.save.

Related:

• How to output each coordinate information as .pkl #86
• Question: Obtaining Bones and Joints #88

If you can verify and see if this is the solution for the issues above @geopavlakos.

[dhingratul]

The concatenation doesn't work fyi.

[Kaszanas]

The concatenation doesn't work fyi.

I am just now testing the solution. Do you know how to make it work?

[Kaszanas]

I think this works now @dhingratul I am not aware of the precise mechanism of how these values should be mutated mathematically. but given the description

You can get the 3D coordinates of the hand keypoints in the camera frame by adding pred_cam_t_full (which is calculated here) to out['pred_keypoints_3d'] (which is calculated here). Then you can export the sum in a pkl file.

I think this is how it is supposed to work.

specifically this piece of code handles this addition for each of the keypoints:

                    # Iterating over all of the samples in the batch:
                    for i in range(len(pred_keypoints_3d)):
                        batch_element_keypoints = pred_keypoints_3d[i]
                        batch_element_cam_t = pred_cam_t_full[i]

                        for keypoint_idx in range(len(batch_element_keypoints)):
                            # Adding the camera translation to the keypoints:
                            batch_element_keypoints[keypoint_idx] += batch_element_cam_t

Some input from @geopavlakos would surely be appreciated. I am also pretty sure that this can be done with some simpler numpy calls, but then the understanding of the dimensionality is lost for me.

Apparently this could be:

pred_keypoints_3d += pred_cam_t_full[:, np.newaxis, :]

But it would be great to get a review from someone.

[dhingratul]

I am still getting around this, but did you visualize and check ?

[Kaszanas]

I am still getting around this, but did you visualize and check ?

Didn't have time (or the knowledge), I've stopped at running through debugger and seeing the results as numbers.

Shapes matched. I think (2,3) and (2,21,3). So adding an empty axis and elementwise adding the last dimension makes sense?

[Kaszanas]

I am still getting around this, but did you visualize and check ?

Based on the discussion in #88 (comment), I think we need some input from @geopavlakos @dhingratul