point cloud data

Question

point cloud data

neverland-haha opened this issue 2 years ago · 11 comments

What feature or requirment is this related to? Please describe
This code example relates to the feature or requirement mentioned on #<github_issue_here>

!pip install mapillary 
import mapillary as mly
mly.interface.set_access_token('MLY|')
response = mly.interface.image_from_key(id,fields = ['sfm_cluster'])
sfm_cluster_url = 
https:\\/\\/scontent-iad3-1.xx.fbcdn.net\\/m1\\/v\\/t0.40383-6\\/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a

real_url = 'https:\\scontent-iad3-1.xx.fbcdn.net/m1/v/t0.40383-6/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a'

I want to get the point of cloud of each image so I can reconstruction the camera position and get better visualization, positioning, however. When I try to get the url that links to sfm_cluster, it returns to me a binary stream. After that , I write it content into a txt file,
but it becomes messy code. Is there any way to decode the messy code or api that I can get the *.ply format file?
Thank you!

Additional context

Answer 1 · 2022-09-10T08:58:15.000Z

Yeah, right now, there isn't any mechanism in the SDK to decode the contents from the URL that sfm_cluster returns. @cbeddow would you know of a way to do this?

Answer 2 · 2022-09-10T11:17:52.000Z

Yeah, right now, there isn't any mechanism in the SDK to decode the contents from the URL that sfm_cluster returns. @cbeddow would you know of a way to do this?

So, it means that there is no solution to decode the contents of this url? I wonder if I could know it encoding type, then Maybe I can try to decode myself?

Answer 3 · 2022-09-10T11:38:47.000Z

@neverland-haha, there is a way to decode the contents. I meant to say that this solution has not yet been integrated into the SDK.

The sfm_cluster property returns a link that points to an encoded .ply format file.

Can you please share the file here that you have right now? I can look into this and integrate a solution to decode the SFM as well.

Answer 4 · 2022-09-10T11:47:27.000Z

@neverland-haha, there is a way to decode the contents. I meant to say that this solution has not yet been integrated into the SDK.

The sfm_cluster property returns a link that points to an encoded .ply format file.

Can you please share the file here that you have right now? I can look into this and integrate a solution to decode the SFM as well.

This this the url tha sfm_cluster returns.
https:\\scontent-iad3-1.xx.fbcdn.net/m1/v/t0.40383-6/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a

Answer 5 · 2022-09-10T12:11:37.000Z

@neverland-haha, I was able to write a quick python script for this.

import requests
import json
import zlib

def main():
    # change URL here as per need
    sfm_content_url = 'https://scontent-iad3-1.xx.fbcdn.net/m1/v/t0.40383-6/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a'

    # decompress sfm content, and return dict
    print(decompress_sfm_content(sfm_content_url))

def decompress_sfm_content(sfm_content_url: str) -> dict:
    # make get request with python and get content

    response = requests.get(sfm_content_url)
    content = response.content

    # decompress zlib content
    return json.loads(zlib.decompress(content).decode('utf-8'))

if __name__ == '__main__':
    main()

I'll be adding this function soon in the SDK. Please let me know if this resolves your concern.

And thanks for opening this issue! I learned something new today.

Answer 6 · 2022-09-10T13:18:30.000Z

@neverland-haha, I was able to write a quick python script for this.

import requests
import json
import zlib

def main():
    # change URL here as per need
    sfm_content_url = 'https://scontent-iad3-1.xx.fbcdn.net/m1/v/t0.40383-6/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a'

    # decompress sfm content, and return dict
    print(decompress_sfm_content(sfm_content_url))

def decompress_sfm_content(sfm_content_url: str) -> dict:
    # make get request with python and get content

    response = requests.get(sfm_content_url)
    content = response.content

    # decompress zlib content
    return json.loads(zlib.decompress(content).decode('utf-8'))

if __name__ == '__main__':
    main()

I'll be adding this function soon in the SDK. Please let me know if this resolves your concern.

And thanks for opening this issue! I learned something new today.

It does solve my problem, Thank you very much!

Answer 7 · 2022-09-11T07:05:25.000Z

@neverland-haha, I'll close when I finally merge the PR for this function in the SDK. Reopening for now.

Answer 8 · 2022-09-11T14:42:47.000Z

@neverland-haha, I'll close when I finally merge the PR for this function in the SDK. Reopening for now.

import mapillary as mly
import requests 
import requests
import json
import zlib

mly.interface.set_access_token('MLY|5175886149161658|4099af83a040b4c64a3891bdcc8314cf')
data = mly.interface.image_from_key('1239407999827329',fields = ['sfm_cluster'])
result = eval(data)
url = result['features']['properties']['sfm_cluster']['url'].replace('\\','')
response = requests.get(url,stream=True,timeout=10)
info = json.loads(zlib.decompress(response.content).decode('utf-8'))
str_info = str(info)
length = len(str_info)
real_info = str_info[1:length-1]
dict_info = eval(real_info)
print(dict_info)

The dict_info contains too much infomation, but I want to retrive the earing angle and the pitch angle of the camera from the point cloud data. How can I know that point that I want?The mly.interface.image_from_key function returns me the id of point of cloud. But I can't find it in dict_info.

Answer 9 · 2022-09-12T18:58:37.000Z

Hi, @neverland-haha! I haven't yet had the time to explore this, but some intuition that I can help with maybe.

After storing the data as a JSON, I found there are 4 keys in the dictionary.

The shots and points are the most heavy data keys.

I'm not sure myself what to do for the earing angle or the pitch angle in this case (I still lack some more advanced GIS knowledge), but I found this field that may (?) help you,

@cbeddow what would you suggest to do here in this case?

Answer 10 · 2022-09-12T22:32:11.000Z

@Rubix982 @neverland-haha in the latest JSON example above, if you need the camera bearing angle, you will work with the rotation of the image: https://opensfm.org/docs/cam_coord_system.html

It should be the Z-Axis is telling you which angle the camera is facing. I am not sure if that is a 0 to 360 value though. The full data there you see is designed to work with OpenSfM library in Python so I suggest to go from Mapillary Python SDK to OpenSfM to work with the results you get from the JSON: https://opensfm.org/docs/dataset.html#reconstruction-file-format

Answer 11 · 2022-09-13T02:57:44.000Z

@Rubix982 @neverland-haha in the latest JSON example above, if you need the camera bearing angle, you will work with the rotation of the image: https://opensfm.org/docs/cam_coord_system.html

It should be the Z-Axis is telling you which angle the camera is facing. I am not sure if that is a 0 to 360 value though. The full data there you see is designed to work with OpenSfM library in Python so I suggest to go from Mapillary Python SDK to OpenSfM to work with the results you get from the JSON: https://opensfm.org/docs/dataset.html#reconstruction-file-format

Thanks, I' ll try and give you the answer. But As the doc says: Y-Axis is telling the angle the camera is up or down, however, when I change the units from rad to degree,some angle even greater than 90°. I'll check the process again.