Points to Lidar Channels

[sinjax]

Hey guys,

Is there a way to map lidar points to the lidar channel that produced them? Specifically those channels reported here: https://hesaiweb2019.blob.core.chinacloudapi.cn/uploads/Pandar64_User's_Manual.pdf

I've tried something simple along the lines of:

theta = np.arctan2(points[..., 2], points[..., 1])

but the visualisations don't look quite right when i colour points red > 3 * np.pi / 180 and colour points yellow <3 * np.pi/180. According to the hesai data sheet I was expecting to see 4 distinct bands but that didn't work out :) attached is what I see (also included a green rectangle whose corner is (0, 0, 0) which is 100 units long

My impression is that the provided lidar pose is in fact the baselink on the vehicle as opposed to the center of the lidar which is in turn making my "find the lidar channel" logic work incorrectly. The reason I think this is because if I add around 2m/3m to the "lidar_to_ego" corrected points I get the following:

But of course "about 3m" isn't quite the whole story because the lidary unit has a little tilt as well :) I guess what I need is the transform from baselink to the lidar sensor?

I've also attached the notebook I used to produce the images
view.tar.gz

Any pointers?

[sinjax]

clarifications as I investigate this further. In the notebook above I have applied the lidar_to_ego transform. However, from what I can see the 0,0,0 point of the un transformed lidar points from the spinning sensor are on the ground... which implies something funny is going on :)

[sinjax]

Idea! I can use the timestamp of the points to group them into columns and more easily understand which point is which channel! Right! Shall let you know if that works

[xpchuan-95]

Idea! I can use the timestamp of the points to group them into columns and more easily understand which point is which channel! Right! Shall let you know if that works

That works. I will implement a function to compute column and row later. @nisseknudsen

[sinjax]

Just an update on this, I couldn't quite get the timestamp grouping approach to work ... The issue many timestamp groups are missing returns ... So while you can always get an order of a group of timestamps there's no reliable way to get a one to one correspondence with a row.

@xpchuan-95 if there is a way to implement the row/col mapping of a point I'd be happy to contribute if you can point me in the right direction..

I'm scratching my head on this one but I think without the transform to the middle of the lidar scan I'm not sure how I could do this :). I'd love to hear your ideas

[sinjax]

hmm looking at the hesai64 docs (and some hesai ros sourcecode here) it looks like there is a direct mapping from the timestamp to a channel, however:

• this is only true if the lidar is running in Single mode (otherwise 2 points will share the same timestamp)
• this mapping can only be constructed if the packet timestamp is also known

sooo this leads me back to my original "yaw == channel" idea :D but that necessitates knowing the actual lidar transform :) which i do not know...

sadness.

[xpchuan-95]

hmm looking at the hesai64 docs (and some hesai ros sourcecode here) it looks like there is a direct mapping from the timestamp to a channel, however:

• this is only true if the lidar is running in Single mode (otherwise 2 points will share the same timestamp)
• this mapping can only be constructed if the packet timestamp is also known

sooo this leads me back to my original "yaw == channel" idea :D but that necessitates knowing the actual lidar transform :) which i do not know...

sadness.

I have checked my resource, it will take some time to get laser id, so what do you use laser id for? for generating range map?

[sinjax]

Thanks! Sad to hear it will take a while but look forward to it :). In the mean time I'm working on a registration based approach to at least get the true lidar position.

We're aiming to do some domain adaptation experiments for which the channel would be useful to create some baselines

[xpchuan-95]

Thanks! Sad to hear it will take a while but look forward to it :). In the mean time I'm working on a registration based approach to at least get the true lidar position.

We're aiming to do some domain adaptation experiments for which the channel would be useful to create some baselines

extrinsic_calibration.txt
you can use this yaml file to figure out the relative extrinsic pose between two sensors. If you want to get raw Lidar points you can transform points to camera frame first and then use this calibration file to transform points in camera frame to "main_pandar64" or "front_gt", the calibration is done under static situation.

But in fact the points in pandaset is compensated by moving compensation. Using above way can't get exact raw pointcloud, but a similar pointcloud.

[sinjax]

ah hah! :D interesting! yes i think i can use that... course if the pre-motion compensated frames could be used to assign a point to a channel that would be more accurate ... I understand that'd be tricky to get into your guys pipelines but I'd be super appreciative of it so I suppose treat this as a vote towards that making its way into your guys sprints :)

... but yeah this extrinsic transforms might just be enough (and probably better than the registration I could cook up ;))
I'll have a play and let you know

[xpchuan-95]

Also we make it to get laser id, it is from source data and accuracy, but only for Pandar64, so you need to set_sensor(0).
https://drive.google.com/drive/folders/1PL6N2QSNFjXQTFqpcM1IShOhnJCgSz73

[sinjax]

!! Nice that was fast! Great, can I have access to the drive? I've requested access.

Thank you so much for the effort, I'll investigate this as soon as I have access

[xpchuan-95]

!! Nice that was fast! Great, can I have access to the drive? I've requested access.

Thank you so much for the effort, I'll investigate this as soon as I have access

you can't access google driver link? it's public.

[sinjax]

The link works now, I'll play with it this evening, thanks again!

[sinjax]

Victory! Thank you @xpchuan-95 ! :)

[xpchuan-95]

Great!