[pyrealsense2] 1. Is there a way to export .ply file with both color(from color frame) & vertex normals? 2. can I clamp in x, y direction ?

[AlexSeongJu-sr]

---

| Camera Model | D435i |
| Firmware Version | (Open RealSense Viewer --> Click info) |
|API version |
| Operating System & Version | Linux (Ubuntu 18.04) |
| Kernel Version (Linux Only) | (e.g. 4.14.13) | |
| Language | /python/ } |

Issue Description

Hi. I'm quite new to using pyrealsense2 library, and having trouble in saving .ply file with both color & vertex normals.
As I see from export_to_ply.py, there is a filter named 'save_to_ply', but with this, I cloud only save pointcloud(.ply) with normals but color. Here is a part of my code.

pipe.wait_for_frames()

frames = pipe.wait_for_frames()
aligned_frames = align.process(frames)
color = aligned_frames.get_color_frame()
depth = aligned_frames.get_depth_frame()
colorized = colorizer.process(aligned_frames)

# Create save_to_ply object
ply = rs.save_to_ply("1.ply")

# Set options to the desired values
# In this example we'll generate a textual PLY with normals (mesh is already created by default)
ply.set_option(rs.save_to_ply.option_ply_binary, False)
ply.set_option(rs.save_to_ply.option_ply_normals, True)
ply.set_option(rs.save_to_ply.option_ignore_color, False)
ply.set_option(rs.save_to_ply.option_ply_threshold, True)


print("Saving to 1.ply...")
# Apply the processing block to the frameset which contains the depth frame and the texture
ply.process(colorized)

with this, I could only get colors from depth frame, not colors from color frame.
How can I get color from color_frame when saving .ply files?

There is also another way of saving .ply file using points like below.

for i in range(30):
    pipe.wait_for_frames()

frames = pipe.wait_for_frames()

# Get aligned frames
aligned_depth_frame = frames.get_depth_frame()  
color_frame = frames.get_color_frame()
color_image = np.array(color_frame.get_data())

pc.map_to(color_frame)
# Generate the pointcloud and texture mappings
points = pc.calculate(aligned_depth_frame)
print("Saving to 1.ply...")
points.export_to_ply("1.ply", color_frame)
print("Done")

However, in this way, I couldn't get vertex normals.
I see that with realsense-viewr, I can get both normals & color from color frame. but I wanna do it with code.

1. How can I get both vertex normals & color from color_frame in pyrealsense2?

2. another question. How can I clamp in x, y direction when saving .ply files? I coud do depth clamping with threshold filter like this

thre = rs.threshold_filter(0.1, 1.8)
.
.
.
filtered = thre.process(depth_frame)

However, I also want to clamp in x, y directions also and save the clamped point cloud. Is there any functions related to this ? I could find depth claping in realsense-viewer, but not with x, y directions.
Also, I wanna do it in python code, not realsense viewer.

Thank u for reading.

[MartyG-RealSense]

There is a Python example program called export_ply_example. It has Mesh creation set to True by default, and sets saving of Normals to True.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/export_ply_example.py

If you want to set a 'bounding box' for your Python point cloud, this link may be helpful:

#2769

[AlexSeongJu-sr]

@MartyG-RealSense
Hi. thanks for your help. I've also read that example, but it saves point cloud with color from depth frame, not color frame. I want point cloud with real colors, not depth colors. (i don't want a human face with blue/red color like Avatar by James Cameron)
do u have any suggestions ?

I think there should be some way to initialize colorizer with color stream so that when using colorizer, the output frame should get colors from color_frame, not depth_frame.

27 # (alternatively, texture can be obtained from color or infrared stream)
28 colorizer = rs.colorizer()
....
33 colorized = colorizer.process(frames) ---> this processes 'frames' with depth color

2. Thanks for POINT CLOUD OF A PREDEFINED REGION USING PYREALSENSE2 #2769. I'll check on that.

[MartyG-RealSense]

Are you attempting to align depth to RGB color, please?

The link below has an example of a Python user who aligned depth to color and then generated a point cloud.

#5834

[AlexSeongJu-sr]

@MartyG-RealSense
I've read that issue, but I think my case is different. As I mentioned above, I know how to align depth to color. below is one option.

for i in range(30):
pipe.wait_for_frames()

frames = pipe.wait_for_frames()
aligned_frames = align.process(frames)
aligned_depth_frame = aligned_frames.get_depth_frame()
color_frame = aligned_frames.get_color_frame()
pc.map_to(color_frame)
points = pc.calculate(aligned_depth_frame)
print("Saving to 1.ply...")
points.export_to_ply("1.ply", color_frame)
print("Done")

However, this 1.ply file doesn't have vertex normals in it.
Also, when using save_to_ply object like in export_ply_exampe.py, this doesn't have colors from color frame.

left image is what I got from export_ply_example.py
right image is what I got from realsense-viewer.
I wanna get .ply with both color and vertex normal with python code, like I got from realsense-viewer(right image).

but what I found is

1. ply with color, not with vertex normal(like above code)
2. ply with vertex norma, not with rgb color (like in export_to_ply.py)

How can I save .ply file with both rgb color & vertex normals at the same time?

[MartyG-RealSense]

I had a careful look at your code and compared it to similar examples and could not find anything obviously wrong with it. I am admittedly not an advanced-level Python programmer though. I will refer this question to @dorodnic for advice.

[AlexSeongJu-sr]

@MartyG-RealSense
I would really appreciate that.
thanks for ur help so much.

[MartyG-RealSense]

Case closed due to no further comments received.

[usarawgi911]

Same issue as mentioned of @djflstkddk
Will really appreciate any help! Thank you

[MartyG-RealSense]

@usarawgi911 Can you start a new issue please by clicking on the New Issue button on the forum's front page and provide a link to this page in your question? Thanks!

https://github.com/IntelRealSense/librealsense/issues

[TheDech]

For those who are still trying to solve the issue of saving their pcd in rgb color instead of
colorized = colorizer.process(aligned_frames)
ply.process(colorized)
use
aligned_frames = align.process(frames)
ply.process(aligned_frames)

[MartyG-RealSense]

Thanks so much @TheDech for sharing your solution!