multiple Realsense depth camera’s appear to be accessible from different source identifiers/locations (especially using DirectShow on windows). I am not interested in hardware sync so will be using an enforced timestamp to associate with the capture.
Intel RealSense Depth Camera D435i: https://www.intelrealsense.com/depth-camera-d435i
Intel RealSense SDK 2.0 code: https://github.com/IntelRealSense/librealsense
I installed the RealSense SDK 2.23.0 version for my Windows 10 environment. Looks like all the example projects in github focus on C++. However we have wrapper python examples which does provide the stuff we need.
Build Python wrapper and tests using CMake & Visual Studio on Windows 10
Build the RealSense python wrapper from source code
- Git clone the librealsense2 source code
- Install Visual Studio Community edition for compiling the source
- Run CMake Gui “cmake-gui.exe”.
- “Browse Source” and configure the path. Example “C:/Dev/librealsense”
- Create “build” folder on librealsense path folder. Click “Browse Build” button and configure the path. Example “C:/Dev/librealsense/build”.
- Check on “Grouped” and “Advanced” checkbox
- Click “Configure” button.
- On “Specify the generator for this project”, Select “Visual Studio 16 2019”
- Then click “Finish” button.
- After configuration is done, on “BUILD” node, check “BUILD_PYTHON_BINDINGS” checkbox button.
- Then click “Configure” button again.
- In “Ungrouped Entries”, identify variable “PYTHON_EXECUTABLE”, and change the path to anaconda python.exe in your virtual environment path as
“C:/Python3/python.exe” or “C:/ProgramData/Anaconda3/envs/cv3/python.exe”
- Then click “Generate” button, after all is done, click “Open Project”.
- Alternately, open Visual Studio IDE, select the librealsense2.sln solution and build all in Release or Debug mode.
- The libraries are built and available in folder c:/Dev/librealsense/build/Release if you built in Release mode
- cp pybackend2.cp35-win_amd64.pyd pybackend2.pyd
- cp pyrealsense2.cp35-win_amd64.pyd pyrealsense2.pyd
- Copy the libraries to the relevant environment
- copy “pybackend2.pyd”, “pyrealsense2.pyd” and “realsense2.dll” to “DLLs” folder. Example “C:/ProgramData/Anaconda3/envs/cv3/DLLs”
- copy “realsense2.lib” to “libs” folder. Example “C:/ProgramData/Anaconda3/envs/cv3/libs”
- Test the python bindings and realsense libraries
- Activate your Anaconda environment. Example: conda activate cv3
- Test the python opencv RGB & Depth visualization code and see the results as per the image below
Test Example: Opencv RGB & Depth viewer
cd C:/Dev/librealsense/wrappers\python\example python opencv_viewer_example.py
Now lets write a test code for Multiple Camera Setup
Test 1: enlist all the devices
import pyrealsense2 as rs import numpy as np import cv2 DS5_product_ids = ["0AD1", "0AD2", "0AD3", "0AD4", "0AD5", "0AF6", "0AFE", "0AFF", "0B00", "0B01", "0B03", "0B07","0B3A"] def find_device_that_supports_advanced_mode() : ctx = rs.context() ds5_dev = rs.device() devices = ctx.query_devices() print("D: ", devices) devs =  for dev in devices: if dev.supports(rs.camera_info.product_id) and str(dev.get_info(rs.camera_info.product_id)) in DS5_product_ids: if dev.supports(rs.camera_info.name): print("Found device that supports advanced mode:", dev.get_info(rs.camera_info.name), " -> ", dev) devs.append(dev) #raise Exception("No device that supports advanced mode was found") return devs devs = find_device_that_supports_advanced_mode()
(cv3) λ python senserealtest.py D: <pyrealsense2.device_list object at 0x00000237B1C3B688> Found device that supports advanced mode: Intel RealSense D435I -> <pyrealsense2.device: Intel RealSense D435I (S/N: 843112070672)> Found device that supports advanced mode: Intel RealSense D435I -> <pyrealsense2.device: Intel RealSense D435I (S/N: 843112070632)> Found device that supports advanced mode: Intel RealSense D435I -> <pyrealsense2.device: Intel RealSense D435I (S/N: 843112070689)>
Test 2: Stream images from all the devices and show
import pyrealsense2 as rs import numpy as np import cv2 from realsense_device_manager import DeviceManager def visualise_measurements(frames_devices): """ Calculate the cumulative pointcloud from the multiple devices Parameters: ----------- frames_devices : dict The frames from the different devices keys: str Serial number of the device values: [frame] frame: rs.frame() The frameset obtained over the active pipeline from the realsense device """ for (device, frame) in frames_devices.items(): color_image = np.asarray(frame[rs.stream.color].get_data()) text_str = device cv2.putText(color_image, text_str, (50,50), cv2.FONT_HERSHEY_PLAIN, 2, (0,255,0) ) # Visualise the results text_str = 'Color image from RealSense Device Nr: ' + device cv2.namedWindow(text_str) cv2.imshow(text_str, color_image) cv2.waitKey(1) # Define some constants resolution_width = 1280 # pixels resolution_height = 720 # pixels frame_rate = 15 # fps dispose_frames_for_stablisation = 30 # frames try: # Enable the streams from all the intel realsense devices rs_config = rs.config() rs_config.enable_stream(rs.stream.depth, resolution_width, resolution_height, rs.format.z16, frame_rate) rs_config.enable_stream(rs.stream.infrared, 1, resolution_width, resolution_height, rs.format.y8, frame_rate) rs_config.enable_stream(rs.stream.color, resolution_width, resolution_height, rs.format.bgr8, frame_rate) # Use the device manager class to enable the devices and get the frames device_manager = DeviceManager(rs.context(), rs_config) device_manager.enable_all_devices() # Allow some frames for the auto-exposure controller to stablise while 1: frames = device_manager.poll_frames() visualise_measurements(frames) except KeyboardInterrupt: print("The program was interupted by the user. Closing the program...") finally: device_manager.disable_streams() cv2.destroyAllWindows()
This will show you multiple windows, one for each camera connected to the system
That should be good for today !!!
References can be found at the following places:
- Intel Multicam: https://dev.intelrealsense.com/docs/multiple-depth-cameras-configuration
- Github Ex: https://github.com/IntelRealSense/librealsense/tree/master/examples
- Documentation: https://github.com/IntelRealSense/librealsense/tree/master/doc
- Issues & Test Code: https://github.com/IntelRealSense/librealsense/issues/1735