Skip to main content
Skip table of contents

Rapid Digital Imaging - Stereo cameras for 3D and Machine Vision Applications

3D modeling and machine vision applications often require the capture of data from two or more camera sources simultaneously. The spatial separation between the two or more different points of view produced by multiple cameras are interleaved in time. The timing between the camera frames is the key. The more in sync the frames are, the better the accuracy of the model generated. If a scene is static, or the system is moving slowly (<2 knots), there is a lot more leeway in the timing between the cameras.

Figure 1. An autonomous subsea system with prcessor decision making using camera images

Note: The processor, storage and NTP server can also be topside. Cameras are connected to the system via Ethernet and everything is powered from 20-32Vdc. LEDs plug into the back of the camera and are powered and controlled directly.

Figure 2. Seaeye Falcon with Stereo Rayfin and LEDs

How to time sync data from multiple Rayfin cameras

With SubC Rayfin cameras, there are two methods of capturing stereo pairs of camera data that our customers are using on their projects:

  1. IP video frame sync: frame timing is embedded within IP streams

  2. Digital stills sync: images have a capture timestamp in EXIF

Additionally, there is XVS: hardware exposure sync. In this mode, one camera operates as the Primary. This is not available off the shelf but can be developed within a contract.

IP video frame sync

Rayfin cameras use RTSP to transmit live video frames. Each frame has a timestamp embedded. Frames are at approx. 30Hz. For more about RTSP video and framerates, please see this article.

Steps to time synchronize live IP video frames within your processing system:

  1. Synchronize all cameras to the NTP server in the system via the included software or API:

    • EnableNTP:{NTP server IP address} returns the TimeZone and SystemDateTime for confirmation
      TimeZone:Etc/GMT
      SystemDateTime:12/16/2022 9:46:32 PM

  2. On the processor, open each camera RTSP stream: rtsp:/{rayfin_ip}:5540/ch0 (ex. rtsp://192.168.2.34:5540/ch0)

  3. Capture / save frames from multiple RTSP streams

    • Frames are 30 FPS approximately

    • Frame timing has a slight bit of variability of up to approx. 15 milliseconds

    • Each frame has a timestamp

  4. Process each pair of frames for 3D data / and build on the position model

    • For data quality control, in the processing software, compare frame timings to check that they are not too far apart.

The Rayfin is capable of framerates lower than 30FPS by setting the shutter speed slower than 1/30. Ensure your shutter speed is <= 1/30 to maintain 30FPS framerate. Higher shutter speeds will also reduce motion blur. We recommend 1/125 or higher for moving scenes/vehicles.

Figure 3. Whitecap ROV3D with dual Rayfin RTSP feeds

Digital still sync

Digital stills can generate higher resolution models through a few advantages over video frames:

  • Resolution: Digital stills are higher resolution than video frames (12.3MP in the Rayfin Mk2).

  • Sharpness: Digital stills can trigger a strobe to enable more light on the scene and further reduce motion blur. Note, SubC LEDs are TTL synchronized to the camera exposure time and are plug-and-play with the camera hardware and software.

  • Timing: the precision of the photos should be high as they are using the NTP clock source timing directly to start exposures.

Similar to IP video frame sync, digital still sync would use the NTP clock source to keep each camera in sync. The drawback is a lower frequency using this method (2-3 Hz). The steps to get started with digital still sync are as follows:

  1. Setup the cameras for this session

    1. To help with file sorting, set your still name to include the ${ff} tag for milliseconds. Ex. StillName:Cam1_Location2_${yyyy}-${MM}-${dd}_${hh}${mm}${ss}${ff}

    2. (optional) Re-sync your NTP time to each camera. EnableNTP:{NTP server IP address}

    3. (optional) Send the USBL GPS string to the camera

    4. Over TCP (port 8889) or UDP (port 8886), send the USBL $--GGA string to each Rayfin camera

    5. Send StartLogging and each camera will log the GPS

    6. It will use the last GPS string to store the Latitude and Longitude in photo EXIF data

  2. Take synchronized photos on an interval

    1. Read the current time on the processing system, which is synchronized to the same NTP clock as the cameras. Ie. The current time is 1145:03:240

    2. Accounting for command latency (Ethernet network dependent), send the command to each camera to take a photo at a specific time.

      1. TakeStill:01/15/2023 11-45-03 {MM/dd/yyyy HH-mm-ss} (current version)

      2. TakeStill:11-45-03-400 {HH-mm-ss-fff} (in development version)

    3. The digital stills store locally on the camera and can be accessed in real-time, or they can store on the Ethernet network onto a NAS.

    4. The processor can take the still pairs and run them through an algorithm

    5. Repeat this on an interval within the processing system

Focus settings

When using cameras in stereo configuration, focus often has to be fixed and calibrated between cameras. The API focus commands are available in the Rayfin API documentation.

We recommend a focus distance of approx 0.2 - which gives a wide depth of field. We also recommend testing this for your given expected distance in water. It has to be tested in water, as the Rayfin camera optics are optimized for water. The command would be UpdateFocus:0.2

XVS

Hardware synchronization of frames can be accomplished with a vertical sync. The advantage of this method is that each exposure at the sensor level is timed in sync. The XVS signal is sent from a Primary camera to a Secondary camera. Using XVS in combination with one of the above capture methods would ensure that the capture timing of the exposures is very precise. This would be for high-precision 3D models, fast-moving vehicles or other applications where up to 15 milliseconds between frames can make a difference in data.

Reference material

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close