Skip to main content

RDI SDK documentation

Overview​

The RDI (Raptor Drone Integration) SDK enables integration of custom drone platforms into Raptor Ace. This documentation provides the information required to develop, build, deploy and validate a RDI plugin that connects a drone platform to Raptor Ace.

Included in the SDK​

  • API headers: rdi_api.h and rdi_api_lens_param.h
  • Example plugin: Example plugin with full source code
  • This documentation: Integration guide

Quick start​

Minimal plugin example​

The following code demonstrates the minimum required for a working RDI plugin:

#include "rdi_api.h"

// Lifecycle functions
rdi_result_t init_func(rdi_func_t* func, rdi_context_t* context) {
    // Initialize plugin, allocate resources
    return RDI_RESULT_OK;
}

rdi_result_t run_func(rdi_func_t* func, rdi_context_t* context) {
    // Process one frame, submit it using func->frame_submit()
    return RDI_RESULT_OK;
}

rdi_result_t shutdown_func(rdi_func_t* func, rdi_context_t* context) {
    // Clean up resources
    return RDI_RESULT_OK;
}

// Plugin declaration
static const rdi_plugin_decl_t plugin_decl = {
    .api_version = RDI_API_VERSION,
    .init = init_func,
    .run = run_func,
    .shutdown = shutdown_func,
    .name = "Example Drone",
    .version = "1.0",
    .description = "Example drone platform",
    .config = "{}"
};

// Required export function
RDI_PLUGIN_EXPORT const rdi_plugin_decl_t* rdi_get_plugin_decl(void) {
    return &plugin_decl;
}

Build requirements​

Plugins must be compiled as a shared library (.so) with:

  • C99 or later (plugin implementations may require later standards)
  • Exported symbols marked with RDI_PLUGIN_EXPORT

Example CMake configuration:

add_library(rdiplugin_example SHARED example.c)
target_link_libraries(rdiplugin_example PRIVATE ...)
set_target_properties(rdiplugin_example PROPERTIES
    PREFIX ""
    SUFFIX ".so"
)

System overview​

Core structures​

The RDI system refers to the runtime component within Raptor Ace that loads the plugin, manages its lifecycle, and processes the frames submitted by the plugin.

The RDI API is built around four key structures:

  • RDI Declaration (rdi_plugin_decl_t) - Structure containing plugin's lifecycle functions and metadata, provided to the RDI system via the rdi_get_plugin_decl() function
  • RDI Context (rdi_context_t) - Provides connection information and stores plugin data for one stream instance. See Context fields for details
  • RDI Functions (rdi_func_t) - API functions provided by the RDI system for frame allocation, time parsing, etc.
  • RDI Frame (rdi_frame_t) - Structure for providing frame image and metadata to the RDI system

Plugin operation​

The RDI system loads the plugin shared library (.so) and obtains the plugin declaration (see Plugin configuration).

The RDI system then manages stream instances by calling lifecycle functions:

  • Initialization: The init() function is called once to establish connection to the drone and set up resources. Must complete before returning.
  • Runtime loop: The run() function is called repeatedly after successful initialization for the plugin to fetch frames from the drone. The plugin is then expected to populate RDI frames with image data as well as the necessary metadata and submit them to the RDI system.
  • Shutdown: The shutdown() function is called once to close connections and clean up resources. Must complete before returning.

The RDI system processes submitted frames by applying lens undistortion (depending on the lens model) and timestamp corrections to ensure a smooth video stream. The processed frames are then integrated into Raptor Ace for GNSS-denied navigation and video presentation. See the Lens models section for detailed information on supported lens models and when undistortion is applied.

Plugin responsibilities​

The plugin is responsible for:

  1. Connecting to the drone, e.g. via RTSP
  2. Decoding video frames to one of the supported image formats (GRAY8, RGB8, YUV420, YUV444)
  3. Extracting metadata, e.g. field of view, timestamp
  4. Providing camera calibration (lens model and parameters)
  5. Submitting frames to RDI via the provided API functions
  6. Cleaning up resources and closing connections

Context fields​

User data (userdata):

Plugin-managed pointer for storing plugin state (e.g., connection handles, buffers, configuration) for the stream instance. Persists across init(), run(), and shutdown() calls for the stream. Typically allocated in init() and freed in shutdown().

Stream location (filepath and url):

The context provides stream location in one of two mutually exclusive ways:

  • Opening file (filepath is non-NULL):

    • Absolute file path on file system (null-terminated UTF-8 string)
    • Example: /home/user/video.mp4
    • url and all URL component fields are NULL
    • Use filepath directly to open the file

  • Opening URL (url is non-NULL):

    • Complete URL, including file:// URLs (null-terminated UTF-8 string). file:// will be treated as a URL
    • URL is parsed into components: url_scheme, url_hostname, url_port, url_path, etc. (see rdi_api.h)
    • All URL component fields are non-NULL (empty string for components not present in the URL)
    • For network streams (e.g., rtsp://192.168.1.100:8554/stream), use url or URL components for connection
    • For file:// URLs (e.g., file:///home/user/video.mp4), use url_path to get the actual file path (/home/user/video.mp4)
// Example: handling both regular filepath and file:// URL
if (context->filepath || (context->url_scheme && strcmp(context->url_scheme, "file") == 0)) {
    // Local file - handle both regular filepath and file:// URL
    const char* path = context->filepath ? context->filepath : context->url_path;
    open_file(path);
}
else if (context->url) {
    // URL-based stream
    if (context->url_scheme && strcmp(context->url_scheme, "rtsp") == 0) {
        connect_rtsp(context->url);
    }
    else {
        // Handle other URL schemes...
    }
}

Plugin configuration​

The plugin must provide a declaration structure (rdi_plugin_decl_t) containing lifecycle functions, metadata, and configuration. This declaration is provided to the RDI system via the exported rdi_get_plugin_decl() function.

Important: The plugin must export the rdi_get_plugin_decl symbol using the RDI_PLUGIN_EXPORT macro.

Configuration field​

The config field is a null-terminated UTF-8 string containing a JSON document that specifies processing options affecting how the RDI system handles frames from the plugin.

If no configuration is needed, typically use an empty JSON object: .config = "{}"

Stream lifecycle​

1. Initialization phase​

When: Stream is opened

Function: init(rdi_func_t* func, rdi_context_t* context)

Tasks:

  • Parse the stream location from the context (see Context fields)
  • Establish connection to drone or open video file
  • Initialize video decoder
  • Allocate internal data structures
  • Store state in context->userdata

Return Values:

  • RDI_RESULT_OK - Initialization successful
  • RDI_RESULT_ERROR - Initialization failed

2. Run phase​

When: Called repeatedly to process frames

Function: run(rdi_func_t* func, rdi_context_t* context)

Tasks:

  • Fetch one frame from the drone
  • Decode the frame to a supported format
  • Allocate an RDI frame using func->frame_alloc()
  • Fill in image data, timestamp and lens parameters
  • Submit the frame using func->frame_submit() or drop it using func->frame_drop()

Return Values:

  • RDI_RESULT_OK - Frame processed successfully
  • RDI_RESULT_STREAM_COMPLETED - Clean end of stream
  • RDI_RESULT_ERROR
  • RDI_RESULT_ERROR_STREAM_DISRUPTED - Stream broken/disconnected (e.g., network error, connection lost)

3. Shutdown phase​

When: Application shutdown

Function: shutdown(rdi_func_t* func, rdi_context_t* context)

Tasks:

  • Close connection to drone
  • Free all stream-related resources
  • Clean up context->userdata

Return Values:

  • RDI_RESULT_OK - Shutdown successful

Lens models​

The RDI system supports multiple lens models to represent different camera types. The plugin must specify which lens model matches the camera that provides the video feed and provide the corresponding parameters with the submitted RDI frame. The lens parameters that need to be provided depend on the lens model used, as defined in rdi_api_lens_param.h. The RDI system uses this information to decide if and how to perform lens undistortion, converting the distorted image from the camera into an ideal pinhole projection. If the plugin or drone system performs undistortion on the image, the pinhole model should be used to pass through the undistortion step in the RDI system.

Camera intrinsics in the form of focal length parameters expressed as pixel units (fx, fy) as well as the principal point (cx, cy) are expected to be provided with the submitted RDI frame for all models. For some lenses the principal point moves with changes to the focal length. For these models it is expected that the plugin provides cx and cy parameters tailored for the given focal length and then use the appropriate lens model based on the lens distortion. The principal point are given in image pixel coordinates, with the origin at the upper-left corner of the image. The cx parameter increases with x to the right and cy increases with y downward.

For lenses with very high field of view, undistortion may be insufficient and tracking performance can be negatively affected. For these cases it is recommended that the plugin performs the undistortion and crops the image until all distortion effects are removed.

It is the processed image that is used for both for GNSS-denied navigation as well as displayed in the video feed for the Raptor Ace user.

Supported models​

Pinhole​

A simple pinhole model with no distortion. This can be set if the plugin or drone performs undistortion on the image or if the camera lens can be approximated as a pinhole (performance of the GNSS denied navigation can be affected if a distorted image is provided using this model). Focal length parameters expressed as pixel units as well as the principal point are still expected to be provided with the submitted RDI frame.

Distorted​

A model that supports distorted images given that the distortion parameters for radial distortion are provided. The RDI system will apply undistortion to convert the image to a pinhole model. The undistorted image will be used for GNSS denied navigation as well as displayed in the video feed for the Raptor Ace user.

Fisheye​

For higher field of view fisheye lenses, the fisheye model should be used. The RDI system will apply undistortion to convert the image to a pinhole model. The undistorted image will be used for GNSS denied navigation as well as displayed in the video feed for the Raptor Ace user. Notice that undistortion may not be sufficient for very high FOV fisheye lenses and it is recommended that the plugin performs the necessary processing to achieve an undistorted image before submitting it to the RDI system.

Important: Fisheye lenses are not directly supported by Raptor Ace (refer to Raptor Ace Product Specification for more information). If using this lens model, thorough testing and validation of results from initialization, navigation and point extraction should be performed. Use of the fisheye lens model is at own risk and performance stated in the Raptor Ace Product Specification cannot be guaranteed.

Plugin requirements​

For each frame, the plugin must set:

  1. Lens model type (rdiframe->lens.model) - Which model to use
  2. Image dimensions (rdiframe->lens.width, rdiframe->lens.height) - Must match the image size
  3. Model parameters - Camera intrinsics and distortion parameters depending on the model used, as defined in rdi_api_lens_param.h

Build and deployment​

Build requirements​

Output format: The plugin must be built as a shared library (.so) with the naming convention rdiplugin_<plugin-name>.so.

C standard: C99 or later is required for the RDI API headers. Plugin implementations may require later standards depending on their specific needs.

Symbol export: The plugin must export exactly one symbol: rdi_get_plugin_decl. This is accomplished using the RDI_PLUGIN_EXPORT macro defined in rdi_api.h:

#include "rdi_api.h"

// Required export - provides plugin declaration to RDI system
RDI_PLUGIN_EXPORT const rdi_plugin_decl_t* rdi_get_plugin_decl(void) {
    return &plugin_decl;
}

Plugin installation​

The plugin shared library must be installed in the following location within the Raptor Ace package:

RaptorAce/rdiplugin/rdiplugin_<plugin-name>.so

Dependencies: Any additional shared libraries that the plugin depends on must also be placed in the RaptorAce/rdiplugin/ directory. Standard system libraries do not need to be included.

Testing and validation​

Thorough testing and validation are critical to ensure the plugin performs correctly and meets operational requirements.

Conformance testing​

The Raptor Ace application can be launched as a command-line tool to perform conformance testing of the plugin. This validates that the plugin correctly implements the RDI API and provides properly formatted data suitable for GNSS denied navigation within Raptor Ace.

Basic API implementation test (plugin only):

./RaptorAce --rdiplugintest=/path/to/rdiplugin_example.so

This basic test verifies:

  • Plugin loads successfully
  • Declaration structure is valid
  • Lifecycle functions are properly implemented

Full stream health test (plugin + stream):

./RaptorAce --rdiplugintest=RaptorAce/rdiplugin/rdiplugin_example.so \
           --rditeststream=rtsp://user:pass@192.168.1.100:8554/stream

Adding the --rditeststream parameter runs additional tests that verify:

  • Stream connection is established successfully
  • Frame data arrives at the expected rate without excessive frame drops
  • RDI frames contain valid image data and lens parameters

Plugin location requirements:

  • Basic test: The plugin can be located anywhere on the filesystem during development
  • Full stream health test: The plugin must be in its final installation location (as described in Build and deployment)

Optional parameters:

--rditestduration=<seconds>      # Test duration (default: 10)
--rditestexpectedfps=<fps>       # Expected frame rate (default: 30)

Complete example:

./RaptorAce --rdiplugintest=/RaptorAce/rdiplugin/rdiplugin_example.so \
           --rditeststream=rtsp://user:pass@192.168.1.100:8554/stream \
           --rditestduration=10 \
           --rditestexpectedfps=30

Functional testing​

For comprehensive functional testing, launch the full Raptor Ace application with the plugin. Note: 3D data supplied by Vantor is required to perform initialization, tracking, and coordinate extraction.

Refer to the Raptor Ace User Guide for detailed instructions on how to prepare the application and perform initialization, tracking, and coordinate extraction steps correctly.

Coordinate extraction validation​

Extracted coordinates must be carefully validated against reference coordinates to verify plugin performance:

  1. Reference measurements: Obtain reference coordinates using high-precision equipment operated by skilled personnel

  2. Comparison testing: Extract coordinates using Raptor Ace with the plugin and compare against reference measurements. Critical: Ensure that both the extracted coordinates and reference measurements are expressed in the same coordinate reference system and epoch. Skilled personnel should be consulted to perform any necessary coordinate transformations to maintain accurate and correct results

  3. Accuracy verification: Verify that coordinate extraction accuracy meets the required specifications

  4. Comprehensive coverage testing: Perform repeated tests that cover the operational envelope, including various zoom levels and slant angles

Operational envelope considerations​

Important: The operational envelope may differ from standard Raptor Ace specifications depending on:

  • Drone platform characteristics: Camera quality, gimbal characteristics, stabilization, flight dynamics
  • Plugin implementation: Stream health, lens calibration

Recommendation: Conduct thorough field testing across the expected operational conditions to characterize the actual performance envelope for the specific drone platform and plugin implementation.