Server Integration Guide

Introduction

The Raptor Guide Server provides a network-accessible interface to the Raptor Guide SDK for visual-inertial pose estimation. This guide covers how to integrate client applications with the Guide Server using either the high-level Python client API or directly via Protocol Buffers API over ZeroMQ.

For detailed SDK concepts, see:

• SDK integration details: Integration guide
• Coordinate and attitude conventions: Conventions
• Protocol specification: guide.proto

Architecture Overview

System Components

┌─────────────────┐         ZeroMQ          ┌──────────────────┐
│                 │    (REQ/REP Pattern)    │                  │
│  Client App     │◄───────────────────────►│  Guide Server    │
│                 │    Protocol Buffers     │                  │
└─────────────────┘                         └──────────────────┘
                                                     │
                                                     ▼
                                            ┌──────────────────┐
                                            │  Raptor Guide    │
                                            │      SDK         │
                                            └──────────────────┘
                                                     │
                                                     ▼
                                            ┌──────────────────┐
                                            │   3D Maps        │
                                            │   (.r3db files)  │
                                            └──────────────────┘

Protocol and Operations

This section describes the communication protocol, message format, and available operations.

Transport Layer

The Guide Server uses ZeroMQ with a REQ/REP (Request-Reply) pattern:

• Client: Uses a ZMQ_REQ socket
• Server: Uses a ZMQ_REP socket
• Pattern: Strict request-response - client sends one request, receives one reply
• Endpoint: Standard ZeroMQ URI format (e.g., tcp://localhost:5555)

Message Format

All messages are serialized using Protocol Buffers version 3 (proto3). The server uses a request/response envelope pattern:

• Request: GuideRequest with a request_id and one of several request types (init, update, render, etc.)
• Response: GuideResponse with matching request_id and corresponding response type

Communication Flow:

1. Client creates GuideRequest, sets unique request_id, populates one request field
2. Serializes to bytes and sends via ZeroMQ
3. Server processes and creates GuideResponse with matching request_id
4. Client receives, deserializes, and dispatches based on WhichOneof("response")

For complete message definitions, see: guide.proto

Coordinate System Configuration

The server supports two coordinate reference frames (GEODETIC and ECEF) with customizable geodetic options for units and vertical datum.

See: Integration guide - "Loading Configuration from JSON" and "Geodetic Coordinate Options" sections

For coordinate conventions, quaternion format, and transformations, see: Conventions

Available Operations

The server exposes SDK functionality and additional service-specific operations through Protocol Buffer messages. Each operation has a corresponding Python client method for easy integration.

Operations Summary

SDK Component	Server Request	Server Response	Python Client Method
Guide(...) constructor	GuideInitRequest	GuideInitResponse	client.initialize_guide()
Guide::updatePose(...)	UpdatePoseRequest	UpdatePoseResponse	client.update_pose()
Guide::renderReferenceImage(...)	RenderReferenceImageRequest	RenderReferenceImageResponse	client.render_reference_image()
Guide::~Guide() destructor	GuideDestroyRequest	GuideDestroyResponse	client.destroy_guide()
raptor::guide::Config (get)	GetConfigRequest	GetConfigResponse	client.get_config()
raptor::guide::Config (set)	SetConfigRequest	SetConfigResponse	client.set_config()
N/A (server-specific)	GetStatusRequest	GetStatusResponse	client.get_status()
N/A (server-specific)	QueryAvailableMapsRequest	QueryAvailableMapsResponse	client.query_available_maps()

Core SDK Operations

GuideInit - Create guide instance with maps and image dimensions

UpdatePose - Refine pose using image matching

• Returns PoseOutput with confidence and refined pose
• Quality assessment: See Integration guide for confidence interpretation and recommended thresholds

RenderReferenceImage - Generate synthetic view from map

DestroyGuide - Clean up guide instance and free resources

Server Management Operations

GetStatus - Check server health, version, and license status

• Returns: server_version, guide_sdk_version, guide_initialized, license_status

QueryAvailableMaps - Discover maps available on the server

• Returns: List of directories with available .r3db and .3tz maps
• Each map entry includes: map_id, relative_path, and base_dir
• Map IDs can be used with initialize_guide(map_ids=[...]) for convenient map selection

GetConfig - Retrieve current server configuration

• Returns: geo_resource_path, reference_frame, detail_stage_1, detail_stage_2, etc.

SetConfig - Update server configuration (affects future guide instances)

Important

Configuration changes are not persistent across server restarts. If your application requires deviations from the server's default configuration, you must call set_config() after connecting and before initializing the guide.

For complete message definitions and field details, see:

• Protocol specification: guide.proto
• SDK integration details: Integration guide (quality metrics, confidence thresholds, best practices)
• Coordinate conventions: Conventions (position/attitude formats, reference frames)

Client Integration

Prerequisites: The Raptor Guide Server should already be installed and running. Contact your system administrator for the server address and port if you don't have this information.

Option 1: Python Client API

The Python client provides a high-level, Pythonic interface to the Guide Server.

Installation

Recommendation: Use a virtual environment

# Create a virtual environment
python3 -m venv raptor_guide_env

# Activate it
source raptor_guide_env/bin/activate  # On Linux/macOS

Install client module

pip install raptor_guide_client-0.1.0-py3-none-any.whl

Dependencies: The client requires:

• Python 3.9 or later
• pyzmq (ZeroMQ Python bindings)
• protobuf (Protocol Buffers)

These are automatically installed when you install the client package.

Running the Example

A complete working example is provided in example_basic_usage.py.

Prerequisites:

• Guide Server must be running and accessible
• Install the client module (see above)
• Install Pillow for image handling: pip install Pillow

Run with default settings:

python example_basic_usage.py

The example demonstrates:

• Connecting to the server and querying status
• Discovering available maps
• Initializing a guide with a map
• Performing pose updates
• Rendering reference images (optional)

For detailed example documentation, see: python/raptor_guide_client_example/README.md

Basic Usage

Recommended approach: Configure maps server-side

from raptor.guide_service.client import GuideClient

# Connect to the server (replace with your server address)
with GuideClient("tcp://localhost:5555") as client:
    # Verify server is responding
    status = client.get_status()
    print(f"Connected to server version: {status['server_version']}")

    # Configure server with maps and other settings (one-time setup)
    client.set_config(
        geo_resource_path="/path/to/geo/resources",
        reference_frame="ECEF",
        map_paths=["/data/maps/map1.r3db", "/data/maps/map2.3tz"]
    )

    # Initialize guide with just image dimensions
    client.initialize_guide(image_width=1920, image_height=1080)

    # Update pose
    ok, result = client.update_pose(
        image_data=image_bytes,  # 8-bit grayscale
        h_fov=1.39, v_fov=1.12,  # radians
        position=[x, y, z],      # ECEF or Geodetic (see conventions.md)
        attitude=[qx, qy, qz, qw]  # quaternion (see conventions.md)
    )

    if ok and result['confidence'] > 0.75:
        # Use refined pose
        refined_position = result['position']
        refined_attitude = result['attitude']

Alternative: Per-request specifications

# Example 1: Explicit map paths (backward compatibility)
client.initialize_guide(
    image_width=1920,
    image_height=1080,
    map_paths=["/data/maps/map1.r3db"]
)

# Example 2: Map IDs (resolved server-side from query_available_maps)
maps_info = client.query_available_maps()
# Extract map IDs from the response
map_ids = [...]  # e.g., ["map_001", "map_002"]
client.initialize_guide(
    image_width=1920,
    image_height=1080,
    map_ids=map_ids
)

# Example 3: Per-request coordinate epoch (mission-specific)
# Useful in high-precision applications where tectonic plate shift could affect accuracy
client.initialize_guide(
    image_width=1920,
    image_height=1080,
    coordinate_epoch=2025.5  # Override server epoch for tectonic correction
)

Option 2: Raw Protocol Buffers over ZeroMQ

For non-Python languages or when you need fine-grained control over the protocol.

Minimal example:

import zmq
from raptor.guide_service.client.proto import guide_pb2

context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect("tcp://localhost:5555")
socket.setsockopt(zmq.RCVTIMEO, 60000)

# Set server configuration (one-time setup)
req = guide_pb2.GuideRequest(request_id="0")
config = guide_pb2.Config()
config.geo_resource_path = "/path/to/geo/resources"
config.reference_frame = guide_pb2.ECEF
config.map_paths.extend(["/data/maps/map1.r3db", "/data/maps/map2.3tz"])
req.set_config_request.config.CopyFrom(config)
socket.send(req.SerializeToString())
resp = guide_pb2.GuideResponse()
resp.ParseFromString(socket.recv())

# Initialize guide using server config
req = guide_pb2.GuideRequest(request_id="1")
req.init_request.image_width = 1920
req.init_request.image_height = 1080
socket.send(req.SerializeToString())
resp = guide_pb2.GuideResponse()
resp.ParseFromString(socket.recv())

# Alternative: Override with per-request map_paths (backward compatibility)
req = guide_pb2.GuideRequest(request_id="2")
req.init_request.image_width = 1920
req.init_request.image_height = 1080
req.init_request.map_paths.extend(["/data/maps/map1.r3db"])
socket.send(req.SerializeToString())
resp = guide_pb2.GuideResponse()
resp.ParseFromString(socket.recv())

# Update pose
req = guide_pb2.GuideRequest(request_id="3")
req.update_pose_request.image_data = image_bytes
req.update_pose_request.h_fov = 1.39
req.update_pose_request.v_fov = 1.12
req.update_pose_request.position.x, req.update_pose_request.position.y, req.update_pose_request.position.z = x, y, z
req.update_pose_request.attitude.x, req.update_pose_request.attitude.y = qx, qy
req.update_pose_request.attitude.z, req.update_pose_request.attitude.w = qz, qw
socket.send(req.SerializeToString())
resp = guide_pb2.GuideResponse()
resp.ParseFromString(socket.recv())

For complete examples in Python and C++, see:

• Python: python/raptor_guide_client_example/example_basic_usage.py
• C++: apps/raptor_guide_server/examples/client_example.cpp

Map Configuration Strategies

The Guide Server supports three approaches for specifying maps when initializing a guide:

1. Server-Side Configuration (Recommended)

Maps are configured once on the server and reused for all guide instances.

Advantages:

• Simplest client code
• Consistent map selection across all clients
• No per-request overhead
• Ideal for production deployments

Usage:

# One-time setup
client.set_config(map_paths=["/data/maps/map1.r3db", "/data/maps/map2.3tz"])

# All subsequent initializations use these maps
client.initialize_guide(image_width=1920, image_height=1080)

2. Per-Request Map Paths (Backward Compatibility)

Clients specify absolute paths to maps in each initialization request.

Advantages:

• Flexible per-request map selection
• Backward compatible with older clients
• Useful for testing different map combinations

Usage:

client.initialize_guide(
    image_width=1920,
    image_height=1080,
    map_paths=["/data/maps/map1.r3db"]
)

3. Per-Request Map IDs (Convenient)

Clients use map IDs discovered via query_available_maps() to select maps.

Advantages:

• No need to know absolute paths
• Convenient for interactive applications
• Server-side resolution of IDs to paths (language-agnostic)
• Ideal for CLI tools and exploratory workflows

Usage:

# Discover available maps
maps_info = client.query_available_maps()

# Extract map IDs and use them
map_ids = ["map_001", "map_002"]
client.initialize_guide(
    image_width=1920,
    image_height=1080,
    map_ids=map_ids
)

Precedence and Fallback

When initializing a guide:

1. If map_ids are provided, they are resolved to absolute paths server-side
2. If map_paths are provided, they are used directly
3. If neither are provided, the server uses maps from its base configuration
4. If no maps are available from any source, initialization fails with an error

Coordinate Epoch Configuration

The coordinate epoch is used for tectonic plate correction to improve accuracy when the mission epoch differs from the reference map epoch. Uses GSRM2.1 model for tectonic plate movement.

note

Tectonic plate shift is typically only a few centimeters per year in most locations, so coordinate epoch can be omitted for many applications unless high precision is required.

Server-Side Default (Recommended: Omit)

Recommended approach: Leave coordinate_epoch unset in server configuration. This allows clients to specify mission-specific epochs per-request:

client.set_config(
    geo_resource_path="/path/to/geo",
    reference_frame="ECEF"
    # coordinate_epoch omitted - clients will specify per-request if needed
)

# Clients specify coordinate_epoch per-request for their mission
client.initialize_guide(image_width=1920, image_height=1080, coordinate_epoch=2025.5)

Optional: If you need a server-wide default, you can set it:

client.set_config(
    geo_resource_path="/path/to/geo",
    reference_frame="ECEF",
    coordinate_epoch=2020.0  # Server default
)

# Clients can override per-request for their mission epoch
client.initialize_guide(image_width=1920, image_height=1080, coordinate_epoch=2025.5)

Per-Request Override (Recommended for High-Precision)

For high-precision applications, specify the mission-specific epoch when initializing a guide:

from datetime import datetime

# High-precision real-time mission: use today's date
today = datetime.now().year + (datetime.now().timetuple().tm_yday - 1) / 365.25
client.initialize_guide(
    image_width=1920,
    image_height=1080,
    coordinate_epoch=today  # Current date for high-precision accuracy
)

# High-precision post-processing: use recording date
recording_date = 2024.5  # June 1, 2024
client.initialize_guide(
    image_width=1920,
    image_height=1080,
    coordinate_epoch=recording_date  # Recording date for historical data
)

# For most applications, coordinate_epoch can be omitted
client.initialize_guide(
    image_width=1920,
    image_height=1080
    # coordinate_epoch not specified - no tectonic correction applied
)

Limitations of Tectonic Plate Shift Correction

The coordinate epoch correction models smooth, long-term tectonic plate movement. It does NOT:

• Model deformation in zones along tectonic plate borders (where complex, non-linear deformation occurs)
• Compensate for sudden shifts such as earthquakes or landslides
• Account for local subsidence or uplift from other geological processes

For applications in tectonically active regions or areas with known deformation, additional validation or local ground control points may be necessary to achieve the required accuracy.

Best Practices

Connection Management

1. Use timeouts: Set receive/send timeouts to prevent indefinite blocking
2. Test connectivity: Use GetStatus to verify server is responsive before initialization
3. Clean up resources: Always destroy guides and disconnect (use context manager pattern)

Image Preparation

1. Grayscale conversion: Ensure images are 8-bit single-channel grayscale
2. Consistent dimensions: All images must match the dimensions specified in initialize_guide()
3. Accurate FOV: Provide accurate field of view values matching your camera calibration

Quality Assessment and Performance

For detailed guidance on:

• Quality metrics (confidence recommended, FOM deprecated) and recommended thresholds
• Pose covariance usage
• Performance tuning (detail levels, max_time_ms)
• Reference frame selection (ECEF vs GEODETIC)

See: Integration guide

Troubleshooting

Connection Issues

Problem: ConnectionError or TimeoutError

Solutions:

1. Verify server is running: ps aux | grep raptor_guide_server
2. Check server address and port match client configuration
3. Test network connectivity: telnet localhost 5555
4. Increase client timeout if needed: GuideClient(timeout=120000)
5. Check firewall rules

Initialization Failures

Common causes and solutions:

Error	Cause	Solution
"File not found"	Invalid map paths	Use absolute paths accessible by server
"Geo resource path not found"	Missing geo resources	Verify directory exists and is readable
"Out of memory"	Insufficient memory	Reduce maps
"License validation failed"	License issues	Check with client.get_status()

Pose Update Failures

For detailed troubleshooting of:

• Low confidence scores
• PoseOutput.result = FAILED
• Poor matching performance
• Image quality issues

See: Integration guide (troubleshooting section)

References

Documentation

• Protocol Specification: guide.proto - Complete protobuf message definitions
• SDK Integration Guide: Integration guide - Quality metrics, confidence thresholds, performance tuning, troubleshooting
• Coordinate Conventions: Conventions - Reference frames, position/attitude formats, quaternion conventions
• License Management: License management guide - Fingerprint generation, license installation/replacement, troubleshooting

Example Code

• Python Client (Complete): example_basic_usage.py