PX4 User Guide
  • Introduction
  • Getting Started
    • Basic Concepts
    • Vehicles/Frames
    • Flight Controllers
    • Sensors
    • Radio Systems
    • Flight Modes
    • Vehicle Status Notifications
      • LED Meanings
      • Tune/Sound Meanings
      • Preflight Checks
    • Payloads & Cameras
    • Flight Reporting
  • Basic Assembly
    • Mounting the Flight Controller
    • Mounting the GPS/Compass
    • Vibration Isolation
    • Cable Wiring
    • CUAV Pixhawk V6X Wiring QuickStart
    • CUAV V5+ Wiring Quickstart
    • CUAV V5 nano Wiring Quickstart
    • Holybro Pixhawk 6C Wiring Quickstart
    • Holybro Pixhawk 6X Wiring Quickstart
    • Holybro Pixhawk 5X Wiring Quickstart
    • Holybro Pixhawk 4 Wiring Quickstart - Discontinued
    • Holybro Pixhawk 4 Mini Wiring Quickstart - Discontinued
    • Holybro Durandal Wiring Quickstart
    • Holybro Pix32 v5 Wiring Quickstart
    • Cube Wiring Quickstart
    • Pixracer Wiring Quickstart
    • mRo (3DR) Pixhawk Wiring Quickstart
  • Standard Configuration
    • Firmware
    • Airframe
    • Sensor Orientation
    • Compass
    • Gyroscope
    • Accelerometer
    • Airspeed
    • Level Horizon Calibration
    • Radio Setup
    • Joystick Setup
    • Flight Modes
    • Battery
    • Safety
      • Failsafe Simulation
    • ESC Calibration
    • Actuators
    • Autotune
  • Vehicle Types & Setup
    • Multicopters
      • Multicopter Config/Tuning
        • MC Filter/Control Latency Tuning
        • MC PID Tuning (Manual/Basic)
        • MC PID Tuning Guide (Manual/Advanced)
        • MC Setpoint Tuning (Trajectory Generator)
          • MC Jerk-limited Type Trajectory
        • Multicopter Racer Setup
      • X500 v2 (Pixhawk 6C)
      • X500 v2 (Pixhawk 5X)
      • X500 (Pixhawk 4)
      • S500 V2 (Pixhawk 4)
      • DJI F450 (CUAV v5+)
      • DJI F450 (CUAV v5 nano)
      • QAV250 (Pixhawk4 Mini) - Discontinued
      • DJI F450 + RTK (Pixhawk 3 Pro)
      • QAV250 (Pixhawk Mini)
      • QAV-R 5" Racer (Pixracer)
      • Omnicopter
    • Planes
      • Fixed Wing Config/Tuning
        • Fixedwing PID Tuning Guide
        • Fixedwing Advanced Tuning Guide
        • Fixedwing Trimming Guide
      • Reptile Dragon 2 (ARK6X)
      • Turbo Timber Evolution (Pixhawk 4 Mini)
      • Wing Wing Z84 (Pixracer)
    • VTOL
      • VTOL Config/Tuning
        • QuadPlane Configuration
        • Back-transition Tuning
        • VTOL w/o Airspeed Sensor
        • VTOL Weather Vane
      • Standard VTOL
        • FunCub QuadPlane (Pixhawk)
        • Ranger QuadPlane (Pixhawk)
        • Falcon Vertigo QuadPlane (Dropix)
      • Tailsitter VTOL
        • Build: TBS Caipiroshka Tailsitter Build (Pixracer)
      • Tiltrotor VTOL
        • Build: Convergence Tiltrotor (Pixfalcon)
    • Airships (experimental)
    • Autogyros (experimental)
      • ThunderFly Auto-G2 (Holybro pix32)
    • Balloons (experimental)
    • Helicopter (experimental)
      • Helicopter Config/Tuning
    • Rovers (experimental)
      • Traxxas Stampede
    • Submarines (experimental)
      • BlueROV2
    • Airframes Reference
  • Flying
    • First Flight Guidelines
    • Flying 101
    • Missions
      • Package Delivery Mission
    • GeoFence
    • Safety Point Planning
    • Flight Modes
      • Position Mode (MC)
      • Altitude Mode (MC)
      • Manual/Stabilized Mode (MC)
      • Acro Mode (MC)
      • Orbit Mode (MC)
      • Position Mode (FW)
      • Altitude Mode (FW)
      • Stabilized Mode (FW)
      • Acro Mode (FW)
      • Manual Mode (FW)
      • Takeoff Mode
      • Land Mode
      • Return Mode
      • Hold Mode
      • Mission Mode
      • Follow Me Mode
      • Offboard Mode
    • Terrain Following/Holding
  • Flight Log Analysis
    • Log Analysis using Flight Review
    • Log Analysis using PlotJuggler
  • Advanced Configuration
    • Finding/Updating Parameters
    • Full Parameter Reference
    • ECL/EKF Overview & Tuning
    • Flight Termination Configuration
    • Bootloader Flashing onto Betaflight Systems
    • Land Detector Configuration
    • Prearm/Arm/Disarm Configuration
    • IMU Factory Calibration
    • Sensor Thermal Compensation
    • Compass Power Compensation
    • Advanced Controller Orientation
    • Static Pressure Buildup
    • Serial Port Configuration
    • MAVLink Telemetry (OSD/GCS)
    • PX4 Ethernet Setup
    • Bootloader Update
  • Hardware (Drones&Parts)
    • Complete Vehicles
      • ModalAI Starling
      • PX4 Vision Kit
      • MindRacer BNF & RTF
        • MindRacer 210
        • NanoMind 110
      • Holybro Kopis 2
      • Bitcraze Crazyflie 2.1
    • Flight Controllers (Autopilots)
      • Pixhawk Series
        • Silicon Errata
      • Pixhawk Standard Autopilots
        • CUAV Pixhawk V6X (FMUv6X)
        • Holybro Pixhawk 6X (FMUv6X)
        • Holybro Pixhawk 6C (FMUv6C)
        • Holybro Pixhawk 6C Mini(FMUv6C)
        • Holybro Pix32 v6 (FMUv6C)
        • Holybro Pixhawk 5X (FMUv5X)
        • Holybro Pixhawk 4 (FMUv5) - Discontinued
        • Holybro Pixhawk 4 Mini (FMUv5) - Discontinued
        • Drotek Pixhawk 3 Pro (FMUv4pro)
        • mRo Pixracer (FMUv4)
        • Hex Cube Black (FMUv3)
        • mRo Pixhawk (FMUv3)
        • Holybro Pixhawk Mini (FMUv3) - Discontinued
      • Manufacturer-Supported Autopilots
        • AirMind MindPX
        • AirMind MindRacer
        • ARK Electronics ARKV6X
        • CUAV X7
        • CUAV Nora
        • CUAV V5+ (FMUv5)
        • CUAV V5 nano (FMUv5)
        • CUAV Pixhack v3 (FMUv3)
        • CubePilot Cube Orange+ (CubePilot)
        • CubePilot Cube Orange (CubePilot)
        • CubePilot Cube Yellow (CubePilot)
        • Holybro Kakute H7v2
        • Holybro Kakute H7mini
        • Holybro Kakute H7
        • Holybro Durandal
        • Holybro Pix32 v5
        • ModalAI Flight Core v1
        • ModalAI VOXL Flight
        • ModalAI VOXL 2
        • mRobotics-X2.1 (FMUv2)
        • mRo Control Zero F7)
        • NXP RDDRONE-FMUK66 FMU
        • Sky-Drones AIRLink
        • SPRacing SPRacingH7EXTREME
        • ThePeach FCC-K1
        • ThePeach FCC-R1
      • Experimental Autopilots
        • BeagleBone Blue
        • Raspberry Pi 2/3 Navio2
        • Raspberry Pi 2/3/4 PilotPi
          • PilotPi with Raspberry Pi OS
          • PilotPi with Ubuntu Server
      • Discontinued Autopilots/Vehicles
        • Drotek Dropix (FMUv2)
        • Omnibus F4 SD
        • BetaFPV Beta75X 2S Brushless Whoop
        • Bitcraze Crazyflie 2.0
        • Aerotenna OcPoC-Zynq Mini
        • CUAV v5
        • Holybro Kakute F7 (Discontinued)
        • Holybro Pixfalcon
        • Holybro pix32 (FMUv2)
        • mRo AUAV-X2
        • 3DR Pixhawk 1
        • Snapdragon Flight
        • Intel® Aero RTF Drone (Discontinued)
      • Pixhawk Autopilot Bus (PAB) & Carriers
        • ARK Electronics Pixhawk Autopilot Bus Carrier
    • Flight Controller Peripherals
      • ADSB/FLARM (Traffic Avoidance)
      • Air Traffic Avoidance: ADSB/FLARM
      • Air Traffic Avoidance: UTM
      • Airspeed Sensors
        • TFSlot Airspeed Sensor
      • Barometers
      • Camera
      • Distance Sensors (Rangefinders)
        • Lightware SFxx Lidar
        • Ainstein US-D1 Standard Radar Altimeter
        • LeddarOne Lidar
        • Benewake TFmini Lidar
        • Lidar-Lite
        • TeraRanger
        • Lanbao PSK-CM8JL65-CC5
        • Avionics Anonymous Laser Altimeter UAVCAN Interface
      • ESCs & Motors
        • PWM ESCs and Servos
        • DShot ESCs
        • OneShot ESCs and Servos
        • DroneCAN ESCs
          • Zubax Telega
          • PX4 Sapog ESC Firmware
            • Holybro Kotleta
            • Zubax Orel
        • VESC
      • TBS Crossfire (CRSF) Telemetry
      • FrSky Telemetry
      • Gimbal (Mount) Configuration
      • GPS/Compass
        • ARK GPS
        • Holybro DroneCAN M8N GPS
        • LOCOSYS Hawk A1 GNSS
        • Hex Here2
        • Holybro M8N & M9N GPS
        • Sky-Drones SmartAP GPS
      • Grippers
        • Servo Gripper
      • Optical Flow
        • ARK Flow
        • PMW3901
        • PX4FLOW (Deprecated)
      • Precision Landing
      • Parachute
      • Power Modules/PDB
        • CUAV HV pm
        • CUAV CAN PMU
        • Holybro PM02
        • Holybro PM07
        • Holybro PM06 V2
        • Holybro PM02D (digital)
        • Holybro PM03D (digital)
        • Pomegranate Systems Power Module
        • Sky-Drones SmartAP PDB
      • Satellite Coms (Iridium/RockBlock)
      • Telemetry Radios
        • SiK Radio
          • RFD900 (SiK) Telemetry Radio
          • HolyBro (SIK) Telemetry Radio
        • Telemetry Wifi
          • ESP8266 WiFi Module
          • ESP32 WiFi Module
          • 3DR Telemetry Wifi (Discontinued)
        • Microhard Serial Telemetry Radio
          • ARK Electron Microhard Serial Telemetry Radio
          • Holybro Microhard P900 Telemetry Radio
        • CUAV P8 Telemetry Radio
        • HolyBro XBP9X - Discontinued
      • RTK GPS
        • ARK RTK GPS
        • RTK GPS Heading with Dual u-blox F9P
        • CUAV C-RTK
        • CUAV C-RTK2 PPK/RTK GNSS
        • CUAV C-RTK 9Ps
        • Femtones MINI2 Receiver
        • Freefly RTK GPS
        • Holybro H-RTK-F9P
        • Holybro H-RTK-M8P
        • Holybro H-RTK Unicore UM982 GPS
        • Locosys Hawk R1
        • Locosys Hawk R2
        • Septentrio AsteRx-RIB
        • Septentrio mosaic-go
        • Trimble MB-Two
        • CubePilot Here+ (Discontined)
      • Remote ID
      • Smart Batteries
        • Rotoye Batmon Battery Smartification Kit
      • Tachometers (Revolution Counters)
        • ThunderFly TFRPM01 Tachometer Sensor
      • I2C Peripherals
        • I2C bus accelerators
        • TFI2CADT01 I2C address translator
      • CAN Peripherals
      • DroneCAN Peripherals
        • PX4 DroneCAN Firmware
        • ARK CANnode
    • Companion Computers
      • Pixhawk + Companion Setup
        • RasPi Pixhawk Companion
      • Companion Computer Peripherals
      • Holybro Pixhawk RPI CM4 Baseboard
      • Auterion Skynode
      • Computer Vision
        • Obstacle Avoidance
        • Safe Landing
        • Collision Prevention
        • Path Planning Interface
        • Motion Capture (MoCap)
        • Visual Inertial Odometry (VIO)
          • Realsense T265 Tracking Camera (VIO)
      • Video Streaming
  • Development
    • Getting Started
      • Recommended Hardware/Setup
      • Toolchain Installation
        • MacOS Setup
        • Ubuntu Setup
        • Windows Setup
        • Visual Studio Code IDE
        • Other/Generic Tools
      • Building the Code
      • Writing your First Application
      • Application/Module Template
    • Concepts
      • PX4 Architecture
      • PX4 Flight Stack Architecture
        • Controller Diagrams
      • Events Interface
      • Flight Modes
      • Flight Tasks
      • Control Allocation
      • PWM limit state machine
      • System Startup
      • SD Card Layout
    • Simulation
      • jMAVSim Simulation
        • Multi-Vehicle Sim with JMAVSim
      • Gazebo Simulation
        • Vehicles
        • Multi-Vehicle Sim
      • Gazebo Classic Simulation
        • Vehicles
        • Worlds
        • Multi-Vehicle Sim
      • FlightGear Simulation
        • FlightGear Vehicles
        • Multi-Vehicle Sim with FlightGear
      • JSBSim Simulation
      • AirSim Simulation
      • Multi-Vehicle Simulation
      • Simulate Failsafes
      • HITL Simulation
      • Simulation-In-Hardware
    • Hardware
      • Flight Controller Reference Design
      • Manufacturer’s Board Support Guide
      • Flight Controller Porting Guide
        • PX4 Board Configuration (kconfig)
        • NuttX Board Porting Guide
      • Serial Port Mapping
      • Airframes
        • Adding a New Airframe
      • Device Drivers
      • Telemetry Radio
        • SiK Radio
      • Sensor and Actuator I/O
        • DroneCAN
        • I2C Bus
        • UART/Serial Ports
          • Port-Configurable Serial Drivers
      • RTK GPS (Integration)
    • Middleware
      • uORB Messaging
      • uORB Graph
      • uORB Message Reference
        • ActionRequest
        • ActuatorArmed
        • ActuatorControlsStatus
        • ActuatorMotors
        • ActuatorOutputs
        • ActuatorServos
        • ActuatorServosTrim
        • ActuatorTest
        • AdcReport
        • Airspeed
        • AirspeedValidated
        • AirspeedWind
        • AutotuneAttitudeControlStatus
        • BatteryStatus
        • ButtonEvent
        • CameraCapture
        • CameraStatus
        • CameraTrigger
        • CellularStatus
        • CollisionConstraints
        • CollisionReport
        • ControlAllocatorStatus
        • Cpuload
        • DebugArray
        • DebugKeyValue
        • DebugValue
        • DebugVect
        • DifferentialPressure
        • DistanceSensor
        • Ekf2Timestamps
        • EscReport
        • EscStatus
        • EstimatorAidSource1d
        • EstimatorAidSource2d
        • EstimatorAidSource3d
        • EstimatorBias
        • EstimatorBias3d
        • EstimatorEventFlags
        • EstimatorGpsStatus
        • EstimatorInnovations
        • EstimatorSelectorStatus
        • EstimatorSensorBias
        • EstimatorStates
        • EstimatorStatus
        • EstimatorStatusFlags
        • Event
        • FailsafeFlags
        • FailureDetectorStatus
        • FollowTarget
        • FollowTargetEstimator
        • FollowTargetStatus
        • GeneratorStatus
        • GeofenceResult
        • GimbalControls
        • GimbalDeviceAttitudeStatus
        • GimbalDeviceInformation
        • GimbalDeviceSetAttitude
        • GimbalManagerInformation
        • GimbalManagerSetAttitude
        • GimbalManagerSetManualControl
        • GimbalManagerStatus
        • GpioConfig
        • GpioIn
        • GpioOut
        • GpioRequest
        • GpsDump
        • GpsInjectData
        • Gripper
        • HealthReport
        • HeaterStatus
        • HomePosition
        • HoverThrustEstimate
        • InputRc
        • InternalCombustionEngineStatus
        • IridiumsbdStatus
        • IrlockReport
        • LandingGear
        • LandingGearWheel
        • LandingTargetInnovations
        • LandingTargetPose
        • LaunchDetectionStatus
        • LedControl
        • LogMessage
        • LoggerStatus
        • MagWorkerData
        • MagnetometerBiasEstimate
        • ManualControlSetpoint
        • ManualControlSwitches
        • MavlinkLog
        • MavlinkTunnel
        • Mission
        • MissionResult
        • ModeCompleted
        • MountOrientation
        • NavigatorMissionItem
        • NormalizedUnsignedSetpoint
        • NpfgStatus
        • ObstacleDistance
        • OffboardControlMode
        • OnboardComputerStatus
        • OrbTest
        • OrbTestLarge
        • OrbTestMedium
        • OrbitStatus
        • ParameterUpdate
        • Ping
        • PositionControllerLandingStatus
        • PositionControllerStatus
        • PositionSetpoint
        • PositionSetpointTriplet
        • PowerButtonState
        • PowerMonitor
        • PpsCapture
        • PwmInput
        • Px4ioStatus
        • QshellReq
        • QshellRetval
        • RadioStatus
        • RateCtrlStatus
        • RcChannels
        • RcParameterMap
        • Rpm
        • RtlTimeEstimate
        • SatelliteInfo
        • SensorAccel
        • SensorAccelFifo
        • SensorBaro
        • SensorCombined
        • SensorCorrection
        • SensorGnssRelative
        • SensorGps
        • SensorGyro
        • SensorGyroFft
        • SensorGyroFifo
        • SensorHygrometer
        • SensorMag
        • SensorOpticalFlow
        • SensorPreflightMag
        • SensorUwb
        • SensorSelection
        • SensorsStatus
        • SensorsStatusImu
        • SystemPower
        • TakeoffStatus
        • TaskStackInfo
        • TecsStatus
        • TelemetryStatus
        • TiltrotorExtraControls
        • TimesyncStatus
        • TrajectoryBezier
        • TrajectorySetpoint
        • TrajectoryWaypoint
        • TransponderReport
        • TuneControl
        • UavcanParameterRequest
        • UavcanParameterValue
        • UlogStream
        • UlogStreamAck
        • UwbDistance
        • UwbGrid
        • VehicleAcceleration
        • VehicleAirData
        • VehicleAngularAccelerationSetpoint
        • VehicleAngularVelocity
        • VehicleAttitude
        • VehicleAttitudeSetpoint
        • VehicleCommand
        • VehicleCommandAck
        • VehicleConstraints
        • VehicleControlMode
        • VehicleGlobalPosition
        • VehicleImu
        • VehicleImuStatus
        • VehicleLandDetected
        • VehicleLocalPosition
        • VehicleLocalPositionSetpoint
        • VehicleMagnetometer
        • VehicleOdometry
        • VehicleOpticalFlow
        • VehicleOpticalFlowVel
        • VehicleRatesSetpoint
        • VehicleRoi
        • VehicleStatus
        • VehicleThrustSetpoint
        • VehicleTorqueSetpoint
        • VehicleTrajectoryBezier
        • VehicleTrajectoryWaypoint
        • VtolVehicleStatus
        • Wind
        • YawEstimatorStatus
      • MAVLink Messaging
      • uXRCE-DDS (PX4-ROS 2/DDS Bridge)
    • Modules & Commands
      • Autotune
      • Commands
      • Communication
      • Controllers
      • Drivers
        • Airspeed Sensor
        • Baro
        • Distance Sensor
        • IMU
        • INS
        • Magnetometer
        • Optical Flow
        • Rpm Sensor
        • Transponder
      • Estimators
      • Simulations
      • System
      • Template
    • Debugging/Logging
      • FAQ
      • Consoles/Shells
        • MAVLink Shell
        • System Console
      • Debugging with GDB
        • SWD Debug Port
        • JLink Probe
        • Black Magic/DroneCode Probe
        • STLink Probe
        • Hardfault Debugging
      • Debugging with Eclipse
      • Failure Injection
      • Sensor/Topic Debugging
      • Simulation Debugging
      • Sending Debug Values
      • System-wide Replay
      • Profiling
      • Binary Size Profiling
      • Logging
      • Flight Log Analysis
      • ULog File Format
    • Tutorials
      • Long-distance Video Streaming
      • Connecting an RC Receiver on Linux
    • Advanced Topics
      • Parameters & Configs
      • Package Delivery Architecture
      • Computer Vision
        • Motion Capture (VICON, Optitrack, NOKOV)
      • Installing driver for Intel RealSense R200
      • Switching State Estimators
      • Out-of-Tree Modules
      • STM32 Bootloader
      • System Tunes
      • Advanced Linux Installation Cases
      • Windows Cygwin Toolchain Maintenance
      • Unsupported Developer Setup
        • CentOS Linux
        • Arch Linux
        • Windows VM Toolchain
        • Windows Cygwin Toolchain
        • Qt Creator IDE
    • Platform Testing and CI
      • Test Flights
        • Test MC_01 - Manual Modes
        • Test MC_02 - Full Autonomous
        • Test MC_03 - Auto Manual Mix
        • Test MC_04 - Failsafe Testing
        • Test MC_05 - Indoor Flight (Manual Modes)
      • Unit Tests
      • Continuous Integration
      • MAVSDK Integration Testing
      • ROS Integration Testing
      • Docker Containers
      • Maintenance
  • Drone Apps & APIs
    • Offboard Control from Linux
    • ROS
      • ROS 2
        • ROS 2 User Guide
        • ROS 2 Offboard Control Example
        • ROS 2 Multi Vehicle Simulation
      • ROS 1 with MAVROS
        • ROS/MAVROS Installation Guide
        • ROS/MAVROS Offboard Example (C++)
        • ROS/MAVROS Offboard Example (Python)
        • ROS/MAVROS Sending Custom Messages
        • ROS/MAVROS with Gazebo Classic Simulation
        • Gazebo Classic OctoMap Models with ROS 1
        • ROS/MAVROS Installation on RPi
        • External Position Estimation (Vision/Motion based)
    • DroneKit
  • Contribution (&Dev Call)
    • Dev Call
    • Support
    • Source Code Management
      • GIT Examples
    • Documentation
    • Translation
    • Terminology/Notation
    • Licenses
  • Releases
    • 1.14
    • 1.13
    • 1.12
Powered by GitBook
On this page
  • Suggested Setup
  • Camera Mounting
  • Companion Setup
  • ROS VIO node
  • PX4 Tuning
  • Check/Verify VIO Estimate
  • Troubleshooting
  • Developer Information
  • Further Information
  1. Hardware (Drones&Parts)
  2. Companion Computers
  3. Computer Vision

Visual Inertial Odometry (VIO)

PreviousMotion Capture (MoCap)NextRealsense T265 Tracking Camera (VIO)

Last updated 1 year ago

Visual Inertial Odometry (VIO) is a technique used for estimating the 3D pose (local position and orientation) and velocity of a moving vehicle relative to a local starting position. It is commonly used to navigate a vehicle in situations where GPS is absent or unreliable (e.g. indoors, or when flying under a bridge).

VIO uses to estimate vehicle pose from camera images, combined with inertial measurements from the vehicle IMU (to correct for errors associated with rapid vehicle movement resulting in poor image capture).

This topic gives guidance on configuring PX4 and a companion computer for a VIO setup.

:::note The suggested setup uses ROS for routing VIO information to PX4. However, PX4 itself does not care about the source of messages, provided they are provided via the appropriate . :::

Suggested Setup

A hardware and software setup for VIO is suggested in the sections below as an illustration of how to interface a VIO system with PX4. It makes use of an off-the-shelf tracking camera and a companion computer running ROS. ROS is used to read odometry information from the camera and supply it to PX4.

An example of a suitable tracking camera is the .

Camera Mounting

Attach the camera to the companion computer and mount it to the frame:

  • Mount the camera with lenses pointing down if at all possible (default).

  • Cameras are typically very sensitive to vibration; a soft mounting is recommended (e.g. using vibration isolation foam).

Companion Setup

To setup ROS and PX4:

  • On the companion computer, install and configure .

  • Implement and run a ROS node to read data from the camera and publish the VIO odometry using MAVROS.

    • See the section below for details of the requirements for this node.

  • Follow the instructions for tuning the PX4 EKF2 estimator.

  • Verify the connection to the flight controller.

    :::tip You can use the QGroundControl to verify that you're getting ODOMETRY or VISION_POSITION_ESTIMATE messages (or check for HEARTBEAT messages that have the component id 197 (MAV_COMP_ID_VISUAL_INERTIAL_ODOMETRY)). :::

  • before your first flight!

ROS VIO node

In this suggested setup, a ROS node is required to

  1. interface with the chosen camera or sensor hardware,

  2. produce odometry messages containing the position estimate, which will be sent to PX4 using MAVROS, and

  3. publish messages to indicate the VIO system status.

The implementation of the ROS node will be specific to the camera used and will need to be developed to use the interface and drivers appropriate for the camera.

  • MAV_STATE_ACTIVE when the VIO system is functioning as expected,

  • MAV_STATE_CRITICAL when the VIO system is functioning, but with low confidence, and

  • MAV_STATE_FLIGHT_TERMINATION when the system has failed or the estimate confidence is unacceptably low.

PX4 Tuning

The following parameters must be set to use external position information with EKF2.

Parameter
Setting for External Position Estimation

Set horizontal position fusion, vertical vision fusion, velocity fusion, and yaw fusion according to your desired fusion model.

Set to Vision to use the vision as the reference sensor for altitude estimation.

Set the position of the vision sensor with respect to the vehicle's body frame.

These can be set in QGroundControl > Vehicle Setup > Parameters > EKF2 (remember to reboot the flight controller in order for parameter changes to take effect).

Tuning EKF2_EV_DELAY

Technically this can be set to 0 if there is correct timestamping (not just arrival time) and timesync (e.g. NTP) between MoCap and (for example) ROS computers. In reality, this may need some empirical tuning because delays in the communication chain are very setup-specific. It is rare that a system is set up with an entirely synchronised chain!

A rough estimate of the delay can be obtained from logs by checking the offset between IMU rates and the EV rates:

The value can further be tuned by varying the parameter to find the value that yields the lowest EKF innovations during dynamic maneuvers.

Check/Verify VIO Estimate

Perform the following checks to verify that VIO is working properly before your first flight:

  • Yaw the vehicle until the quaternion of the ODOMETRY message is very close to a unit quaternion (w=1, x=y=z=0).

    • At this point, the body frame is aligned with the reference frame of the external pose system.

    • If you do not manage to get a quaternion close to the unit quaternion without rolling or pitching your vehicle, your frame probably still has a pitch or roll offset. Do not proceed if this is the case and check your coordinate frames again.

  • Once aligned, you can pick the vehicle up from the ground and you should see the position's z coordinate decrease. Moving the vehicle in the forward direction should increase the position's x coordinate. Moving the vehicle to the right should increase the y coordinate.

  • Check that linear velocities in the message are expressed in the FRD body frame reference frame.

  • Set the PX4 parameter MAV_ODOM_LP back to 0. PX4 will stop streaming the ODOMETRY message back.

If those steps are consistent, you can try your first flight:

  1. Put the vehicle on the ground and start streaming ODOMETRY feedback (as above). Lower your throttle stick and arm the motors.

    At this point, with the left stick at the lowest position, switch to position control. You should have a green light. The green light tells you that position feedback is available and position control is now activated.

  2. Put the throttle stick in the middle (the dead zone) so that the vehicle maintains its altitude. Raising the stick will increase the reference altitude while lowering the value will decrease it. Similarly, the other stick will change the position over the ground.

  3. Increase the value of the throttle stick and the vehicle will take off. Move it back to the middle immediately afterwards.

  4. Confirm that the vehicle can hold its position.

Troubleshooting

First, make sure MAVROS is able to connect successfully to the flight controller.

If it is connecting properly common problems/solutions are:

  • Problem: I get drift / flyaways when the drone flies, but not when I carry it around with the props off.

  • Problem: I get toilet-bowling when VIO is enabled.

  • Problem: I want to use vision position to do loop closing, and also want to run GPS.

    • This is really difficult, because when they disagree it will confuse the EKF. From testing it is more reliable to just use vision velocity (if you figure out a way to make this configuration reliable, let us know).

Developer Information

This topic also explains how to configure VIO for use with the LPE Estimator (deprecated).

Further Information

The odometry messages should be of the type and published to the topic /mavros/odometry/out.

System status messages of the type should be published to the topic /mavros/companion_process/status. These should identify the component as MAV_COMP_ID_VISUAL_INERTIAL_ODOMETRY (197) and indicate the state of the system. Recommended status values are:

Set to the difference between the timestamp of the measurement and the "actual" capture time. For more information see .

, ,

For more detailed/additional information, see: .

is the Vision Position Estimator delay relative to IMU measurements. In other words, it is the difference between the vision system timestamp and the "actual" capture time that would have been recorded by the IMU clock (the "base clock" for EKF2).

:::note A plot of external data vs. onboard estimate (as above) can be generated using or similar flight analysis tools. :::

Set the PX4 parameter MAV_ODOM_LP to 1. PX4 will then stream back the received external pose as MAVLink messages. You can check these MAVLink messages with the QGroundControl

If using the try soft-mounting it (this camera is very sensitive to high-frequency vibrations).

Make sure the orientation of the camera matches the transform in the launch file. Use the QGroundControl to verify that the velocities in the ODOMETRY message coming from MAVROS are aligned to the FRD coordinate system.

Developers who are interested in extending this implementation (or writing a different one, which might not depend on ROS) should see .

nav_msgs/Odometry
mavros_msgs/CompanionProcessStatus
ODOMETRY
MAVLink Inspector
T265
MAVLink Inspector
Using Vision or Motion Capture Systems for Position Estimation
below
computer vision
Visual Odometry
Intel® RealSense™ Tracking Camera T265
MAVROS
MAVLink Inspector
VIO ROS node
below
Verify that VIO is set up correctly
MAVLink Interface
FlightPlot
ECL/EKF Overview & Tuning > External Vision System
ECL/EKF Overview & Tuning > External Vision System
ekf2_ev_delay log
EKF2_EV_DELAY
EKF2_EV_CTRL
EKF2_HGT_REF
EKF2_EV_DELAY
EKF2_EV_POS_X
EKF2_EV_POS_Y
EKF2_EV_POS_Z