An OptimumT case study in curiosity-driven artificial intelligence

Coordinating autonomous aircraft requires continuous decisions in a changing environment. A moving target changes direction. Tracking aircraft must adjust their flight controls. The learning system must turn those interactions into increasingly effective behaviour.

This project brought those capabilities together in a configurable simulation testbed. Combining curiosity-driven reinforcement learning with aircraft flight dynamics, it demonstrated two autonomous UAVs tracking a third aircraft along a changing trajectory.

The work forms part of the research experience behind OptimumT’s expertise in reinforcement learning, autonomous systems and multi-agent coordination.

The challenge: learning when feedback arrives late

Reinforcement learning relies on feedback: an agent takes an action, receives a reward and adjusts its behaviour. In UAV target tracking, the value of an action may become apparent only after several subsequent manoeuvres.

That delay makes it difficult to connect an individual control decision with a successful outcome. A tracking agent needs a way to keep learning throughout the pursuit, including periods when task-specific feedback is sparse.

The project addressed this challenge by incorporating an additional learning signal: intrinsic curiosity.

The solution: make unexpected outcomes a reason to learn

The Intrinsic Curiosity Module predicts what will happen when an agent takes an action. It then compares that prediction with the observed result. A larger discrepancy produces a stronger internal reward, encouraging the agent to explore situations it has yet to understand.

Combined with the external tracking reward, this creates two complementary incentives: pursue the mission objective and learn more about the environment.

The testbed connects three capabilities.

A moving target controlled by AI. An Advantage Actor-Critic model, or A2C, guides the target aircraft. Its learned policy generates a changing trajectory, creating an active tracking challenge.

Tracking agents that share learning. Two UAVs use Asynchronous Advantage Actor-Critic, or A3C, enhanced with the curiosity module. Each agent contributes updates from its experience to a shared model, allowing learning to progress across multiple agents.

Learning settings that adapt during training. A Self-Reflective Curiosity-Weighted tuning mechanism monitors reward trends, policy randomness and training stability. It adjusts settings such as curiosity strength, exploration and learning rates to balance the search for new behaviours with the use of established strategies.

Together, these components connect individual exploration with coordinated control.

The engineering: closing the loop between AI and flight

Making the architecture work required reliable communication between the learning algorithms and the flight simulation.

The project integrated FlightGear with the JSBSim Flight Dynamics Model. FlightGear supplied the visual environment and multiplayer functionality; JSBSim modelled aircraft behaviour, including aerodynamics, propulsion and control surfaces.

A communication interface continuously exchanged control commands and flight-state information. The agents issued throttle, rudder, elevator and aileron commands, then received updated information about aircraft position, speed and orientation.

This continuous loop connected AI decisions to simulated aircraft motion. It also made the system observable: live flight views and three-dimensional trajectory plots showed how the target and tracking aircraft moved through the shared environment.

The results: coordinated tracking in action

The completed testbed demonstrated one target aircraft and two autonomous trackers operating together in real time.

As the target moved through straight flight, turns and changes in direction, the tracking UAVs adjusted their behaviour to follow its trajectory. Flight visualisations and trajectory plots captured the aircraft’s responses and their spatial relationships throughout the demonstration.

The reported training results showed cumulative reward rising from approximately −110 to 300 over roughly 900,000 iterations. Policy entropy—the measure of randomness in action selection—declined as training progressed, showing a transition toward more consistent decisions.

The project delivered an integrated environment for curiosity-driven learning, adaptive tuning and coordinated UAV control, with the tools to observe both training progress and flight behaviour.

The OptimumT connection

Direct involvement in this research informs OptimumT’s approach to applied AI: connecting learning algorithms, domain-specific simulation and system engineering around a concrete operational challenge.

The project demonstrates the value of that combination. An idea from reinforcement learning became a working environment in which multiple autonomous agents could explore, share experience and respond to a moving target. That experience underpins the expertise OptimumT brings to intelligent systems that must make decisions as conditions change.

Research provenance: Conducted at Dundalk Institute of Technology, with funding from Ireland’s Higher Education Authority through the Technological University Transformation Fund.

Adapted from Jawad Mahmood, Muhammad Adil Raja, John Loane and Fergal McCaffery, “A Configurable Intrinsic Curiosity Module for a Testbed for Developing Intelligent Swarm UAVs,” Machine Learning with Applications, 21 (2025), 100714. © 2025 The Authors. Published by Elsevier Ltd. Licensed under CC BY 4.0. The research has been condensed and rewritten for this case study.