Grasping Temperature: Thermal Feedback in VR Robot Teleoperation

This paper presents a proof-of-concept of a robotic teleoperation system, that provides the human operator a thermal sense in addition to the visual sense. With a sensor suite comprising a stereo camera, 360° camera and long-wave infra-red camera, our demonstrator pushes the boundaries of virtual-reality situational awareness by bringing not...

Meaningful human control of drones: exploring human–machine teaming, informed by four different ethical perspectives

A human-centric approach to the design and deployment of AI systems aims to support and augment human capabilities. This sounds worthwhile indeed. But what could this look like in a military context? We explored a human-centric approach to the design and deployment of highly autonomous, unarmed Unmanned Aerial Vehicle (UAV), or drone, and an...

Spectral Classification and Change Detection for Hyperspectral Imaging (V1717)

This paper shows the Netherlands contribution to NATO SET-240 on Exploitation of Longwave Infrared Airborne Hyperspectral Data, that focusses on the detection of soil disturbances, buried objects, and nonburied (surface) objects. Together with other hyperspectral feasibility research on maritime target detection and machine learning. Hyperbolic...

Onboard ROI selection for aerial surveillance using a high resolution, high framerate camera

With the latest advances in image sensor technology, cameras are able to generate video with tens of megapixels per frame. These high resolution videos streams offer great potential to be used in the surveillance domain. For ground based systems, gigapixel streams are already used with great effect as illustrated by the ICME 2019 crowd counting...

Spatiotemporal detection of maritime targets using neural networks

Automatic detection and tracking of maritime targets in imagery can greatly increase situation awareness on naval vessels. Various methods for detection and tracking have been proposed so far, both for reasoning as well as for learning approaches. Learning approaches have the promise to outperform reasoning approaches. They typically detect...

Evaluation of super-resolution algorithms for mosaic hyperspectral imagery

Hyperspectral imaging sensors acquire images in a large number of spectral bands, unlike traditional electro-optical and infrared sensors which sample only one or few bands. Hyperspectral mosaic sensors acquire an image of all spectral bands in one shot. Using a patterned array of spectral filters they measure different wavelength bands at...

Human-Robot Interaction During Virtual Reality Mediated Teleoperation: How Environment Information Affects Spatial Task Performance and Operator Situation Awareness

Virtual Reality (VR) mediated teleoperation is a relatively new field in robotics which means little is known about the Human-Robot Interaction (HRI). The objective of this study was to investigate the effects of environmental information presentation on Human-Robot Team (HRT) task performance and operator Situation Awareness (oSA). Method. The...

Vessel Track Summarization by Viewpoint Selection

For military operations, the availability of high-quality imaging information from Electro-Optical / Infrared (EO/IR) sensors is of vital importance. This information can be used for timely detection and identification of threatening vessels in an environment with a large amount of neutral vessels. EO/IR sensors provide imagery of all vessels at...

Virtual Reality Conferencing: Multi-user immersive VR experiences on the web

Virtual Reality (VR) and 360-degree video are set to become part of the future social environment, enriching and enhancing the way we share experiences and collaborate remotely. While Social VR applications are getting more momentum, most services regarding Social VR focus on animated avatars. In this demo, we present our efforts towards Social...

Maritime detection framework 2.0: A new approach of maritime target detection in electro-optical sensors

Detecting maritime targets with electro-optical sensors is an active area of research. One current trend is to automate target detection through image processing or computer vision. Automation of target detection will decrease the number of people required for lower-level tasks, which frees capacity for higher-level tasks. A second trend is that...

Incremental Learning-Based Adaptive Object Recognition for Mobile Robots

3D visual understanding of the surrounding environment is vital for successful mobile robotic tasks such as autonomous navigation or general object interaction. However, current systems have limited perceptual capabilities in the sense that they are not very well adaptable to unknown environments. Human operators, on the other hand, are experts...

Long-term behavior understanding based on the expert-based combination of short-term observations in high-resolution CCTV

The bottleneck in situation awareness is no longer in the sensing domain but rather in the data interpretation domain, since the number of sensors is rapidly increasing and it is not affordable to increase human data-analysis capacity at the same rate. Automatic image analysis can assist a human analyst by alerting when an event of interest...

Improvements in ship tracking in electro-optical and infrared data using appearance

Naval ships have camera systems available to assist in performing their operational tasks. Some include automatic detection and tracking, assisting an operator by keeping a ship in view or by keeping collected information about ships. Tracking errors limit the use of camera information. When keeping a ship in view, an operator has to re-Target a...