Showing 1–50 of 136 results for author: Hoffmann, M

Registration by Regression (RbR): a framework for interpretable and flexible atlas registration

Authors: Karthik Gopinath, Xiaoling Hu, Malte Hoffmann, Oula Puonti, Juan Eugenio Iglesias

Abstract: In human neuroimaging studies, atlas registration enables mapping MRI scans to a common coordinate frame, which is necessary to aggregate data from multiple subjects. Machine learning registration methods have achieved excellent speed and accuracy but lack interpretability. More recently, keypoint-based methods have been proposed to tackle this issue, but their accuracy is still subpar, particular… ▽ More In human neuroimaging studies, atlas registration enables mapping MRI scans to a common coordinate frame, which is necessary to aggregate data from multiple subjects. Machine learning registration methods have achieved excellent speed and accuracy but lack interpretability. More recently, keypoint-based methods have been proposed to tackle this issue, but their accuracy is still subpar, particularly when fitting nonlinear transforms. Here we propose Registration by Regression (RbR), a novel atlas registration framework that is highly robust and flexible, conceptually simple, and can be trained with cheaply obtained data. RbR predicts the (x,y,z) atlas coordinates for every voxel of the input scan (i.e., every voxel is a keypoint), and then uses closed-form expressions to quickly fit transforms using a wide array of possible deformation models, including affine and nonlinear (e.g., Bspline, Demons, invertible diffeomorphic models, etc.). Robustness is provided by the large number of voxels informing the registration and can be further increased by robust estimators like RANSAC. Experiments on independent public datasets show that RbR yields more accurate registration than competing keypoint approaches, while providing full control of the deformation model. △ Less

Submitted 25 April, 2024; originally announced April 2024.

Comments: 11 pages, 3 figures

Closed Loop Interactive Embodied Reasoning for Robot Manipulation

Authors: Michal Nazarczuk, Jan Kristof Behrens, Karla Stepanova, Matej Hoffmann, Krystian Mikolajczyk

Abstract: Embodied reasoning systems integrate robotic hardware and cognitive processes to perform complex tasks typically in response to a natural language query about a specific physical environment. This usually involves changing the belief about the scene or physically interacting and changing the scene (e.g. 'Sort the objects from lightest to heaviest'). In order to facilitate the development of such s… ▽ More Embodied reasoning systems integrate robotic hardware and cognitive processes to perform complex tasks typically in response to a natural language query about a specific physical environment. This usually involves changing the belief about the scene or physically interacting and changing the scene (e.g. 'Sort the objects from lightest to heaviest'). In order to facilitate the development of such systems we introduce a new simulating environment that makes use of MuJoCo physics engine and high-quality renderer Blender to provide realistic visual observations that are also accurate to the physical state of the scene. Together with the simulator we propose a new benchmark composed of 10 classes of multi-step reasoning scenarios that require simultaneous visual and physical measurements. Finally, we develop a new modular Closed Loop Interactive Reasoning (CLIER) approach that takes into account the measurements of non-visual object properties, changes in the scene caused by external disturbances as well as uncertain outcomes of robotic actions. We extensively evaluate our reasoning approach in simulation and in the real world manipulation tasks with a success rate above 76% and 64%, respectively. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Interactive Learning of Physical Object Properties Through Robot Manipulation and Database of Object Measurements

Authors: Andrej Kruzliak, Jiri Hartvich, Shubhan P. Patni, Lukas Rustler, Jan Kristof Behrens, Fares J. Abu-Dakka, Krystian Mikolajczyk, Ville Kyrki, Matej Hoffmann

Abstract: This work presents a framework for automatically extracting physical object properties, such as material composition, mass, volume, and stiffness, through robot manipulation and a database of object measurements. The framework involves exploratory action selection to maximize learning about objects on a table. A Bayesian network models conditional dependencies between object properties, incorporat… ▽ More This work presents a framework for automatically extracting physical object properties, such as material composition, mass, volume, and stiffness, through robot manipulation and a database of object measurements. The framework involves exploratory action selection to maximize learning about objects on a table. A Bayesian network models conditional dependencies between object properties, incorporating prior probability distributions and uncertainty associated with measurement actions. The algorithm selects optimal exploratory actions based on expected information gain and updates object properties through Bayesian inference. Experimental evaluation demonstrates effective action selection compared to a baseline and correct termination of the experiments if there is nothing more to be learned. The algorithm proved to behave intelligently when presented with trick objects with material properties in conflict with their appearance. The robot pipeline integrates with a logging module and an online database of objects, containing over 24,000 measurements of 63 objects with different grippers. All code and data are publicly available, facilitating automatic digitization of objects and their physical properties through exploratory manipulations. △ Less

Submitted 10 April, 2024; originally announced April 2024.

Comments: 8 pages, 8 figures

ACM Class: I.2.9

O-RAN for Energy-Efficient Serving Cluster Formulation in User-Centric Cell-Free MMIMO

Authors: Marcin Hoffmann, Paweł Kryszkiewicz

Abstract: The 6G Massive Multiple-Input Multiple-Output (MMIMO) networks can follow the so-called User-Centric Cell-Free (UCCF) architecture, where a single user is served by multiple Access Points (APs) coordinated by the Central Processing Unit (CPU). In this paper, we propose how O-RAN functionalities, i.e., rApp-xApp pair, can be used for energy-efficient Serving Cluster Formulation (SCF). Simulation st… ▽ More The 6G Massive Multiple-Input Multiple-Output (MMIMO) networks can follow the so-called User-Centric Cell-Free (UCCF) architecture, where a single user is served by multiple Access Points (APs) coordinated by the Central Processing Unit (CPU). In this paper, we propose how O-RAN functionalities, i.e., rApp-xApp pair, can be used for energy-efficient Serving Cluster Formulation (SCF). Simulation studies show up to 37\% gain in Energy Efficiency (EE) of the proposed solution over the state-of-the-art Network-Centric (NC) designs. △ Less

Submitted 28 March, 2024; originally announced March 2024.

Comments: Accepted for presentation during The 2nd Workshop on Next-generation Open and Programmable Radio Access Networks (NG-OPERA), organized in conjunction with IEEE International Conference on Computer Communications, May 20, 2024

Boosting Skull-Stripping Performance for Pediatric Brain Images

Authors: William Kelley, Nathan Ngo, Adrian V. Dalca, Bruce Fischl, Lilla Zöllei, Malte Hoffmann

Abstract: Skull-stripping is the removal of background and non-brain anatomical features from brain images. While many skull-stripping tools exist, few target pediatric populations. With the emergence of multi-institutional pediatric data acquisition efforts to broaden the understanding of perinatal brain development, it is essential to develop robust and well-tested tools ready for the relevant data proces… ▽ More Skull-stripping is the removal of background and non-brain anatomical features from brain images. While many skull-stripping tools exist, few target pediatric populations. With the emergence of multi-institutional pediatric data acquisition efforts to broaden the understanding of perinatal brain development, it is essential to develop robust and well-tested tools ready for the relevant data processing. However, the broad range of neuroanatomical variation in the developing brain, combined with additional challenges such as high motion levels, as well as shoulder and chest signal in the images, leaves many adult-specific tools ill-suited for pediatric skull-stripping. Building on an existing framework for robust and accurate skull-stripping, we propose developmental SynthStrip (d-SynthStrip), a skull-stripping model tailored to pediatric images. This framework exposes networks to highly variable images synthesized from label maps. Our model substantially outperforms pediatric baselines across scan types and age cohorts. In addition, the <1-minute runtime of our tool compares favorably to the fastest baselines. We distribute our model at https://w3id.org/synthstrip. △ Less

Submitted 26 February, 2024; originally announced February 2024.

Comments: 5 pages, 5 figures, 1 table, skull-stripping, brain extraction, newborn, infant, toddler, pediatric MRI, machine learning, accepted by the IEEE International Symposium on Biomedical Imaging

Radar-Based Recognition of Static Hand Gestures in American Sign Language

Authors: Christian Schuessler, Wenxuan Zhang, Johanna Bräunig, Marcel Hoffmann, Michael Stelzig, Martin Vossiek

Abstract: In the fast-paced field of human-computer interaction (HCI) and virtual reality (VR), automatic gesture recognition has become increasingly essential. This is particularly true for the recognition of hand signs, providing an intuitive way to effortlessly navigate and control VR and HCI applications. Considering increased privacy requirements, radar sensors emerge as a compelling alternative to cam… ▽ More In the fast-paced field of human-computer interaction (HCI) and virtual reality (VR), automatic gesture recognition has become increasingly essential. This is particularly true for the recognition of hand signs, providing an intuitive way to effortlessly navigate and control VR and HCI applications. Considering increased privacy requirements, radar sensors emerge as a compelling alternative to cameras. They operate effectively in low-light conditions without capturing identifiable human details, thanks to their lower resolution and distinct wavelength compared to visible light. While previous works predominantly deploy radar sensors for dynamic hand gesture recognition based on Doppler information, our approach prioritizes classification using an imaging radar that operates on spatial information, e.g. image-like data. However, generating large training datasets required for neural networks (NN) is a time-consuming and challenging process, often falling short of covering all potential scenarios. Acknowledging these challenges, this study explores the efficacy of synthetic data generated by an advanced radar ray-tracing simulator. This simulator employs an intuitive material model that can be adjusted to introduce data diversity. Despite exclusively training the NN on synthetic data, it demonstrates promising performance when put to the test with real measurement data. This emphasizes the practicality of our methodology in overcoming data scarcity challenges and advancing the field of automatic gesture recognition in VR and HCI applications. △ Less

Submitted 20 February, 2024; originally announced February 2024.

Comments: 5 pages, 6 figures. Accepted to IEEE Radarconf2024

Online Elasticity Estimation and Material Sorting Using Standard Robot Grippers

Authors: Shubhan P. Patni, Pavel Stoudek, Hynek Chlup, Matej Hoffmann

Abstract: Standard robot grippers are not designed for material recognition. We experimentally evaluated the accuracy with which material properties can be estimated through object compression by two standard parallel jaw grippers and a force/torque sensor mounted at the robot wrist, with a professional biaxial compression device used as reference. Gripper effort versus position curves were obtained and tra… ▽ More Standard robot grippers are not designed for material recognition. We experimentally evaluated the accuracy with which material properties can be estimated through object compression by two standard parallel jaw grippers and a force/torque sensor mounted at the robot wrist, with a professional biaxial compression device used as reference. Gripper effort versus position curves were obtained and transformed into stress/strain curves. The modulus of elasticity was estimated at different strain points and the effect of multiple compression cycles (precycling), compression speed, and the gripper surface area on estimation was studied. Viscoelasticity was estimated using the energy absorbed in a compression/decompression cycle, the Kelvin-Voigt, and Hunt-Crossley models. We found that: (1) slower compression speeds improved elasticity estimation, while precycling or surface area did not; (2) the robot grippers, even after calibration, were found to have a limited capability of delivering accurate estimates of absolute values of Young's modulus and viscoelasticity; (3) relative ordering of material characteristics was largely consistent across different grippers; (4) despite the nonlinear characteristics of deformable objects, fitting linear stress/strain approximations led to more stable results than local estimates of Young's modulus; (5) the Hunt-Crossley model worked best to estimate viscoelasticity, from a single object compression. A two-dimensional space formed by elasticity and viscoelasticity estimates obtained from a single grasp is advantageous for the discrimination of the object material properties. We demonstrated the applicability of our findings in a mock single stream recycling scenario, where plastic, paper, and metal objects were correctly separated from a single grasp, even when compressed at different locations on the object. The data and code are publicly available. △ Less

Submitted 8 April, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Comments: 22 pages, 17 figures

ACM Class: I.2.9

Recognition of Unit Segment and Polyline Graphs is $\exists\mathbb{R}$-Complete

Authors: Michael Hoffmann, Tillmann Miltzow, Simon Weber, Lasse Wulf

Abstract: Given a set of objects O in the plane, the corresponding intersection graph is defined as follows. A vertex is created for each object and an edge joins two vertices whenever the corresponding objects intersect. We study here the case of unit segments and polylines with exactly k bends. In the recognition problem, we are given a graph and want to decide whether the graph can be represented as the… ▽ More Given a set of objects O in the plane, the corresponding intersection graph is defined as follows. A vertex is created for each object and an edge joins two vertices whenever the corresponding objects intersect. We study here the case of unit segments and polylines with exactly k bends. In the recognition problem, we are given a graph and want to decide whether the graph can be represented as the intersection graph of certain geometric objects. In previous work it was shown that various recognition problems are $\exists\mathbb{R}$-complete, leaving unit segments and polylines as few remaining natural cases. We show that recognition for both families of objects is $\exists\mathbb{R}$-complete. △ Less

Submitted 4 January, 2024; originally announced January 2024.

Comments: 18 pages, 15 figures

SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI

Authors: Benjamin Billot, Neel Dey, Daniel Moyer, Malte Hoffmann, Esra Abaci Turk, Borjan Gagoski, Ellen Grant, Polina Golland

Abstract: Rigid motion tracking is paramount in many medical imaging applications where movements need to be detected, corrected, or accounted for. Modern strategies rely on convolutional neural networks (CNN) and pose this problem as rigid registration. Yet, CNNs do not exploit natural symmetries in this task, as they are equivariant to translations (their outputs shift with their inputs) but not to rotati… ▽ More Rigid motion tracking is paramount in many medical imaging applications where movements need to be detected, corrected, or accounted for. Modern strategies rely on convolutional neural networks (CNN) and pose this problem as rigid registration. Yet, CNNs do not exploit natural symmetries in this task, as they are equivariant to translations (their outputs shift with their inputs) but not to rotations. Here we propose EquiTrack, the first method that uses recent steerable SE(3)-equivariant CNNs (E-CNN) for motion tracking. While steerable E-CNNs can extract corresponding features across different poses, testing them on noisy medical images reveals that they do not have enough learning capacity to learn noise invariance. Thus, we introduce a hybrid architecture that pairs a denoiser with an E-CNN to decouple the processing of anatomically irrelevant intensity features from the extraction of equivariant spatial features. Rigid transforms are then estimated in closed-form. EquiTrack outperforms state-of-the-art learning and optimisation methods for motion tracking in adult brain MRI and fetal MRI time series. Our code is available at https://github.com/BBillot/EquiTrack. △ Less

Submitted 22 April, 2024; v1 submitted 20 December, 2023; originally announced December 2023.

Comments: under review

HARMONIOUS -- Human-like reactive motion control and multimodal perception for humanoid robots

Authors: Jakub Rozlivek, Alessandro Roncone, Ugo Pattacini, Matej Hoffmann

Abstract: For safe and effective operation of humanoid robots in human-populated environments, the problem of commanding a large number of Degrees of Freedom (DoF) while simultaneously considering dynamic obstacles and human proximity has still not been solved. We present a new reactive motion controller that commands two arms of a humanoid robot and three torso joints (17 DoF in total). We formulate a quad… ▽ More For safe and effective operation of humanoid robots in human-populated environments, the problem of commanding a large number of Degrees of Freedom (DoF) while simultaneously considering dynamic obstacles and human proximity has still not been solved. We present a new reactive motion controller that commands two arms of a humanoid robot and three torso joints (17 DoF in total). We formulate a quadratic program that seeks joint velocity commands respecting multiple constraints while minimizing the magnitude of the velocities. We introduce a new unified treatment of obstacles that dynamically maps visual and proximity (pre-collision) and tactile (post-collision) obstacles as additional constraints to the motion controller, in a distributed fashion over surface of the upper-body of the iCub robot (with 2000 pressure-sensitive receptors). The bio-inspired controller: (i) produces human-like minimum jerk movement profiles; (ii) gives rise to a robot with whole-body visuo-tactile awareness, resembling peripersonal space representations. The controller was extensively experimentally validated, including a physical human-robot interaction scenario. △ Less

Submitted 5 December, 2023; originally announced December 2023.

Comments: 14 pages, 7 figures

ACM Class: I.2.9

Open RAN xApps Design and Evaluation: Lessons Learnt and Identified Challenges

Authors: Marcin Hoffmann, Salim Janji, Adam Samorzewski, Lukasz Kulacz, Cezary Adamczyk, Marcin Dryjański, Pawel Kryszkiewicz, Adrian Kliks, Hanna Bogucka

Abstract: Open Radio Access Networks (RAN) offer diverse economic opportunities. A transition to a flexible, modular approach within the disaggregated RAN framework is crucial, involving careful planning of RAN architecture and the deployment of specialized software applications. Collaboration across sectors is essential for efficiency and reliability, with the open-source community driving innovation. This… ▽ More Open Radio Access Networks (RAN) offer diverse economic opportunities. A transition to a flexible, modular approach within the disaggregated RAN framework is crucial, involving careful planning of RAN architecture and the deployment of specialized software applications. Collaboration across sectors is essential for efficiency and reliability, with the open-source community driving innovation. This paper explores challenges for third-party application developers in Open RAN. It provides a comparative analysis of solutions, focusing on xApp development and implementation. Challenges arise in two areas: the complexities of xApp development, particularly for advanced use cases like beam management, and issues in low-level software implementation within open platforms. In conclusion, key challenges must promote academia-industry collaboration in Open RAN. This paper shares early lessons from xApp development, guiding the field's evolution. △ Less

Submitted 7 November, 2023; originally announced November 2023.

Comments: Accepted for publication in IEEE Journal on Selected Areas in Communications

Open-World Lifelong Graph Learning

Authors: Marcel Hoffmann, Lukas Galke, Ansgar Scherp

Abstract: We study the problem of lifelong graph learning in an open-world scenario, where a model needs to deal with new tasks and potentially unknown classes. We utilize Out-of-Distribution (OOD) detection methods to recognize new classes and adapt existing non-graph OOD detection methods to graph data. Crucially, we suggest performing new class detection by combining OOD detection methods with informatio… ▽ More We study the problem of lifelong graph learning in an open-world scenario, where a model needs to deal with new tasks and potentially unknown classes. We utilize Out-of-Distribution (OOD) detection methods to recognize new classes and adapt existing non-graph OOD detection methods to graph data. Crucially, we suggest performing new class detection by combining OOD detection methods with information aggregated from the graph neighborhood. Most OOD detection methods avoid determining a crisp threshold for deciding whether a vertex is OOD. To tackle this problem, we propose a Weakly-supervised Relevance Feedback (Open-WRF) method, which decreases the sensitivity to thresholds in OOD detection. We evaluate our approach on six benchmark datasets. Our results show that the proposed neighborhood aggregation method for OOD scores outperforms existing methods independent of the underlying graph neural network. Furthermore, we demonstrate that our Open-WRF method is more robust to threshold selection and analyze the influence of graph neighborhood on OOD detection. The aggregation and threshold methods are compatible with arbitrary graph neural networks and OOD detection methods, making our approach versatile and applicable to many real-world applications. △ Less

Submitted 19 October, 2023; originally announced October 2023.

Why white noise is not enough? On using radio front-end models while designing 6G PHY

Authors: Pawel Kryszkiewicz, Pawel Sroka, Marcin Hoffmann, Marcin Wachowiak

Abstract: From generation to generation there are increasing requirements for wireless standards both in terms of spectral and energy efficiency. While up to now the layered wireless transceiver architecture worked allowing for, e.g., separation of channel decoding algorithms from front-end design, this may need reconsideration in the 6G era. Especially the hardware-originated distortions have to be taken i… ▽ More From generation to generation there are increasing requirements for wireless standards both in terms of spectral and energy efficiency. While up to now the layered wireless transceiver architecture worked allowing for, e.g., separation of channel decoding algorithms from front-end design, this may need reconsideration in the 6G era. Especially the hardware-originated distortions have to be taken into account while designing other layer algorithms as the high throughput and energy efficiency requirements will push these devices to their limit revealing their nonlinear characteristics. This position paper will shed some light on new degrees of freedom while cross-layer designing and controlling multicarrier and multiantenna transceivers of 6G systems. △ Less

Submitted 28 June, 2023; originally announced June 2023.

Journal ref: Journal of Telecommunications and Information Technology, 2/2023

Super-Resolution Radar Imaging with Sparse Arrays Using a Deep Neural Network Trained with Enhanced Virtual Data

Authors: Christian Schuessler, Marcel Hoffmann, Martin Vossiek

Abstract: This paper introduces a method based on a deep neural network (DNN) that is perfectly capable of processing radar data from extremely thinned radar apertures. The proposed DNN processing can provide both aliasing-free radar imaging and super-resolution. The results are validated by measuring the detection performance on realistic simulation data and by evaluating the Point-Spread-function (PSF) an… ▽ More This paper introduces a method based on a deep neural network (DNN) that is perfectly capable of processing radar data from extremely thinned radar apertures. The proposed DNN processing can provide both aliasing-free radar imaging and super-resolution. The results are validated by measuring the detection performance on realistic simulation data and by evaluating the Point-Spread-function (PSF) and the target-separation performance on measured point-like targets. Also, a qualitative evaluation of a typical automotive scene is conducted. It is shown that this approach can outperform state-of-the-art subspace algorithms and also other existing machine learning solutions. The presented results suggest that machine learning approaches trained with sufficiently sophisticated virtual input data are a very promising alternative to compressed sensing and subspace approaches in radar signal processing. The key to this performance is that the DNN is trained using realistic simulation data that perfectly mimic a given sparse antenna radar array hardware as the input. As ground truth, ultra-high resolution data from an enhanced virtual radar are simulated. Contrary to other work, the DNN utilizes the complete radar cube and not only the antenna channel information at certain range-Doppler detections. After training, the proposed DNN is capable of sidelobe- and ambiguity-free imaging. It simultaneously delivers nearly the same resolution and image quality as would be achieved with a fully occupied array. △ Less

Submitted 16 June, 2023; originally announced June 2023.

Comments: 15 pages, 12 figures, Accepted to IEEE Journal of Microwaves

Beam Management Driven by Radio Environment Maps in O-RAN Architecture

Authors: Marcin Hoffmann, Pawel Kryszkiewicz

Abstract: The Massive Multiple-Input Multiple-Output (M-MIMO) is considered as one of the key technologies in 5G, and future 6G networks. From the perspective of, e.g., channel estimation, especially for high-speed users it is easier to implement an M-MIMO network exploiting a static set of beams, i.e., Grid of Beams (GoB). While considering GoB it is important to properly assign users to the beams, i.e., t… ▽ More The Massive Multiple-Input Multiple-Output (M-MIMO) is considered as one of the key technologies in 5G, and future 6G networks. From the perspective of, e.g., channel estimation, especially for high-speed users it is easier to implement an M-MIMO network exploiting a static set of beams, i.e., Grid of Beams (GoB). While considering GoB it is important to properly assign users to the beams, i.e., to perform Beam Management (BM). BM can be enhanced by taking into account historical knowledge about the radio environment, e.g., to avoid radio link failures. The aim of this paper is to propose such a BM algorithm, that utilizes location-dependent data stored in a Radio Environment Map (REM). It utilizes received power maps, and user mobility patterns to optimize the BM process in terms of Reinforcement Learning (RL) by using the Policy Iteration method under different goal functions, e.g., maximization of received power or minimization of beam reselections while avoiding radio link failures. The proposed solution is compliant with the Open Radio Access Network (O-RAN) architecture, enabling its practical implementation. Simulation studies have shown that the proposed BM algorithm can significantly reduce the number of beam reselections or radio link failures compared to the baseline algorithm. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Journal ref: 2023 IEEE International Conference on Communications Workshops (ICC Workshops)

The Number of Edges in Maximal 2-planar Graphs

Authors: Michael Hoffmann, Meghana M. Reddy

Abstract: A graph is $2$-planar if it has local crossing number two, that is, it can be drawn in the plane such that every edge has at most two crossings. A graph is maximal $2$-planar if no edge can be added such that the resulting graph remains $2$-planar. A $2$-planar graph on $n$ vertices has at most $5n-10$ edges, and some (maximal) $2$-planar graphs -- referred to as optimal $2$-planar -- achieve this… ▽ More A graph is $2$-planar if it has local crossing number two, that is, it can be drawn in the plane such that every edge has at most two crossings. A graph is maximal $2$-planar if no edge can be added such that the resulting graph remains $2$-planar. A $2$-planar graph on $n$ vertices has at most $5n-10$ edges, and some (maximal) $2$-planar graphs -- referred to as optimal $2$-planar -- achieve this bound. However, in strong contrast to maximal planar graphs, a maximal $2$-planar graph may have fewer than the maximum possible number of edges. In this paper, we determine the minimum edge density of maximal $2$-planar graphs by proving that every maximal $2$-planar graph on $n\ge 5$ vertices has at least $2n$ edges. We also show that this bound is tight, up to an additive constant. The lower bound is based on an analysis of the degree distribution in specific classes of drawings of the graph. The upper bound construction is verified by carefully exploring the space of admissible drawings using computer support. △ Less

Submitted 15 March, 2023; originally announced March 2023.

Comments: This work (without appendix) is available at the 39th International Symposium on Computational Geometry (SoCG 2023)

ACM Class: G.2.1; G.2.2

Drawings of Complete Multipartite Graphs Up to Triangle Flips

Authors: Oswin Aichholzer, Man-Kwun Chiu, Hung P. Hoang, Michael Hoffmann, Jan Kynčl, Yannic Maus, Birgit Vogtenhuber, Alexandra Weinberger

Abstract: For a drawing of a labeled graph, the rotation of a vertex or crossing is the cyclic order of its incident edges, represented by the labels of their other endpoints. The extended rotation system (ERS) of the drawing is the collection of the rotations of all vertices and crossings. A drawing is simple if each pair of edges has at most one common point. Gioan's Theorem states that for any two simple… ▽ More For a drawing of a labeled graph, the rotation of a vertex or crossing is the cyclic order of its incident edges, represented by the labels of their other endpoints. The extended rotation system (ERS) of the drawing is the collection of the rotations of all vertices and crossings. A drawing is simple if each pair of edges has at most one common point. Gioan's Theorem states that for any two simple drawings of the complete graph $K_n$ with the same crossing edge pairs, one drawing can be transformed into the other by a sequence of triangle flips (a.k.a. Reidemeister moves of Type 3). This operation refers to the act of moving one edge of a triangular cell formed by three pairwise crossing edges over the opposite crossing of the cell, via a local transformation. We investigate to what extent Gioan-type theorems can be obtained for wider classes of graphs. A necessary (but in general not sufficient) condition for two drawings of a graph to be transformable into each other by a sequence of triangle flips is that they have the same ERS. As our main result, we show that for the large class of complete multipartite graphs, this necessary condition is in fact also sufficient. We present two different proofs of this result, one of which is shorter, while the other one yields a polynomial time algorithm for which the number of needed triangle flips for graphs on $n$ vertices is bounded by $O(n^{16})$. The latter proof uses a Carathéodory-type theorem for simple drawings of complete multipartite graphs, which we believe to be of independent interest. Moreover, we show that our Gioan-type theorem for complete multipartite graphs is essentially tight in the sense that having the same ERS does not remain sufficient when removing or adding very few edges. △ Less

Submitted 13 March, 2023; originally announced March 2023.

Comments: Abstract shortened for arxiv. This work (without appendix) is available at the 39th International Symposium on Computational Geometry (SoCG 2023)

Efficient Visuo-Haptic Object Shape Completion for Robot Manipulation

Authors: Lukas Rustler, Jiri Matas, Matej Hoffmann

Abstract: For robot manipulation, a complete and accurate object shape is desirable. Here, we present a method that combines visual and haptic reconstruction in a closed-loop pipeline. From an initial viewpoint, the object shape is reconstructed using an implicit surface deep neural network. The location with highest uncertainty is selected for haptic exploration, the object is touched, the new information… ▽ More For robot manipulation, a complete and accurate object shape is desirable. Here, we present a method that combines visual and haptic reconstruction in a closed-loop pipeline. From an initial viewpoint, the object shape is reconstructed using an implicit surface deep neural network. The location with highest uncertainty is selected for haptic exploration, the object is touched, the new information from touch and a new point cloud from the camera are added, object position is re-estimated and the cycle is repeated. We extend Rustler et al. (2022) by using a new theoretically grounded method to determine the points with highest uncertainty, and we increase the yield of every haptic exploration by adding not only the contact points to the point cloud but also incorporating the empty space established through the robot movement to the object. Additionally, the solution is compact in that the jaws of a closed two-finger gripper are directly used for exploration. The object position is re-estimated after every robot action and multiple objects can be present simultaneously on the table. We achieve a steady improvement with every touch using three different metrics and demonstrate the utility of the better shape reconstruction in grasping experiments on the real robot. On average, grasp success rate increases from 63.3% to 70.4% after a single exploratory touch and to 82.7% after five touches. The collected data and code are publicly available (https://osf.io/j6rkd/, https://github.com/ctu-vras/vishac) △ Less

Submitted 8 March, 2023; originally announced March 2023.

Perirobot space representation for HRI: measuring and designing collaborative workspace coverage by diverse sensors

Authors: Jakub Rozlivek, Petr Svarny, Matej Hoffmann

Abstract: Two regimes permitting safe physical human-robot interaction, speed and separation monitoring and safety-rated monitored stop, depend on reliable perception of the space surrounding the robot. This can be accomplished by visual sensors (like cameras, RGB-D cameras, LIDARs), proximity sensors, or dedicated devices used in industrial settings like pads that are activated by the presence of the opera… ▽ More Two regimes permitting safe physical human-robot interaction, speed and separation monitoring and safety-rated monitored stop, depend on reliable perception of the space surrounding the robot. This can be accomplished by visual sensors (like cameras, RGB-D cameras, LIDARs), proximity sensors, or dedicated devices used in industrial settings like pads that are activated by the presence of the operator. The deployment of a particular solution is often ad hoc and no unified representation of the interaction space or its coverage by the different sensors exists. In this work, we make first steps in this direction by defining the spaces to be monitored, representing all sensor data as information about occupancy and using occupancy-based metrics to calculate how a particular sensor covers the workspace. We demonstrate our approach in two (multi-)sensor-placement experiments in three static scenes and one experiment in a dynamic scene. The occupancy representation allow to compare the effectiveness of various sensor setups. Therefore, this approach can serve as a prototyping tool to establish the sensor setup that provides the most efficient coverage for the given metrics and sensor representations. △ Less

Submitted 20 December, 2023; v1 submitted 4 March, 2023; originally announced March 2023.

Comments: 8 pages, 12 figures

ACM Class: I.2.9

Journal ref: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Signaling Storm Detection in IIoT Network based on the Open RAN Architecture

Authors: Marcin Hoffmann, Pawel Kryszkiewicz

Abstract: The Industrial Internet of Things devices due to their low cost and complexity are exposed to being hacked and utilized to attack the network infrastructure causing a so-called Signaling Storm. In this paper, we propose to utilize the Open Radio Access Network (O-RAN) architecture, to monitor the control plane messages in order to detect the activity of adversaries at its early stage. The Industrial Internet of Things devices due to their low cost and complexity are exposed to being hacked and utilized to attack the network infrastructure causing a so-called Signaling Storm. In this paper, we propose to utilize the Open Radio Access Network (O-RAN) architecture, to monitor the control plane messages in order to detect the activity of adversaries at its early stage. △ Less

Submitted 16 February, 2023; originally announced February 2023.

Journal ref: IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)

Gaze Cueing and the Role of Presence in Human-Robot Interaction

Authors: Kassandra Friebe, Kristina Malinovska, Sabina Samporova, Matej Hoffmann

Abstract: Gaze cueing is a fundamental part of social interactions, and broadly studied using Posner task based gaze cueing paradigms. While studies using human stimuli consistently yield a gaze cueing effect, results from studies using robotic stimuli are inconsistent. Typically, these studies use virtual agents or pictures of robots. As previous research has pointed to the significance of physical presenc… ▽ More Gaze cueing is a fundamental part of social interactions, and broadly studied using Posner task based gaze cueing paradigms. While studies using human stimuli consistently yield a gaze cueing effect, results from studies using robotic stimuli are inconsistent. Typically, these studies use virtual agents or pictures of robots. As previous research has pointed to the significance of physical presence in human-robot interaction, it is of fundamental importance to understand its yet unexplored role in interactions with gaze cues. This paper investigates whether the physical presence of the iCub humanoid robot affects the strength of the gaze cueing effect in human-robot interaction. We exposed 42 participants to a gaze cueing task. We asked participants to react as quickly and accurately as possible to the appearance of a target stimulus that was either congruently or incongruently cued by the gaze of a copresent iCub robot or a virtual version of the same robot. Analysis of the reaction time measurements showed that participants were consistently affected by their robot interaction partner's gaze, independently on the way the robot was presented. Additional analyses of participants' ratings of the robot's anthropomorphism, animacy and likeability further add to the impression that presence does not play a significant role in simple gaze based interactions. Together our findings open up interesting discussions about the possibility to generalize results from studies using virtual agents to real life interactions with copresent robots. △ Less

Submitted 8 February, 2023; originally announced February 2023.

Comments: 13 pages, 4 figures

Journal ref: 2022, International Conference on Social Robotics (ICSR). Lecture Notes in Computer Science, vol 13817. Springer, Cham, pp. 402-414

Anatomy-aware and acquisition-agnostic joint registration with SynthMorph

Authors: Malte Hoffmann, Andrew Hoopes, Douglas N. Greve, Bruce Fischl, Adrian V. Dalca

Abstract: Affine image registration is a cornerstone of medical-image analysis. While classical algorithms can achieve excellent accuracy, they solve a time-consuming optimization for every image pair. Deep-learning (DL) methods learn a function that maps an image pair to an output transform. Evaluating the function is fast, but capturing large transforms can be challenging, and networks tend to struggle if… ▽ More Affine image registration is a cornerstone of medical-image analysis. While classical algorithms can achieve excellent accuracy, they solve a time-consuming optimization for every image pair. Deep-learning (DL) methods learn a function that maps an image pair to an output transform. Evaluating the function is fast, but capturing large transforms can be challenging, and networks tend to struggle if a test-image characteristic shifts from the training domain, such as resolution. Most affine methods are agnostic to anatomy, meaning the registration will be inaccurate if algorithms consider all structures in the image. We address these shortcomings with SynthMorph, an easy-to-use DL tool for joint affine-deformable registration of any brain image without preprocessing, right off the MRI scanner. First, we leverage a strategy to train networks with wildly varying images synthesized from label maps, yielding robust performance across acquisition specifics unseen at training. Second, we optimize the spatial overlap of select anatomical labels. This enables networks to distinguish anatomy of interest from irrelevant structures, removing the need for preprocessing that excludes content which would impinge on anatomy-specific registration. Third, we combine the affine model with a deformable hypernetwork that lets users choose the optimal deformation-field regularity for their specific data, at registration time, in a fraction of the time required by classical methods. We rigorously analyze how competing architectures learn affine transforms and compare state-of-the-art registration tools across an extremely diverse set of neuroimaging data, aiming to truly capture the behavior of methods in the real world. SynthMorph demonstrates consistent and improved accuracy. It is available at https://w3id.org/synthmorph, as a single complete end-to-end solution for registration of brain MRI. △ Less

Submitted 4 January, 2024; v1 submitted 26 January, 2023; originally announced January 2023.

Comments: 33 pages, 22 figures, 4 tables, affine registration, deformable registration, deep learning, hypernetwork, domain shift, neuroimaging

Data Consistent Deep Rigid MRI Motion Correction

Authors: Nalini M. Singh, Neel Dey, Malte Hoffmann, Bruce Fischl, Elfar Adalsteinsson, Robert Frost, Adrian V. Dalca, Polina Golland

Abstract: Motion artifacts are a pervasive problem in MRI, leading to misdiagnosis or mischaracterization in population-level imaging studies. Current retrospective rigid intra-slice motion correction techniques jointly optimize estimates of the image and the motion parameters. In this paper, we use a deep network to reduce the joint image-motion parameter search to a search over rigid motion parameters alo… ▽ More Motion artifacts are a pervasive problem in MRI, leading to misdiagnosis or mischaracterization in population-level imaging studies. Current retrospective rigid intra-slice motion correction techniques jointly optimize estimates of the image and the motion parameters. In this paper, we use a deep network to reduce the joint image-motion parameter search to a search over rigid motion parameters alone. Our network produces a reconstruction as a function of two inputs: corrupted k-space data and motion parameters. We train the network using simulated, motion-corrupted k-space data generated with known motion parameters. At test-time, we estimate unknown motion parameters by minimizing a data consistency loss between the motion parameters, the network-based image reconstruction given those parameters, and the acquired measurements. Intra-slice motion correction experiments on simulated and realistic 2D fast spin echo brain MRI achieve high reconstruction fidelity while providing the benefits of explicit data consistency optimization. Our code is publicly available at https://www.github.com/nalinimsingh/neuroMoCo. △ Less

Submitted 16 November, 2023; v1 submitted 24 January, 2023; originally announced January 2023.

Comments: Presented at MIDL 2023. 14 pages, 6 figures. Keywords: motion correction, magnetic resonance imaging, deep learning

Path Planning for Concentric Tube Robots: a Toolchain with Application to Stereotactic Neurosurgery

Authors: Matthias K. Hoffmann, Willem Esterhuizen, Karl Worthmann, Kathrin Flaßkamp

Abstract: We present a toolchain for solving path planning problems for concentric tube robots through obstacle fields. First, ellipsoidal sets representing the target area and obstacles are constructed from labelled point clouds. Then, the nonlinear and highly nonconvex optimal control problem is solved by introducing a homotopy on the obstacle positions where at one extreme of the parameter the obstacles… ▽ More We present a toolchain for solving path planning problems for concentric tube robots through obstacle fields. First, ellipsoidal sets representing the target area and obstacles are constructed from labelled point clouds. Then, the nonlinear and highly nonconvex optimal control problem is solved by introducing a homotopy on the obstacle positions where at one extreme of the parameter the obstacles are removed from the operating space, and at the other extreme they are located at their intended positions. We present a detailed example (with more than a thousand obstacles) from stereotactic neurosurgery with real-world data obtained from labelled MPRI scans. △ Less

Submitted 28 November, 2022; originally announced November 2022.

Comments: 8 pages, 7 figures. Paper under review

MSC Class: 49M20 65H20 92C50 70E60 34H05 ACM Class: I.6.3; J.2; J.3

Reducing a Set of Regular Expressions and Analyzing Differences of Domain-specific Statistic Reporting

Authors: Tobias Kalmbach, Marcel Hoffmann, Nicolas Lell, Ansgar Scherp

Abstract: Due to the large amount of daily scientific publications, it is impossible to manually review each one. Therefore, an automatic extraction of key information is desirable. In this paper, we examine STEREO, a tool for extracting statistics from scientific papers using regular expressions. By adapting an existing regular expression inclusion algorithm for our use case, we decrease the number of regu… ▽ More Due to the large amount of daily scientific publications, it is impossible to manually review each one. Therefore, an automatic extraction of key information is desirable. In this paper, we examine STEREO, a tool for extracting statistics from scientific papers using regular expressions. By adapting an existing regular expression inclusion algorithm for our use case, we decrease the number of regular expressions used in STEREO by about $33.8\%$. We reveal common patterns from the condensed rule set that can be used for the creation of new rules. We also apply STEREO, which was previously trained in the life-sciences and medical domain, to a new scientific domain, namely Human-Computer-Interaction (HCI), and re-evaluate it. According to our research, statistics in the HCI domain are similar to those in the medical domain, although a higher percentage of APA-conform statistics were found in the HCI domain. Additionally, we compare extraction on PDF and LaTeX source files, finding LaTeX to be more reliable for extraction. △ Less

Submitted 25 March, 2023; v1 submitted 24 November, 2022; originally announced November 2022.

Learning body models: from humans to humanoids

Authors: Matej Hoffmann

Abstract: Humans and animals excel in combining information from multiple sensory modalities, controlling their complex bodies, adapting to growth, failures, or using tools. These capabilities are also highly desirable in robots. They are displayed by machines to some extent. Yet, the artificial creatures are lagging behind. The key foundation is an internal representation of the body that the agent - human… ▽ More Humans and animals excel in combining information from multiple sensory modalities, controlling their complex bodies, adapting to growth, failures, or using tools. These capabilities are also highly desirable in robots. They are displayed by machines to some extent. Yet, the artificial creatures are lagging behind. The key foundation is an internal representation of the body that the agent - human, animal, or robot - has developed. The mechanisms of operation of body models in the brain are largely unknown and even less is known about how they are constructed from experience after birth. In collaboration with developmental psychologists, we conducted targeted experiments to understand how infants acquire first "sensorimotor body knowledge". These experiments inform our work in which we construct embodied computational models on humanoid robots that address the mechanisms behind learning, adaptation, and operation of multimodal body representations. At the same time, we assess which of the features of the "body in the brain" should be transferred to robots to give rise to more adaptive and resilient, self-calibrating machines. We extend traditional robot kinematic calibration focusing on self-contained approaches where no external metrology is needed: self-contact and self-observation. Problem formulation allowing to combine several ways of closing the kinematic chain simultaneously is presented, along with a calibration toolbox and experimental validation on several robot platforms. Finally, next to models of the body itself, we study peripersonal space - the space immediately surrounding the body. Again, embodied computational models are developed and subsequently, the possibility of turning these biologically inspired representations into safe human-robot collaboration is studied. △ Less

Submitted 6 November, 2022; originally announced November 2022.

Comments: 34 pages, 5 figures. Habilitation thesis, Faculty of Electrical Engineering, Czech Technical University in Prague (2021)

MSC Class: 68T40 ACM Class: I.2.9

Federated Learning-Based Interference Modeling for Vehicular Dynamic Spectrum Access

Authors: Marcin Hoffmann, Pawel Kryszkiewicz, Adrian Kliks

Abstract: A platoon-based driving is a technology allowing vehicles to follow each other at close distances to, e.g., save fuel. However, it requires reliable wireless communications to adjust their speeds. Recent studies have shown that the frequency band dedicated for vehicle-to-vehicle communications can be too busy for intra-platoon communications. Thus it is reasonable to use additional spectrum resour… ▽ More A platoon-based driving is a technology allowing vehicles to follow each other at close distances to, e.g., save fuel. However, it requires reliable wireless communications to adjust their speeds. Recent studies have shown that the frequency band dedicated for vehicle-to-vehicle communications can be too busy for intra-platoon communications. Thus it is reasonable to use additional spectrum resources, of low occupancy, i.e., secondary spectrum channels. The challenge is to model the interference in those channels to enable proper channel selection. In this paper, we propose a two-layered Radio Environment Map (REM) that aims at providing platoons with accurate location-dependent interference models by using the Federated Learning approach. Each platoon is equipped with a Local REM that is updated on the basis of raw interference samples and previous interference model stored in the Global REM. The model in global REM is obtained by merging models reported by platoons. The nodes exchange only parameters of interference models, reducing the required control channel capacity. Moreover, in the proposed architecture platoon can utilize Local REM to predict channel occupancy, even when the connection to the Global REM is temporarily unavailable. The proposed system is validated via computer simulations considering non-trivial interference patterns. △ Less

Submitted 3 October, 2022; originally announced October 2022.

Report number: vol 492. Springer, Cham

Journal ref: MobiQuitous 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

Radio Environment Map and Deep Q-Learning for 5G Dynamic Point Blanking

Authors: Marcin Hoffmann, Paweł Kryszkiewicz

Abstract: Dynamic Point Blanking (DPB) is one of the Coordinated MultiPoint (CoMP) techniques, where some Base Stations (BSs) can be temporarily muted, e.g., to improve the cell-edge users throughput. In this paper, it is proposed to obtain the muting pattern that improves cell-edge users throughput with the use of a Deep Q-Learning. The Deep Q-Learning agent is trained on location-dependent data. Simulatio… ▽ More Dynamic Point Blanking (DPB) is one of the Coordinated MultiPoint (CoMP) techniques, where some Base Stations (BSs) can be temporarily muted, e.g., to improve the cell-edge users throughput. In this paper, it is proposed to obtain the muting pattern that improves cell-edge users throughput with the use of a Deep Q-Learning. The Deep Q-Learning agent is trained on location-dependent data. Simulation studies have shown that the proposed solution improves cell-edge user throughput by about 20.6%. △ Less

Submitted 26 October, 2022; v1 submitted 24 September, 2022; originally announced September 2022.

Journal ref: 2022 International Conference on Software, Telecommunications and Computer Networks (SoftCOM), 2022, pp. 1-3

Blind and Channel-agnostic Equalization Using Adversarial Networks

Authors: Vincent Lauinger, Manuel Hoffmann, Jonas Ney, Norbert Wehn, Laurent Schmalen

Abstract: Due to the rapid development of autonomous driving, the Internet of Things and streaming services, modern communication systems have to cope with varying channel conditions and a steadily rising number of users and devices. This, and the still rising bandwidth demands, can only be met by intelligent network automation, which requires highly flexible and blind transceiver algorithms. To tackle thos… ▽ More Due to the rapid development of autonomous driving, the Internet of Things and streaming services, modern communication systems have to cope with varying channel conditions and a steadily rising number of users and devices. This, and the still rising bandwidth demands, can only be met by intelligent network automation, which requires highly flexible and blind transceiver algorithms. To tackle those challenges, we propose a novel adaptive equalization scheme, which exploits the prosperous advances in deep learning by training an equalizer with an adversarial network. The learning is only based on the statistics of the transmit signal, so it is blind regarding the actual transmit symbols and agnostic to the channel model. The proposed approach is independent of the equalizer topology and enables the application of powerful neural network based equalizers. In this work, we prove this concept in simulations of different -- both linear and nonlinear -- transmission channels and demonstrate the capability of the proposed blind learning scheme to approach the performance of non-blind equalizers. Furthermore, we provide a theoretical perspective and highlight the challenges of the approach. △ Less

Submitted 15 September, 2022; originally announced September 2022.

Comments: Accepted and to be presented at the IEEE GLOBECOM 2022 conference

Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions

Authors: Sebastian Kiefer, Mareike Hoffmann

Abstract: Interactive Machine Learning (IML) shall enable intelligent systems to interactively learn from their end-users, and is quickly becoming more and more important. Although it puts the human in the loop, interactions are mostly performed via mutual explanations that miss contextual information. Furthermore, current model-agnostic IML strategies like CAIPI are limited to 'destructive' feedback, meani… ▽ More Interactive Machine Learning (IML) shall enable intelligent systems to interactively learn from their end-users, and is quickly becoming more and more important. Although it puts the human in the loop, interactions are mostly performed via mutual explanations that miss contextual information. Furthermore, current model-agnostic IML strategies like CAIPI are limited to 'destructive' feedback, meaning they solely allow an expert to prevent a learner from using irrelevant features. In this work, we propose a novel interaction framework called Semantic Interactive Learning for the text domain. We frame the problem of incorporating constructive and contextual feedback into the learner as a task to find an architecture that (a) enables more semantic alignment between humans and machines and (b) at the same time helps to maintain statistical characteristics of the input domain when generating user-defined counterexamples based on meaningful corrections. Therefore, we introduce a technique called SemanticPush that is effective for translating conceptual corrections of humans to non-extrapolating training examples such that the learner's reasoning is pushed towards the desired behavior. In several experiments, we show that our method clearly outperforms CAIPI, a state of the art IML strategy, in terms of Predictive Performance as well as Local Explanation Quality in downstream multi-class classification tasks. △ Less

Submitted 7 September, 2022; originally announced September 2022.

Hey, robot! An investigation of getting robot's attention through touch

Authors: Hagen Lehmann, Adam Rojik, Kassandra Friebe, Matej Hoffmann

Abstract: Touch is a key part of interaction and communication between humans, but has still been little explored in human-robot interaction. In this work, participants were asked to approach and touch a humanoid robot on the hand (Nao - 26 participants; Pepper - 28 participants) to get its attention. We designed reaction behaviors for the robot that consisted in four different combinations of arm movements… ▽ More Touch is a key part of interaction and communication between humans, but has still been little explored in human-robot interaction. In this work, participants were asked to approach and touch a humanoid robot on the hand (Nao - 26 participants; Pepper - 28 participants) to get its attention. We designed reaction behaviors for the robot that consisted in four different combinations of arm movements with the touched hand moving forward or back and the other hand moving forward or staying in place, with simultaneous leaning back, followed by looking at the participant. We studied which reaction of the robot people found the most appropriate and what was the reason for their choice. For both robots, the preferred reaction of the robot hand being touched was moving back. For the other hand, no movement at all was rated most natural for the Pepper, while it was movement forward for the Nao. A correlation between the anxiety subscale of the participants' personality traits and the passive to active/aggressive nature of the robot reactions was found. Most participants noticed the leaning back and rated it positively. Looking at the participant was commented on positively by some participants in unstructured comments. We also analyzed where and how participants spontaneously touched the robot on the hand. In summary, the touch reaction behaviors designed here are good candidates to be deployed more generally in social robots, possibly including incidental touch in crowded environments. The robot size constitutes one important factor shaping how the robot reaction is perceived. △ Less

Submitted 9 February, 2023; v1 submitted 1 September, 2022; originally announced September 2022.

Comments: 14 pages, 4 figures; 'International Conference on Social Robotics (ICSR)'. Lecture Notes in Computer Science, vol 13817. Springer, Cham, pp. 388-401 (2022)

Human keypoint detection for close proximity human-robot interaction

Authors: Jan Docekal, Jakub Rozlivek, Jiri Matas, Matej Hoffmann

Abstract: We study the performance of state-of-the-art human keypoint detectors in the context of close proximity human-robot interaction. The detection in this scenario is specific in that only a subset of body parts such as hands and torso are in the field of view. In particular, (i) we survey existing datasets with human pose annotation from the perspective of close proximity images and prepare and make… ▽ More We study the performance of state-of-the-art human keypoint detectors in the context of close proximity human-robot interaction. The detection in this scenario is specific in that only a subset of body parts such as hands and torso are in the field of view. In particular, (i) we survey existing datasets with human pose annotation from the perspective of close proximity images and prepare and make publicly available a new Human in Close Proximity (HiCP) dataset; (ii) we quantitatively and qualitatively compare state-of-the-art human whole-body 2D keypoint detection methods (OpenPose, MMPose, AlphaPose, Detectron2) on this dataset; (iii) since accurate detection of hands and fingers is critical in applications with handovers, we evaluate the performance of the MediaPipe hand detector; (iv) we deploy the algorithms on a humanoid robot with an RGB-D camera on its head and evaluate the performance in 3D human keypoint detection. A motion capture system is used as reference. The best performing whole-body keypoint detectors in close proximity were MMPose and AlphaPose, but both had difficulty with finger detection. Thus, we propose a combination of MMPose or AlphaPose for the body and MediaPipe for the hands in a single framework providing the most accurate and robust detection. We also analyse the failure modes of individual detectors -- for example, to what extent the absence of the head of the person in the image degrades performance. Finally, we demonstrate the framework in a scenario where a humanoid robot interacting with a person uses the detected 3D keypoints for whole-body avoidance maneuvers. △ Less

Submitted 9 February, 2023; v1 submitted 15 July, 2022; originally announced July 2022.

Comments: 8 pages 8 figures

ACM Class: I.2.9; I.4.9; I.2.10

Journal ref: IEEE-RAS International Conference on Humanoid Robots (Humanoids 2022)

Bounding and computing obstacle numbers of graphs

Authors: Martin Balko, Steven Chaplick, Robert Ganian, Siddharth Gupta, Michael Hoffmann, Pavel Valtr, Alexander Wolff

Abstract: An obstacle representation of a graph $G$ consists of a set of pairwise disjoint simply-connected closed regions and a one-to-one mapping of the vertices of $G$ to points such that two vertices are adjacent in $G$ if and only if the line segment connecting the two corresponding points does not intersect any obstacle. The obstacle number of a graph is the smallest number of obstacles in an obstacle… ▽ More An obstacle representation of a graph $G$ consists of a set of pairwise disjoint simply-connected closed regions and a one-to-one mapping of the vertices of $G$ to points such that two vertices are adjacent in $G$ if and only if the line segment connecting the two corresponding points does not intersect any obstacle. The obstacle number of a graph is the smallest number of obstacles in an obstacle representation of the graph in the plane such that all obstacles are simple polygons. It is known that the obstacle number of each $n$-vertex graph is $O(n \log n)$ [Balko, Cibulka, and Valtr, 2018] and that there are $n$-vertex graphs whose obstacle number is $Ω(n/(\log\log n)^2)$ [Dujmović and Morin, 2015]. We improve this lower bound to $Ω(n/\log\log n)$ for simple polygons and to $Ω(n)$ for convex polygons. To obtain these stronger bounds, we improve known estimates on the number of $n$-vertex graphs with bounded obstacle number, solving a conjecture by Dujmović and Morin. We also show that if the drawing of some $n$-vertex graph is given as part of the input, then for some drawings $Ω(n^2)$ obstacles are required to turn them into an obstacle representation of the graph. Our bounds are asymptotically tight in several instances. We complement these combinatorial bounds by two complexity results. First, we show that computing the obstacle number of a graph $G$ is fixed-parameter tractable in the vertex cover number of $G$. Second, we show that, given a graph $G$ and a simple polygon $P$, it is NP-hard to decide whether $G$ admits an obstacle representation using $P$ as the only obstacle. △ Less

Submitted 21 January, 2024; v1 submitted 30 June, 2022; originally announced June 2022.

Single-grasp deformable object discrimination: the effect of gripper morphology, sensing modalities, and action parameters

Authors: Michal Pliska, Shubhan Patni, Michal Mares, Pavel Stoudek, Zdenek Straka, Karla Stepanova, Matej Hoffmann

Abstract: In haptic object discrimination, the effect of gripper embodiment, action parameters, and sensory channels has not been systematically studied. We used two anthropomorphic hands and two 2-finger grippers to grasp two sets of deformable objects. On the object classification task, we found: (i) among classifiers, SVM on sensory features and LSTM on raw time series performed best across all grippers;… ▽ More In haptic object discrimination, the effect of gripper embodiment, action parameters, and sensory channels has not been systematically studied. We used two anthropomorphic hands and two 2-finger grippers to grasp two sets of deformable objects. On the object classification task, we found: (i) among classifiers, SVM on sensory features and LSTM on raw time series performed best across all grippers; (ii) faster compression speeds degraded performance; (iii) generalization to different grasping configurations was limited; transfer to different compression speeds worked well for the Barrett Hand only. Visualization of the feature spaces using PCA showed that the gripper morphology and the action parameters were the main source of variance, rendering generalization across embodiment or grasp configurations very hard. On the highly challenging dataset consisting of polyurethane foams alone, only the Barrett Hand achieved excellent performance. Tactile sensors can thus provide a key advantage even if recognition is based on stiffness rather than shape. The dataset with 24000 measurements is publicly available. △ Less

Submitted 2 February, 2024; v1 submitted 13 April, 2022; originally announced April 2022.

Comments: 12 pages, 9 figures

ACM Class: I.2.9

Learning the Effect of Registration Hyperparameters with HyperMorph

Authors: Andrew Hoopes, Malte Hoffmann, Douglas N. Greve, Bruce Fischl, John Guttag, Adrian V. Dalca

Abstract: We introduce HyperMorph, a framework that facilitates efficient hyperparameter tuning in learning-based deformable image registration. Classical registration algorithms perform an iterative pair-wise optimization to compute a deformation field that aligns two images. Recent learning-based approaches leverage large image datasets to learn a function that rapidly estimates a deformation for a given… ▽ More We introduce HyperMorph, a framework that facilitates efficient hyperparameter tuning in learning-based deformable image registration. Classical registration algorithms perform an iterative pair-wise optimization to compute a deformation field that aligns two images. Recent learning-based approaches leverage large image datasets to learn a function that rapidly estimates a deformation for a given image pair. In both strategies, the accuracy of the resulting spatial correspondences is strongly influenced by the choice of certain hyperparameter values. However, an effective hyperparameter search consumes substantial time and human effort as it often involves training multiple models for different fixed hyperparameter values and may lead to suboptimal registration. We propose an amortized hyperparameter learning strategy to alleviate this burden by learning the impact of hyperparameters on deformation fields. We design a meta network, or hypernetwork, that predicts the parameters of a registration network for input hyperparameters, thereby comprising a single model that generates the optimal deformation field corresponding to given hyperparameter values. This strategy enables fast, high-resolution hyperparameter search at test-time, reducing the inefficiency of traditional approaches while increasing flexibility. We also demonstrate additional benefits of HyperMorph, including enhanced robustness to model initialization and the ability to rapidly identify optimal hyperparameter values specific to a dataset, image contrast, task, or even anatomical region, all without the need to retrain models. We make our code publicly available at http://hypermorph.voxelmorph.net. △ Less

Submitted 30 March, 2022; originally announced March 2022.

Comments: Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) at https://www.melba-journal.org

SynthStrip: Skull-Stripping for Any Brain Image

Authors: Andrew Hoopes, Jocelyn S. Mora, Adrian V. Dalca, Bruce Fischl, Malte Hoffmann

Abstract: The removal of non-brain signal from magnetic resonance imaging (MRI) data, known as skull-stripping, is an integral component of many neuroimage analysis streams. Despite their abundance, popular classical skull-stripping methods are usually tailored to images with specific acquisition properties, namely near-isotropic resolution and T1-weighted (T1w) MRI contrast, which are prevalent in research… ▽ More The removal of non-brain signal from magnetic resonance imaging (MRI) data, known as skull-stripping, is an integral component of many neuroimage analysis streams. Despite their abundance, popular classical skull-stripping methods are usually tailored to images with specific acquisition properties, namely near-isotropic resolution and T1-weighted (T1w) MRI contrast, which are prevalent in research settings. As a result, existing tools tend to adapt poorly to other image types, such as stacks of thick slices acquired with fast spin-echo (FSE) MRI that are common in the clinic. While learning-based approaches for brain extraction have gained traction in recent years, these methods face a similar burden, as they are only effective for image types seen during the training procedure. To achieve robust skull-stripping across a landscape of imaging protocols, we introduce SynthStrip, a rapid, learning-based brain-extraction tool. By leveraging anatomical segmentations to generate an entirely synthetic training dataset with anatomies, intensity distributions, and artifacts that far exceed the realistic range of medical images, SynthStrip learns to successfully generalize to a variety of real acquired brain images, removing the need for training data with target contrasts. We demonstrate the efficacy of SynthStrip for a diverse set of image acquisitions and resolutions across subject populations, ranging from newborn to adult. We show substantial improvements in accuracy over popular skull-stripping baselines -- all with a single trained model. Our method and labeled evaluation data are available at https://w3id.org/synthstrip. △ Less

Submitted 26 July, 2022; v1 submitted 18 March, 2022; originally announced March 2022.

Comments: 19 pages, 9 figures, 7 tables, skull stripping, brain extraction, image synthesis, MRI-contrast agnosticism, deep learning, final published version

Journal ref: Neuroimage 260, 2022

Effect of Active and Passive Protective Soft Skins on Collision Forces in Human-robot Collaboration

Authors: Petr Svarny, Jakub Rozlivek, Lukas Rustler, Martin Sramek, Ozgur Deli, Michael Zillich, Matej Hoffmann

Abstract: Soft electronic skins are one of the means to turn an industrial manipulator into a collaborative robot. For manipulators that are already fit for physical human-robot collaboration, soft skins can make them safer. In this work, we study the after impact behavior of two collaborative manipulators (UR10e and KUKA LBR iiwa) and one classical industrial manipulator (KUKA Cybertech), in presence or ab… ▽ More Soft electronic skins are one of the means to turn an industrial manipulator into a collaborative robot. For manipulators that are already fit for physical human-robot collaboration, soft skins can make them safer. In this work, we study the after impact behavior of two collaborative manipulators (UR10e and KUKA LBR iiwa) and one classical industrial manipulator (KUKA Cybertech), in presence or absence of an industrial protective skin (AIRSKIN). In addition, we isolate the effects of the passive padding and the active contribution of the sensor to robot reaction. We present a total of 2250 collision measurements and study the impact force, contact duration, clamping force, and impulse. The dataset is publicly available. We summarize our results as follows. For transient collisions, the passive skin properties lowered the impact forces by about 40 %. During quasi-static contact, the effect of skin covers -- active or passive -- cannot be isolated from the collision detection and reaction by the collaborative robots. Important effects of the stop categories triggered by the active protective skin were found. We systematically compare the different settings and the empirically established safe velocities with prescriptions by the ISO/TS 15066. In some cases, up to the quadruple of the ISO/TS 15066 prescribed velocity can comply with the impact force limits and thus be considered safe. We propose an extension of the formulas relating impact force and permissible velocity that take into account the stiffness and compressible thickness of the protective cover, leading to better predictions of the collision forces. At the same time, this work emphasizes the need for in situ measurements as all the factors we studied -- presence of active/passive skin, safety stop settings, robot collision reaction, impact direction, and, of course, velocity -- have effects on the force evolution after impact. △ Less

Submitted 19 May, 2022; v1 submitted 18 March, 2022; originally announced March 2022.

Comments: 18 pages, 15 figures

ACM Class: I.2.9

Journal ref: Robotics and Computer-Integrated Manufacturing 2022, Volume 78, 102363

Active Visuo-Haptic Object Shape Completion

Authors: Lukas Rustler, Jens Lundell, Jan Kristof Behrens, Ville Kyrki, Matej Hoffmann

Abstract: Recent advancements in object shape completion have enabled impressive object reconstructions using only visual input. However, due to self-occlusion, the reconstructions have high uncertainty in the occluded object parts, which negatively impacts the performance of downstream robotic tasks such as grasping. In this work, we propose an active visuo-haptic shape completion method called Act-VH that… ▽ More Recent advancements in object shape completion have enabled impressive object reconstructions using only visual input. However, due to self-occlusion, the reconstructions have high uncertainty in the occluded object parts, which negatively impacts the performance of downstream robotic tasks such as grasping. In this work, we propose an active visuo-haptic shape completion method called Act-VH that actively computes where to touch the objects based on the reconstruction uncertainty. Act-VH reconstructs objects from point clouds and calculates the reconstruction uncertainty using IGR, a recent state-of-the-art implicit surface deep neural network. We experimentally evaluate the reconstruction accuracy of Act-VH against five baselines in simulation and in the real world. We also propose a new simulation environment for this purpose. The results show that Act-VH outperforms all baselines and that an uncertainty-driven haptic exploration policy leads to higher reconstruction accuracy than a random policy and a policy driven by Gaussian Process Implicit Surfaces. As a final experiment, we evaluate Act-VH and the best reconstruction baseline on grasping 10 novel objects. The results show that Act-VH reaches a significantly higher grasp success rate than the baseline on all objects. Together, this work opens up the door for using active visuo-haptic shape completion in more complex cluttered scenes. △ Less

Submitted 17 March, 2022; originally announced March 2022.

Comments: 8 pages, 7 figures

Journal ref: IEEE Robotics and Automation Letters 7 (2) 2022, 5254-5261

Body Models in Humans and Robots

Authors: Matej Hoffmann, Matthew R. Longo

Abstract: Neurocognitive models of higher-level somatosensory processing have emphasised the role of stored body representations in interpreting real-time sensory signals coming from the body (Longo, Azanon and Haggard, 2010; Tame, Azanon and Longo, 2019). The need for such stored representations arises from the fact that immediate sensory signals coming from the body do not specify metric details about bod… ▽ More Neurocognitive models of higher-level somatosensory processing have emphasised the role of stored body representations in interpreting real-time sensory signals coming from the body (Longo, Azanon and Haggard, 2010; Tame, Azanon and Longo, 2019). The need for such stored representations arises from the fact that immediate sensory signals coming from the body do not specify metric details about body size and shape. Several aspects of somatoperception, therefore, require that immediate sensory signals be combined with stored body representations. This basic problem is equally true for humanoid robots and, intriguingly, neurocognitive models developed to explain human perception are strikingly similar to those developed independently for localizing touch on humanoid robots, such as the iCub, equipped with artificial electronic skin on the majority of its body surface (Roncone et al., 2014; Hoffmann, 2021). In this chapter, we will review the key features of these models, discuss their similarities and differences to each other, and to other models in the literature. Using robots as embodied computational models is an example of synthetic methodology or 'understanding by building' (e.g., Hoffmann and Pfeifer, 2018), computational embodied neuroscience (Caligiore et al., 2010) or 'synthetic psychology of the self' (Prescott and Camilleri, 2019). Such models have the advantage that they need to be worked out into every detail, making any theory explicit and complete. There is also an additional way of (pre)validating such a theory other than comparing to the biological or psychological phenomenon studied by simply verifying that a particular implementation really performs the task: can the robot localize where it is being touched (see https://youtu.be/pfse424t5mQ)? △ Less

Submitted 20 January, 2022; originally announced January 2022.

Comments: 14 pages, 2 figures

Journal ref: The Routledge Handbook of Bodily Awareness 2022

Frequency Selection for Platoon Communications in Secondary Spectrum Using Radio Environment Maps

Authors: Marcin Hoffmann, Pawel Kryszkiewicz, Adrian Kliks

Abstract: Platoon-based driving is an idea that vehicles follow each other at a close distance, in order to increase road throughput and fuel savings. This requires reliable wireless communications to adjust the speeds of vehicles. Although there is a dedicated frequency band for vehicle-to-vehicle (V2V) communications, studies have shown that it is too congested to provide reliable transmission for the pla… ▽ More Platoon-based driving is an idea that vehicles follow each other at a close distance, in order to increase road throughput and fuel savings. This requires reliable wireless communications to adjust the speeds of vehicles. Although there is a dedicated frequency band for vehicle-to-vehicle (V2V) communications, studies have shown that it is too congested to provide reliable transmission for the platoons. Additional spectrum resources, i.e., secondary spectrum channels, can be utilized when these are not occupied by other users. Characteristics of interference in these channels are usually location-dependent and can be stored in the so-called Radio Environment Maps (REMs). This paper aims to design REM, in order to support the selection of secondary spectrum channel for intra-platoon communications. We propose to assess the channel's quality in terms of outage probability computed, with the use of estimated interference distributions stored in REM. A frequency selection algorithm that minimizes the number of channel switches along the planned platoon route is proposed. Additionally, the REM creation procedure is shown that reduces the number of database entries using (Density-Based Spatial Clustering of Applications with Noise) DBSCAN algorithm. The proposals are tested using real IQ samples captured on a real road. Application of the DBSCAN clustering to the constructed REM provided 7% reduction in its size. Utilization of the proposed channel selection algorithm resulted in a 35 times reduction of channel switches concerning channel assignment performed independently in every location. △ Less

Submitted 30 December, 2021; originally announced December 2021.

Journal ref: IEEE Transactions on Intelligent Transportation Systems ( Early Access ), 2021

Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes

Authors: Lukas Galke, Iacopo Vagliano, Benedikt Franke, Tobias Zielke, Marcel Hoffmann, Ansgar Scherp

Abstract: Lifelong graph learning deals with the problem of continually adapting graph neural network (GNN) models to changes in evolving graphs. We address two critical challenges of lifelong graph learning in this work: dealing with new classes and tackling imbalanced class distributions. The combination of these two challenges is particularly relevant since newly emerging classes typically resemble only… ▽ More Lifelong graph learning deals with the problem of continually adapting graph neural network (GNN) models to changes in evolving graphs. We address two critical challenges of lifelong graph learning in this work: dealing with new classes and tackling imbalanced class distributions. The combination of these two challenges is particularly relevant since newly emerging classes typically resemble only a tiny fraction of the data, adding to the already skewed class distribution. We make several contributions: First, we show that the amount of unlabeled data does not influence the results, which is an essential prerequisite for lifelong learning on a sequence of tasks. Second, we experiment with different label rates and show that our methods can perform well with only a tiny fraction of annotated nodes. Third, we propose the gDOC method to detect new classes under the constraint of having an imbalanced class distribution. The critical ingredient is a weighted binary cross-entropy loss function to account for the class imbalance. Moreover, we demonstrate combinations of gDOC with various base GNN models such as GraphSAGE, Simplified Graph Convolution, and Graph Attention Networks. Lastly, our k-neighborhood time difference measure provably normalizes the temporal changes across different graph datasets. With extensive experimentation, we find that the proposed gDOC method is consistently better than a naive adaption of DOC to graphs. Specifically, in experiments using the smallest history size, the out-of-distribution detection score of gDOC is 0.09 compared to 0.01 for DOC. Furthermore, gDOC achieves an Open-F1 score, a combined measure of in-distribution classification and out-of-distribution detection, of 0.33 compared to 0.25 of DOC (32% increase). △ Less

Submitted 9 May, 2023; v1 submitted 20 December, 2021; originally announced December 2021.

Comments: Accepted manuscript (after peer review, before copy-editing). Published article available at https://doi.org/10.1016/j.neunet.2023.04.022

ACM Class: I.2.6

Journal ref: Neural Networks 164 (2023) 156-176

The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients

Authors: Bhakti Baheti, Satrajit Chakrabarty, Hamed Akbari, Michel Bilello, Benedikt Wiestler, Julian Schwarting, Evan Calabrese, Jeffrey Rudie, Syed Abidi, Mina Mousa, Javier Villanueva-Meyer, Brandon K. K. Fields, Florian Kofler, Russell Takeshi Shinohara, Juan Eugenio Iglesias, Tony C. W. Mok, Albert C. S. Chung, Marek Wodzinski, Artur Jurgas, Niccolo Marini, Manfredo Atzori, Henning Muller, Christoph Grobroehmer, Hanna Siebert, Lasse Hansen , et al. (48 additional authors not shown)

Abstract: Registration of longitudinal brain MRI scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registr… ▽ More Registration of longitudinal brain MRI scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registration (BraTS-Reg) challenge, as the first public benchmark environment for deformable registration algorithms focusing on estimating correspondences between pre-operative and follow-up scans of the same patient diagnosed with a diffuse brain glioma. The BraTS-Reg data comprise de-identified multi-institutional multi-parametric MRI (mpMRI) scans, curated for size and resolution according to a canonical anatomical template, and divided into training, validation, and testing sets. Clinical experts annotated ground truth (GT) landmark points of anatomical locations distinct across the temporal domain. Quantitative evaluation and ranking were based on the Median Euclidean Error (MEE), Robustness, and the determinant of the Jacobian of the displacement field. The top-ranked methodologies yielded similar performance across all evaluation metrics and shared several methodological commonalities, including pre-alignment, deep neural networks, inverse consistency analysis, and test-time instance optimization per-case basis as a post-processing step. The top-ranked method attained the MEE at or below that of the inter-rater variability for approximately 60% of the evaluated landmarks, underscoring the scope for further accuracy and robustness improvements, especially relative to human experts. The aim of BraTS-Reg is to continue to serve as an active resource for research, with the data and online evaluation tools accessible at https://bratsreg.github.io/. △ Less

Submitted 17 April, 2024; v1 submitted 13 December, 2021; originally announced December 2021.

Anomaly Detection in IR Images of PV Modules using Supervised Contrastive Learning

Authors: Lukas Bommes, Mathis Hoffmann, Claudia Buerhop-Lutz, Tobias Pickel, Jens Hauch, Christoph Brabec, Andreas Maier, Ian Marius Peters

Abstract: Increasing deployment of photovoltaic (PV) plants requires methods for automatic detection of faulty PV modules in modalities, such as infrared (IR) images. Recently, deep learning has become popular for this. However, related works typically sample train and test data from the same distribution ignoring the presence of domain shift between data of different PV plants. Instead, we frame fault dete… ▽ More Increasing deployment of photovoltaic (PV) plants requires methods for automatic detection of faulty PV modules in modalities, such as infrared (IR) images. Recently, deep learning has become popular for this. However, related works typically sample train and test data from the same distribution ignoring the presence of domain shift between data of different PV plants. Instead, we frame fault detection as more realistic unsupervised domain adaptation problem where we train on labelled data of one source PV plant and make predictions on another target plant. We train a ResNet-34 convolutional neural network with a supervised contrastive loss, on top of which we employ a k-nearest neighbor classifier to detect anomalies. Our method achieves a satisfactory area under the receiver operating characteristic (AUROC) of 73.3 % to 96.6 % on nine combinations of four source and target datasets with 2.92 million IR images of which 8.5 % are anomalous. It even outperforms a binary cross-entropy classifier in some cases. With a fixed decision threshold this results in 79.4 % and 77.1 % correctly classified normal and anomalous images, respectively. Most misclassified anomalies are of low severity, such as hot diodes and small hot spots. Our method is insensitive to hyperparameter settings, converges quickly and reliably detects unknown types of anomalies making it well suited for practice. Possible uses are in automatic PV plant inspection systems or to streamline manual labelling of IR datasets by filtering out normal images. Furthermore, our work serves the community with a more realistic view on PV module fault detection using unsupervised domain adaptation to develop more performant methods with favorable generalization capabilities. △ Less

Submitted 6 December, 2021; originally announced December 2021.

Deeptime: a Python library for machine learning dynamical models from time series data

Authors: Moritz Hoffmann, Martin Scherer, Tim Hempel, Andreas Mardt, Brian de Silva, Brooke E. Husic, Stefan Klus, Hao Wu, Nathan Kutz, Steven L. Brunton, Frank Noé

Abstract: Generation and analysis of time-series data is relevant to many quantitative fields ranging from economics to fluid mechanics. In the physical sciences, structures such as metastable and coherent sets, slow relaxation processes, collective variables dominant transition pathways or manifolds and channels of probability flow can be of great importance for understanding and characterizing the kinetic… ▽ More Generation and analysis of time-series data is relevant to many quantitative fields ranging from economics to fluid mechanics. In the physical sciences, structures such as metastable and coherent sets, slow relaxation processes, collective variables dominant transition pathways or manifolds and channels of probability flow can be of great importance for understanding and characterizing the kinetic, thermodynamic and mechanistic properties of the system. Deeptime is a general purpose Python library offering various tools to estimate dynamical models based on time-series data including conventional linear learning methods, such as Markov state models (MSMs), Hidden Markov Models and Koopman models, as well as kernel and deep learning approaches such as VAMPnets and deep MSMs. The library is largely compatible with scikit-learn, having a range of Estimator classes for these different models, but in contrast to scikit-learn also provides deep Model classes, e.g. in the case of an MSM, which provide a multitude of analysis methods to compute interesting thermodynamic, kinetic and dynamical quantities, such as free energies, relaxation times and transition paths. The library is designed for ease of use but also easily maintainable and extensible code. In this paper we introduce the main features and structure of the deeptime software. △ Less

Submitted 11 December, 2021; v1 submitted 28 October, 2021; originally announced October 2021.

Journal ref: Machine Learning: Science and Technology, Volume 3, Number 1, 2021

Rapid head-pose detection for automated slice prescription of fetal-brain MRI

Authors: Malte Hoffmann, Esra Abaci Turk, Borjan Gagoski, Leah Morgan, Paul Wighton, M. Dylan Tisdall, Martin Reuter, Elfar Adalsteinsson, P. Ellen Grant, Lawrence L. Wald, André J. W. van der Kouwe

Abstract: In fetal-brain MRI, head-pose changes between prescription and acquisition present a challenge to obtaining the standard sagittal, coronal and axial views essential to clinical assessment. As motion limits acquisitions to thick slices that preclude retrospective resampling, technologists repeat ~55-second stack-of-slices scans (HASTE) with incrementally reoriented field of view numerous times, ded… ▽ More In fetal-brain MRI, head-pose changes between prescription and acquisition present a challenge to obtaining the standard sagittal, coronal and axial views essential to clinical assessment. As motion limits acquisitions to thick slices that preclude retrospective resampling, technologists repeat ~55-second stack-of-slices scans (HASTE) with incrementally reoriented field of view numerous times, deducing the head pose from previous stacks. To address this inefficient workflow, we propose a robust head-pose detection algorithm using full-uterus scout scans (EPI) which take ~5 seconds to acquire. Our ~2-second procedure automatically locates the fetal brain and eyes, which we derive from maximally stable extremal regions (MSERs). The success rate of the method exceeds 94% in the third trimester, outperforming a trained technologist by up to 20%. The pipeline may be used to automatically orient the anatomical sequence, removing the need to estimate the head pose from 2D views and reducing delays during which motion can occur. △ Less

Submitted 8 October, 2021; originally announced October 2021.

Comments: 19 pages, 10 figures, 2 tables, fetal MRI, head-pose detection, MSER, scan automation, scan prescription, slice positioning, final published version

Journal ref: Int J Imaging Syst Technol, 31 (3), 2021, 1136-1154

Module-Power Prediction from PL Measurements using Deep Learning

Authors: Mathis Hoffmann, Johannes Hepp, Bernd Doll, Claudia Buerhop-Lutz, Ian Marius Peters, Christoph Brabec, Andreas Maier, Vincent Christlein

Abstract: The individual causes for power loss of photovoltaic modules are investigated for quite some time. Recently, it has been shown that the power loss of a module is, for example, related to the fraction of inactive areas. While these areas can be easily identified from electroluminescense (EL) images, this is much harder for photoluminescence (PL) images. With this work, we close the gap between powe… ▽ More The individual causes for power loss of photovoltaic modules are investigated for quite some time. Recently, it has been shown that the power loss of a module is, for example, related to the fraction of inactive areas. While these areas can be easily identified from electroluminescense (EL) images, this is much harder for photoluminescence (PL) images. With this work, we close the gap between power regression from EL and PL images. We apply a deep convolutional neural network to predict the module power from PL images with a mean absolute error (MAE) of 4.4% or 11.7WP. Furthermore, we depict that regression maps computed from the embeddings of the trained network can be used to compute the localized power loss. Finally, we show that these regression maps can be used to identify inactive regions in PL images as well. △ Less

Submitted 31 August, 2021; originally announced August 2021.

Hafnia-based Double Layer Ferroelectric Tunnel Junctions as Artificial Synapses for Neuromorphic Computing

Authors: Benjamin Max, Michael Hoffmann, Halid Mulaosmanovic, Stefan Slesazeck, Thomas Mikolajick

Abstract: Ferroelectric tunnel junctions (FTJ) based on hafnium zirconium oxide (Hf1-xZrxO2; HZO) are a promising candidate for future applications, such as low-power memories and neuromorphic computing. The tunneling electroresistance (TER) is tunable through the polarization state of the HZO film. To circumvent the challenge of fabricating thin ferroelectric HZO layers in the tunneling range of 1-3 nm ran… ▽ More Ferroelectric tunnel junctions (FTJ) based on hafnium zirconium oxide (Hf1-xZrxO2; HZO) are a promising candidate for future applications, such as low-power memories and neuromorphic computing. The tunneling electroresistance (TER) is tunable through the polarization state of the HZO film. To circumvent the challenge of fabricating thin ferroelectric HZO layers in the tunneling range of 1-3 nm range, ferroelectric/dielectric double layer sandwiched between two symmetric metal electrodes are used. Due to the decoupling of the ferroelectric polarization storage layer and a dielectric tunneling layer with a higher bandgap, a significant TER ratio between the two polarization states is obtained. By exploiting previously reported switching behaviour and the gradual tunability of the resistance, FTJs can be used as potential candidates for the emulation of synapses for neuromorphic computing in spiking neural networks. The implementation of two major components of a synapse are shown: long term depression/potentiation by varying the amplitude/width/number of voltage pulses applied to the artificial FTJ synapse, and spike-timing-dependent-plasticity curves by applying time-delayed voltages at each electrode. These experimental findings show the potential of spiking neural networks and neuromorphic computing that can be implemented with hafnia-based FTJs. △ Less

Submitted 2 July, 2021; originally announced July 2021.

Journal ref: ACS Applied Electronic Materials 2 12 2020 4023-4033

Informed Machine Learning for Improved Similarity Assessment in Process-Oriented Case-Based Reasoning

Authors: Maximilian Hoffmann, Ralph Bergmann

Abstract: Currently, Deep Learning (DL) components within a Case-Based Reasoning (CBR) application often lack the comprehensive integration of available domain knowledge. The trend within machine learning towards so-called Informed machine learning can help to overcome this limitation. In this paper, we therefore investigate the potential of integrating domain knowledge into Graph Neural Networks (GNNs) tha… ▽ More Currently, Deep Learning (DL) components within a Case-Based Reasoning (CBR) application often lack the comprehensive integration of available domain knowledge. The trend within machine learning towards so-called Informed machine learning can help to overcome this limitation. In this paper, we therefore investigate the potential of integrating domain knowledge into Graph Neural Networks (GNNs) that are used for similarity assessment between semantic graphs within process-oriented CBR applications. We integrate knowledge in two ways: First, a special data representation and processing method is used that encodes structural knowledge about the semantic annotations of each graph node and edge. Second, the message-passing component of the GNNs is constrained by knowledge on legal node mappings. The evaluation examines the quality and training time of the extended GNNs, compared to the stock models. The results show that both extensions are capable of providing better quality, shorter training times, or in some configurations both advantages at once. △ Less

Submitted 30 June, 2021; originally announced June 2021.

Comments: Accepted at the IJCAI-21 workshop on Deep Learning, Case-Based Reasoning, and AutoML: Present and Future Synergies

ARMORY: Fully Automated and Exhaustive Fault Simulation on ARM-M Binaries

Authors: Max Hoffmann, Falk Schellenberg, Christof Paar

Abstract: Embedded systems are ubiquitous. However, physical access of users and likewise attackers makes them often threatened by fault attacks: a single fault during the computation of a cryptographic primitive can lead to a total loss of system security. This can have serious consequences, e.g., in safetycritical systems, including bodily harm and catastrophic technical failures. However, countermeasures… ▽ More Embedded systems are ubiquitous. However, physical access of users and likewise attackers makes them often threatened by fault attacks: a single fault during the computation of a cryptographic primitive can lead to a total loss of system security. This can have serious consequences, e.g., in safetycritical systems, including bodily harm and catastrophic technical failures. However, countermeasures often focus on isolated fault models and high layers of abstraction. This leads to a dangerous sense of security, because exploitable faults that are only visible at machine code level might not be covered by countermeasures. In this work we present ARMORY, a fully automated open source framework for exhaustive fault simulation on binaries of the ubiquitous ARM-M class. It allows engineers and analysts to efficiently scan a binary for potential weaknesses against arbitrary combinations of multi-variate fault injections under a large variety of fault models. Using ARMORY, we demonstrate the power of fully automated fault analysis and the dangerous implications of applying countermeasures without knowledge of physical addresses and offsets. We exemplarily analyze two case studies, which are highly relevant for practice: a DFA on AES (cryptographic) and a secure bootloader (non-cryptographic). Our results show that indeed numerous exploitable faults found by ARMORY which occur in the actual implementations are easily missed in manual inspection. Crucially, most faults are only visible when taking machine code information, i.e., addresses and offsets, into account. Surprisingly, we show that a countermeasure that protects against one type of fault can actually largely increase the vulnerability to other fault models. Our work demonstrates the need for countermeasures that, at least in their evaluation, are not restricted to isolated fault models and consider low-level information [...]. △ Less

Submitted 28 May, 2021; originally announced May 2021.

Interactive $G^1$ and $G^2$ Hermite Interpolation Using Extended Log-aesthetic Curves

Authors: Ferenc Nagy, Norimasa Yoshida, Miklós Hoffmann

Abstract: In the field of aesthetic design, log-aesthetic curves have a significant role to meet the high industrial requirements. In this paper, we propose a new interactive $G^1$ Hermite interpolation method based on the algorithm of Yoshida et al. with a minor boundary condition. In this novel approach, we compute an extended log-aesthetic curve segment that may include inflection point (S-shaped curve)… ▽ More In the field of aesthetic design, log-aesthetic curves have a significant role to meet the high industrial requirements. In this paper, we propose a new interactive $G^1$ Hermite interpolation method based on the algorithm of Yoshida et al. with a minor boundary condition. In this novel approach, we compute an extended log-aesthetic curve segment that may include inflection point (S-shaped curve) or cusp. The curve segment is defined by its endpoints, a tangent vector at the first point, and a tangent direction at the second point. The algorithm also determines the shape parameter of the log-aesthetic curve based on the length of the first tangent that provides control over the curvature of the first point and makes the method capable of joining log-aesthetic curve segments with $G^2$ continuity. △ Less

Submitted 20 April, 2021; originally announced May 2021.

Report number: Vol 19

Journal ref: Computer-Aided Design and Applications, 2022