-
Towards Generalist Robot Learning from Internet Video: A Survey
Authors:
Robert McCarthy,
Daniel C. H. Tan,
Dominik Schmidt,
Fernando Acero,
Nathan Herr,
Yilun Du,
Thomas G. Thuruthel,
Zhibin Li
Abstract:
This survey presents an overview of methods for learning from video (LfV) in the context of reinforcement learning (RL) and robotics. We focus on methods capable of scaling to large internet video datasets and, in the process, extracting foundational knowledge about the world's dynamics and physical human behaviour. Such methods hold great promise for developing general-purpose robots.
We open w…
▽ More
This survey presents an overview of methods for learning from video (LfV) in the context of reinforcement learning (RL) and robotics. We focus on methods capable of scaling to large internet video datasets and, in the process, extracting foundational knowledge about the world's dynamics and physical human behaviour. Such methods hold great promise for developing general-purpose robots.
We open with an overview of fundamental concepts relevant to the LfV-for-robotics setting. This includes a discussion of the exciting benefits LfV methods can offer (e.g., improved generalization beyond the available robot data) and commentary on key LfV challenges (e.g., challenges related to missing information in video and LfV distribution shifts). Our literature review begins with an analysis of video foundation model techniques that can extract knowledge from large, heterogeneous video datasets. Next, we review methods that specifically leverage video data for robot learning. Here, we categorise work according to which RL knowledge modality benefits from the use of video data. We additionally highlight techniques for mitigating LfV challenges, including reviewing action representations that address the issue of missing action labels in video.
Finally, we examine LfV datasets and benchmarks, before concluding the survey by discussing challenges and opportunities in LfV. Here, we advocate for scalable approaches that can leverage the full range of available data and that target the key benefits of LfV. Overall, we hope this survey will serve as a comprehensive reference for the emerging field of LfV, catalysing further research in the area, and ultimately facilitating progress towards obtaining general-purpose robots.
△ Less
Submitted 30 April, 2024;
originally announced April 2024.
-
Monocular UAV Localisation with Deep Learning and Uncertainty Propagation
Authors:
Xueyan Oh,
Ryan Lim,
Leonard Loh,
Chee How Tan,
Shaohui Foong,
U-Xuan Tan
Abstract:
In this paper, we propose a ground-based monocular UAV localisation system that detects and localises an LED marker attached to the underside of a UAV. Our system removes the need for extensive infrastructure and calibration unlike existing technologies such as UWB, radio frequency and multi-camera systems often used for localisation in GPS-denied environment. To improve deployablity for real-worl…
▽ More
In this paper, we propose a ground-based monocular UAV localisation system that detects and localises an LED marker attached to the underside of a UAV. Our system removes the need for extensive infrastructure and calibration unlike existing technologies such as UWB, radio frequency and multi-camera systems often used for localisation in GPS-denied environment. To improve deployablity for real-world applications without the need to collect extensive real dataset, we train a CNN on synthetic binary images as opposed to using real images in existing monocular UAV localisation methods, and factor in the camera's zoom to allow tracking of UAVs flying at further distances. We propose NoisyCutout algorithm for augmenting synthetic binary images to simulate binary images processed from real images and show that it improves localisation accuracy as compared to using existing salt-and-pepper and Cutout augmentation methods. We also leverage uncertainty propagation to modify the CNN's loss function and show that this also improves localisation accuracy. Real-world experiments are conducted to evaluate our methods and we achieve an overall 3D RMSE of approximately 0.41m.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
Value Functions are Control Barrier Functions: Verification of Safe Policies using Control Theory
Authors:
Daniel C. H. Tan,
Fernando Acero,
Robert McCarthy,
Dimitrios Kanoulas,
Zhibin Li
Abstract:
Guaranteeing safe behaviour of reinforcement learning (RL) policies poses significant challenges for safety-critical applications, despite RL's generality and scalability. To address this, we propose a new approach to apply verification methods from control theory to learned value functions. By analyzing task structures for safety preservation, we formalize original theorems that establish links b…
▽ More
Guaranteeing safe behaviour of reinforcement learning (RL) policies poses significant challenges for safety-critical applications, despite RL's generality and scalability. To address this, we propose a new approach to apply verification methods from control theory to learned value functions. By analyzing task structures for safety preservation, we formalize original theorems that establish links between value functions and control barrier functions. Further, we propose novel metrics for verifying value functions in safe control tasks and practical implementation details to improve learning. Our work presents a novel method for certificate learning, which unlocks a diversity of verification techniques from control theory for RL policies, and marks a significant step towards a formal framework for the general, scalable, and verifiable design of RL-based control systems. Code and videos are available at this https url: https://rl-cbf.github.io/
△ Less
Submitted 5 December, 2023; v1 submitted 6 June, 2023;
originally announced June 2023.
-
Optimizing Airbnb Search Journey with Multi-task Learning
Authors:
Chun How Tan,
Austin Chan,
Malay Haldar,
Jie Tang,
Xin Liu,
Mustafa Abdool,
Huiji Gao,
Liwei He,
Sanjeev Katariya
Abstract:
At Airbnb, an online marketplace for stays and experiences, guests often spend weeks exploring and comparing multiple items before making a final reservation request. Each reservation request may then potentially be rejected or cancelled by the host prior to check-in. The long and exploratory nature of the search journey, as well as the need to balance both guest and host preferences, present uniq…
▽ More
At Airbnb, an online marketplace for stays and experiences, guests often spend weeks exploring and comparing multiple items before making a final reservation request. Each reservation request may then potentially be rejected or cancelled by the host prior to check-in. The long and exploratory nature of the search journey, as well as the need to balance both guest and host preferences, present unique challenges for Airbnb search ranking. In this paper, we present Journey Ranker, a new multi-task deep learning model architecture that addresses these challenges. Journey Ranker leverages intermediate guest actions as milestones, both positive and negative, to better progress the guest towards a successful booking. It also uses contextual information such as guest state and search query to balance guest and host preferences. Its modular and extensible design, consisting of four modules with clear separation of concerns, allows for easy application to use cases beyond the Airbnb search ranking context. We conducted offline and online testing of the Journey Ranker and successfully deployed it in production to four different Airbnb products with significant business metrics improvements.
△ Less
Submitted 28 May, 2023;
originally announced May 2023.
-
Perceptive Locomotion with Controllable Pace and Natural Gait Transitions Over Uneven Terrains
Authors:
Daniel Chee Hian Tan,
Jenny Zhang,
Michael,
Chuah,
Zhibin Li
Abstract:
This work developed a learning framework for perceptive legged locomotion that combines visual feedback, proprioceptive information, and active gait regulation of foot-ground contacts. The perception requires only one forward-facing camera to obtain the heightmap, and the active regulation of gait paces and traveling velocity are realized through our formulation of CPG-based high-level imitation o…
▽ More
This work developed a learning framework for perceptive legged locomotion that combines visual feedback, proprioceptive information, and active gait regulation of foot-ground contacts. The perception requires only one forward-facing camera to obtain the heightmap, and the active regulation of gait paces and traveling velocity are realized through our formulation of CPG-based high-level imitation of foot-ground contacts. Through this framework, an end-user has the ability to command task-level inputs to control different walking speeds and gait frequencies according to the traversal of different terrains, which enables more reliable negotiation with encountered obstacles. The results demonstrated that the learned perceptive locomotion policy followed task-level control inputs with intended behaviors, and was robust in presence of unseen terrains and external force perturbations. A video demonstration can be found at https://youtu.be/OTzlWzDfAe8, and the codebase at https://github.com/jennyzzt/perceptual-locomotion.
△ Less
Submitted 30 January, 2023; v1 submitted 25 January, 2023;
originally announced January 2023.
-
Using Deep Learning with Large Aggregated Datasets for COVID-19 Classification from Cough
Authors:
Esin Darici Haritaoglu,
Nicholas Rasmussen,
Daniel C. H. Tan,
Jennifer Ranjani J.,
Jaclyn Xiao,
Gunvant Chaudhari,
Akanksha Rajput,
Praveen Govindan,
Christian Canham,
Wei Chen,
Minami Yamaura,
Laura Gomezjurado,
Aaron Broukhim,
Amil Khanzada,
Mert Pilanci
Abstract:
The Covid-19 pandemic has been one of the most devastating events in recent history, claiming the lives of more than 5 million people worldwide. Even with the worldwide distribution of vaccines, there is an apparent need for affordable, reliable, and accessible screening techniques to serve parts of the World that do not have access to Western medicine. Artificial Intelligence can provide a soluti…
▽ More
The Covid-19 pandemic has been one of the most devastating events in recent history, claiming the lives of more than 5 million people worldwide. Even with the worldwide distribution of vaccines, there is an apparent need for affordable, reliable, and accessible screening techniques to serve parts of the World that do not have access to Western medicine. Artificial Intelligence can provide a solution utilizing cough sounds as a primary screening mode for COVID-19 diagnosis. This paper presents multiple models that have achieved relatively respectable performance on the largest evaluation dataset currently presented in academic literature. Through investigation of a self-supervised learning model (Area under the ROC curve, AUC = 0.807) and a convolutional nerual network (CNN) model (AUC = 0.802), we observe the possibility of model bias with limited datasets. Moreover, we observe that performance increases with training data size, showing the need for the worldwide collection of data to help combat the Covid-19 pandemic with non-traditional means.
△ Less
Submitted 29 March, 2022; v1 submitted 5 January, 2022;
originally announced January 2022.
-
Online Semi-Supervised Concept Drift Detection with Density Estimation
Authors:
Chang How Tan,
Vincent CS Lee,
Mahsa Salehi
Abstract:
Concept drift is formally defined as the change in joint distribution of a set of input variables X and a target variable y. The two types of drift that are extensively studied are real drift and virtual drift where the former is the change in posterior probabilities p(y|X) while the latter is the change in distribution of X without affecting the posterior probabilities. Many approaches on concept…
▽ More
Concept drift is formally defined as the change in joint distribution of a set of input variables X and a target variable y. The two types of drift that are extensively studied are real drift and virtual drift where the former is the change in posterior probabilities p(y|X) while the latter is the change in distribution of X without affecting the posterior probabilities. Many approaches on concept drift detection either assume full availability of data labels, y or handle only the virtual drift. In a streaming environment, the assumption of full availability of data labels, y is questioned. On the other hand, approaches that deal with virtual drift failed to address real drift. Rather than improving the state-of-the-art methods, this paper presents a semi-supervised framework to deal with the challenges above. The objective of the proposed framework is to learn from streaming environment with limited data labels, y and detect real drift concurrently. This paper proposes a novel concept drift detection method utilizing the densities of posterior probabilities in partially labeled streaming environments. Experimental results on both synthetic and realworld datasets show that our proposed semi-supervised framework enables the detection of concept drift in such environment while achieving comparable prediction performance to the state-of-the-art methods.
△ Less
Submitted 10 November, 2019; v1 submitted 24 September, 2019;
originally announced September 2019.
-
Key Recovery Attack on Rank Quasi-Cyclic Code-based Signature Scheme
Authors:
Terry Shue Chien Lau,
Chik How Tan
Abstract:
Rank Quasi-Cyclic Signature (RQCS) is a rank metric code-based signature scheme based on the Rank Quasi-Cyclic Syndrome Decoding (RQCSD) problem proposed by Song et al. in [2]. Their paper was accepted in the 22nd International Conference on Practice and Theory of Public Key Cryptography (PKC 2019). They have also shown that RQCS is EUF-CMA in the random oracle model. This short paper describes ho…
▽ More
Rank Quasi-Cyclic Signature (RQCS) is a rank metric code-based signature scheme based on the Rank Quasi-Cyclic Syndrome Decoding (RQCSD) problem proposed by Song et al. in [2]. Their paper was accepted in the 22nd International Conference on Practice and Theory of Public Key Cryptography (PKC 2019). They have also shown that RQCS is EUF-CMA in the random oracle model. This short paper describes how to recover the secret key in RQCS with practical simulations. Our experimental results show that we are able to recover the secret key of RQCS in less than 41 seconds for all the proposed schemes at 128-bit, 192-bit and 256-bit security level.
△ Less
Submitted 1 February, 2019;
originally announced February 2019.
-
On the near prime-order MNT curves
Authors:
Duc-Phong Le,
Nadia El Mrabet,
Safia Haloui,
Chik How Tan
Abstract:
In their seminar paper, Miyaji, Nakabayashi and Takano introduced the first method to construct families of prime-order elliptic curves with small embedding degrees, namely k = 3, 4, and 6. These curves, so-called MNT curves, were then extended by Scott and Barreto, and also Galbraith, McKee and Valenca to near prime-order curves with the same embedding degrees. In this paper, we extend the method…
▽ More
In their seminar paper, Miyaji, Nakabayashi and Takano introduced the first method to construct families of prime-order elliptic curves with small embedding degrees, namely k = 3, 4, and 6. These curves, so-called MNT curves, were then extended by Scott and Barreto, and also Galbraith, McKee and Valenca to near prime-order curves with the same embedding degrees. In this paper, we extend the method of Scott and Barreto to introduce an explicit and simple algorithm that is able to generate all families of MNT curves with any given cofactor. Furthermore, we analyze the number of potential families of these curves that could be obtained for a given embedding degree $k$ and a cofactor h. We then discuss the generalized Pell equations that allow us to construct particular curves. Finally, we provide statistics of the near prime-order MNT curves.
△ Less
Submitted 7 June, 2018;
originally announced June 2018.
-
Six Constructions of Difference Families
Authors:
Cunsheng Ding,
Chik How Tan,
Yin Tan
Abstract:
In this paper, six constructions of difference families are presented. These constructions make use of difference sets, almost difference sets and disjoint difference families, and give new point of views of relationships among these combinatorial objects. Most of the constructions work for all finite groups. Though these constructions look simple, they produce many difference families with new pa…
▽ More
In this paper, six constructions of difference families are presented. These constructions make use of difference sets, almost difference sets and disjoint difference families, and give new point of views of relationships among these combinatorial objects. Most of the constructions work for all finite groups. Though these constructions look simple, they produce many difference families with new parameters. In addition to the six new constructions, new results about intersection numbers are also derived.
△ Less
Submitted 13 November, 2014;
originally announced November 2014.
-
A new construction of differentially 4-uniform permutations over $F_{2^{2k}}$
Authors:
Jie Peng,
Chik How Tan,
Qichun Wang
Abstract:
Permutations over $F_{2^{2k}}$ with low differential uniform, high algebraic degree and high nonlinearity are of great cryptographical importance since they can be chosen as the substitution boxes (S-boxes) for many block ciphers. A well known example is that the Advanced Encryption Standard (AES) chooses a differentially 4-uniform permutation, the multiplicative inverse function, as its S-box. In…
▽ More
Permutations over $F_{2^{2k}}$ with low differential uniform, high algebraic degree and high nonlinearity are of great cryptographical importance since they can be chosen as the substitution boxes (S-boxes) for many block ciphers. A well known example is that the Advanced Encryption Standard (AES) chooses a differentially 4-uniform permutation, the multiplicative inverse function, as its S-box. In this paper, we present a new construction of differentially 4-uniformity permutations over even characteristic finite fields and obtain many new CCZ-inequivalent functions. All the functions are switching neighbors in the narrow sense of the multiplicative inverse function and have the optimal algebraic degree and high nonlinearity.
△ Less
Submitted 18 July, 2014;
originally announced July 2014.
-
Further Refinements of Miller Algorithm on Edwards curves
Authors:
Duc-Phong Le,
Chik How Tan
Abstract:
Recently, Edwards curves have received a lot of attention in the cryptographic community due to their fast scalar multiplication algorithms. Then, many works on the application of these curves to pairing-based cryptography have been introduced. Xu and Lin (CT-RSA, 2010) presented refinements to improve the Miller algorithm that is central role compute pairings on Edwards curves. In this paper, we…
▽ More
Recently, Edwards curves have received a lot of attention in the cryptographic community due to their fast scalar multiplication algorithms. Then, many works on the application of these curves to pairing-based cryptography have been introduced. Xu and Lin (CT-RSA, 2010) presented refinements to improve the Miller algorithm that is central role compute pairings on Edwards curves. In this paper, we study further refinements to Miller algorithm. Our approach is generic, hence it allow to compute both Weil and Tate pairings on pairing-friendly Edwards curves of any embedding degree. We analyze and show that our algorithm is faster than the original Miller algorithm and the Xu-Lin's refinements.
△ Less
Submitted 4 August, 2014; v1 submitted 13 May, 2013;
originally announced May 2013.
-
Breaking the Estream Finalists and AES Modes of Operation Faster than Exhaustive Search
Authors:
Khoongming Khoo,
Chik How Tan
Abstract:
This paper explores the time-memory-data trade-off attack on stream and block ciphers.
This paper explores the time-memory-data trade-off attack on stream and block ciphers.
△ Less
Submitted 22 October, 2012; v1 submitted 1 July, 2012;
originally announced July 2012.
-
On a Class of Quadratic Polynomials with no Zeros and its Application to APN Functions
Authors:
Carl Bracken,
Chik How Tan,
Tan Yin
Abstract:
We show that the there exists an infinite family of APN functions of the form $F(x)=x^{2^{s}+1} + x^{2^{k+s}+2^k} + cx^{2^{k+s}+1} + c^{2^k}x^{2^k + 2^s} + δx^{2^{k}+1}$, over $\gf_{2^{2k}}$, where $k$ is an even integer and $\gcd(2k,s)=1, 3\nmid k$. This is actually a proposed APN family of Lilya Budaghyan and Claude Carlet who show in \cite{carlet-1} that the function is APN when there exists…
▽ More
We show that the there exists an infinite family of APN functions of the form $F(x)=x^{2^{s}+1} + x^{2^{k+s}+2^k} + cx^{2^{k+s}+1} + c^{2^k}x^{2^k + 2^s} + δx^{2^{k}+1}$, over $\gf_{2^{2k}}$, where $k$ is an even integer and $\gcd(2k,s)=1, 3\nmid k$. This is actually a proposed APN family of Lilya Budaghyan and Claude Carlet who show in \cite{carlet-1} that the function is APN when there exists $c$ such that the polynomial $y^{2^s+1}+cy^{2^s}+c^{2^k}y+1=0$ has no solutions in the field $\gf_{2^{2k}}$. In \cite{carlet-1} they demonstrate by computer that such elements $c$ can be found over many fields, particularly when the degree of the field is not divisible by 3. We show that such $c$ exists when $k$ is even and $3\nmid k$ (and demonstrate why the $k$ odd case only re-describes an existing family of APN functions). The form of these coefficients is given so that we may write the infinite family of APN functions.
△ Less
Submitted 14 October, 2011;
originally announced October 2011.
