Showing 1–50 of 528 results for author: Zhang, Z

Goal-Oriented Sequential Bayesian Experimental Design for Causal Learning

Authors: Zheyu Zhang, Jiayuan Dong, Jie Liu, Xun Huan

Abstract: We present GO-CBED, a goal-oriented Bayesian framework for sequential causal experimental design. Unlike conventional approaches that select interventions aimed at inferring the full causal model, GO-CBED directly maximizes the expected information gain (EIG) on user-specified causal quantities of interest, enabling more targeted and efficient experimentation. The framework is both non-myopic, opt… ▽ More We present GO-CBED, a goal-oriented Bayesian framework for sequential causal experimental design. Unlike conventional approaches that select interventions aimed at inferring the full causal model, GO-CBED directly maximizes the expected information gain (EIG) on user-specified causal quantities of interest, enabling more targeted and efficient experimentation. The framework is both non-myopic, optimizing over entire intervention sequences, and goal-oriented, targeting only model aspects relevant to the causal query. To address the intractability of exact EIG computation, we introduce a variational lower bound estimator, optimized jointly through a transformer-based policy network and normalizing flow-based variational posteriors. The resulting policy enables real-time decision-making via an amortized network. We demonstrate that GO-CBED consistently outperforms existing baselines across various causal reasoning and discovery tasks-including synthetic structural causal models and semi-synthetic gene regulatory networks-particularly in settings with limited experimental budgets and complex causal mechanisms. Our results highlight the benefits of aligning experimental design objectives with specific research goals and of forward-looking sequential planning. △ Less

Submitted 9 July, 2025; originally announced July 2025.

Comments: 10 pages, 6 figures

A New and Efficient Debiased Estimation of General Treatment Models by Balanced Neural Networks Weighting

Authors: Zeqi Wu, Meilin Wang, Wei Huang, Zheng Zhang

Abstract: Estimation and inference of treatment effects under unconfounded treatment assignments often suffer from bias and the `curse of dimensionality' due to the nonparametric estimation of nuisance parameters for high-dimensional confounders. Although debiased state-of-the-art methods have been proposed for binary treatments under particular treatment models, they can be unstable for small sample sizes.… ▽ More Estimation and inference of treatment effects under unconfounded treatment assignments often suffer from bias and the `curse of dimensionality' due to the nonparametric estimation of nuisance parameters for high-dimensional confounders. Although debiased state-of-the-art methods have been proposed for binary treatments under particular treatment models, they can be unstable for small sample sizes. Moreover, directly extending them to general treatment models can lead to computational complexity. We propose a balanced neural networks weighting method for general treatment models, which leverages deep neural networks to alleviate the curse of dimensionality while retaining optimal covariate balance through calibration, thereby achieving debiased and robust estimation. Our method accommodates a wide range of treatment models, including average, quantile, distributional, and asymmetric least squares treatment effects, for discrete, continuous, and mixed treatments. Under regularity conditions, we show that our estimator achieves rate double robustness and $\sqrt{N}$-asymptotic normality, and its asymptotic variance achieves the semiparametric efficiency bound. We further develop a statistical inference procedure based on weighted bootstrap, which avoids estimating the efficient influence/score functions. Simulation results reveal that the proposed method consistently outperforms existing alternatives, especially when the sample size is small. Applications to the 401(k) dataset and the Mother's Significant Features dataset further illustrate the practical value of the method for estimating both average and quantile treatment effects under binary and continuous treatments, respectively. △ Less

Submitted 5 July, 2025; originally announced July 2025.

Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift

Authors: Zhen Zhang, Xin Liu, Shaoli Wang, Jiaye Teng

Abstract: Covariate shift occurs when the distribution of input features differs between the training and testing phases. In covariate shift, estimating an unknown function's moment is a classical problem that remains under-explored, despite its common occurrence in real-world scenarios. In this paper, we investigate the minimax lower bound of the problem when the source and target distributions are known.… ▽ More Covariate shift occurs when the distribution of input features differs between the training and testing phases. In covariate shift, estimating an unknown function's moment is a classical problem that remains under-explored, despite its common occurrence in real-world scenarios. In this paper, we investigate the minimax lower bound of the problem when the source and target distributions are known. To achieve the minimax optimal bound (up to a logarithmic factor), we propose a two-stage algorithm. Specifically, it first trains an optimal estimator for the function under the source distribution, and then uses a likelihood ratio reweighting procedure to calibrate the moment estimator. In practice, the source and target distributions are typically unknown, and estimating the likelihood ratio may be unstable. To solve this problem, we propose a truncated version of the estimator that ensures double robustness and provide the corresponding upper bound. Extensive numerical studies on synthetic examples confirm our theoretical findings and further illustrate the effectiveness of our proposed method. △ Less

Submitted 29 June, 2025; originally announced June 2025.

Optimal Convergence Rates of Deep Neural Network Classifiers

Authors: Zihan Zhang, Lei Shi, Ding-Xuan Zhou

Abstract: In this paper, we study the binary classification problem on $[0,1]^d$ under the Tsybakov noise condition (with exponent $s \in [0,\infty]$) and the compositional assumption. This assumption requires the conditional class probability function of the data distribution to be the composition of $q+1$ vector-valued multivariate functions, where each component function is either a maximum value functio… ▽ More In this paper, we study the binary classification problem on $[0,1]^d$ under the Tsybakov noise condition (with exponent $s \in [0,\infty]$) and the compositional assumption. This assumption requires the conditional class probability function of the data distribution to be the composition of $q+1$ vector-valued multivariate functions, where each component function is either a maximum value function or a Hölder-$β$ smooth function that depends only on $d_*$ of its input variables. Notably, $d_*$ can be significantly smaller than the input dimension $d$. We prove that, under these conditions, the optimal convergence rate for the excess 0-1 risk of classifiers is $$ \left( \frac{1}{n} \right)^{\frac{β\cdot(1\wedgeβ)^q}{{\frac{d_*}{s+1}+(1+\frac{1}{s+1})\cdotβ\cdot(1\wedgeβ)^q}}}\;\;\;, $$ which is independent of the input dimension $d$. Additionally, we demonstrate that ReLU deep neural networks (DNNs) trained with hinge loss can achieve this optimal convergence rate up to a logarithmic factor. This result provides theoretical justification for the excellent performance of ReLU DNNs in practical classification tasks, particularly in high-dimensional settings. The technique used to establish these results extends the oracle inequality presented in our previous work. The generalized approach is of independent interest. △ Less

Submitted 17 June, 2025; originally announced June 2025.

Single Index Bandits: Generalized Linear Contextual Bandits with Unknown Reward Functions

Authors: Yue Kang, Mingshuo Liu, Bongsoo Yi, Jing Lyu, Zhi Zhang, Doudou Zhou, Yao Li

Abstract: Generalized linear bandits have been extensively studied due to their broad applicability in real-world online decision-making problems. However, these methods typically assume that the expected reward function is known to the users, an assumption that is often unrealistic in practice. Misspecification of this link function can lead to the failure of all existing algorithms. In this work, we addre… ▽ More Generalized linear bandits have been extensively studied due to their broad applicability in real-world online decision-making problems. However, these methods typically assume that the expected reward function is known to the users, an assumption that is often unrealistic in practice. Misspecification of this link function can lead to the failure of all existing algorithms. In this work, we address this critical limitation by introducing a new problem of generalized linear bandits with unknown reward functions, also known as single index bandits. We first consider the case where the unknown reward function is monotonically increasing, and propose two novel and efficient algorithms, STOR and ESTOR, that achieve decent regrets under standard assumptions. Notably, our ESTOR can obtain the nearly optimal regret bound $\tilde{O}_T(\sqrt{T})$ in terms of the time horizon $T$. We then extend our methods to the high-dimensional sparse setting and show that the same regret rate can be attained with the sparsity index. Next, we introduce GSTOR, an algorithm that is agnostic to general reward functions, and establish regret bounds under a Gaussian design assumption. Finally, we validate the efficiency and effectiveness of our algorithms through experiments on both synthetic and real-world datasets. △ Less

Submitted 15 June, 2025; originally announced June 2025.

Residual Reweighted Conformal Prediction for Graph Neural Networks

Authors: Zheng Zhang, Jie Bao, Zhixin Zhou, Nicolo Colombo, Lixin Cheng, Rui Luo

Abstract: Graph Neural Networks (GNNs) excel at modeling relational data but face significant challenges in high-stakes domains due to unquantified uncertainty. Conformal prediction (CP) offers statistical coverage guarantees, but existing methods often produce overly conservative prediction intervals that fail to account for graph heteroscedasticity and structural biases. While residual reweighting CP vari… ▽ More Graph Neural Networks (GNNs) excel at modeling relational data but face significant challenges in high-stakes domains due to unquantified uncertainty. Conformal prediction (CP) offers statistical coverage guarantees, but existing methods often produce overly conservative prediction intervals that fail to account for graph heteroscedasticity and structural biases. While residual reweighting CP variants address some of these limitations, they neglect graph topology, cluster-specific uncertainties, and risk data leakage by reusing training sets. To address these issues, we propose Residual Reweighted GNN (RR-GNN), a framework designed to generate minimal prediction sets with provable marginal coverage guarantees. RR-GNN introduces three major innovations to enhance prediction performance. First, it employs Graph-Structured Mondrian CP to partition nodes or edges into communities based on topological features, ensuring cluster-conditional coverage that reflects heterogeneity. Second, it uses Residual-Adaptive Nonconformity Scores by training a secondary GNN on a held-out calibration set to estimate task-specific residuals, dynamically adjusting prediction intervals according to node or edge uncertainty. Third, it adopts a Cross-Training Protocol, which alternates the optimization of the primary GNN and the residual predictor to prevent information leakage while maintaining graph dependencies. We validate RR-GNN on 15 real-world graphs across diverse tasks, including node classification, regression, and edge weight prediction. Compared to CP baselines, RR-GNN achieves improved efficiency over state-of-the-art methods, with no loss of coverage. △ Less

Submitted 9 June, 2025; originally announced June 2025.

Individual Treatment Effect: Prediction Intervals and Sharp Bounds

Authors: Zhehao Zhang, Thomas S. Richardson

Abstract: Individual treatment effect (ITE) is often regarded as the ideal target of inference in causal analyses and has been the focus of several recent studies. In this paper, we describe the intrinsic limits regarding what can be learned concerning ITEs given data from large randomized experiments. We consider when a valid prediction interval for the ITE is informative and when it can be bounded away fr… ▽ More Individual treatment effect (ITE) is often regarded as the ideal target of inference in causal analyses and has been the focus of several recent studies. In this paper, we describe the intrinsic limits regarding what can be learned concerning ITEs given data from large randomized experiments. We consider when a valid prediction interval for the ITE is informative and when it can be bounded away from zero. The joint distribution over potential outcomes is only partially identified from a randomized trial. Consequently, to be valid, an ITE prediction interval must be valid for all joint distribution consistent with the observed data and hence will in general be wider than that resulting from knowledge of this joint distribution. We characterize prediction intervals in the binary treatment and outcome setting, and extend these insights to models with continuous and ordinal outcomes. We derive sharp bounds on the probability mass function (pmf) of the individual treatment effect (ITE). Finally, we contrast prediction intervals for the ITE and confidence intervals for the average treatment effect (ATE). This also leads to the consideration of Fisher versus Neyman null hypotheses. While confidence intervals for the ATE shrink with increasing sample size due to its status as a population parameter, prediction intervals for the ITE generally do not vanish, leading to scenarios where one may reject the Neyman null yet still find evidence consistent with the Fisher null, highlighting the challenges of individualized decision-making under partial identification. △ Less

Submitted 9 June, 2025; originally announced June 2025.

Sharp Gap-Dependent Variance-Aware Regret Bounds for Tabular MDPs

Authors: Shulun Chen, Runlong Zhou, Zihan Zhang, Maryam Fazel, Simon S. Du

Abstract: We consider the gap-dependent regret bounds for episodic MDPs. We show that the Monotonic Value Propagation (MVP) algorithm achieves a variance-aware gap-dependent regret bound of… ▽ More We consider the gap-dependent regret bounds for episodic MDPs. We show that the Monotonic Value Propagation (MVP) algorithm achieves a variance-aware gap-dependent regret bound of $$\tilde{O}\left(\left(\sum_{Δ_h(s,a)>0} \frac{H^2 \log K \land \mathtt{Var}_{\max}^{\text{c}}}{Δ_h(s,a)} +\sum_{Δ_h(s,a)=0}\frac{ H^2 \land \mathtt{Var}_{\max}^{\text{c}}}{Δ_{\mathrm{min}}} + SAH^4 (S \lor H) \right) \log K\right),$$ where $H$ is the planning horizon, $S$ is the number of states, $A$ is the number of actions, and $K$ is the number of episodes. Here, $Δ_h(s,a) =V_h^* (a) - Q_h^* (s, a)$ represents the suboptimality gap and $Δ_{\mathrm{min}} := \min_{Δ_h (s,a) > 0} Δ_h(s,a)$. The term $\mathtt{Var}_{\max}^{\text{c}}$ denotes the maximum conditional total variance, calculated as the maximum over all $(π, h, s)$ tuples of the expected total variance under policy $π$ conditioned on trajectories visiting state $s$ at step $h$. $\mathtt{Var}_{\max}^{\text{c}}$ characterizes the maximum randomness encountered when learning any $(h, s)$ pair. Our result stems from a novel analysis of the weighted sum of the suboptimality gap and can be potentially adapted for other algorithms. To complement the study, we establish a lower bound of $$Ω\left( \sum_{Δ_h(s,a)>0} \frac{H^2 \land \mathtt{Var}_{\max}^{\text{c}}}{Δ_h(s,a)}\cdot \log K\right),$$ demonstrating the necessity of dependence on $\mathtt{Var}_{\max}^{\text{c}}$ even when the maximum unconditional total variance (without conditioning on $(h, s)$) approaches zero. △ Less

Submitted 6 June, 2025; originally announced June 2025.

Comments: 30 pages

Reconstruction and Prediction of Volterra Integral Equations Driven by Gaussian Noise

Authors: Zhihao Xu, Saisai Ding, Zhikun Zhang, Xiangjun Wang

Abstract: Integral equations are widely used in fields such as applied modeling, medical imaging, and system identification, providing a powerful framework for solving deterministic problems. While parameter identification for differential equations has been extensively studied, the focus on integral equations, particularly stochastic Volterra integral equations, remains limited. This research addresses the… ▽ More Integral equations are widely used in fields such as applied modeling, medical imaging, and system identification, providing a powerful framework for solving deterministic problems. While parameter identification for differential equations has been extensively studied, the focus on integral equations, particularly stochastic Volterra integral equations, remains limited. This research addresses the parameter identification problem, also known as the equation reconstruction problem, in Volterra integral equations driven by Gaussian noise. We propose an improved deep neural networks framework for estimating unknown parameters in the drift term of these equations. The network represents the primary variables and their integrals, enhancing parameter estimation accuracy by incorporating inter-output relationships into the loss function. Additionally, the framework extends beyond parameter identification to predict the system's behavior outside the integration interval. Prediction accuracy is validated by comparing predicted and true trajectories using a 95% confidence interval. Numerical experiments demonstrate the effectiveness of the proposed deep neural networks framework in both parameter identification and prediction tasks, showing robust performance under varying noise levels and providing accurate solutions for modeling stochastic systems. △ Less

Submitted 1 June, 2025; originally announced June 2025.

Projection Pursuit Density Ratio Estimation

Authors: Meilin Wang, Wei Huang, Mingming Gong, Zheng Zhang

Abstract: Density ratio estimation (DRE) is a paramount task in machine learning, for its broad applications across multiple domains, such as covariate shift adaptation, causal inference, independence tests and beyond. Parametric methods for estimating the density ratio possibly lead to biased results if models are misspecified, while conventional non-parametric methods suffer from the curse of dimensionali… ▽ More Density ratio estimation (DRE) is a paramount task in machine learning, for its broad applications across multiple domains, such as covariate shift adaptation, causal inference, independence tests and beyond. Parametric methods for estimating the density ratio possibly lead to biased results if models are misspecified, while conventional non-parametric methods suffer from the curse of dimensionality when the dimension of data is large. To address these challenges, in this paper, we propose a novel approach for DRE based on the projection pursuit (PP) approximation. The proposed method leverages PP to mitigate the impact of high dimensionality while retaining the model flexibility needed for the accuracy of DRE. We establish the consistency and the convergence rate for the proposed estimator. Experimental results demonstrate that our proposed method outperforms existing alternatives in various applications. △ Less

Submitted 1 June, 2025; originally announced June 2025.

Estimation of Gender Wage Gap in the University of North Carolina System

Authors: Zihan Zhang, Jan Hannig

Abstract: Gender pay equity remains an open challenge in academia despite decades of movements. Prior studies, however, have relied largely on descriptive regressions, leaving causal analysis underexplored. This study examines gender-based wage disparities among tenure-track faculty in the University of North Carolina system using both parametric and non-parametric causal inference methods. In particular, w… ▽ More Gender pay equity remains an open challenge in academia despite decades of movements. Prior studies, however, have relied largely on descriptive regressions, leaving causal analysis underexplored. This study examines gender-based wage disparities among tenure-track faculty in the University of North Carolina system using both parametric and non-parametric causal inference methods. In particular, we employed propensity score matching and causal forests to estimate the causal effect of gender on academic salary while controlling for university type, discipline, titles, working years, and scholarly productivity metrics. The results indicate that on average female professors earn approximately 6% less than their male colleagues with similar qualifications and positions. △ Less

Submitted 29 May, 2025; originally announced May 2025.

Offline Clustering of Linear Bandits: Unlocking the Power of Clusters in Data-Limited Environments

Authors: Jingyuan Liu, Zeyu Zhang, Xuchuang Wang, Xutong Liu, John C. S. Lui, Mohammad Hajiesmaili, Carlee Joe-Wong

Abstract: Contextual linear multi-armed bandits are a learning framework for making a sequence of decisions, e.g., advertising recommendations for a sequence of arriving users. Recent works have shown that clustering these users based on the similarity of their learned preferences can significantly accelerate the learning. However, prior work has primarily focused on the online setting, which requires conti… ▽ More Contextual linear multi-armed bandits are a learning framework for making a sequence of decisions, e.g., advertising recommendations for a sequence of arriving users. Recent works have shown that clustering these users based on the similarity of their learned preferences can significantly accelerate the learning. However, prior work has primarily focused on the online setting, which requires continually collecting user data, ignoring the offline data widely available in many applications. To tackle these limitations, we study the offline clustering of bandits (Off-ClusBand) problem, which studies how to use the offline dataset to learn cluster properties and improve decision-making across multiple users. The key challenge in Off-ClusBand arises from data insufficiency for users: unlike the online case, in the offline case, we have a fixed, limited dataset to work from and thus must determine whether we have enough data to confidently cluster users together. To address this challenge, we propose two algorithms: Off-C$^2$LUB, which we analytically show performs well for arbitrary amounts of user data, and Off-CLUB, which is prone to bias when data is limited but, given sufficient data, matches a theoretical lower bound that we derive for the offline clustered MAB problem. We experimentally validate these results on both real and synthetic datasets. △ Less

Submitted 25 May, 2025; originally announced May 2025.

Optimal Treatment Allocations Accounting for Population Differences

Authors: Wei Zhang, Zhiwei Zhang, Aiyi Liu

Abstract: The treatment allocation mechanism in a randomized clinical trial can be optimized by maximizing the nonparametric efficiency bound for a specific measure of treatment effect. Optimal treatment allocations which may or may not depend on baseline covariates have been derived for a variety of effect measures focusing on the trial population, the patient population represented by the trial participan… ▽ More The treatment allocation mechanism in a randomized clinical trial can be optimized by maximizing the nonparametric efficiency bound for a specific measure of treatment effect. Optimal treatment allocations which may or may not depend on baseline covariates have been derived for a variety of effect measures focusing on the trial population, the patient population represented by the trial participants. Frequently, clinical trial data are used to estimate treatment effects in a target population that is related to but different from the trial population. This article provides optimal treatment allocations that account for the impact of such population differences. We consider three cases with different data configurations: transportation, generalization, and post-stratification. Our results indicate that, for general effect measures, optimal treatment allocations may depend on the covariate distribution in the target population but not on the configuration of data or information that describes the target covariate distribution. For estimating average treatment effects, there is a unique covariate-dependent allocation that achieves maximal efficiency regardless of the target covariate distribution and the associated data configuration. △ Less

Submitted 21 May, 2025; originally announced May 2025.

LiDDA: Data Driven Attribution at LinkedIn

Authors: John Bencina, Erkut Aykutlug, Yue Chen, Zerui Zhang, Stephanie Sorenson, Shao Tang, Changshuai Wei

Abstract: Data Driven Attribution, which assigns conversion credits to marketing interactions based on causal patterns learned from data, is the foundation of modern marketing intelligence and vital to any marketing businesses and advertising platform. In this paper, we introduce a unified transformer-based attribution approach that can handle member-level data, aggregate-level data, and integration of exte… ▽ More Data Driven Attribution, which assigns conversion credits to marketing interactions based on causal patterns learned from data, is the foundation of modern marketing intelligence and vital to any marketing businesses and advertising platform. In this paper, we introduce a unified transformer-based attribution approach that can handle member-level data, aggregate-level data, and integration of external macro factors. We detail the large scale implementation of the approach at LinkedIn, showcasing significant impact. We also share learning and insights that are broadly applicable to the marketing and ad tech fields. △ Less

Submitted 21 May, 2025; v1 submitted 14 May, 2025; originally announced May 2025.

Diffusion-based supervised learning of generative models for efficient sampling of multimodal distributions

Authors: Hoang Tran, Zezhong Zhang, Feng Bao, Dan Lu, Guannan Zhang

Abstract: We propose a hybrid generative model for efficient sampling of high-dimensional, multimodal probability distributions for Bayesian inference. Traditional Monte Carlo methods, such as the Metropolis-Hastings and Langevin Monte Carlo sampling methods, are effective for sampling from single-mode distributions in high-dimensional spaces. However, these methods struggle to produce samples with the corr… ▽ More We propose a hybrid generative model for efficient sampling of high-dimensional, multimodal probability distributions for Bayesian inference. Traditional Monte Carlo methods, such as the Metropolis-Hastings and Langevin Monte Carlo sampling methods, are effective for sampling from single-mode distributions in high-dimensional spaces. However, these methods struggle to produce samples with the correct proportions for each mode in multimodal distributions, especially for distributions with well separated modes. To address the challenges posed by multimodality, we adopt a divide-and-conquer strategy. We start by minimizing the energy function with initial guesses uniformly distributed within the prior domain to identify all the modes of the energy function. Then, we train a classifier to segment the domain corresponding to each mode. After the domain decomposition, we train a diffusion-model-assisted generative model for each identified mode within its support. Once each mode is characterized, we employ bridge sampling to estimate the normalizing constant, allowing us to directly adjust the ratios between the modes. Our numerical examples demonstrate that the proposed framework can effectively handle multimodal distributions with varying mode shapes in up to 100 dimensions. An application to Bayesian inverse problem for partial differential equations is also provided. △ Less

Submitted 20 April, 2025; originally announced May 2025.

A Unified Approach to Covariate Adjustment for Survival Endpoints in Randomized Clinical Trials

Authors: Zhiwei Zhang, Ya Wang, Dong Xi

Abstract: Covariate adjustment aims to improve the statistical efficiency of randomized trials by incorporating information from baseline covariates. Popular methods for covariate adjustment include analysis of covariance for continuous endpoints and standardized logistic regression for binary endpoints. For survival endpoints, while some covariate adjustment methods have been developed for specific effect… ▽ More Covariate adjustment aims to improve the statistical efficiency of randomized trials by incorporating information from baseline covariates. Popular methods for covariate adjustment include analysis of covariance for continuous endpoints and standardized logistic regression for binary endpoints. For survival endpoints, while some covariate adjustment methods have been developed for specific effect measures, they are not commonly used in practice for various reasons, including high demands for theoretical and methodological sophistication as well as computational skills. This article describes an augmentation approach to covariate adjustment for survival endpoints that is relatively easy to understand and widely applicable to different effect measures. This approach involves augmenting a given treatment effect estimator in a way that preserves interpretation, consistency, and asymptotic normality. The optimal augmentation term, which minimizes asymptotic variance, can be estimated using various statistical and machine learning methods. Simulation results demonstrate that the augmentation approach can bring substantial gains in statistical efficiency. This approach has been implemented in an R package named \texttt{sleete}, which is described in detail and illustrated with real data. △ Less

Submitted 8 May, 2025; originally announced May 2025.

Testing Conditional Independence via Density Ratio Regression

Authors: Chunrong Ai, Zixuan Xu, Zheng Zhang

Abstract: This paper develops a conditional independence (CI) test from a conditional density ratio (CDR) for weakly dependent data. The main contribution is presenting a closed-form expression for the estimated conditional density ratio function with good finite-sample performance. The key idea is exploiting the linear sieve combined with the quadratic norm. Matsushita et al. (2022) exploited the linear si… ▽ More This paper develops a conditional independence (CI) test from a conditional density ratio (CDR) for weakly dependent data. The main contribution is presenting a closed-form expression for the estimated conditional density ratio function with good finite-sample performance. The key idea is exploiting the linear sieve combined with the quadratic norm. Matsushita et al. (2022) exploited the linear sieve to estimate the unconditional density ratio. We must exploit the linear sieve twice to estimate the conditional density ratio. First, we estimate an unconditional density ratio with an unweighted sieve least-squares regression, as done in Matsushita et al. (2022), and then the conditional density ratio with a weighted sieve least-squares regression, where the weights are the estimated unconditional density ratio. The proposed test has several advantages over existing alternatives. First, the test statistic is invariant to the monotone transformation of the data distribution and has a closed-form expression that enhances computational speed and efficiency. Second, the conditional density ratio satisfies the moment restrictions. The estimated ratio satisfies the empirical analog of those moment restrictions. As a result, the estimated density ratio is unlikely to have extreme values. Third, the proposed test can detect all deviations from conditional independence at rates arbitrarily close to $n^{-1/2}$ , and the local power loss is independent of the data dimension. A small-scale simulation study indicates that the proposed test outperforms the alternatives in various dependence structures. △ Less

Submitted 24 April, 2025; originally announced April 2025.

The underlap coefficient as a measure of a biomarker's discriminatory ability

Authors: Zhaoxi Zhang, Vanda Inacio, Miguel de Carvalho

Abstract: The first step in evaluating a potential diagnostic biomarker is to examine the variation in its values across different disease groups. In a three-class disease setting, the volume under the receiver operating characteristic surface and the three-class Youden index are commonly used summary measures of a biomarker's discriminatory ability. However, these measures rely on a stochastic ordering ass… ▽ More The first step in evaluating a potential diagnostic biomarker is to examine the variation in its values across different disease groups. In a three-class disease setting, the volume under the receiver operating characteristic surface and the three-class Youden index are commonly used summary measures of a biomarker's discriminatory ability. However, these measures rely on a stochastic ordering assumption for the distributions of biomarker outcomes across the three groups. This assumption can be restrictive, particularly when covariates are involved, and its violation may lead to incorrect conclusions about a biomarker's ability to distinguish between the three disease classes. Even when a stochastic ordering exists, the order may vary across different biomarkers in discovery studies involving dozens or even thousands of candidate biomarkers, complicating automated ranking. To address these challenges and complement existing measures, we propose the underlap coefficient, a novel summary index of a biomarker's ability to distinguish between three (or more) disease groups, and study its properties. Additionally, we introduce Bayesian nonparametric estimators for both the unconditional underlap coefficient and its covariate-specific counterpart. These estimators are broadly applicable to a wide range of biomarkers and populations. A simulation study reveals a good performance of the proposed estimators across a range of conceivable scenarios. We illustrate the proposed approach through an application to an Alzheimer's disease (AD) dataset aimed to assess how four potential AD biomarkers distinguish between individuals with normal cognition, mild impairment, and dementia, and how and if age and gender impact this discriminatory ability. △ Less

Submitted 17 April, 2025; v1 submitted 16 April, 2025; originally announced April 2025.

Follow-the-Perturbed-Leader Approaches Best-of-Both-Worlds for the m-Set Semi-Bandit Problems

Authors: Jingxin Zhan, Yuchen Xin, Chenjie Sun, Zhihua Zhang

Abstract: We consider a common case of the combinatorial semi-bandit problem, the $m$-set semi-bandit, where the learner exactly selects $m$ arms from the total $d$ arms. In the adversarial setting, the best regret bound, known to be $\mathcal{O}(\sqrt{nmd})$ for time horizon $n$, is achieved by the well-known Follow-the-Regularized-Leader (FTRL) policy. However, this requires to explicitly compute the arm-… ▽ More We consider a common case of the combinatorial semi-bandit problem, the $m$-set semi-bandit, where the learner exactly selects $m$ arms from the total $d$ arms. In the adversarial setting, the best regret bound, known to be $\mathcal{O}(\sqrt{nmd})$ for time horizon $n$, is achieved by the well-known Follow-the-Regularized-Leader (FTRL) policy. However, this requires to explicitly compute the arm-selection probabilities via optimizing problems at each time step and sample according to them. This problem can be avoided by the Follow-the-Perturbed-Leader (FTPL) policy, which simply pulls the $m$ arms that rank among the $m$ smallest (estimated) loss with random perturbation. In this paper, we show that FTPL with a Fréchet perturbation also enjoys the near optimal regret bound $\mathcal{O}(\sqrt{nm}(\sqrt{d\log(d)}+m^{5/6}))$ in the adversarial setting and approaches best-of-both-world regret bounds, i.e., achieves a logarithmic regret for the stochastic setting. Moreover, our lower bounds show that the extra factors are unavoidable with our approach; any improvement would require a fundamentally different and more challenging method. △ Less

Submitted 7 July, 2025; v1 submitted 9 April, 2025; originally announced April 2025.

Feature splitting parallel algorithm for Dantzig selectors

Authors: Xiaofei Wu, Yue Chao, Rongmei Liang, Shi Tang, Zhiming Zhang

Abstract: The Dantzig selector is a widely used and effective method for variable selection in ultra-high-dimensional data. Feature splitting is an efficient processing technique that involves dividing these ultra-high-dimensional variable datasets into manageable subsets that can be stored and processed more easily on a single machine. This paper proposes a variable splitting parallel algorithm for solving… ▽ More The Dantzig selector is a widely used and effective method for variable selection in ultra-high-dimensional data. Feature splitting is an efficient processing technique that involves dividing these ultra-high-dimensional variable datasets into manageable subsets that can be stored and processed more easily on a single machine. This paper proposes a variable splitting parallel algorithm for solving both convex and nonconvex Dantzig selectors based on the proximal point algorithm. The primary advantage of our parallel algorithm, compared to existing parallel approaches, is the significantly reduced number of iteration variables, which greatly enhances computational efficiency and accelerates the convergence speed of the algorithm. Furthermore, we show that our solution remains unchanged regardless of how the data is partitioned, a property referred to as partitioninsensitive. In theory, we use a concise proof framework to demonstrate that the algorithm exhibits linear convergence. Numerical experiments indicate that our algorithm performs competitively in both parallel and nonparallel environments. The R package for implementing the proposed algorithm can be obtained at https://github.com/xfwu1016/PPADS. △ Less

Submitted 3 April, 2025; originally announced April 2025.

Non-parametric Quantile Regression and Uniform Inference with Unknown Error Distribution

Authors: Haoze Hou, Wei Huang, Zheng Zhang

Abstract: This paper studies the non-parametric estimation and uniform inference for the conditional quantile regression function (CQRF) with covariates exposed to measurement errors. We consider the case that the distribution of the measurement error is unknown and allowed to be either ordinary or super smooth. We estimate the density of the measurement error by the repeated measurements and propose the de… ▽ More This paper studies the non-parametric estimation and uniform inference for the conditional quantile regression function (CQRF) with covariates exposed to measurement errors. We consider the case that the distribution of the measurement error is unknown and allowed to be either ordinary or super smooth. We estimate the density of the measurement error by the repeated measurements and propose the deconvolution kernel estimator for the CQRF. We derive the uniform Bahadur representation of the proposed estimator and construct the uniform confidence bands for the CQRF, uniformly in the sense for all covariates and a set of quantile indices, and establish the theoretical validity of the proposed inference. A data-driven approach for selecting the tuning parameter is also included. Monte Carlo simulations and a real data application demonstrate the usefulness of the proposed method. △ Less

Submitted 2 April, 2025; originally announced April 2025.

Analytic DAG Constraints for Differentiable DAG Learning

Authors: Zhen Zhang, Ignavier Ng, Dong Gong, Yuhang Liu, Mingming Gong, Biwei Huang, Kun Zhang, Anton van den Hengel, Javen Qinfeng Shi

Abstract: Recovering the underlying Directed Acyclic Graph (DAG) structures from observational data presents a formidable challenge, partly due to the combinatorial nature of the DAG-constrained optimization problem. Recently, researchers have identified gradient vanishing as one of the primary obstacles in differentiable DAG learning and have proposed several DAG constraints to mitigate this issue. By deve… ▽ More Recovering the underlying Directed Acyclic Graph (DAG) structures from observational data presents a formidable challenge, partly due to the combinatorial nature of the DAG-constrained optimization problem. Recently, researchers have identified gradient vanishing as one of the primary obstacles in differentiable DAG learning and have proposed several DAG constraints to mitigate this issue. By developing the necessary theory to establish a connection between analytic functions and DAG constraints, we demonstrate that analytic functions from the set $\{f(x) = c_0 + \sum_{i=1}^{\infty}c_ix^i | \forall i > 0, c_i > 0; r = \lim_{i\rightarrow \infty}c_{i}/c_{i+1} > 0\}$ can be employed to formulate effective DAG constraints. Furthermore, we establish that this set of functions is closed under several functional operators, including differentiation, summation, and multiplication. Consequently, these operators can be leveraged to create novel DAG constraints based on existing ones. Using these properties, we design a series of DAG constraints and develop an efficient algorithm to evaluate them. Experiments in various settings demonstrate that our DAG constraints outperform previous state-of-the-art comparators. Our implementation is available at https://github.com/zzhang1987/AnalyticDAGLearning. △ Less

Submitted 24 March, 2025; originally announced March 2025.

Comments: Accepted to ICLR 2025

Journal ref: ICLR 2025

Alignment of Continuous Brain Connectivity

Authors: Martin Cole, Yang Xiang, Will Consagra, Anuj Srivastava, Xing Qiu, Zhengwu Zhang

Abstract: Brain networks are typically represented by adjacency matrices, where each node corresponds to a brain region. In traditional brain network analysis, nodes are assumed to be matched across individuals, but the methods used for node matching often overlook the underlying connectivity information. This oversight can result in inaccurate node alignment, leading to inflated edge variability and reduce… ▽ More Brain networks are typically represented by adjacency matrices, where each node corresponds to a brain region. In traditional brain network analysis, nodes are assumed to be matched across individuals, but the methods used for node matching often overlook the underlying connectivity information. This oversight can result in inaccurate node alignment, leading to inflated edge variability and reduced statistical power in downstream connectivity analyses. To overcome this challenge, we propose a novel framework for registering high resolution continuous connectivity (ConCon), defined as a continuous function on a product manifold space specifically, the cortical surface capturing structural connectivity between all pairs of cortical points. Leveraging ConCon, we formulate an optimal diffeomorphism problem to align both connectivity profiles and cortical surfaces simultaneously. We introduce an efficient algorithm to solve this problem and validate our approach using data from the Human Connectome Project (HCP). Results demonstrate that our method substantially improves the accuracy and robustness of connectome-based analyses compared to existing techniques. △ Less

Submitted 19 March, 2025; originally announced March 2025.

Comments: 12 pages of main text and 8 pages of supplemental material. 10 figures in main text and 3 figures in supplemental material

Competing-risk Weibull survival model with multiple causes

Authors: Kai Wang, Yuqin Mu, Shenyi Zhang, Zhengjun Zhang, Chengxiu Ling

Abstract: The failure of a system can result from the simultaneous effects of multiple causes, where assigning a specific cause may be inappropriate or unavailable. Examples include contributing causes of death in epidemiology and the aetiology of neurodegenerative diseases like Alzheimer's. We propose a parametric Weibull accelerated failure time model for multiple causes, incorporating a data-driven, indi… ▽ More The failure of a system can result from the simultaneous effects of multiple causes, where assigning a specific cause may be inappropriate or unavailable. Examples include contributing causes of death in epidemiology and the aetiology of neurodegenerative diseases like Alzheimer's. We propose a parametric Weibull accelerated failure time model for multiple causes, incorporating a data-driven, individualized, and time-varying winning probability (relative importance) matrix. Using maximum likelihood estimation and the expectation-maximization (EM) algorithm, our approach enables simultaneous estimation of regression coefficients and relative cause importance, ensuring consistency and asymptotic normality. A simulation study and an application to Alzheimer's disease demonstrate its effectiveness in addressing cause-mixture problems and identifying informative biomarker combinations, with comparisons to Weibull and Cox proportional hazards models. △ Less

Submitted 12 March, 2025; originally announced March 2025.

Max-Linear Tail Regression

Authors: Liujun Chen, Deyuan Li, Zhengjun Zhang

Abstract: The relationship between a response variable and its covariates can vary significantly, especially in scenarios where covariates take on extremely high or low values. This paper introduces a max-linear tail regression model specifically designed to capture such extreme relationships. To estimate the regression coefficients within this framework, we propose a novel M-estimator based on extreme valu… ▽ More The relationship between a response variable and its covariates can vary significantly, especially in scenarios where covariates take on extremely high or low values. This paper introduces a max-linear tail regression model specifically designed to capture such extreme relationships. To estimate the regression coefficients within this framework, we propose a novel M-estimator based on extreme value theory. The consistency and asymptotic normality of our proposed estimator are rigorously established under mild conditions. Simulation results demonstrate that our estimation method outperforms the conditional least squares approach. We validate the practical applicability of our model through two case studies: one using financial data and the other using rainfall data. △ Less

Submitted 21 February, 2025; originally announced February 2025.

A Non-Asymptotic Theory of Seminorm Lyapunov Stability: From Deterministic to Stochastic Iterative Algorithms

Authors: Zaiwei Chen, Sheng Zhang, Zhe Zhang, Shaan Ul Haque, Siva Theja Maguluri

Abstract: We study the problem of solving fixed-point equations for seminorm-contractive operators and establish foundational results on the non-asymptotic behavior of iterative algorithms in both deterministic and stochastic settings. Specifically, in the deterministic setting, we prove a fixed-point theorem for seminorm-contractive operators, showing that iterates converge geometrically to the kernel of t… ▽ More We study the problem of solving fixed-point equations for seminorm-contractive operators and establish foundational results on the non-asymptotic behavior of iterative algorithms in both deterministic and stochastic settings. Specifically, in the deterministic setting, we prove a fixed-point theorem for seminorm-contractive operators, showing that iterates converge geometrically to the kernel of the seminorm. In the stochastic setting, we analyze the corresponding stochastic approximation (SA) algorithm under seminorm-contractive operators and Markovian noise, providing a finite-sample analysis for various stepsize choices. A benchmark for equation solving is linear systems of equations, where the convergence behavior of fixed-point iteration is closely tied to the stability of linear dynamical systems. In this special case, our results provide a complete characterization of system stability with respect to a seminorm, linking it to the solution of a Lyapunov equation in terms of positive semi-definite matrices. In the stochastic setting, we establish a finite-sample analysis for linear Markovian SA without requiring the Hurwitzness assumption. Our theoretical results offer a unified framework for deriving finite-sample bounds for various reinforcement learning algorithms in the average reward setting, including TD($λ$) for policy evaluation (which is a special case of solving a Poisson equation) and Q-learning for control. △ Less

Submitted 19 February, 2025; originally announced February 2025.

A Finite Sample Analysis of Distributional TD Learning with Linear Function Approximation

Authors: Yang Peng, Kaicheng Jin, Liangyu Zhang, Zhihua Zhang

Abstract: In this paper, we study the finite-sample statistical rates of distributional temporal difference (TD) learning with linear function approximation. The aim of distributional TD learning is to estimate the return distribution of a discounted Markov decision process for a given policy π. Previous works on statistical analysis of distributional TD learning mainly focus on the tabular case. In contras… ▽ More In this paper, we study the finite-sample statistical rates of distributional temporal difference (TD) learning with linear function approximation. The aim of distributional TD learning is to estimate the return distribution of a discounted Markov decision process for a given policy π. Previous works on statistical analysis of distributional TD learning mainly focus on the tabular case. In contrast, we first consider the linear function approximation setting and derive sharp finite-sample rates. Our theoretical results demonstrate that the sample complexity of linear distributional TD learning matches that of classic linear TD learning. This implies that, with linear function approximation, learning the full distribution of the return from streaming data is no more difficult than learning its expectation (value function). To derive tight sample complexity bounds, we conduct a fine-grained analysis of the linear-categorical Bellman equation and employ the exponential stability arguments for products of random matrices. Our results provide new insights into the statistical efficiency of distributional reinforcement learning algorithms. △ Less

Submitted 13 May, 2025; v1 submitted 19 February, 2025; originally announced February 2025.

Performance Evaluation of Large Language Models in Statistical Programming

Authors: Xinyi Song, Kexin Xie, Lina Lee, Ruizhe Chen, Jared M. Clark, Hao He, Haoran He, Jie Min, Xinlei Zhang, Simin Zheng, Zhiyang Zhang, Xinwei Deng, Yili Hong

Abstract: The programming capabilities of large language models (LLMs) have revolutionized automatic code generation and opened new avenues for automatic statistical analysis. However, the validity and quality of these generated codes need to be systematically evaluated before they can be widely adopted. Despite their growing prominence, a comprehensive evaluation of statistical code generated by LLMs remai… ▽ More The programming capabilities of large language models (LLMs) have revolutionized automatic code generation and opened new avenues for automatic statistical analysis. However, the validity and quality of these generated codes need to be systematically evaluated before they can be widely adopted. Despite their growing prominence, a comprehensive evaluation of statistical code generated by LLMs remains scarce in the literature. In this paper, we assess the performance of LLMs, including two versions of ChatGPT and one version of Llama, in the domain of SAS programming for statistical analysis. Our study utilizes a set of statistical analysis tasks encompassing diverse statistical topics and datasets. Each task includes a problem description, dataset information, and human-verified SAS code. We conduct a comprehensive assessment of the quality of SAS code generated by LLMs through human expert evaluation based on correctness, effectiveness, readability, executability, and the accuracy of output results. The analysis of rating scores reveals that while LLMs demonstrate usefulness in generating syntactically correct code, they struggle with tasks requiring deep domain understanding and may produce redundant or incorrect results. This study offers valuable insights into the capabilities and limitations of LLMs in statistical programming, providing guidance for future advancements in AI-assisted coding systems for statistical analysis. △ Less

Submitted 18 February, 2025; originally announced February 2025.

Comments: 27 pages, 8 figures

Gaussian Approximation and Multiplier Bootstrap for Stochastic Gradient Descent

Authors: Marina Sheshukova, Sergey Samsonov, Denis Belomestny, Eric Moulines, Qi-Man Shao, Zhuo-Song Zhang, Alexey Naumov

Abstract: In this paper, we establish non-asymptotic convergence rates in the central limit theorem for Polyak-Ruppert-averaged iterates of stochastic gradient descent (SGD). Our analysis builds on the result of the Gaussian approximation for nonlinear statistics of independent random variables of Shao and Zhang (2022). Using this result, we prove the non-asymptotic validity of the multiplier bootstrap for… ▽ More In this paper, we establish non-asymptotic convergence rates in the central limit theorem for Polyak-Ruppert-averaged iterates of stochastic gradient descent (SGD). Our analysis builds on the result of the Gaussian approximation for nonlinear statistics of independent random variables of Shao and Zhang (2022). Using this result, we prove the non-asymptotic validity of the multiplier bootstrap for constructing the confidence sets for the optimal solution of an optimization problem. In particular, our approach avoids the need to approximate the limiting covariance of Polyak-Ruppert SGD iterates, which allows us to derive approximation rates in convex distance of order up to $1/\sqrt{n}$. △ Less

Submitted 10 February, 2025; originally announced February 2025.

MSC Class: 60F05; 62L20; 93E35

Sparsity-Based Interpolation of External, Internal and Swap Regret

Authors: Zhou Lu, Y. Jennifer Sun, Zhiyu Zhang

Abstract: Focusing on the expert problem in online learning, this paper studies the interpolation of several performance metrics via $φ$-regret minimization, which measures the total loss of an algorithm by its regret with respect to an arbitrary action modification rule $φ$. With $d$ experts and $T\gg d$ rounds in total, we present a single algorithm achieving the instance-adaptive $φ$-regret bound \begin{… ▽ More Focusing on the expert problem in online learning, this paper studies the interpolation of several performance metrics via $φ$-regret minimization, which measures the total loss of an algorithm by its regret with respect to an arbitrary action modification rule $φ$. With $d$ experts and $T\gg d$ rounds in total, we present a single algorithm achieving the instance-adaptive $φ$-regret bound \begin{equation*} \tilde O\left(\min\left\{\sqrt{d-d^{\mathrm{unif}}_φ+1},\sqrt{d-d^{\mathrm{self}}_φ}\right\}\cdot\sqrt{T}\right), \end{equation*} where $d^{\mathrm{unif}}_φ$ is the maximum amount of experts modified identically by $φ$, and $d^{\mathrm{self}}_φ$ is the amount of experts that $φ$ trivially modifies to themselves. By recovering the optimal $O(\sqrt{T\log d})$ external regret bound when $d^{\mathrm{unif}}_φ=d$, the standard $\tilde O(\sqrt{T})$ internal regret bound when $d^{\mathrm{self}}_φ=d-1$ and the optimal $\tilde O(\sqrt{dT})$ swap regret bound in the worst case, we improve upon existing algorithms in the intermediate regimes. In addition, the computational complexity of our algorithm matches that of the standard swap-regret minimization algorithm due to (Blum and Mansour, 2007). Technically, building on the well-known reduction from $φ$-regret minimization to external regret minimization on stochastic matrices, our main idea is to further convert the latter to online linear regression using Haar-wavelet-inspired matrix features. Then, by associating the complexity of each $φ$ instance with its sparsity under the feature representation, we apply techniques from comparator-adaptive online learning to exploit the sparsity in this regression subroutine. △ Less

Submitted 17 June, 2025; v1 submitted 6 February, 2025; originally announced February 2025.

Comments: COLT 2025. Equal contribution, alphabetical order

CausalCOMRL: Context-Based Offline Meta-Reinforcement Learning with Causal Representation

Authors: Zhengzhe Zhang, Wenjia Meng, Haoliang Sun, Gang Pan

Abstract: Context-based offline meta-reinforcement learning (OMRL) methods have achieved appealing success by leveraging pre-collected offline datasets to develop task representations that guide policy learning. However, current context-based OMRL methods often introduce spurious correlations, where task components are incorrectly correlated due to confounders. These correlations can degrade policy performa… ▽ More Context-based offline meta-reinforcement learning (OMRL) methods have achieved appealing success by leveraging pre-collected offline datasets to develop task representations that guide policy learning. However, current context-based OMRL methods often introduce spurious correlations, where task components are incorrectly correlated due to confounders. These correlations can degrade policy performance when the confounders in the test task differ from those in the training task. To address this problem, we propose CausalCOMRL, a context-based OMRL method that integrates causal representation learning. This approach uncovers causal relationships among the task components and incorporates the causal relationships into task representations, enhancing the generalizability of RL agents. We further improve the distinction of task representations from different tasks by using mutual information optimization and contrastive learning. Utilizing these causal task representations, we employ SAC to optimize policies on meta-RL benchmarks. Experimental results show that CausalCOMRL achieves better performance than other methods on most benchmarks. △ Less

Submitted 2 February, 2025; originally announced February 2025.

Zeroth-order Informed Fine-Tuning for Diffusion Model: A Recursive Likelihood Ratio Optimizer

Authors: Tao Ren, Zishi Zhang, Zehao Li, Jingyang Jiang, Shentao Qin, Guanghao Li, Yan Li, Yi Zheng, Xinping Li, Min Zhan, Yijie Peng

Abstract: The probabilistic diffusion model (DM), generating content by inferencing through a recursive chain structure, has emerged as a powerful framework for visual generation. After pre-training on enormous unlabeled data, the model needs to be properly aligned to meet requirements for downstream applications. How to efficiently align the foundation DM is a crucial task. Contemporary methods are either… ▽ More The probabilistic diffusion model (DM), generating content by inferencing through a recursive chain structure, has emerged as a powerful framework for visual generation. After pre-training on enormous unlabeled data, the model needs to be properly aligned to meet requirements for downstream applications. How to efficiently align the foundation DM is a crucial task. Contemporary methods are either based on Reinforcement Learning (RL) or truncated Backpropagation (BP). However, RL and truncated BP suffer from low sample efficiency and biased gradient estimation respectively, resulting in limited improvement or, even worse, complete training failure. To overcome the challenges, we propose the Recursive Likelihood Ratio (RLR) optimizer, a zeroth-order informed fine-tuning paradigm for DM. The zeroth-order gradient estimator enables the computation graph rearrangement within the recursive diffusive chain, making the RLR's gradient estimator an unbiased one with the lower variance than other methods. We provide theoretical guarantees for the performance of the RLR. Extensive experiments are conducted on image and video generation tasks to validate the superiority of the RLR. Furthermore, we propose a novel prompt technique that is natural for the RLR to achieve a synergistic effect. △ Less

Submitted 24 March, 2025; v1 submitted 1 February, 2025; originally announced February 2025.

Outcome Regression Methods for Analyzing Hybrid Control Studies: Balancing Bias and Variability

Authors: Zhiwei Zhang, Jialuo Liu, Wei Liu

Abstract: There is growing interest in a hybrid control design in which a randomized controlled trial is augmented with an external control arm from a previous trial or real world data. Existing methods for analyzing hybrid control studies include various downweighting and propensity score methods as well as methods that combine downweighting with propensity score stratification. In this article, we describ… ▽ More There is growing interest in a hybrid control design in which a randomized controlled trial is augmented with an external control arm from a previous trial or real world data. Existing methods for analyzing hybrid control studies include various downweighting and propensity score methods as well as methods that combine downweighting with propensity score stratification. In this article, we describe and discuss methods that make use of an outcome regression model (possibly in addition to a propensity score model). Specifically, we consider an augmentation method, a G-computation method, and a weighted regression method, and note that the three methods provide different bias-variance trade-offs. The methods are compared with each other and with existing methods in a simulation study. Simulation results indicate that weighted regression compares favorably with other model-based methods that seek to improve efficiency by incorporating external control data. The methods are illustrated using two examples from urology and infectious disease. △ Less

Submitted 28 January, 2025; originally announced January 2025.

The typicality principle and its implications for statistics and data science

Authors: Yiran Jiang, Zeyu Zhang, Ryan Martin, Chuanhai Liu

Abstract: A central focus of data science is the transformation of empirical evidence into knowledge. As such, the key insights and scientific attitudes of deep thinkers like Fisher, Popper, and Tukey are expected to inspire exciting new advances in machine learning and artificial intelligence in years to come. Along these lines, the present paper advances a novel {\em typicality principle} which states, ro… ▽ More A central focus of data science is the transformation of empirical evidence into knowledge. As such, the key insights and scientific attitudes of deep thinkers like Fisher, Popper, and Tukey are expected to inspire exciting new advances in machine learning and artificial intelligence in years to come. Along these lines, the present paper advances a novel {\em typicality principle} which states, roughly, that if the observed data is sufficiently ``atypical'' in a certain sense relative to a posited theory, then that theory is unwarranted. This emphasis on typicality brings familiar but often overlooked background notions like model-checking to the inferential foreground. One instantiation of the typicality principle is in the context of parameter estimation, where we propose a new typicality-based regularization strategy that leans heavily on goodness-of-fit testing. The effectiveness of this new regularization strategy is illustrated in three non-trivial examples where ordinary maximum likelihood estimation fails miserably. We also demonstrate how the typicality principle fits within a bigger picture of reliable and efficient uncertainty quantification. △ Less

Submitted 24 January, 2025; originally announced January 2025.

Physics-Informed Machine Learning for Efficient Reconfigurable Intelligent Surface Design

Authors: Zhen Zhang, Jun Hui Qiu, Jun Wei Zhang, Hui Dong Li, Dong Tang, Qiang Cheng, Wei Lin

Abstract: Reconfigurable intelligent surface (RIS) is a two-dimensional periodic structure integrated with a large number of reflective elements, which can manipulate electromagnetic waves in a digital way, offering great potentials for wireless communication and radar detection applications. However, conventional RIS designs highly rely on extensive full-wave EM simulations that are extremely time-consumin… ▽ More Reconfigurable intelligent surface (RIS) is a two-dimensional periodic structure integrated with a large number of reflective elements, which can manipulate electromagnetic waves in a digital way, offering great potentials for wireless communication and radar detection applications. However, conventional RIS designs highly rely on extensive full-wave EM simulations that are extremely time-consuming. To address this challenge, we propose a machine-learning-assisted approach for efficient RIS design. An accurate and fast model to predict the reflection coefficient of RIS element is developed by combining a multi-layer perceptron neural network (MLP) and a dual-port network, which can significantly reduce tedious EM simulations in the network training. A RIS has been practically designed based on the proposed method. To verify the proposed method, the RIS has also been fabricated and measured. The experimental results are in good agreement with the simulation results, which validates the efficacy of the proposed method in RIS design. △ Less

Submitted 20 January, 2025; originally announced January 2025.

A Regularized Online Newton Method for Stochastic Convex Bandits with Linear Vanishing Noise

Authors: Jingxin Zhan, Yuchen Xin, Kaicheng Jin, Zhihua Zhang

Abstract: We study a stochastic convex bandit problem where the subgaussian noise parameter is assumed to decrease linearly as the learner selects actions closer and closer to the minimizer of the convex loss function. Accordingly, we propose a Regularized Online Newton Method (RONM) for solving the problem, based on the Online Newton Method (ONM) of arXiv:2406.06506. Our RONM reaches a polylogarithmic regr… ▽ More We study a stochastic convex bandit problem where the subgaussian noise parameter is assumed to decrease linearly as the learner selects actions closer and closer to the minimizer of the convex loss function. Accordingly, we propose a Regularized Online Newton Method (RONM) for solving the problem, based on the Online Newton Method (ONM) of arXiv:2406.06506. Our RONM reaches a polylogarithmic regret in the time horizon $n$ when the loss function grows quadratically in the constraint set, which recovers the results of arXiv:2402.12042 in linear bandits. Our analyses rely on the growth rate of the precision matrix $Σ_t^{-1}$ in ONM and we find that linear growth solves the question exactly. These analyses also help us obtain better convergence rates when the loss function grows faster. We also study and analyze two new bandit models: stochastic convex bandits with noise scaled to a subgaussian parameter function and convex bandits with stochastic multiplicative noise. △ Less

Submitted 19 January, 2025; originally announced January 2025.

Parallel ADMM Algorithm with Gaussian Back Substitution for High-Dimensional Quantile Regression and Classification

Authors: Xiaofei Wu, Dingzi Guo, Rongmei Liang, Zhimin Zhang

Abstract: In the field of high-dimensional data analysis, modeling methods based on quantile loss function are highly regarded due to their ability to provide a comprehensive statistical perspective and effective handling of heterogeneous data. In recent years, many studies have focused on using the parallel alternating direction method of multipliers (P-ADMM) to solve high-dimensional quantile regression a… ▽ More In the field of high-dimensional data analysis, modeling methods based on quantile loss function are highly regarded due to their ability to provide a comprehensive statistical perspective and effective handling of heterogeneous data. In recent years, many studies have focused on using the parallel alternating direction method of multipliers (P-ADMM) to solve high-dimensional quantile regression and classification problems. One efficient strategy is to reformulate the quantile loss function by introducing slack variables. However, this reformulation introduces a theoretical challenge: even when the regularization term is convex, the convergence of the algorithm cannot be guaranteed. To address this challenge, this paper proposes the Gaussian Back-Substitution strategy, which requires only a simple and effective correction step that can be easily integrated into existing parallel algorithm frameworks, achieving a linear convergence rate. Furthermore, this paper extends the parallel algorithm to handle some novel quantile loss classification models. Numerical simulations demonstrate that the proposed modified P-ADMM algorithm exhibits excellent performance in terms of reliability and efficiency. △ Less

Submitted 12 January, 2025; originally announced January 2025.

SyNPar: Synthetic Null Data Parallelism for High-Power False Discovery Rate Control in High-Dimensional Variable Selection

Authors: Changhu Wang, Ziheng Zhang, Jingyi Jessica Li

Abstract: Balancing false discovery rate (FDR) and statistical power to ensure reliable discoveries is a key challenge in high-dimensional variable selection. Although several FDR control methods have been proposed, most involve perturbing the original data, either by concatenating knockoff variables or splitting the data into two halves, both of which can lead to a loss of power. In this paper, we introduc… ▽ More Balancing false discovery rate (FDR) and statistical power to ensure reliable discoveries is a key challenge in high-dimensional variable selection. Although several FDR control methods have been proposed, most involve perturbing the original data, either by concatenating knockoff variables or splitting the data into two halves, both of which can lead to a loss of power. In this paper, we introduce a novel approach called Synthetic Null Parallelism (SyNPar), which controls the FDR in high-dimensional variable selection while preserving the original data. SyNPar generates synthetic null data from a model fitted to the original data and modified to reflect the null hypothesis. It then applies the same estimation procedure in parallel to both the original and synthetic null data to estimate coefficients that indicate feature importance. By comparing the coefficients estimated from the null data with those from the original data, SyNPar effectively identifies false positives, functioning as a numerical analog of a likelihood ratio test. We provide theoretical guarantees for FDR control at any desired level while ensuring that the power approaches one with high probability asymptotically. SyNPar is straightforward to implement and can be applied to a wide range of statistical models, including high-dimensional linear regression, generalized linear models, Cox models, and Gaussian graphical models. Through extensive simulations and real data applications, we demonstrate that SyNPar outperforms state-of-the-art methods, including knockoffs and data-splitting methods, in terms of FDR control, power, and computational efficiency. △ Less

Submitted 9 January, 2025; originally announced January 2025.

Modeling Cell Type Developmental Trajectory using Multinomial Unbalanced Optimal Transport

Authors: Junhao Zhu, Kevin Zhang, Zhaolei Zhang, Dehan Kong

Abstract: Single-cell trajectory analysis aims to reconstruct the biological developmental processes of cells as they evolve over time, leveraging temporal correlations in gene expression. During cellular development, gene expression patterns typically change and vary across different cell types. A significant challenge in this analysis is that RNA sequencing destroys the cell, making it impossible to track… ▽ More Single-cell trajectory analysis aims to reconstruct the biological developmental processes of cells as they evolve over time, leveraging temporal correlations in gene expression. During cellular development, gene expression patterns typically change and vary across different cell types. A significant challenge in this analysis is that RNA sequencing destroys the cell, making it impossible to track gene expression across multiple stages for the same cell. Recent advances have introduced the use of optimal transport tools to model the trajectory of individual cells. In this paper, our focus shifts to a question of greater practical importance: we examine the differentiation of cell types over time. Specifically, we propose a novel method based on discrete unbalanced optimal transport to model the developmental trajectory of cell types. Our method detects biological changes in cell types and infers their transitions to different states by analyzing the transport matrix. We validated our method using single-cell RNA sequencing data from mouse embryonic fibroblasts. The results accurately identified major developmental changes in cell types, which were corroborated by experimental evidence. Furthermore, the inferred transition probabilities between cell types are highly congruent to biological ground truth. △ Less

Submitted 6 January, 2025; originally announced January 2025.

NeuroPMD: Neural Fields for Density Estimation on Product Manifolds

Authors: William Consagra, Zhiling Gu, Zhengwu Zhang

Abstract: We propose a novel deep neural network methodology for density estimation on product Riemannian manifold domains. In our approach, the network directly parameterizes the unknown density function and is trained using a penalized maximum likelihood framework, with a penalty term formed using manifold differential operators. The network architecture and estimation algorithm are carefully designed to… ▽ More We propose a novel deep neural network methodology for density estimation on product Riemannian manifold domains. In our approach, the network directly parameterizes the unknown density function and is trained using a penalized maximum likelihood framework, with a penalty term formed using manifold differential operators. The network architecture and estimation algorithm are carefully designed to handle the challenges of high-dimensional product manifold domains, effectively mitigating the curse of dimensionality that limits traditional kernel and basis expansion estimators, as well as overcoming the convergence issues encountered by non-specialized neural network methods. Extensive simulations and a real-world application to brain structural connectivity data highlight the clear advantages of our method over the competing alternatives. △ Less

Submitted 6 January, 2025; originally announced January 2025.

Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networks

Authors: Carlos Misael Madrid Padilla, Oscar Hernan Madrid Padilla, Yik Lun Kei, Zhi Zhang, Yanzhen Chen

Abstract: This paper addresses the problems of conditional variance estimation and confidence interval construction in nonparametric regression using dense networks with the Rectified Linear Unit (ReLU) activation function. We present a residual-based framework for conditional variance estimation, deriving nonasymptotic bounds for variance estimation under both heteroscedastic and homoscedastic settings. We… ▽ More This paper addresses the problems of conditional variance estimation and confidence interval construction in nonparametric regression using dense networks with the Rectified Linear Unit (ReLU) activation function. We present a residual-based framework for conditional variance estimation, deriving nonasymptotic bounds for variance estimation under both heteroscedastic and homoscedastic settings. We relax the sub-Gaussian noise assumption, allowing the proposed bounds to accommodate sub-Exponential noise and beyond. Building on this, for a ReLU neural network estimator, we derive non-asymptotic bounds for both its conditional mean and variance estimation, representing the first result for variance estimation using ReLU networks. Furthermore, we develop a ReLU network based robust bootstrap procedure (Efron, 1992) for constructing confidence intervals for the true mean that comes with a theoretical guarantee on the coverage, providing a significant advancement in uncertainty quantification and the construction of reliable confidence intervals in deep learning settings. △ Less

Submitted 31 December, 2024; v1 submitted 29 December, 2024; originally announced December 2024.

Decoupled Functional Central Limit Theorems for Two-Time-Scale Stochastic Approximation

Authors: Yuze Han, Xiang Li, Jiadong Liang, Zhihua Zhang

Abstract: In two-time-scale stochastic approximation (SA), two iterates are updated at different rates, governed by distinct step sizes, with each update influencing the other. Previous studies have demonstrated that the convergence rates of the error terms for these updates depend solely on their respective step sizes, a property known as decoupled convergence. However, a functional version of this decoupl… ▽ More In two-time-scale stochastic approximation (SA), two iterates are updated at different rates, governed by distinct step sizes, with each update influencing the other. Previous studies have demonstrated that the convergence rates of the error terms for these updates depend solely on their respective step sizes, a property known as decoupled convergence. However, a functional version of this decoupled convergence has not been explored. Our work fills this gap by establishing decoupled functional central limit theorems for two-time-scale SA, offering a more precise characterization of its asymptotic behavior. To achieve these results, we leverage the martingale problem approach and establish tightness as a crucial intermediate step. Furthermore, to address the interdependence between different time scales, we introduce an innovative auxiliary sequence to eliminate the primary influence of the fast-time-scale update on the slow-time-scale update. △ Less

Submitted 14 January, 2025; v1 submitted 22 December, 2024; originally announced December 2024.

Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models

Authors: Zijun Chen, Wenbo Hu, Guande He, Zhijie Deng, Zheng Zhang, Richang Hong

Abstract: Multimodal large language models (MLLMs) combine visual and textual data for tasks such as image captioning and visual question answering. Proper uncertainty calibration is crucial, yet challenging, for reliable use in areas like healthcare and autonomous driving. This paper investigates representative MLLMs, focusing on their calibration across various scenarios, including before and after visual… ▽ More Multimodal large language models (MLLMs) combine visual and textual data for tasks such as image captioning and visual question answering. Proper uncertainty calibration is crucial, yet challenging, for reliable use in areas like healthcare and autonomous driving. This paper investigates representative MLLMs, focusing on their calibration across various scenarios, including before and after visual fine-tuning, as well as before and after multimodal training of the base LLMs. We observed miscalibration in their performance, and at the same time, no significant differences in calibration across these scenarios. We also highlight how uncertainty differs between text and images and how their integration affects overall uncertainty. To better understand MLLMs' miscalibration and their ability to self-assess uncertainty, we construct the IDK (I don't know) dataset, which is key to evaluating how they handle unknowns. Our findings reveal that MLLMs tend to give answers rather than admit uncertainty, but this self-assessment improves with proper prompt adjustments. Finally, to calibrate MLLMs and enhance model reliability, we propose techniques such as temperature scaling and iterative prompt optimization. Our results provide insights into improving MLLMs for effective and responsible deployment in multimodal applications. Code and IDK dataset: https://github.com/hfutml/Calibration-MLLM. △ Less

Submitted 25 December, 2024; v1 submitted 19 December, 2024; originally announced December 2024.

Comments: Accepted to COLING 2025

Adjusting auxiliary variables under approximate neighborhood interference

Authors: Xin Lu, Yuhao Wang, Zhiheng Zhang

Abstract: Randomized experiments are the gold standard for causal inference. However, traditional assumptions, such as the Stable Unit Treatment Value Assumption (SUTVA), often fail in real-world settings where interference between units is present. Network interference, in particular, has garnered significant attention. Structural models, like the linear-in-means model, are commonly used to describe interf… ▽ More Randomized experiments are the gold standard for causal inference. However, traditional assumptions, such as the Stable Unit Treatment Value Assumption (SUTVA), often fail in real-world settings where interference between units is present. Network interference, in particular, has garnered significant attention. Structural models, like the linear-in-means model, are commonly used to describe interference; but they rely on the correct specification of the model, which can be restrictive. Recent advancements in the literature, such as the Approximate Neighborhood Interference (ANI) framework, offer more flexible approaches by assuming negligible interference from distant units. In this paper, we introduce a general framework for regression adjustment for the network experiments under the ANI assumption. This framework expands traditional regression adjustment by accounting for imbalances in network-based covariates, ensuring precision improvement, and providing shorter confidence intervals. We establish the validity of our approach using a design-based inference framework, which relies solely on randomization of treatment assignments for inference without requiring correctly specified outcome models. △ Less

Submitted 29 November, 2024; originally announced November 2024.

Comments: 46 pages

Anytime Acceleration of Gradient Descent

Authors: Zihan Zhang, Jason D. Lee, Simon S. Du, Yuxin Chen

Abstract: This work investigates stepsize-based acceleration of gradient descent with {\em anytime} convergence guarantees. For smooth (non-strongly) convex optimization, we propose a stepsize schedule that allows gradient descent to achieve convergence guarantees of $O(T^{-1.119})$ for any stopping time $T$, where the stepsize schedule is predetermined without prior knowledge of the stopping time. This res… ▽ More This work investigates stepsize-based acceleration of gradient descent with {\em anytime} convergence guarantees. For smooth (non-strongly) convex optimization, we propose a stepsize schedule that allows gradient descent to achieve convergence guarantees of $O(T^{-1.119})$ for any stopping time $T$, where the stepsize schedule is predetermined without prior knowledge of the stopping time. This result provides an affirmative answer to a COLT open problem \citep{kornowski2024open} regarding whether stepsize-based acceleration can yield anytime convergence rates of $o(T^{-1})$. We further extend our theory to yield anytime convergence guarantees of $\exp(-Ω(T/κ^{0.893}))$ for smooth and strongly convex optimization, with $κ$ being the condition number. △ Less

Submitted 8 December, 2024; v1 submitted 26 November, 2024; originally announced November 2024.

Comments: v2: We improve the convergence rate from $O(T^{-1.03})$ to O(T^{-1.119}) through more precise computations

Unlocking the Potential of Text-to-Image Diffusion with PAC-Bayesian Theory

Authors: Eric Hanchen Jiang, Yasi Zhang, Zhi Zhang, Yixin Wan, Andrew Lizarraga, Shufan Li, Ying Nian Wu

Abstract: Text-to-image (T2I) diffusion models have revolutionized generative modeling by producing high-fidelity, diverse, and visually realistic images from textual prompts. Despite these advances, existing models struggle with complex prompts involving multiple objects and attributes, often misaligning modifiers with their corresponding nouns or neglecting certain elements. Recent attention-based methods… ▽ More Text-to-image (T2I) diffusion models have revolutionized generative modeling by producing high-fidelity, diverse, and visually realistic images from textual prompts. Despite these advances, existing models struggle with complex prompts involving multiple objects and attributes, often misaligning modifiers with their corresponding nouns or neglecting certain elements. Recent attention-based methods have improved object inclusion and linguistic binding, but still face challenges such as attribute misbinding and a lack of robust generalization guarantees. Leveraging the PAC-Bayes framework, we propose a Bayesian approach that designs custom priors over attention distributions to enforce desirable properties, including divergence between objects, alignment between modifiers and their corresponding nouns, minimal attention to irrelevant tokens, and regularization for better generalization. Our approach treats the attention mechanism as an interpretable component, enabling fine-grained control and improved attribute-object alignment. We demonstrate the effectiveness of our method on standard benchmarks, achieving state-of-the-art results across multiple metrics. By integrating custom priors into the denoising process, our method enhances image quality and addresses long-standing challenges in T2I diffusion models, paving the way for more reliable and interpretable generative models. △ Less

Submitted 25 November, 2024; originally announced November 2024.

Dense ReLU Neural Networks for Temporal-spatial Model

Authors: Carlos Misael Madrid Padilla, Zhi Zhang, Xiaokai Luo, Daren Wang, Oscar Hernan Madrid Padilla

Abstract: In this paper, we focus on fully connected deep neural networks utilizing the Rectified Linear Unit (ReLU) activation function for nonparametric estimation. We derive non-asymptotic bounds that lead to convergence rates, addressing both temporal and spatial dependence in the observed measurements. By accounting for dependencies across time and space, our models better reflect the complexities of r… ▽ More In this paper, we focus on fully connected deep neural networks utilizing the Rectified Linear Unit (ReLU) activation function for nonparametric estimation. We derive non-asymptotic bounds that lead to convergence rates, addressing both temporal and spatial dependence in the observed measurements. By accounting for dependencies across time and space, our models better reflect the complexities of real-world data, enhancing both predictive performance and theoretical robustness. We also tackle the curse of dimensionality by modeling the data on a manifold, exploring the intrinsic dimensionality of high-dimensional data. We broaden existing theoretical findings of temporal-spatial analysis by applying them to neural networks in more general contexts and demonstrate that our proof techniques are effective for models with short-range dependence. Our empirical simulations across various synthetic response functions underscore the superior performance of our method, outperforming established approaches in the existing literature. These findings provide valuable insights into the strong capabilities of dense neural networks (Dense NN) for temporal-spatial modeling across a broad range of function classes. △ Less

Submitted 9 June, 2025; v1 submitted 15 November, 2024; originally announced November 2024.

Performance Analysis of uRLLC in scalable Cell-free Radio Access Network System

Authors: Ziyang Zhang, Dongming Wang, Yunxiang Guo, Yang Cao, Xiaohu You

Abstract: As a critical component of beyond fifth-generation (B5G) and sixth-generation (6G) mobile communication systems, ultra-reliable low-latency communication (uRLLC) imposes stringent requirements on latency and reliability. In recent years, with the improvement of mobile communication network, centralized and distributed processing schemes for cellfree massive multiple-input multiple-output (CF-mMIMO… ▽ More As a critical component of beyond fifth-generation (B5G) and sixth-generation (6G) mobile communication systems, ultra-reliable low-latency communication (uRLLC) imposes stringent requirements on latency and reliability. In recent years, with the improvement of mobile communication network, centralized and distributed processing schemes for cellfree massive multiple-input multiple-output (CF-mMIMO) have attracted significant research attention. This paper investigates the performance of a novel scalable cell-free radio access network (CF-RAN) architecture featuring multiple edge distributed units (EDUs) under the finite block length regime. Closed expressions for the upper and lower bounds of its expected spectral efficiency (SE) performance are derived, where centralized and fully distributed deployment can be treated as two special cases, respectively. Furthermore, the spatial distribution of user equipments (UEs) and remote radio units (RRUs) is examined and the analysis reveals that the interleaving RRUs deployment associated with the EDU can enhance SE performance under finite block length constraints with specific transmission error probability. The paper also compares Monte Carlo simulation results with multi-RRU clustering-based collaborative processing, validating the accuracy of the space-time exchange theory in the scalable CF-RAN scenario. By deploying scalable EDUs, a practical trade-off between latency and reliability can be achieved through spatial degree-of-freedom (DoF), offering a distributed and scalable realization of the space-time exchange theory. △ Less

Submitted 12 December, 2024; v1 submitted 13 November, 2024; originally announced November 2024.

See it, Think it, Sorted: Large Multimodal Models are Few-shot Time Series Anomaly Analyzers

Authors: Jiaxin Zhuang, Leon Yan, Zhenwei Zhang, Ruiqi Wang, Jiawei Zhang, Yuantao Gu

Abstract: Time series anomaly detection (TSAD) is becoming increasingly vital due to the rapid growth of time series data across various sectors. Anomalies in web service data, for example, can signal critical incidents such as system failures or server malfunctions, necessitating timely detection and response. However, most existing TSAD methodologies rely heavily on manual feature engineering or require e… ▽ More Time series anomaly detection (TSAD) is becoming increasingly vital due to the rapid growth of time series data across various sectors. Anomalies in web service data, for example, can signal critical incidents such as system failures or server malfunctions, necessitating timely detection and response. However, most existing TSAD methodologies rely heavily on manual feature engineering or require extensive labeled training data, while also offering limited interpretability. To address these challenges, we introduce a pioneering framework called the Time Series Anomaly Multimodal Analyzer (TAMA), which leverages the power of Large Multimodal Models (LMMs) to enhance both the detection and interpretation of anomalies in time series data. By converting time series into visual formats that LMMs can efficiently process, TAMA leverages few-shot in-context learning capabilities to reduce dependence on extensive labeled datasets. Our methodology is validated through rigorous experimentation on multiple real-world datasets, where TAMA consistently outperforms state-of-the-art methods in TSAD tasks. Additionally, TAMA provides rich, natural language-based semantic analysis, offering deeper insights into the nature of detected anomalies. Furthermore, we contribute one of the first open-source datasets that includes anomaly detection labels, anomaly type labels, and contextual description, facilitating broader exploration and advancement within this critical field. Ultimately, TAMA not only excels in anomaly detection but also provides a comprehensive approach for understanding the underlying causes of anomalies, pushing TSAD forward through innovative methodologies and insights. △ Less

Submitted 4 November, 2024; originally announced November 2024.

Comments: Under review

Difference-in-Differences with Time-varying Continuous Treatments using Double/Debiased Machine Learning

Authors: Michel F. C. Haddad, Martin Huber, Lucas Z. Zhang

Abstract: We propose a difference-in-differences (DiD) method for a time-varying continuous treatment and multiple time periods. Our framework assesses the average treatment effect on the treated (ATET) when comparing two non-zero treatment doses. The identification is based on a conditional parallel trend assumption imposed on the mean potential outcome under the lower dose, given observed covariates and p… ▽ More We propose a difference-in-differences (DiD) method for a time-varying continuous treatment and multiple time periods. Our framework assesses the average treatment effect on the treated (ATET) when comparing two non-zero treatment doses. The identification is based on a conditional parallel trend assumption imposed on the mean potential outcome under the lower dose, given observed covariates and past treatment histories. We employ kernel-based ATET estimators for repeated cross-sections and panel data adopting the double/debiased machine learning framework to control for covariates and past treatment histories in a data-adaptive manner. We also demonstrate the asymptotic normality of our estimation approach under specific regularity conditions. In a simulation study, we find a compelling finite sample performance of undersmoothed versions of our estimators in setups with several thousand observations. △ Less

Submitted 28 October, 2024; originally announced October 2024.