Bonassi Fabio

While scaling laws have established a fundamental framework for foundation models in natural language processing, their applicability to electrocardiogram (ECG) models remains poorly characterized. Indeed, recent studies do not always yield consistent downstream gains as one increases the model size or pre-training dataset size of ECG models, leaving the exact roles of architectural inductive biases, pre-training paradigms, and expected improvements with size largely unanswered. In this work, we systematically investigate neural and loss-to-loss scaling laws within the ECG domain. By pre-training over 120 models (ranging from 20K to 200M parameters) on the large-scale CODE dataset (2.3M records), we decouple the effects of model architecture (ResNet vs. Transformer) and pre-training paradigm, namely supervised learning (SL) versus self-supervised learning (SSL). We found that (i) SL models are data-bottlenecked in-distribution, whereas SSL models scale robustly across both model and data sizes; (ii) for out-of-distribution (OOD) generalization, ResNets are 1.3 to 2.5 times more parameter-efficient than Transformers, while SSL is up to 16 times more data-efficient and achieves up to 7.6 times higher transfer efficiency than SL on unseen clinical tasks; (iii) across the observed scales, ResNet-based models generally achieve the lowest OOD loss, with SSL dominating on unseen clinical tasks and self-supervised Transformers overtaking at very large model sizes. Our results suggest that the path to effective ECG foundation models lies in the strategic alignment of architecture and paradigm rather than brute-force scaling.

Recently, several direct Data-Driven Predictive Control (DDPC) methods have been proposed, advocating the possibility of designing predictive controllers from historical input-output trajectories without the need to identify a model. In this work, we show their equivalence to a (relaxed) indirect approach, allowing us to reformulate direct methods in terms of estimated parameters and covariance matrices. This allows us to provide further insights into how these direct predictive control methods work, showing that, for unconstrained problems, the direct methods are equivalent to subspace predictive control with a reduced weight on the tracking cost, and analyzing the impact of the data length on tuning strategies. Via a numerical experiment, we also illustrate why the performance of direct DDPC methods with fixed regularization tends to degrade as the number of training samples increases.

The goal of this paper is to provide a system identification-friendly introduction to the Structured State-space Models (SSMs). These models have become recently popular in the machine learning community since, owing to their parallelizability, they can be efficiently and scalably trained to tackle extremely-long sequence classification and regression problems. Interestingly, SSMs appear as an effective way to learn deep Wiener models, which allows to reframe SSMs as an extension of a model class commonly used in system identification. In order to stimulate a fruitful exchange of ideas between the machine learning and system identification communities, we deem it useful to summarize the recent contributions on the topic in a structured and accessible form. At last, we highlight future research directions for which this community could provide impactful contributions.

This brief addresses the design of a Nonlinear Model Predictive Control (NMPC) strategy for exponentially incremental Input-to-State Stable (ISS) systems. In particular, a novel formulation is devised, which does not necessitate the onerous computation of terminal ingredients, but rather relies on the explicit definition of a minimum prediction horizon ensuring closed-loop stability. The designed methodology is particularly suited for the control of systems learned by Recurrent Neural Networks (RNNs), which are known for their enhanced modeling capabilities and for which the incremental ISS properties can be studied thanks to simple algebraic conditions. The approach is applied to Gated Recurrent Unit (GRU) networks, providing also a method for the design of a tailored state observer with convergence guarantees. The resulting control architecture is tested on a benchmark system, demonstrating its good control performances and efficient applicability.

This doctoral thesis aims to establish a theoretically-sound framework for the adoption of Recurrent Neural Network (RNN) models in the context of nonlinear system identification and model-based control design. The idea, long advocated by practitioners, of exploiting the remarkable modeling performances of RNNs to learn black-box models of unknown nonlinear systems, and then using such models to synthesize model-based control laws, has already shown considerable potential in many practical applications. On the other hand, the adoption of these architectures by the control systems community has been so far limited, mainly because the generality of these architectures makes it difficult to attain general properties and to build solid theoretical foundations for their safe and profitable use for control design. To address these gaps, we first provide a control engineer-friendly description of the most common RNN architectures, i.e., Neural NARXs (NNARXs), Gated Recurrent Units (GRUs), and Long Short-Term Memory networks (LSTMs), as well as their training procedure. The stability properties of these architectures are then analyzed, using common nonlinear systems’ stability notions such as the Input-to-State Stability (ISS), the Input-to-State Practical Stability (ISPS), and the Incremental Input-to-State Stability (δISS). In particular, sufficient conditions for these properties are devised for the considered RNN architectures, and it is shown how to enforce these conditions during the training procedure, in order to learn provenly stable RNN models. Model-based control strategies are then synthesized for these models. In particular, nonlinear model predictive control schemes are first designed: in this context, the model’s δISS is shown to enable the attainment of nominal closed-loop stability and, under a suitable design of the control scheme, also robust asymptotic zero-error output regulation. Then, an alternative computationally-lightweight control scheme, based on the internal model control strategy, is proposed, and its closed-loop properties are discussed. The performances of these control schemes are tested on several nonlinear benchmark systems, demonstrating the potentiality of the proposed framework. Finally, some fundamental issues for the practical implementation of RNN-based control strategies are mentioned. In particular, we discuss the need for the safety verification of RNN models and their adaptation in front of changes of the plant’s behavior, the definition of RNN structures that exploit qualitative physical knowledge of the system to boost the performances and interpretability of these models, and the problem of designing control schemes that are robust to the unavoidable plant-model mismatch.