Tag: transformers
All the talks with the tag "transformers".
PIDformer - Transformer Meets Control Theory
Pankaj KumarPublished: at 11:00 AMThis talk discusses a paper that addresses two key flaws of Transformer architectures - input corruption and rank collapse in output representation, by framing self-attention as a state-space model and revealing its tendency towards lower-rank outputs and sensitivity to input perturbations and introducing a Proportional-Integral-Derivative (PID) feedback control system to enhance robustness and representation capacity.
Are Transformers Effective for Time Series Forecasting?
Jyotirmaya ShivottamPublished: at 07:30 PMThis talk discusses a recent paper that compares the effectiveness of transformer-based architectures to simple linear (NN) models for long-term time series forecasting tasks. The paper concludes that linear models outperform transformers in these tasks and provides a hypothesis for this observation. We will also briefly discuss some recent papers that use transformers for time series forecasting and end with a discussion on a literature gap in this domain.
Exploring Long-term (Time-)Series Forecasting (LTSF) using Echo State Networks (ESNs) and comparisons with Single-Layer Perceptron (SLP), MLP, LSTM and especially Attention-based methods
Jyotirmaya ShivottamPublished: at 12:14 AMThis talk will explore Echo State Networks (ESNs) and their applications in Long-term (Time-)Series Forecasting (LTSF). We will compare ESNs with Single-Layer Perceptron (SLP), Multi-Layer Perceptron (MLP), Long Short-Term Memory (LSTM) networks, and especially attention-based methods for LTSF.
End-to-end Object Detection with Transformers
Rucha Bhalchandra JoshiPublished: at 02:00 PMThis talk presents DETR, a new method for object detection that streamlines the pipeline by removing the need for components like non-maximum suppression and anchor generation.
End-to-end Object Detection with Transformers
Rucha Bhalchandra JoshiPublished: at 02:00 PMThis talk presents DETR, a new method for object detection that streamlines the pipeline by removing the need for components like non-maximum suppression and anchor generation.