Tag: Control Theory
All the talks with the tag "Control Theory".
PIDformer - Transformer Meets Control Theory
Pankaj KumarPublished: at 11:00 AMThis talk discusses a paper that addresses two key flaws of Transformer architectures - input corruption and rank collapse in output representation, by framing self-attention as a state-space model and revealing its tendency towards lower-rank outputs and sensitivity to input perturbations and introducing a Proportional-Integral-Derivative (PID) feedback control system to enhance robustness and representation capacity.