A fundamental question in neuroscience is to understand how information is represented in the activity of tens of thousands of neurons in the brain. Towards this end, low-rank matrix and tensor decompositions are commonly used to identify correlates of behavior in high-dimensional neural data. In this talk I will first present a novel tensor decomposition based on the slice rank which is able to disentangle mixed modes of covarying patterns in data tensors. Second, to compliment this statistical approach, I will present our recent dynamical systems modelling of neural activity over learning. Rather than factorizing data tensors themselves, we instead fit a dynamical system to the data, while constraining the tensor of parameters to be low rank. Together these projects highlight how applications in neural data can inspire new classes of low-rank models.