This paper proposes a new class of time varying models for which a vector of unknown parameters may vary stochastically or deterministically over time or be a mixture of both types. There are novel features to this class and its econometric treatment differs from the existing literature which typically separates stochastic and deterministic time variation in the parameters. Estimation methods for the former are often based on Bayesian resampling algorithms whereas nonparametric estimation methods are usually employed for fitting unknown deterministic functional forms. This paper develops instead a unified approach based on orthonormal series decompositions to estimating time variation irrespective of whether that variation is stochastic or deterministic. The proposed procedure has wide applicability, covering linear and nonlinear time series models as well as stochastic trends. Consistent estimators of the time varying structures are developed and the limit theory for each of the settings is established. A notable outcome is that unit root time-varying parameters can be estimated with asymptotic validity and fast rates of convergence when the unit root structure is captured by an orthonormal series representation. Other advantages include the flexibility and convenience of the approach in practical implementation. Simulations are conducted to examine finite sample performance and the procedures are illustrated in several real data examples.