Conventional time series analysis, focusing exclusively on a time series at a given scale, lacks the ability to explain the nature of the data-generating process. A process equation that successfully explains daily price changes, for example, is unable to characterize the nature of hourly price changes. On the other hand, statistical properties of monthly price changes are often not fully covered by a model based on daily price changes. In this paper, we simultaneously model regimes of volatilities at multiple time scales through wavelet-domain hidden Markov models. We establish an important stylized property of volatility across different time scales. We call this property asymmetric vertical dependence. It is asymmetric in the sense that a low volatility state (regime) at a long time horizon is most likely followed by low volatility states at shorter time horizons. On the other hand, a high volatility state at long time horizons does not necessarily imply a high volatility state at shorter time horizons. Our analysis provides evidence that volatility is a mixture of high and low volatility regimes, resulting in a distribution that is non-Gaussian. This result has important implications regarding the scaling behavior of volatility, and, consequently, the calculation of risk at different time scales.