Search tips
Search criteria 


Logo of scirepAboutEditorial BoardFor AuthorsScientific Reports
Sci Rep. 2017; 7: 11564.
Published online 2017 September 14. doi:  10.1038/s41598-017-11854-1
PMCID: PMC5599602

Measuring critical transitions in financial markets


Tipping points in complex systems are structural transitions from one state to another. In financial markets these critical points are connected to systemic risks, which have led to financial crisis in the past. Due to this, researchers are studying tipping points with different methods. This paper introduces a new method which bridges the gap between real-world portfolio management and statistical facts in financial markets in order to give more insight into the mechanics of financial markets.


Understanding the links between different assets, economic sectors or even between economic zones gives the opportunity to uncover the internal structures of financial markets. These interactions are part of the complex mechanics, which drive prices at stock exchanges up or down1. A striking effect in observed price time series are sudden trend switches 15, which sometimes affect only a few assets, while at other times a whole market is impacted and moves in a synchronised fashion. Such turmoils are not limited to stock markets but might have ripple effects on the economy of a country or even of the whole world6.

These tipping points are an interdisciplinary phenomenon found in fields with complex networks e.g. physics, biology and climate research7, 8. Scheffer et al.8 describe two possibilities of a regime shift. The first results from an exogenous shock while the other one is endogenous. Especially in financial systems connectivity and homogeneity play a major role in crisis research. A small perturbation can have a cascading effect threatening the whole system. When approaching an endogenous tipping point one can observe critical slowing down, meaning a perturbation to the system state takes a long time to revert back to the preferred local state.

Hence it is of great interest to develop tools in order to identify current market states.

A common method at the disposal of researchers is the use of the correlation matrix between the log-return time-series of assets traded at stock markets. By using Random Matrix Theory (RMT) on such a correlation matrix it has been shown that markets consist of a dominate market mode and different economic sectors, which exhibit common trends919. It has been found that the role of the market mode in emerging markets is more important than in developed economic zones where in comparison the clustering of sectors is more dominant16, 20, 21.

With the financial crisis of 2008 the focus of researchers shifted to the empirical measurement of systemic risk22. In 2010, Billio et al.23 used Principal Component Analysis (PCA) on the correlation matrix derived from the return series with a sliding time window. They showed that for hedge funds, banks, brokers and insurance companies an increasing entanglement was present during the time of crisis in 200823, 24. This showed the prominent role of these market participants.

Following this train of thought, Zheng et al.22 used the eigenvectors of the time dependent correlation matrix consisting of ten US sector indices. They found that by tracing the sum of the four largest eigenvalues over time, one can relate the steepest increase to the systemic risk of the US economy represented by the chosen sector indices. Furthermore, they outlined that the magnitude of an increase corresponds to the interconnectedness of the system and therefore provided a precursor indicator.

In this paper, we introduce a new method which links known empirical facts to portfolio management research in order to empirically quantify market states in the S&P 500.

We start with the assumption, that only a few big market participants are responsible for the major changes in share prices25, 26 and we assume that these investors are not noise traders 27 but follow the principle of risk reduction by diversification2830 and belong to the group of fundamental traders31. The general idea of a diversified portfolio is that it is more robust against turbulences since not all selected assets are influenced by the news regarding one particular sector1, 32. However, determining the correlation between assets faces the problem that time-series in financial markets react on exogenous news and are therefore not stable3336 as commonly assumed1, 37. This forces investors to rebalance their portfolio to keep up with the changes within the market. One can imagine the inverse of rebalancing need as the energy barrier, which prevents an investor from moving to the next energy basin and a new preferred state. This paper aims to measure these critical transition points in time.


Extending the approach by Münnix et al.38, who investigated the similarity between correlation matrices at different times, we define the similarity between two points in time t, t′ as the L 1-norm similarity.

An external file that holds a picture, illustration, etc.
Object name is 41598_2017_11854_Article_Equ1.gif

between two portfolios P(t), P(t′). The entries of the similarity matrix ζ effectively measure the transaction costs an investor has to pay in order to rebalance the portfolio from one point in time to another. We use the daily closing prices of the S&P 500 components from 2000 until the end of 2016 as input data for the creation of mean-variance portfolios with a time window ω of 3 years.

Choosing the parameter ω to be 3 years ensures that the ratio between the number of data-points and the number of assets Q is greater than 1.5 and therefore the correlation matrix is always well behaved.

Figure ((1)1) shows the similarity matrix ζ for the minimal variance portfolios.

Figure 1
Similarity matrix ζ for the minimal variance investor approximation. There is a clear separation in October 2008 and again in October 2011. The subclusters are also a sign of minor changes within the minimum-variance portfolios.

In this matrix, one can identify two clusters of high similarity. The first one spans from October 2008 to September 2011 with an average similarity ζ = 0.58. The next visible cluster ranges from September 2011 until July 2014 and has an average similarity of ζ = 0.48. Before October 2008, the high similarity clusters are overlapping each other and do not show such a sharp structural change as in October 2008, September 2011 and July 2014. One notices smaller transitions, for example in September 2001, but these occur within a moderate similarity level.

In order to get more detailed information on the market phase duration depicted by similarity clusters, we use eigenvalue decomposition on the similarity matrix ζ.

The resulting normed eigenvalues

An external file that holds a picture, illustration, etc.
Object name is 41598_2017_11854_Article_IEq1.gif
correspond to the importance of the eigenvector u k in decreasing order.

These unit eigenvectors are orthogonal to each other and describe the data with a new set of basis vectors. In our case, the eigenvectors have the following interpretation: Each component of an eigenvector corresponds to a point in time where its absolute value signals whether at time t there is a significant contribution for describing a certain level of similarity in the matrix ζ. Since the eigenvectors correspond to a time-series, successive high values in the eigenvector imply a similar level of similarity. Therefore by following the temporal evolvement of u kt, one follows a direction of similarity level. By taking the fourth power of every entry

An external file that holds a picture, illustration, etc.
Object name is 41598_2017_11854_Article_IEq2.gif
, unimportant entries become negligible small while the rest are amplified, which is usually done when calculating the inverse participation ratio39, 40. This allows us to capture the participation of the eigenvector to a market state at that point in time.

In Fig. ((2)2) the first twelve eigenvectors to the fourth power are shown, which cover approximately 82% of the information.

Figure 2
The fourth power of the first twelve eigenvectors
An external file that holds a picture, illustration, etc.
Object name is 41598_2017_11854_Article_IEq3.gif
for the minimum-variance similarity matrix (γ = 0) in comparison to the S&P 500 Index (black line). There are several transition present from 2000–2017. For example,

The eigenvectors k = 1, 3 and 4 are needed to explain most of the similarities between 2000 and 2008 and represent 28% of the overall information. The eigenvectors u 2 and u 5 are representing 17% of the similarity matrix. One notices the separation in October 2008. After this point in time a new subspace is needed to describe the properties of the similarity matrix ζ. This is a sign of structural change in the investor’s minimal-variance portfolio. From October 2008 on, an investor would have had to completely change his portfolio in order to reposition himself in the new market environment. The next six eigenvectors show shorter clusters. In the subspace spanned by the directions of k = 6, 8, 10 one can find a cluster right before the financial crisis of 2008. This phase ranges from November 2007 until October 2008. All other eigenvectors have a significant participation over the whole period of time from 2000 until 2017 and are showing shorter phases with equal similarities, except the eigenvector u 7.

Since this analysis relies on visually inspecting the temporal development of the eigenvectors, we automatized this procedure by using an algorithm (see methods), which maps, from a chosen set of eigenvectors, the significant ones at a time t to a state.

In Fig. ((3)3) the result of this algorithm for k = {0, 1}, k = {0, 1, 2, 3} and k = {0, 1, 2, 3, 4, 5} is shown. A lower state indicates that eigenvectors with smaller eigenvalues are used to construct this state.

Figure 3
States for the minimal variance investor for different sets of eigenvectors. The eigenvectors k = 0, 1 cover 39%, k = 0, ..., 3 cover 57% and k = 0, .., 5 make up 71% of the similarity matrix information. ...

If only the first two eigenvectors are used, the S&P 500 only uses two out of the 22 possible states, which means at least one eigenvector is significant over the whole time. State s = 2 lasts from 2000 until the end of October 2005, after that s = 1 is occupied until October 2008. Between July 2015 and November 2015 the market is switching from s = 1 to s = 2 five times and settles in state s = 1 after that.

Increasing the number of eigenvectors also increases the number of possible states but the number of actual occupied states is much lower than the number of possible states (2|k|). The biggest changes in these three cases occur in October 2005, October 2008, October 2011 and July 2015.

In order to verify whether there is an impact on the S&P 500 by a state change, we performed a Granger causality test41 between the changes in overall trading volume within a week and the absolute weekly state changes.

The results in Table 1 show that the week after a state change, with more than two eigenvectors, is linked to the volume changes in the S&P 500. For lags longer than a week, there is no Granger causality on a 0.05 significance level performed with a F-test.

Table 1
Results of the Granger Causality p-values for four different sets of eigenvectors with up to five weeks of lag.


The approximation of a risk aware investor by the mean-variance model aims to close the gap between stylized facts (volatility clustering, fat tails) and the connection between systemic risks and correlation matrices.

The S&P 500 analysis shows similar transition points to the pure PCA analysis of a correlation matrix22, 24, 42. From 2000 until 2017, there is clear jump in 2008 in all investor approximations γ[set membership][0, 1] supporting research by Preis et al.1, who found the diversity breakdown in times of market stress.

That implies that one can measure the transition point t c of a financial system by applying eigenvalue decomposition on the similarity matrix ζ and determine whether t′ belongs to the same similarity subspace as t < t′. If at t′ another set of eigenvectors is needed, then t′ is a transition point t c. By increasing the number of eigenvectors and therefore gaining descriptive information of the similarity matrix, more transition points can be identified and more subtle distinctions between states can be found. These points in time highlight the investment shifts in the market, which can be small or larger structural changes as it was the case in 2008 and 2011. These critical transitions are linked to volume changes in the S&P 500.

In summary, our method to find critical transition points within a financial market is based on the similarity between mean-variance portfolios at different times. The resulting matrix is analysed by eigenvalue decomposition, which uncovers the temporal development of different similarity levels. As a last step, the subspaces formed by the eigenvectors are mapped to unique states along the time axis. This method allows to find two new aspects of financial markets: Market phases must be classifiable by an investor approximation, times of systemic risk fall into transition phases. These results are complementary to the known stylized facts and should be incorporated when constructing new general financial market models.


In order to approximate an investors risk aversion, we use the mean-variance model by Markowitz43, 44 in its classical form. This is an optimisation problem, where one has to minimise the variance and maximise the return of a portfolio p. Moreover, by restricting the portfolio components p i to be positive for all available assets N, we only allow long positions. The resulting selection problem is solved by minimising the cost-function

An external file that holds a picture, illustration, etc.
Object name is 41598_2017_11854_Article_Equ2.gif

where μ i is the expected return of asset i and C the pearson covariance matrix. The parameter γ is used to balance the trade-off between risk and return. A value of γ = 0 would result in only minimising the variance, while a value of γ = 1 would cause portfolios to be only optimized for maximum return.

We implemented a coordinate descent algorithm as used by Friedman et al.45, 46 for fitting generalised linear models in order to generate efficient portfolios for a specific γ. As proposed by Altenbuchinger et al.47, one can incorporate the equality constraint ipi = 1 in the coordinate descent algorithm by substituting pk = 1 − ∑i ikpi. By calculating the partial derivative and solving for p k one obtains an extreme value for p k and p s. With the help of the second derivative and curve sketching, one can fulfill the constraint p i  0 and a coordinate descent step can be constructed. Iterating this update step in combination with active set cycling46, 48, 49, allows the generation of thousands of portfolios in a reasonable time frame.

The algorithm for mapping a given set of eigenvectors to a state works in 4 steps:

  1. Calculate the participation
    An external file that holds a picture, illustration, etc.
Object name is 41598_2017_11854_Article_IEq6.gif
    and normalize each column to the max-Norm.
  2. Map U to a state matrix S = U > τ, where τ is a given threshold. τ is set to 0.01 in this paper. Each row of S now represents a state.
  3. These states are now mapped to a state number S  s between 1 and the number of unique states found in S, where state 1 is the state combined from the eigenvectors with the lowest eigenvalues.
  4. The state time-series s is returned.

The data we used to create the mean-variance portfolios was downloaded from the WIKI Quandl database50.The financial time-series data for the S&P 500 was downloaded with the QUANDL50 data interface from the WIKI database.

Author Contributions

Author Contributions

J.J. conceived the experiment, J.J. and T.R. conducted the experiment, J.J., T.R., A.E. and I.M. analysed the results. All authors reviewed the manuscript.


Competing Interests

The authors declare that they have no competing interests.


Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


1. Preis T, Kenett DY, Stanley HE, Helbing D, Ben-Jacob E. Quantifying the behavior of stock correlations under market stress. Scientific reports. 2012;2:752. doi: 10.1038/srep00752. [PMC free article] [PubMed] [Cross Ref]
2. Petersen, A. M., Wang, F., Havlin, S. & Stanley, H. E. Market dynamics immediately before and after financial shocks: Quantifying the Omori, productivity, and Bath laws. Physical Review E - Statistical, Nonlinear, and Soft Matter Physics82 (2010). [PubMed]
3. Preis T, Schneider JJ, Stanley HE. Switching processes in financial markets. Proceedings of the National Academy of Sciences of the United States of America. 2011;108:7674–8. doi: 10.1073/pnas.1019484108. [PubMed] [Cross Ref]
4. Preis T, Stanley HE. Switching phenomena in a system with no switches. Journal of Statistical Physics. 2010;138:431–446. doi: 10.1007/s10955-009-9914-y. [Cross Ref]
5. Preis T. Econophysics—complex correlations and trend switchings in financial time series. The European Physical Journal-Special Topics. 2011;194:5–86. doi: 10.1140/epjst/e2011-01397-y. [Cross Ref]
6. Balogh, E., Simonsen, I., Nagy, B. Z. & Néda, Z. Persistent collective trend in stock markets. Physical Review E - Statistical, Nonlinear, and Soft Matter Physics82 (2010). [PubMed]
7. Scheffer M, et al. Early-warning signals for critical transitions. Nature. 2009;461:53–59. doi: 10.1038/nature08227. [PubMed] [Cross Ref]
8. Scheffer M, et al. Anticipating critical transitions. Science (New York, N.Y.) 2012;338:344–8. doi: 10.1126/science.1225244. [PubMed] [Cross Ref]
9. Bouchaud, J. P. & Potters, M. Financial applications of random matrix theory: a short review. arXiv 0910.1205 (2009).
10. Potters M, Bouchaud JP, Laloux L. Financial applications of random matrix theory: Old laces and new pieces. In. Acta Physica Polonica B. 2005;36:2767–2784.
11. Rosenow B, Plerou V, Gopikrishnan P, Amaral LAN, Stanley HE. Application of Random Matrix Theory To Study Cross-Correlations of Stock Prices. International Journal of Theoretical and Applied Finance. 2000;03:399–403. doi: 10.1142/S0219024900000267. [Cross Ref]
12. Utsugi, A., Ino, K. & Oshikawa, M. Random matrix theory analysis of cross correlations in financial markets. Physical Review E - Statistical, Nonlinear, and Soft Matter Physics70 (2004). [PubMed]
13. Kenett, D. Y., Shapira, Y. & Ben-Jacob, E. RMT assessments of the market latent information embedded in the stocks’ raw, normalized, and partial correlations. Journal of Probability and Statistics (2009).
14. Plerou V, Gopikrishnan P, Rosenow B, Nunes Amaral L, Stanley H. Universal and Nonuniversal Properties of Cross Correlations in Financial Time Series. Physical Review Letters. 1999;83:1471–1474. doi: 10.1103/PhysRevLett.83.1471. [Cross Ref]
15. Plerou, V. et al. Random matrix approach to cross correlations in financial data. Physical Review E - Statistical, Nonlinear, and Soft Matter Physics65 (2002). [PubMed]
16. Shen J, Zheng B. Cross-correlation in financial dynamics. EPL (Europhysics Letters) 2009;86:48005. doi: 10.1209/0295-5075/86/48005. [Cross Ref]
17. Laloux, L. & Bouchaud, J.-P. Random Matrix Theory and Financial Correlations. International Journal of Theoretical and Applied Finance3 (2000).
18. Laloux L, Cizeau P, Bouchaud J-P, Potters M. Noise Dressing of Financial Correlation Matrices. Physical Review Letters. 1999;83:1467–1470. doi: 10.1103/PhysRevLett.83.1467. [Cross Ref]
19. Qiu T, Zheng B, Chen G. Financial networks with static and dynamic thresholds. New Journal of Physics. 2010;12:043057. doi: 10.1088/1367-2630/12/4/043057. [Cross Ref]
20. Pan, R. K. & Sinha, S. Collective behavior of stock price movements in an emerging market. Physical Review E - Statistical, Nonlinear, and Soft Matter Physics76 (2007). [PubMed]
21. Jiang XF, Zheng B. Anti-correlation and subsector structure in financial systems. EPL (Europhysics Letters) 2012;97:48006. doi: 10.1209/0295-5075/97/48006. [Cross Ref]
22. Zheng Z, Podobnik B, Feng L, Li B. Changes in cross-correlations as an indicator for systemic risk. Scientific reports. 2012;2:888. doi: 10.1038/srep00888. [PMC free article] [PubMed] [Cross Ref]
23. Billio, M., Getmansky, M. & Lo, A. Measuring systemic risk in the finance and insurance sectors. NBER working paper (2010).
24. Billio M, Getmansky M, Lo AW, Pelizzon L. Econometric measures of connectedness and systemic risk in the finance and insurance sectors. Journal of Financial Economics. 2012;104:535–559. doi: 10.1016/j.jfineco.2011.12.010. [Cross Ref]
25. Coyne, K. P. & Witter, J. W. What makes your stock price go up and down. McKinsey&Company Number 2 (2002).
26. Keim DB, Madhavan A. Anatomy of the trading process empirical evidence on the behavior of institutional traders. Journal of Financial Economics. 1995;37:371–398. doi: 10.1016/0304-405X(94)00799-7. [Cross Ref]
27. Wang F. Overconfidence, Investor Sentiment, and Evolution. Journal of Financial Intermediation. 2001;10:138–170. doi: 10.1006/jfin.2001.0311. [Cross Ref]
28. Meucci A. Managing Diversification. Risk pp 7479 May 2009. 2010;27:74–79.
29. Fragkiskos, A. What is Portfolio Diversification? Alternative Investment Analyst Review 8–18 (2014).
30. Cesarone, F., Moretti, J. & Tardella, F. Does Greater Diversification Really Improve Performance in Portfolio Selection? Available at SSRN 2473630 1–15 (2014).
31. Lux T, Marchesi M. Scaling and criticality in a stochastic multi-agent model of a financial market. Nature. 1999;397:498–500. doi: 10.1038/17290. [Cross Ref]
32. Eckrot A, Jurczyk J, Morgenstern I. Ising model of financial markets with many assets. Physica A: Statistical Mechanics and its Applications. 2016;462:250–254. doi: 10.1016/j.physa.2016.06.045. [Cross Ref]
33. Brock W, Hommes CH, Wagener F. More hedging instruments may destablize markets. Journal of Economic Dynamics & Control. 2009;33:1912–1928. doi: 10.1016/j.jedc.2009.05.004. [Cross Ref]
34. Kenett DY, Preis T, Gur-Gershgoren G, Ben-Jacob E. Dependency Network and Node Influence: Application To the Study of Financial Markets. International Journal of Bifurcation and Chaos. 2012;22:1250181. doi: 10.1142/S0218127412501817. [Cross Ref]
35. Kenett, D. Y., Raddant, M., Lux, T. & Ben-Jacob, E. Evolvement of uniformity and volatility in the stressed global financial village. PLoS ONE7 (2012). [PMC free article] [PubMed]
36. Kenett, D. Y. et al. Index cohesive force analysis reveals that the US market became prone to systemic collapses since 2002. PLoS ONE6 (2011). [PMC free article] [PubMed]
37. Campbell RAJ, Forbes CS, Koedijk KG, Kofman P. Increasing correlations or just fat tails? Journal of Empirical Finance. 2008;15:287–309. doi: 10.1016/j.jempfin.2007.01.001. [Cross Ref]
38. Münnix MC, et al. Identifying states of a financial market. Scientific Reports. 2012;2:644. doi: 10.1038/srep00644. [PMC free article] [PubMed] [Cross Ref]
39. Rosenow B, Plerou V, Gopikrishnan P, Stanley HE. Portfolio Optimization and the Random Magnet Problem. Europhysics Letters. 2002;59:500–506. doi: 10.1209/epl/i2002-00135-4. [Cross Ref]
40. Conlon T, Ruskin HJ, Crane M. Random matrix theory and fund of funds portfolio optimisation. Physica A: Statistical Mechanics and its Applications. 2007;382:565–576. doi: 10.1016/j.physa.2007.04.039. [Cross Ref]
41. Granger C. Testing for causality. Journal of Economic Dynamics and Control. 1980;2:329–352. doi: 10.1016/0165-1889(80)90069-X. [Cross Ref]
42. Kritzman M, Li Y, Page S, Rigobon R. Principal Components as a Measure of Systemic Risk. The Journal of Portfolio Management. 2011;37:112–126. doi: 10.3905/jpm.2011.37.4.112. [Cross Ref]
43. Markowitz H. Portfolio Selection*. The J. Finance. 1952;7:77–91.
44. Markowitz, H. M. Portfolio Selection: Efficient Diversification of Investments (1959).
45. Friedman J, Hastie T, Höfling H, Tibshirani R, et al. Pathwise coordinate optimization. The Annals of Applied Statistics. 2007;1:302–332. doi: 10.1214/07-AOAS131. [Cross Ref]
46. Friedman JH, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software. 2010;33:1–22. doi: 10.18637/jss.v033.i01. [PMC free article] [PubMed] [Cross Ref]
47. Altenbuchinger M, et al. Reference point insensitive molecular data analysis. Bioinformatics. 2017;33:219. doi: 10.1093/bioinformatics/btw598. [PubMed] [Cross Ref]
48. Krishnapuram B, Carin L, Figueiredo MA, Hartemink AJ. Sparse multinomial logistic regression: Fast algorithms and generalization bounds. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2005;27:957–968. doi: 10.1109/TPAMI.2005.127. [PubMed] [Cross Ref]
49. Meier L, Van De Geer S, Bühlmann P. The group lasso for logistic regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 2008;70:53–71. doi: 10.1111/j.1467-9868.2007.00627.x. [Cross Ref]
50. Quandl. Quandl Financial and Economic Data.

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group