Here is a stack of papers about multivariate data analysis (grabbed from a course by Gilbert Saporta that was helded in 2010, CNAM Paris) that I should (have) read.

1. Jensen, D.R. and Ramirez, D.E. (2008). Anomalies in the Foundations of Ridge Regression. International Statistical Review, 76(1), 89-105.
2. Hyvarinen, A. and Oja, E. (2000). Independent component analysis: algorithms and applications. Neural networks, 13, 411-430.
3. Scholkopf, B., Smola, A., and Muller, K.R. (1998). Nonlinear Component Analysis as a Kernel Eigenvalue Problem. Neural Computation, 10, 1299-1319, 1998.
4. Louw, N. and Steel, S.J. (2006). Variable selection in kernel Fisher discriminant analysis by means of recursive feature elimination. Computational Statistics & Data Analysis, 51(3), 2043-2055.
5. Bastien, P., Tenenhaus, M., and Esposito-Vinzi, V. (2005). PLS Generalized linear regression, Computational Statistics & Data Analysis, 48, 17-46.
6. Krieger, A.M. and Green, P.E. (1999). A Generalized Rand-Index Method for Consensus Clustering of Separate Partitions of the Same Data Base. Journal of Classification, 16, 63-89.
7. Diana, G. and Tommasi, C. (2002). Cross validation methods in principal component analysis: a comparison. Statistical Methods and Applications, 11(1), 71-82.
8. Ben-Hur, A., Horn, D., Siegelman, H.T., and Vapnik, V. (2001). Support Vector Clustering. Journal of Machine Learning Research, 2, 125-137.
9. Gardner, S. and Le Roux, N.J. (2005). Extensions of Biplot Methodology to Discriminant Analysis. Journal of Classification, 22(1), 59-86.
10. Tarpey, T. and Kinateder, K.K.J. (2003). Clustering Functional Data. Journal of Classification, 20(1), 93-114.
11. Beh, E.J. and D’Ambra, L. (2009). Some Interpretative Tools for Non-Symmetrical Correspondence Analysis. Journal of Classification, 26(1), 55-76.
12. van Rosmalen, J., Groenen, P.J.F., Trejos, J., and Castillo, W. (2009). Optimization Strategies for Two-Mode Partitioning. Journal of Classification, 26(2), 155-181.
13. Rosipal, R. and Trejo, L.J. (2001). Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space. JMLR Special Issue on Kernel Methods.
14. Vicari, D. and Vichi, M. (2009). Structural Classification Analysis of Three-Way Dissimilarity Data. Journal of Classification, 26(2), 121-154.
15. Buja, A. and Swayne, D. (2002). Visualization Methodology for Multidimensional Scaling. Journal of Classification, 19(1), 7-43.
16. van de Velden, M. (2004). Optimal Scaling of Paired Comparison Data. Journal of Classification, 21(1), 89–109.
17. Ye, J. (2005). Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems. Journal of Machine Learning Research, 6, 483-502.
18. Aguilera, A.M., Escabias, M., and Valderrama, M.J. (2006). Using principal components for estimating logistic regression with high-dimensional multicollinear data. Computational Statistics & Data Analysis, 50(8), 1905-1924.
19. Bougeard, S., Hanafi, M., and Qannari, E.M. (2008). Continuum redundancy–PLS regression: A simple continuum approach. Computational Statistics & Data Analysis, 52(7), 3686-3696.
20. Piccarreta, R. (2008). Classification trees for ordinal variables. Computational Statistics, 23(3), 407-427. See also the rpartOrdinal R package and the associated JSS paper.
21. Escabias, M., Aguilera, A.M., and Valderrama, M.J. (2007). Functional PLS logit regression model. Computational Statistics & Data Analysis, 51(10), 4891- 4902.
22. Brusco, M.J., Singh, R., and Steinley, D. (2009). Variable Neighborhood Search Heuristics for Selecting a Subset of Variables in Principal Component Analysis. Psychometrika, 74(4), 705-726.
23. Hwang, H. (2009). Regularized Generalized Structured Component Analysis. Psychometrika, 74(3), 517-530.
24. Steinley, D. and Brusco, M.J. (2008). Selection of Variables in Cluster Analysis: An Empirical Comparison of Eight Procedures. Psychometrika, 73(1), 125-144.

I should add some additional papers of mine soon. In the meantime, here are two must have books:

• Le Roux, B. and Rouanet, H. (2004). Geometric data analysis: from correspondence analysis to structured data. Kluwer Academic Publishers.
• Greenacre, M. and Blasius, G. (2006). Multiple correspondence analysis and related methods. Chapman and Hall.

This year, I should also try to go deeper into the ade4 package. This monograph by Susan Holmes provides a useful summary of the duality diagram as used in ade4: Multivariate Data Analysis: The French Way (2005).