Polysphere PCA-type Analysis Deeper New Methodology: For Polysphere ~Toy Example Pretty Good But Still Some Distortion

Thanks to B. Eltzner PPCA Principal Nested Spheres Main Goal: Extend Principal Arc Analysis ( to )

Jung et al (2012) Important Landmark: This Motivated Backwards PCA Backwards PCA Replace usual forwards view of PCA Data PC1 (1-d approx)

PC2 (1-d approx of Data-PC1) PC1 U PC2 (2-d approx) PC1 U U PCr (r-d approx) Backwards PCA With a backwards approach to PCA

Data PC1 U U PCr (r-d approx) PC1 U U PC(r-1) PC1 U PC2 (2-d approx) PC1

(1-d approx) Backwards PCA Euclidean Settings: Forwards PCA = Backwards PCA (Pythagorean Theorem,

ANOVA Decomposition) So Not Interesting But Very Different in Non-Euclidean Settings (Backwards is Better !?!) Backwards PCA Important Property of PCA:

Nested Series of Approximations (Often taken for granted) (Desirable in Non-Euclidean Settings) An Interesting Question How generally applicable is

Backwards approach to PCA? An Application: Nonnegative Matrix Factorization = PCA in Positive Orthant Think With 0 Constraints (on both & )

Nonnegative Matrix Factorization Standard NMF But Note Not Nested No Multi-scale Analysis Possible (Scores Plot?!?)

Nonnegative Nested Cone Analysis Same Toy Data Set Rank 1 Approx.

Properly Nested 1 Principal Curve st Linear Regn

Projs Regn Usual Smooth Princl Curve Manifold Learning Key Component: Principal Surfaces

LeBlanc & Tibshirani (1996) Challenge: Can have any dimensional surface, But how to nest??? Proposal: Backwards Approach

An Interesting Question How generally applicable is Backwards approach to PCA? An Attractive Answer An Interesting Question

How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Geometry Singularity Theory

An Interesting Question How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics

Damon and Marron (2014) An Interesting Question How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics

Key Idea: Express Backwards PCA as Nested Series of Constraints General View of Backwards

PCA Define Nested Spaces via Constraints Satisfying More Constraints Smaller Subspaces

General View of Backwards PCA Define Nested Spaces via Constraints E.g. SVD (Singular Value Decomposition =

= Not Mean Centered PCA) (notationally very clean) General View of Backwards PCA Define Nested Spaces via Constraints

E.g. SVD Have Nested Subspaces: General View of Backwards

PCA Define Nested Spaces via Constraints E.g. SVD -th SVD Subspace Scores

Loading Vectors General View of Backwards PCA Define Nested Spaces via Constraints E.g. SVD

Now Define: General View of Backwards PCA Define Nested Spaces via Constraints E.g. SVD

Now Define: Constraint Gives Nested Reduction of Dimn General View of Backwards PCA

Define Nested Spaces via Constraints Backwards PCA Reduce Using Affine Constraints General View of Backwards PCA

Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Use Affine Constraints (Planar Slices) In Ambient Space General View of Backwards

PCA Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces Spline Constraint Within Previous?

{Been Done Already???} General View of Backwards PCA Define Nested Spaces via Constraints

Backwards PCA Principal Nested Spheres Principal Surfaces Other Manifold Data Spaces Sub-Manifold Constraints?? (Algebraic Geometry)

General View of Backwards PCA Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces

Other Manifold Data Spaces Tree Spaces Suitable Constraints??? Vectors of Angles Vectors of Angles as Data Objects

Torus Type Space Useful in Structural Chemistry (sequences of bond angles) Vectors of Angles Vectors of Angles as Data Objects

Slice space with hyperplanes???? (ala Principal Nested Spheres) Vectors of Angles Tiled Embedding In

Vectors of Angles Slice space with hyperplanes???? An Approach: E.g. : Embed

in Vectors of Angles Embedding for (in ):

Note: Different from donut (treats angles same) Vectors of Angles E.g. , Data w/ Single Mode of Varn

Best Fitting Planar Slice gives Bimodal Distn Special Thanks to Eduardo Garca-Portugus Vectors of Angles E.g. , Data w/ Single Mode of Varn

Gets worse for tighter distribution Special Thanks to Eduardo Garca-Portugus Vectors of Angles E.g. , Data w/ Single Mode of Varn

Just slightly tighter gives: Disconnected Approxing Manifold Special Thanks to Eduardo Garca-Portugus Vectors of Angles

Vectors of Angles as Data Objects Slice space with hyperplanes???? Gives clumsy lower d subspaces Algebraic Geometry better slicing?

Vectors of Angles Vectors of Angles as Data Objects Slice space with hyperplanes???? Gives clumsy lower d subspaces Try Another Approach

Torus Space Tiled Embedding In Torus Space Zoomed

In View Torus Space Try To Fit A Geodesic

Challenge: Can Get Arbitrarily Close Torus Space Wilfrid Kendall Idea:

Reward Locality Implementation: Fit Nested Submanifold Centerpoint Geodesic

Simultaneously Torus Space Fit Nested Sub-Manifold Nested Submanifold Fits

Major Motivation of Simultaneous Fits: Barycentric Subspace Approach of Xavier Pennec Pennec (2015) PNS Terminology

Principal Nested Submanifolds Motivation: Flag (Nested Sequence of Subspaces) Want Non-Subspace Versions (e.g. Non-Great Circles on Spheres) PNS Main Idea

Data Objects: Where is a dimensional manifold Consider a nested series of sub-manifolds: where for and Goal: Fit all of simultaneously to

General Background Call each a stratum, so is a manifold stratification To be fit to Conventional Approach 1:

only, i.e. Frchet Mean, in Frchet Mean in Toy Data Set

Frchet Mean in Candidate Fit of 0-dim Stratum (point)

Frchet Mean in Residuals In Sense Of Frchet Frchet Mean in

Frchet Sum of Squares Criterion Frchet Mean in

Minimum Frchet Sum of Squares General Background

Call each a stratum, so is a manifold stratification To be fit to Conventional Approach 2: only, i.e. PC 1 line, in

PC 1 line, in Candidate Fit of 1-dim Stratum (line)

PC 1 line, in Residuals In PC Sense Of Linear Fit

PC 1 line, in Corresponding Residual Sum Of Squares

PC 1 line, in Minimum Of Linear Residual Sum Of

Squares General Background Call each a stratum, so is a manifold stratification

To be fit to New Approach: Simultaneously fit Nested Submanifold (NS) Nested Submanifolds in Fit 0-dim Submanifold

(i.e. mean) Nested Submanifolds in Fit 0-dim Submanifold (i.e. mean)

Nested Submanifolds in Fit 0-dim Submanifold (i.e. mean) Nested Submanifolds in

Fit 0-dim Submanifold (i.e. mean) Nested Submanifolds in Fit 0-dim Submanifold

(i.e. mean) Nested Submanifolds in Fit 0-dim Submanifold (i.e. mean)

Nested Submanifolds in Fit Nested Submanifolds Nested Submanifolds in Fit Nested Submanifolds Nested Submanifolds in

Fit Nested Submanifolds Nested Submanifolds in Fit Nested Submanifolds Nested Submanifolds in Fit Nested Submanifolds

Nested Submanifolds in Fit Nested Submanifolds Nested Submanifolds in Fit Nested Submanifolds

Nested Submanifolds in Fit Nested Submanifolds Nested Submanifolds in Fit Nested Submanifolds Nested Submanifolds in

Fit Nested Submanifolds Nested Submanifolds in Fit Nested Submanifolds Nested Submanifolds in Fit Nested Submanifolds

Nested Submanifold Fits Forwards Fit: Sequentially Fit As:

then then then Usual Explanation of Euclidean PCA Nested Submanifold Fits Backwards Fit:

Sequentially Fit As: then then then Useful for constructing many non-Euclidean analogs of PCA (Via Nested Relations)

Nested Submanifold Fits Key Pennec Observation: Both Forwards and Backwards Approaches Are Greedy Searches Often Sub-Optimal Projection Notation

For let onto denote the projection operator I.e. for where

is the manifold distance Projection Notation For let

onto denote the telescoping projection I.e. for Note: This projection is fundamental to

Backwards PCA methods PNS Components For a given , represent a point by its Nested Submanifold components: where for In the sense that means the shortest

geodesic arc between & NS Components in Candidate Nested Submanifold

NS Components in Candidate NS Components For : NS Components in Candidate NS

Components For : NS Components in Candidate NS Components For :

NS Components in Candidate NS Components For All 3 Nested Submanifold Fits

New Approach: Simultaneously fit Nested Submanifold (NS) Simultaneous Fit Criteria?

Nested Submanifold Fits Simultaneous Fit Criteria? Based on Stratum-Wise Sums of Squares For

define Uses lengths of NS Components: SS of NS Components in

SS of NS Components in SS of NS Components in SS of NS Components in

SS of NS Components in SS of NS Components in

SS of NS Components in SS of NS Components in

SS of NS Components in SS of NS Components in SS of NS Components in

SS of NS Components in SS of NS Components in

SS of NS Components in SS of NS Components in

SS of NS Components in SS of NS Components in SS of NS Components in

Nested Submanifolds in Candidate Nested Submanifold (i.e. point

line) NS Components in NS Candidate 1 NS Components in

NS Candidate 2 (Shifted to Sample Mean) Note: Both

& Decrease NS Components in NS based On PC1

Note: Yet is Constant (Pythagorean Thm)

NS Components in NS based On PC2 Note: is Constant

(Pythagorean Thm) NS Components in NS Candidate 1 NS Components in

NS Candidate 2 NS Components in NS based On PC1

NS Components in NS based On PC2 Nested Submanifold Fits

Simultaneously fit Simultaneous Fit Criterion? Above Suggests Want:

Works for Euclidean PCA (?) Nested Submanifold Fits Simultaneous Fit Criterion?

Above Suggests Want: Important Predecessor Pennec (2016) AUC Criterion: Participant Presentations

Shengjie Chai Cancer Metastesis Di Qin Kernel PCA Yaoyu Chen Introduction to Generative Adversarial Networks