香港科技大学：Latent Tree Models

AAAl 2014 Tutorial Latent tree models Part II: Definition and Properties Nevin L Zhang Dept. of computer Science Engineering The hong Kong Univ of Sci. Tech http://www.cse.ust.hk/lzhang

Latent Tree Models Part II: Definition and Properties Nevin L. Zhang Dept. of Computer Science & Engineering The Hong Kong Univ. of Sci. & Tech. http://www.cse.ust.hk/~lzhang AAAI 2014 Tutorial

Part l: Concept and Properties Latent tree Models Definition Relationship with finite mixture models Relationship with phylogenetictrees Basic Properties AAAl2014 Tutorial Nevin L Zhang HKUST

AAAI 2014 Tutorial Nevin L. Zhang HKUST 2 Part II: Concept and Properties  Latent Tree Models  Definition  Relationship with finite mixture models  Relationship with phylogenetic trees  Basic Properties

Basic Latent Tree ModelS LTM) Bayesian network All variables are discrete Y1 Structure is a rooted tree Y2 Y3 Leaf nodes are observed(manifest X4 variables Internal nodes are not observed X1)(X2)(X3 X5)(X6)(X7 (latent variables) aso known as hierarchical Parameters latent class(HLc)models, HLC models(Zhang JMLR 2004) P(Y1),P(Y2Y1),P(X1Y2),P(2Y2), Semantics P(X1,…,Xn,Y1,…,Ym) I P(Z I parent(Z) parent z∈{x1+XnY1……,Ym}

AAAI 2014 Tutorial Nevin L. Zhang HKUST 3 Basic Latent Tree Models (LTM)  Bayesian network  All variables are discrete  Structure is a rooted tree  Leaf nodes are observed (manifest variables)  Internal nodes are not observed (latent variables)  Parameters:  P(Y1), P(Y2|Y1),P(X1|Y2), P(X2|Y2), …  Semantics: Also known as Hierarchical latent class (HLC) models, HLC models (Zhang. JMLR 2004)

Joint distribution over observed variables Marginalizing out the latent variables in P(X1, ...,Xn, Y1,..., Ym),we get a joint distribution over the observed variables P(X1, ..., Xn) In comparison with bayesian network without latent variables, LTM Is computationally very simple to work with Represent complex relationships among manifest variables What does the structure look like without the latent variables? Y1 X4 X1)(X2)(X3 X5(X6)X7 AAAl2014 Tutorial Nevin L Zhang HKUST

AAAI 2014 Tutorial Nevin L. Zhang HKUST 4  Marginalizing out the latent variables in , we get a joint distribution over the observed variables .  In comparison with Bayesian network without latent variables, LTM:  Is computationally very simple to work with.  Represent complex relationships among manifest variables.  What does the structure look like without the latent variables? Joint Distribution over Observed Variables

Pouch Latent Tree Models(PLTM) An extension of basic ltm (Poon et al. ICML 2010) Rooted tree Internal nodes represent discrete latent variables Each leaf node consists of one or more continuous observed variable called a pouch P(x1,x2y2)=N(x1,x2|2,∑y2) (-25,-25):y=51 (0,0 y2 (25,25) Y23) (Y3(3) Y4(3 10.5 y2 0.51 ∈{5}(2)(0)()(如)( AAAl2014 Tutorial Nevin L Zhang HKUST 5

AAAI 2014 Tutorial Nevin L. Zhang HKUST 5 Pouch Latent Tree Models (PLTM)  An extension of basic LTM  Rooted tree  Internal nodes represent discrete latent variables  Each leaf node consists of one or more continuous observed variable, called a pouch. (Poon et al. ICML 2010)

More general latent variable tree models Some internal nodes can be observed Internal nodes can be continuous Choi et al. JMLR 20I D) Forest @E回 2。 @e@@ @ BsisO G @@ Primary focus of this tutorial the basic ltm AAAl2014 Tutorial Nevin L Zhang HKUST

AAAI 2014 Tutorial Nevin L. Zhang HKUST 6 More General Latent Variable Tree Models  Some internal nodes can be observed  Internal nodes can be continuous  Forest  Primary focus of this tutorial: the basic LTM (Choi et al. JMLR 2011)

Part l: Concept and Properties Latent tree Models Definition Relationship with finite mixture models Relationship with phylogenetictrees Basic Properties AAAl2014 Tutorial Nevin L Zhang HKUST

AAAI 2014 Tutorial Nevin L. Zhang HKUST 7 Part II: Concept and Properties  Latent Tree Models  Definition  Relationship with finite mixture models  Relationship with phylogenetic trees  Basic Properties

Finite Mixture Models( FMM) Gaussian Mixture Models(GMM) Continuous attributes p(x)=∑P(z=k)xz=k)=∑xp(x2=k) p(x2z=k)=N(xuk,∑k) Graphical model z X1,xX2,Xx3,X4,x5,X6,X7,X8×9 AAAl2014 Tutorial Nevin L Zhang HKUST

AAAI 2014 Tutorial Nevin L. Zhang HKUST 8 Finite Mixture Models (FMM)  Gaussian Mixture Models (GMM): Continuous attributes  Graphical model

Finite Mixture Models FMM) GMM with independence assumption Block diagonal co-variable matrix X1 2 X3 X1 X2 0 Y2(3) Graphical Model XI X3 X2 AAAl2014 Tutorial Nevin L Zhang HKUST

AAAI 2014 Tutorial Nevin L. Zhang HKUST 9 Finite Mixture Models (FMM)  GMM with independence assumption  Block diagonal co-variable matrix  Graphical Model

Finite mixture models Latent class models (lcm): Discrete attributes Graphical Model P(x)=P(AA2…An)=∑P(2=k)ⅡP(4z=k) Distribution for cluster k: IIP(A)z Product multinomial distribution =1 All FMMs One latent variable Yielding one partition of data AAAl2014 Tutorial Nevin L Zhang HKUST 10

AAAI 2014 Tutorial Nevin L. Zhang HKUST 10 Finite Mixture Models  Latent class models (LCM): Discrete attributes Distribution for cluster k: Product multinomial distribution:  All FMMs  One latent variable  Yielding one partition of data Graphical Model