Outline

1. Data:
   • Citation network of articles on social network analysis
2. Model:
   • To cluster the articles into groups
   • A modified mixed membership stochastic block model
3. Application:
   • Fit model to citation network
   • Visualise memberships & network

Social Network Analysis

Three main groups of articles

1. Generative models
   • E.g. small-world model, preferential attachment model
2. Exponential random graph models
   • Log-density is a linear combination of summary statistics
3. Community detection and latent models
   • Algorithms that cluster nodes
   • Models that uncover latent structure
   • E.g. stochastic block model, latent space model

Citation Network

• 135 nodes (articles), 1118 edges (citations)
• If A cites B, B cannot cite A

Adjacency Matrix Representation

Arbitrary order

Topological Order

Our data is a directed acyclic graph (DAG)

Clustering/Community Detection

Spinglass algorithm

Clustering/Community Detection

Walktrap algorithm

Group-to-group probabilities

• The densities can be seen as the probability of citing, given the group combination
• Rows or columns don’t need to sum to 1

Stochastic Block Model (SBM)

• Example: Article A cites article B
• Also assume A is in group 1, B in group 2
• P(A cites B, A in 1, B in 2)
  = P(A in 1) x P(B in 2) x P(A cites B | A in 1, B in 2)
  = P(A in 1) x P(B in 2) x 0.1

Stochastic Block Model (SBM)

Holland, Laskey, and Leinhardt (1983), Social Networks

• Ingredients of likelihood
  1. The group-to-group probabilities
  2. The latent groups the articles belong to
• The articles are hard clustered

Mixed Membership SBM

Airoldi et al. (2008), JMLR

• Ingredients of likelihood
  1. The group-to-group probabilities
  2. The latent groups the articles belong to,
     for their pairwise interactions
  3. The memberships of the articles
• The articles are soft clustered

Modifications

• Our proposed model is for DAGs
• The number of latent variables halved
• Topological order as extra parameter as it is not unique

Number of Groups

• Can be modelled e.g. Peixoto (2018)
• Not incorporated in our model (yet)
• Fit model with different numbers of groups

Statistical Inference

• Airoldi et al. (2008) used variational Bayes
  • Fast, but accuracy not guaranteed
• We use a regular Gibbs sampler in MCMC
  • Feasible for the size of our data
• Potentially more efficient / scalable alternatives
  • Stochastic gradient MCMC (Li, Ahn, and Welling 2016)
  • Collapsed Gibbs sampler

Back to Citation Network

• Prior knowledge of 3 main groups - manual clustering
• Fit model with 3, 4, 5 & 6 groups; results for 4 groups

Group-to-group Probabilities

Mixed Memberships

Network plot

Membership projection

Topological Order

For 3, 4, 5 & 6 groups

Summary

• Model
  • A modified mixed membership stochastic block model
  • Suitable for directed acyclic graphs
• Inference
  • Number of latent variables halved in Gibbs sampler
• Application
  • Citation network of articles on social network analysis
  • Revealed 3 main groups (+ 1 miscellaneous group)
• Next
  • Model the number of groups
  • Inference alternatives
  • Apply to other data e.g. software dependencies