In [1]:
import networkx as nx
import matplotlib.pyplot as plt
import warnings
from custom import load_data as cf
from itertools import combinations

warnings.filterwarnings('ignore')

%matplotlib inline
%load_ext autoreload
%autoreload 2
%config InlineBackend.figure_format = 'retina'

Load Data

As usual, let's start by loading some network data. This time round, we have a physician trust network, but slightly modified such that it is undirected rather than directed.

This directed network captures innovation spread among 246 physicians in for towns in Illinois, Peoria, Bloomington, Quincy and Galesburg. The data was collected in 1966. A node represents a physician and an edge between two physicians shows that the left physician told that the righ physician is his friend or that he turns to the right physician if he needs advice or is interested in a discussion. There always only exists one edge between two nodes even if more than one of the listed conditions are true.

In [2]:
# Load the network. This network, while in reality is a directed graph,
# is intentionally converted to an undirected one for simplification.
G = cf.load_physicians_network()
In [3]:
# Make a Circos plot of the graph
from nxviz import CircosPlot

c = CircosPlot(G)
c.draw()

Question

What can you infer about the structure of the graph from the Circos plot?

My answer: The structure is interesting. The graph looks like the physician trust network is comprised of discrete subnetworks.

Structures in a Graph

We can leverage what we have learned in the previous notebook to identify special structures in a graph.

In a network, cliques are one of these special structures.

Cliques

In a social network, cliques are groups of people in which everybody knows everybody.

Questions:

  1. What is the simplest clique?
  2. What is the simplest complex clique?

Let's try implementing a simple algorithm that finds out whether a node is present in a simple complex clique.

In [4]:
# Example code.
def in_triangle(G, node):
    """
    Returns whether a given node is present in a triangle relationship or not.
    """    
    # Then, iterate over every pair of the node's neighbors.
    for nbr1, nbr2 in combinations(G.neighbors(node), 2):
        # Check to see if there is an edge between the node's neighbors.
        # If there is an edge, then the given node is present in a triangle.
        if G.has_edge(nbr1, nbr2):
            # We return because any triangle that is present automatically 
            # satisfies the problem requirements.
            return True
    return False

in_triangle(G, 3)
Out[4]:
True

In reality, NetworkX already has a function that counts the number of triangles that any given node is involved in. This is probably more useful than knowing whether a node is present in a triangle or not, but the above code was simply for practice.

In [5]:
nx.triangles(G, 3)
Out[5]:
3

Exercise

Can you write a function that takes in one node and its associated graph as an input, and returns a list or set of itself + all other nodes that it is in a triangle relationship with? Do not return the triplets, but the set/list of nodes. (5 min.)

Possible Implementation: If I check every pair of my neighbors, any pair that are also connected in the graph are in a triangle relationship with me.

Hint: Python's itertools module has a combinations function that may be useful.

Hint: NetworkX graphs have a .has_edge(node1, node2) function that checks whether an edge exists between two nodes.

Verify your answer by drawing out the subgraph composed of those nodes.

In [6]:
# Possible answer
def get_triangles(G, node):
    neighbors1 = set(G.neighbors(node))
    triangle_nodes = set()
    triangle_nodes.add(node)
    """
    Fill in the rest of the code below.
    """
    for nbr1, nbr2 in combinations(neighbors1, 2):
        if G.has_edge(nbr1, nbr2):
            triangle_nodes.add(nbr1)
            triangle_nodes.add(nbr2)
    return triangle_nodes

# Verify your answer with the following funciton call. Should return something of the form:
# {3, 9, 11, 41, 42, 67}
get_triangles(G, 3)
Out[6]:
{3, 9, 11, 41, 42, 67}
In [7]:
# Then, draw out those nodes.
nx.draw(G.subgraph(get_triangles(G, 3)), with_labels=True)
In [8]:
# Compare for yourself that those are the only triangles that node 3 is involved in.
neighbors3 =  list(G.neighbors(3))
neighbors3.append(3)
nx.draw(G.subgraph(neighbors3), with_labels=True)

Friend Recommendation: Open Triangles

Now that we have some code that identifies closed triangles, we might want to see if we can do some friend recommendations by looking for open triangles.

Open triangles are like those that we described earlier on - A knows B and B knows C, but C's relationship with A isn't captured in the graph.

What are the two general scenarios for finding open triangles that a given node is involved in?

  1. The given node is the centre node.
  2. The given node is one of the termini nodes.

Exercise

Can you write a function that identifies, for a given node, the other two nodes that it is involved with in an open triangle, if there is one? (5 min.)

Note: For this exercise, only consider the case when the node of interest is the centre node.

Possible Implementation: Check every pair of my neighbors, and if they are not connected to one another, then we are in an open triangle relationship.

In [9]:
def get_open_triangles(G, node):
    """
    There are many ways to represent this. One may choose to represent
    only the nodes involved in an open triangle; this is not the 
    approach taken here.
    
    Rather, we have a code that explicitly enumrates every open triangle present.
    """
    open_triangle_nodes = []
    neighbors = set(G.neighbors(node))
    
    for n1, n2 in combinations(neighbors, 2):
        if not G.has_edge(n1, n2):
            open_triangle_nodes.append([n1, node, n2])
    
    return open_triangle_nodes
In [10]:
# # Uncomment the following code if you want to draw out each of the triplets.
# nodes = get_open_triangles(G, 2)
# for i, triplet in enumerate(nodes):
#     fig = plt.figure(i)
#     nx.draw(G.subgraph(triplet), with_labels=True)
print(get_open_triangles(G, 3))
len(get_open_triangles(G, 3))
[[1, 3, 67], [1, 3, 101], [1, 3, 9], [1, 3, 41], [1, 3, 11], [1, 3, 42], [1, 3, 112], [1, 3, 91], [67, 3, 101], [67, 3, 9], [67, 3, 41], [67, 3, 11], [67, 3, 112], [67, 3, 91], [101, 3, 9], [101, 3, 41], [101, 3, 11], [101, 3, 42], [101, 3, 112], [101, 3, 91], [9, 3, 42], [9, 3, 112], [9, 3, 91], [41, 3, 11], [41, 3, 42], [41, 3, 112], [41, 3, 91], [11, 3, 42], [11, 3, 112], [11, 3, 91], [42, 3, 112], [42, 3, 91], [112, 3, 91]]
Out[10]:
33

Triangle closure is also the core idea behind social networks' friend recommendation systems; of course, it's definitely more complicated than what we've implemented here.

Cliques

We have figured out how to find triangles. Now, let's find out what cliques are present in the network. Recall: what is the definition of a clique?

  • NetworkX has a clique-finding algorithm implemented.
  • This algorithm finds all maximally-sized cliques for a given node.
  • Note that maximal cliques of size n include all cliques of size < n
In [11]:
list(nx.find_cliques(G))[0:20]
Out[11]:
[[1, 2],
 [1, 3],
 [1, 4, 5, 6],
 [1, 7],
 [1, 72],
 [2, 41],
 [2, 10],
 [2, 11, 40],
 [2, 11, 39],
 [2, 42, 110],
 [3, 67, 42],
 [3, 101],
 [3, 9, 41],
 [3, 9, 11],
 [3, 112],
 [3, 91],
 [4, 6, 32],
 [4, 6, 109],
 [4, 104],
 [4, 74, 116]]

Exercise

Try writing a function maximal_cliques_of_size(size, G) that implements a search for all maximal cliques of a given size. (3 min.)

In [12]:
def maximal_cliqes_of_size(size, G):
    # Defensive programming check.
    assert isinstance(size, int), "size has to be an integer"
    assert size >= 2, "cliques are of size 2 or greater."
    
    return [i for i in list(nx.find_cliques(G)) if len(i) == size]

maximal_cliqes_of_size(2, G)[0:20]
Out[12]:
[[1, 2],
 [1, 3],
 [1, 7],
 [1, 72],
 [2, 41],
 [2, 10],
 [3, 101],
 [3, 112],
 [3, 91],
 [4, 104],
 [4, 22],
 [4, 59],
 [5, 39],
 [5, 45],
 [5, 91],
 [7, 102],
 [10, 108],
 [10, 54],
 [11, 79],
 [11, 87]]

Connected Components

From Wikipedia:

In graph theory, a connected component (or just component) of an undirected graph is a subgraph in which any two vertices are connected to each other by paths, and which is connected to no additional vertices in the supergraph.

NetworkX also implements a function that identifies connected component subgraphs.

Remember how based on the Circos plot above, we had this hypothesis that the physician trust network may be divided into subgraphs. Let's check that, and see if we can redraw the Circos visualization.

In [13]:
ccsubgraphs = list(nx.connected_component_subgraphs(G))
ccsubgraphs
Out[13]:
[<networkx.classes.graph.Graph at 0x109e1b3c8>,
 <networkx.classes.graph.Graph at 0x109e1bba8>,
 <networkx.classes.graph.Graph at 0x109e1bcf8>,
 <networkx.classes.graph.Graph at 0x109e1b908>]

Exercise

Draw a circos plot of the graph, but now colour and order the nodes by their connected component subgraph. (5 min.)

Recall Circos API:

c = CircosPlot(G, node_order='...', node_color='...')
c.draw()
plt.show()  # or plt.savefig(...)
In [14]:
# Start by labelling each node in the master graph G by some number
# that represents the subgraph that contains the node.
for i, g in enumerate(ccsubgraphs):
    for n in g.nodes():
        G.node[n]['subgraph'] = i
In [15]:
c = CircosPlot(G, node_color='subgraph', node_order='subgraph')
c.draw()
plt.savefig('images/physicians.png', dpi=300)