The authors used Ising lattices to encode the behavior of a Cellular Automata. In order to do this, they used layers of designer Ising blocks, which would map ground state configurations into corresponding states of the CA. I feel however that in their discussion, the Ising model only served as a sufficiently large state vector in generating just as many different configurations of CAs.

That aside, the authors also invoke the use of a Turing machine that interacts with an Ising lattice, such that complete knowledge of the lattice implies complete knowledge of the machine. By arguing that decidability or predictability of $P$, which is dependent on the output of the Turing machine, implies knowledge of the resulting configuration of the CA, but with the contradiction that this can only happen for certain configurations of the CA, they say that $P$ is not decidable, and thus, knowledge of the microscopic states (the Ising lattice underneath) cannot generally predict the macroscopic behavior (Cellular Automata). Thus their claim that complexity is indeed a phenomenon underivable from even a reductionist’s Theory of Everything.

Although I agree with the conclusion of the authors, I do believe that the way they presented their argument was a little lackluster. The physicist in me is more used to seeing plots of measurements of their system for different variables, and the many unfamiliar concepts they used in the layering of their model was highly unintuitive and difficult to follow. I believe that the weakness of the argument is it was constructed in more of a tell fashion. Again, I think it would be nice if there was also some showing of actual simulations and plots which could support their claims.

From Complexity to Perplexity

The author tackles some leading ideas in the field of complexity science, particularly among its proponents at the Santa Fe Institute. The first concept he criticizes is the vision of complexologists of a unifying theory of complexity: a bold claim that suggests that a single theory of complexity could explain complex phenomena in different fields. He then proceeds to question how can a field gun for a unifying theory of complexity, when complexologist themselves cannot agree on a proper definition of complexity. While I cannot deny the fact that there are several ground breaking and interesting insights resulting from complexity science, I personally agree that it might be too farfetched to claim a unified theory. After all, seeing how difficulty a unifying theory is using the reductionist approach (which by definition I guess means these laws will indeed be common to everything in the universe), it might be much more difficult to find a unifying theory among totally different emergent systems.

The author then proceeds to critique Artificial life; computer simulations that use agents following a simple set of rules as a toy model to mimic biological systems. Naomi Oreskes points out one major problem with Artificial life:

"Verification and validation of models of natural systems is impossible."

True, automata models can never capture the full picture of real systems, nor can we definitely say that we have verified the model even if it fits really well with actual data. Verification is best done when you have knowledge of all relevant variables, and have a way to control and isolate the dependent and independent variables. However, I also believe that the author misses the point: complexity science gives us a tool to simplify a physical system, turn it into a model in which we now have control over some parameters, and see how the system behaves in response to these parameters. In the context of traffic systems, this is an invaluable tool since it allows a researcher to mess with traffic without actually screwing over millions of citizens and the economy of a country.

Lastly, the author proceeds to tackle the concept of self-organized criticality (SOC) as a defining feature of complexity. Bak, the proponent of SOC, discovered that his sandpile model organizes itself in such a way that when you look at the distribution of avalanche sizes, a power law distribution would emerge. He then went on to note that other real world phenomenon such as the stock market, earthquakes, extinction of species and even human brain waves also exhibit this power law distribution. According to Bak, this power law distribution was the defining feature of SOC, and thus, complexity. This was criticized by Crutchfield, saying the following:

"If a theory applies to everything, it may really to nothing."
"You need not only statistics but also mechanisms."

Crutchfield is correct in saying that statistics (power law) alone is not enough as a theory for a phenomenon. However, my personal experience working in other complex systems research such as traffic also involves defining mechanisms for the model. These mechanisms limit the validity of predictions that can be obtained from the model itself. In conjunction with statistics, the insights obtained from the model are much more meaningful and insightful.

Complex Systems Science: Dreams of Universality, Reality of Interdisciplinarity

This last paper attempts to deal the crushing blow to complex systems science’s dreams of universality. The authors show that while complex systems science claims that it has found a universal theory for real world systems, the universal trend in the field is the use of computers and computational techniques. They criticize the fact that methodology, rather than theory, is the central denominator to the field of complex systems. Ironically, they arrived at this conclusion using tools that complex systems scientists use: the network.

"To try to become universal, theoretical approaches have to be “translated” into other disciplines."

Collaboration between disciplines manifested as trading zones shows that the collaboration appeared to one-way rather than mutual. Interestingly, interdisciplinarity of the field suffers from the fact that no experimentalists (to the author’s or my knowledge) actually refer to the results of the theorists of complexity science. It may be that despite its attempts at unifying different fields, the different approach at solving problems may have alienated and discouraged collaboration as well.

The take away from all of this is that complex science is far of from being a universal theory, but it has taken some steps in the right direction. Strong interdisciplinarity is key to discovering the possibility (or lack) of a unifying theme for real world systems. As a relatively young science trying to tackle a really large problem (as if reductionism already wasn’t hard enough…), it is reasonable tools and skill building would play a large role in its development. Science shouldn’t really be trigger happy at dismissing emerging theories that show promise, but at the same time, criticisms like this show shortcomings in the approach of a field, and actually provide feedback that can be used to actually improve.

Final thoughts

It is interesting to read all three papers, which actively criticized/supported the field of complexity science. Since my research field somewhat falls within complexity science, I have a rough idea of how the criticisms presented may or may not have merit. I wouldn’t actually say that it was very offensive to me as a researcher, but rather, enlightening that while scientists criticize some claims of complexity science, they also recognize its merits. I see all these articles as a challenge for all complex scientists to up their game, and show that world the impact of this new paradigm of science. Who knows, we might find a universal theory one way or another.

So is complexity science a science? Definitely.

We want to look at the growth of Autonomous Server (AS) networks found in the netdimes project to see how a network grows over time. In particular, we are looking at nodes in the year 2011. First, let us load the data we have downloaded:

In [8]:

%matplotlib inline
%config InlineBackend.figure_format = 'svg'
import numpy as np
import matplotlib.pyplot as plt
import glob
import pandas as pd
import networkx as nx
from statsmodels.distributions.empirical_distribution import ECDF
import seaborn as sns
sns.set_style('whitegrid')
color = sns.color_palette('bright', 10)
month = ['Jan', 'Feb', 'Mar', 'Apr','May','Jun','Jul','Aug','Sept','Oct','Nov','Dec']
edges = glob.glob('ASEdge*')
nodes = glob.glob('ASNode*')
edgelist = pd.read_csv(edges[0], header=None)
nodelist = pd.read_csv(nodes[0], header=None)

Size

Easy stuff first: let’s look at how the size of the network grows over time. Note that this time period is just one year.

In [9]:

size_nodes = [len(pd.read_csv('ASNodes%s_2011.csv.gz'%i, header=None)) for i in range(1,11)]

fig = plt.figure()
ax = fig.add_subplot(111)
ax.set_xlabel("2011")
ax.set_ylabel("Nodes")
ax.plot(size_nodes, linestyle='-', marker='o', label="Magnitude", color='r')
ax.set_xticklabels(month[:10])
plt.legend()

<matplotlib.legend.Legend at 0xd21c518>

So it appears as if the size of the AS network id more or less decreasing throughout the year. This is interesting, as I’d expect 2011 to be a year of strong growth of the internet. Nonetheless, this is server networks we are looking at, and 10 months is also a short timeframe.

Degree Distributions

We are going to use networkx to assemble the internet network. We will do this first for one month of the dataset:

In [10]:

fig = plt.figure()
ax = fig.add_subplot(111)
for i in range(10):
edgelist = pd.read_csv('ASEdges%s_2011.csv.gz'%(i+1), header=None)
nodelist = pd.read_csv('ASNodes%s_2011.csv.gz'%(i+1), header=None)
g = nx.Graph()
g.add_nodes_from(nodelist[0])
g.add_edges_from(tuple(zip(edgelist[0].values, edgelist[1].values)))
degree = nx.degree(g).values()
ecdf_deg = ECDF(degree)
ax.step(ecdf_deg.x, 1-ecdf_deg.y, label=month[i], color=color[i])
ax.set_xscale('log')
ax.set_yscale('log')
ax.set_xlabel("Degree")
ax.set_ylabel("CCDF")
plt.legend()

<matplotlib.legend.Legend at 0x1209d7f0>

We can zoom in at the middle part:

In [11]:

ax.set_xlim(10,1000)
ax.set_ylim(1e-3, 0.1)
fig

The graphs show that there is a general shift towards the lower left part of the curve as the months go by. This is consistent with the observed decrease in size of the networks. At the same time, the degree distributions of the networks still exhibit power law behavior, which is expected: the degree distribution of the network is a manifestation of the real world behavior of how servers connect to each other.

Clustering coefficient

Let us look at the clustering coefficient. This metric will give us an idea how interconnected nodes are as the year progresses.

In [12]:

%config InlineBackend.figure_format = 'jpg'

fig = plt.figure()
ax = fig.add_subplot(111)
for i in range(10):
edgelist = pd.read_csv('ASEdges%s_2011.csv.gz'%(i+1), header=None)
nodelist = pd.read_csv('ASNodes%s_2011.csv.gz'%(i+1), header=None)
g = nx.Graph()
g.add_nodes_from(nodelist[0])
g.add_edges_from(tuple(zip(edgelist[0].values, edgelist[1].values)))
cluster = nx.clustering(g).values()
ecdf_clust = ECDF(cluster)
ax.step(ecdf_clust.x, 1-ecdf_clust.y, label=month[i], color=color[i])
ax.set_xlabel("Clustering Coefficient")
ax.set_ylabel("CCDF")
ax.set_ylim(0.1,1)
plt.legend()

<matplotlib.legend.Legend at 0xa7c1748>

Once again, there is a decreasing trend. Since this is a CCDF, we can say that there are a lot more nodes with low clustering coefficients although nodes are decreasing. This might imply that the nodes being removed are those which have a low tendency to form clusters. It makes sense, as you can think of important nodes have a higher tendency to be a part of clusters, and thus servers that are removed in the network are those that are less important.

I got the earthquake data from Phivolcs website for earthquake that occured between March 1-9, 2016. The data was saved to a csv file to be processed by python. As is, the data did not need that much cleaning. I also used pandas to manage the data.

In [2]:

from **future** import division, print_function
import numpy as np
import networkx as nx
import pandas as pd

earthquake_df = pd.read_csv('seismic.csv')

earthquake_df['Datetime'] = [pd.to_datetime(i) for i in earthquake_df['Date - Time']]
earthquake_df = earthquake_df.sort_values('Datetime', ascending=True, kind='quicksort')
earthquake_df.reset_index(drop=True, inplace=True)
earthquake_df = earthquake_df.drop(['Date - Time', 'Location'], axis=1)

In creating the network, we will need information about the distance between two earthquake epicentres. The euclidean distance is inappropriate for this as the distances are on the surface of the Earth, which is not a flat plane. Instead, we will use the great circle distance or haversine formula, defined as follows:

In [3]:

def haversine(loc1, loc2):
"""
Calculate the great circle distance between two points
on the earth (specified in decimal degrees)
"""
lon1, lat1 = loc1['Longitude'], loc1['Latitude']
lon2, lat2 = loc2['Longitude'], loc2['Latitude'] # convert decimal degrees to radians
lon1, lat1, lon2, lat2 = np.radians([lon1, lat1, lon2, lat2]) # haversine formula
dlon = lon2 - lon1
dlat = lat2 - lat1
a = np.sin(dlat/2)**2 + np.cos(lat1) _ np.cos(lat2) _ np.sin(dlon/2)**2
c = 2 _ np.arcsin(np.sqrt(a))
km = 6367 _ c
return km

Network construction

We will construct the earthquake network using locations of the earthquakes as nodes. The edges between the nodes will depend on the Nearest record breaking criterion which has the following rules:

1. Two earthquakes $i$ and $j$ are connected if the occur consecutively in time.
2. An earthquake $k$ is connected to past earthquakes $i$ if the new earthquake breaks the record $R_{ij}$ of previous earthquake pairs.

An earthquake record of a earthquake $i$ is the current shortest distance between all other earthquake $j$. Thus a new earthquake $k$ is connected to an older earthquake $i$ if $R_{ik}<R_{ij}$.

In [4]:

%matplotlib inline
%config InlineBackend.figure_format = 'svg'
import matplotlib.pyplot as plt
import networkx as nx

loc = earthquake_df[['Longitude', 'Latitude']]

G = nx.DiGraph()

nodes = earthquake_df.index.values
G.add_nodes_from(nodes)
magnitude = [earthquake_df['Mag'][i] for i in nodes]
nx.set_node_attributes(G, 'datetime', earthquake_df['Datetime'].tolist())
nx.set_node_attributes(G, 'record', 1e10)

for i in range(1,len(earthquake_df['Datetime'])):
distance = haversine(loc.iloc[i], loc.iloc[i-1])
G.add_edge(i-1,i, {'distance':distance})
G.node[i-1]['record'] = distance
for j in range(i):
node_dist = haversine(loc.iloc[i], loc.iloc[j])
if node_dist < G.node[j]['record']:
G.node[j]['record'] = node_dist
G.add_edge(j,i, {'distance':node_dist})

In [5]:

import mplleaflet
from mpl_toolkits.basemap import Basemap

fig = plt.figure(figsize=(10,10))
ax = fig.add_subplot(111)
m = Basemap(projection='merc',llcrnrlat=3,urcrnrlat=20, resolution='i',
llcrnrlon=117,urcrnrlon=128, lat_ts=0, ellps='WGS84')
m.drawcountries(zorder=0)
m.fillcontinents(color='#555555')
m.drawmapboundary(fill_color='#111111')

# positions = {i:tuple(loc.iloc[i].values) for i in nodes}

positions = {i:m(\*loc.iloc[i].values) for i in nodes}

s = nx.draw_networkx_nodes(G, ax=ax, pos=positions, node_color=np.array(magnitude),
alpha=0.7, cmap='YlOrRd', with_labels=False)
s.set_zorder(3)
edges = nx.draw_networkx_edges(G, ax=ax, pos=positions, edge_color="#aaaaaa", arrows=True)
edges.set_zorder(2)

plt.show()

Connections in this network are directed. The directionality of links actually implies some possible causal relation, hence, the directionality also tells us the relative sequence of events. Future events do not point to past events, but past events can point to future events.

Degree distributions

The degree distribution is one of the usual metrics by which networks are characterized. The earthquake network created in this entry is a directed graph, wherein the directionality of links is important. Thus, we characterize the in and out degree distributions of the network.

In [6]:

from statsmodels.distributions.empirical_distribution import ECDF
out_deg = np.sort(G.out_degree().values())
in_deg = np.sort(G.in_degree().values())

def degree_distribution(degree):

    frequency = np.arange(len(degree))/float(len(degree))
    ecdf = ECDF(degree)

    return ecdf.x, 1-ecdf.y

fig = plt.figure()
ax = fig.add_subplot(111)

# ax.scatter(degree, 1-frequency)

# ax.step(ecdf.x, 1-ecdf.y)

ax.set_xscale('log')
ax.set_yscale('log')
ax.set_xlabel("Degree")
ax.set_ylabel("CCDF")
ax.step(*degree_distribution(out_deg), label="Out Degree", color='r')
ax.step(*degree_distribution(in_deg), label="In Degree", color='b')
plt.legend()

<matplotlib.legend.Legend at 0xa258a20>

Unfortunately the dataset only contained 69 earthquakes, which is too small to one order of magnitude.

At first glance, it appears that there are more high out-degree nodes than in- degree ones. This tells us that we can link more previous quakes to many different subsequent quakes. On the other hand, there are less connections that can be made as to which previous earthquakes are connected to a future earthquake.

I believe that a possible insight we can get from this small dataset is that a single earthquake can cascade into more earthquakes. This is a reasonable claim since we have an idea of the concept of aftershocks produced by earthquakes. However, the size of the dataset is insufficient to back up any claims we draw from the data.

Magnitude distribution

Finally, we’ll look into the magnitude distribution of the Earthquakes in the recorded period. This analysis is a purely statistical one, independent of the network structure of the earthquakes. Let’s start by plotting the CCDF of the earthquakes.

In [7]:

magnitude = earthquake_df['Mag']

frequency = np.arange(len(magnitude))/float(len(magnitude))
ecdf = ECDF(magnitude)

fig = plt.figure()
ax = fig.add_subplot(111)
ax.set_xscale('log')
ax.set_yscale('log')
ax.set_xlabel("Magnitude")
ax.set_ylabel("CCDF")
ax.step(ecdf.x, 1-ecdf.y, color='r')

plt.legend()

<matplotlib.legend.Legend at 0x8a4feb8>

At first glance, we cannot characterize the distribution that results as a power law. the low slope at low magnitudes means that there are only a few low magnitude earthquakes recorded. This can most likely be attributed to the data gathering of Phivolcs, wherein not all earthquakes are recorded. It would appear that their cutoff is less than magnitude 1.9.

The sharp slope around 3 indicates that most earthquakes that occur are within this magnitude. Unfortunately, as is with the degree distributions, the size of the dataset is not significant enough to draw meaningful analysis from the distribution of data.

Clustering coefficient distribution

In [12]:

cluster = nx.clustering(G.to_undirected()).values()

ecdf_clust = ECDF(cluster)

fig = plt.figure()
ax = fig.add_subplot(111)

# ax.set_xscale('log')

# ax.set_yscale('log')

ax.set_xlabel("Clustering coefficient")
ax.set_ylabel("CDF")
ax.step(ecdf_clust.x, ecdf_clust.y, color='r')

plt.legend(loc='best')

E:\Applications\Anaconda\lib\site-packages\matplotlib\axes\_axes.py:519: UserWarning: No labelled objects found. Use label='...' kwarg on individual plots.
  warnings.warn("No labelled objects found. "

The clustering coefficient distribution has a sharp increase at the center, flattening out at the extreme values. The form of this CDF appears to be gaussian, shifted to the lower clustering coefficients. The clustering coefficients obtained are indeed larger than a randomly connected network, implying the fact that there is an underlying structure in the network. Naturally, we’d expect that this is a consequence of the network construct

Table of Contents

1. Creating an Adjacency Matrix
2. Average Degree
3. Average Clustering Coefficient
4. Average Shortest Path Length
5. References

Creating an Adjacency Matrix

Before the days of networkx, networks were defined using adjacency matrices. The values of the adjacency matrix $A$ are given by \begin{equation} A{ij} = 1 \end{equation} when node $i$ is connected to node $j$. In undirected networks, $A{ij}=A_{ji}$, while this is not necessarily the case for directed networks.

In [1]:

%matplotlib inline
%config InlineBackend.figure_format = 'svg'
from **future** import division, print_function
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

def erdos_renyi(N=10000,p=0.3):
adj = np.zeros((N,N), dtype=bool)

    for k in range(N):
        adj[k,k+1:] = np.random.random(N-(k+1))<p

    adj += adj.transpose()
    return adj

def degree(adj):
k = np.sum(adj, axis=1)
return k

Average Degree

The degree of a node is defined to be the number of edges that node is a part of. For an Erdos-Renyi network, we expect the average degree to be \begin{equation}\langle k \rangle = Np, \end{equation} for large networks $N$.

In [2]:

k = degree(erdos_renyi())
fig = plt.figure()
ax = fig.add_subplot(111)
histogram = ax.hist(k, bins=30, histtype='stepfilled')
ax.set_xlabel("Degree")
ax.set_ylabel("Frequency")

<matplotlib.text.Text at 0x9cce7f0>

In [3]:

N = np.arange(0,4,0.1)
degrees = [np.average(degree(erdos_renyi(int(10**n)))) for n in N]

In [4]:

from scipy import stats

slope, intercept, r_value, p_value, std_err = stats.linregress(10**N,degrees)
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(10**N, 10**N\*slope+intercept, linestyle="--", color='r')
ax.scatter(10**N, degrees, zorder=3)
ax.set_xlabel(r"$N$")
ax.set_ylabel(r"$\langle k \rangle$")

print(r'$\langle k \rangle = %s $' % slope, ", p=", p_value)

$\langle k \rangle = 0.300024590071 $ , p= 1.69578686191e-114

Average Clustering Coefficient

The clustering coefficient $C_i$ of a node $i$ is defined by the equation \begin{equation} C_i = \frac{2t_i}{k_i(k_i-1)}, \end{equation} where $t_i$ is the number of triangles that contain the node, and $k_i$ is the degree of the node.

For an Erdos-Renyi network, the average clustering coefficient for large $N$ reduces to \begin{equation}\langle C \rangle = p. \end{equation}

Counting the number of triangles is not a trivial task, therefore, we will cheat and use networkx to calculate the clustering coefficient of our networks.

In [5]:

def clustering(adj):
import networkx as nx
G = nx.Graph(adj)
clust = nx.clustering(G).values()
return clust

N = np.arange(1,3,0.1)
clust_co = [np.average(clustering(erdos_renyi(int(10**n)))) for n in N]

In [6]:

from scipy import stats

fig = plt.figure()
ax = fig.add_subplot(111)
slope, intercept, r_value, p_value, std_err = stats.linregress(10**N[-6:], clust_co[-6:])
ax.plot(10**N, 10**N\*slope+intercept, linestyle="--", color='r')
ax.scatter(10**N, clust_co, zorder=3)
ax.set_xlabel(r"$N$")
ax.set_ylabel(r"$\langle C \rangle$")

print(r'$p = %s $' % intercept, ", p=", p_value)

$p = 0.299735616273 $ , p= 0.743566602627

Note: obtaining the clustering coefficient can be really slow, with the naive implementation having computational complexity $\mathcal{O}(n^3)$ . This is a resource intensive process and for the sake of brevity, I calculated the clustering coefficient for a smaller network. I obtained a result of $\langle C \rangle = 0.3003$ which is close to $p=0.3$.

Average Shortest Path Length

The shortest path length between two nodes $i$ and $j$ is the path with the least number of edges connecting the two nodes. For an Erdos-Renyi network, the average shortest path length $\langle l \rangle$ is given by

\begin{equation}\langle l \rangle = \frac{\ln{N}}{\ln{\langle k \rangle}}. \end{equation}

In [7]:

N = np.array([int(10**n) for n in np.arange(1,3,0.1)])
degrees = np.array([np.average(degree(erdos_renyi(n))) for n in N])

fig = plt.figure()
ax = fig.add_subplot(111)
slope, intercept, r_value, p_value, std_err = stats.linregress(np.log(degrees[-10:]), np.log(N[-10:]))
ax.plot(degrees, degrees\*np.exp(slope), linestyle="--", color='r')
ax.scatter(degrees, N, zorder=3)
ax.set_ylabel(r"$N$")
ax.set_xlabel(r"$\langle k \rangle$")
ax.set_xscale('log')
ax.set_yscale('log')

print(r'$<l> = %s $' % slope, ", p=", p_value)

$<l> = 1.00407716536 $ , p= 5.62192309984e-18

It’s clear form the figure that there is an exponential relationship between $\langle k \rangle$ and $N$. The slope of the log-log plot obtained corresponds to $\langle l \rangle =1.004$. To verify if the slope of the logarithmic plots is indeed $\langle l \rangle$, we can use the built-in nx.average_shortest_path_length.

In [8]:

import networkx as nx
import numpy as np

def get_spl(N=1000, p=0.3):
G = nx.Graph(nx.erdos_renyi_graph(N,p))
spl = nx.average_shortest_path_length(G)
k = np.average(G.degree().values())
return spl, k

N_nodes = np.array([int(10**i) for i in np.arange(1.5,3.1,0.1)])
results = np.array([get_spl(n) for n in N_nodes])
spl = results[:, 0]
k = results[:, 1]

In [9]:

fig = plt.figure()
ax = fig.add_subplot(111)
slope, intercept, r_value, p_value, std_err = stats.linregress(np.log(N_nodes)/np.log(k), spl)
ax.plot(np.log(N_nodes)/np.log(k), np.log(N_nodes)/np.log(k)\*slope+intercept, linestyle="--", color='r')
ax.scatter(np.log(N_nodes)/np.log(k), spl, zorder=3)
ax.set_xlabel(r"$N$")
ax.set_ylabel(r"$\langle l \rangle$")

# ax.set_xscale('log')

<matplotlib.text.Text at 0xd05ca90>

The obtained result isn’t quite what we are expecting. However, it should be noted that $N=1000$ used is quite small, and the analytical trend was derived for a large random network. It is quite possible that we are unable to replicate the results since the networks sizes used are too small, due to limitations in the memory inefficient implementation of networkx.

References

1. http://dx.doi.org/10.1103/PhysRevE.75.027105
2. http://people.seas.harvard.edu/~babis/tsourICDM08.pdf

This work was made together with Alfred Abella

The Schelling model is one of the early agent-based automata models used to study the formation of black and white communities. As racist as it may sound, it does make sense that a population would naturally segregate into a community with common traits. A more general treatment would probably consider multiple layers such as proximity to a workplace, nationality.

In this model, cells on an $n\times n$ lattice would represent space on the map. A cell may be EMPTY or occupied. When occupied, it can take on two values, BLACK or WHITE, representing the race of agents. We initialize the lattice grids with percent_empty EMPTY cells and percent_white WHITE cells.

In [1]:

from **future** import division
import numpy as np
import matplotlib.pyplot as plt
from numba import jit
from matplotlib import animation

EMPTY = 3
BLACK = 1
WHITE = 2

n = 128
percent_empty = 33.3
percent_white = 33.3
percent_thres = 50
n2 = n**2
grids = np.zeros((n,n))
emp = int((percent_empty / 100) \* n**2)
two = int((percent_white / 100) \* n\*\*2)
tot = emp + two
rand_loc=np.random.permutation(n2)
init_emp=np.unravel_index(rand_loc[:emp+1], (n,n))
init_two=np.unravel_index(rand_loc[emp + 1:tot+1], (n,n))
init_one=np.unravel_index(rand_loc[tot+1:], (n,n))
grids[init_emp]=3
grids[init_two]=2
grids[init_one]=1

grid2 = np.zeros((n+2, n+2))
grid2[1:-1,1:-1] = grids

All agents moves along the grid until it comes within contact of other agents. The agent looks at its neighborhood (in this case, a Moore neighborhood), and if the number of similar neighbors exceeds a threshold, the agent will settle down and stop moving. The rules stated are contained and executed within the function place_indiv().

In [2]:

def place_indiv(grid2):
num_unsat = [0,0]
for i,j in [(i,j) for i in np.arange(1,n+1) for j in np.arange(1,n+1)]:
value = grid2[i,j]
if value != EMPTY:
block = grid2[i - 1:i + 2,j - 1:j + 2]
cnzero = np.sum(block != 0)
cnvalue = np.sum(block == value)
thres = round((percent_thres / 100) \* cnzero)
if cnvalue < thres:
if value == BLACK:
num_unsat[0] += 1
elif value == WHITE:
num_unsat[1] +=1
grid2[i,j] = EMPTY
return grid2, num_unsat

We will run this code for all agents at every timestep until all agents settle down. At the same time, we are going to save a snapshot of the state of the lattice after every 3 timesteps. This is to save on memory since there will be a lot of timesteps before all agents settle down.

In [3]:

FFMpegWriter = animation.writers['ffmpeg']
writer = FFMpegWriter(fps=30, extra_args=['-vcodec', 'libx264'])

fig = plt.figure()
ax = fig.add_subplot(111)
fig.subplots_adjust(left=0, bottom=0, right=1, top=1, wspace=None, hspace=None)
ax.axis('off')
cmap = plt.get_cmap('bone', 3)
im = ax.imshow(grids, cmap=cmap, interpolation='nearest')
count=0
with writer.saving(fig, "writer_test.mp4", 64):
while True:
grid2, num_unsat = place_indiv(grid2)
empty_y, empty_x = np.where(grid2 == EMPTY)
perm = np.random.permutation(len(empty_y)) # perm determines which empty cells are filled
grid2[empty_y[perm[:num_unsat[0] ] ], empty_x[perm[:num_unsat[0] ] ] ] = BLACK
grid2[empty_y[perm[num_unsat[0]:sum(num_unsat)] ], empty_x[perm[num_unsat[0]:sum(num_unsat)] ] ] = WHITE
count += 1
if np.sum(grids!=grid2[1:-1,1:- 1])==0 or count>=1000:

# print count

            break
        else:
            grids = (np.copy(grid2[1:-1,1:- 1])) # remove pad and store to im
            if count%3==0:
                im.set_data(grids)
                writer.grab_frame()

The video below shows the evolution of the Schelling model through time. We can clearly see that from a randomly arranged lattice of BLACK and WHITE agents, communities are formed. It would probably be interesting to characterize the size distributions of these communities.

Fractals are geometric features in which structures are similar when viewed at small localized regions, or the entirety of the structure itself. Fractals are said to be naturally occurring in nature, which is the motivation of applying concepts of fractals in modelling physical systems.

One way to look at fractals is by their fractal dimension, given by \(D = \lim\_{\epsilon \to 0} \dfrac{\log{N}}{\log{1/\epsilon}},\) where $N$ is the number of boxes that contains the structure, and $\epsilon$ is the size of the box. As such, this method is called the box counting method. This is useful for looking at the fractal dimensionality of two dimensional images.

Suppose we want to look at the fractal dimension of this image:

In [1]:

%matplotlib inline
%config InlineBackend.figure_format = 'svg'
import numpy as np
import matplotlib.pyplot as plt
from skimage import color

# img = color.rgb2gray(plt.imread('lightning-fractal.jpg'))

img = plt.imread('lightning-fractal.jpg')[:,:,0]/255.
plt.imshow(img>0.4, cmap="gray")

<matplotlib.image.AxesImage at 0x6d2ecf8>

The following lines of code will count the number of blocks containing the structure $N$ for a given input of $\epsilon$.

In [2]:

def count_N(img, epsilon):
N = 0
for i in range(0, img.shape[0], epsilon):
for j in range(0, img.shape[1], epsilon):
if i==img.shape[0]:
N += (np.sum(img[i:-1, j:j+epsilon])>0)
elif j==img.shape[1]:
N += (np.sum(img[i:i+epsilon, j:-1])>0)
else:
N += (np.sum(img[i:i+epsilon, j:j+epsilon])>0)

# print np.sum(img[i:i+epsilon, j:j+epsilon])>0

    return N

We will then plot $\log{N}$ with $\log{1/\epsilon}$ and if the slope of the resulting plot, representing $D$ appoaches some finite value.

In [3]:

epsilon_values = np.arange(200,0,-1)
N = [count_N(img, epsilon) for epsilon in epsilon_values]

fig = plt.figure()
ax = fig.add_subplot(111)
x = np.log(1./epsilon_values)
y = np.log(N)
ax.scatter(x, y)
ax.set_ylabel(r"$\log{N}$")
ax.set_xlabel(r"$\log{1/\epsilon}$")

<matplotlib.text.Text at 0x6f50ba8>

Getting the fractal dimension

From the definition of the fractal dimension, we can see that it is simply the slope of $\log{N}$ vs $\log{1/\epsilon}$ for large values of $1/\epsilon$. To obtain this, it is best to fit a line at the end of the plot. In this case, we will fit a line to the last 50 points, the slope of which is our fractal dimension.

In [4]:

from scipy import stats
import numpy as np

slope, intercept, r_value, p_value, std_err = stats.linregress(x[-50:],y[-50:])

print 'D = %s' % slope

D = 1.93467964091

Hence, the fractal dimension of the image is $D=1.935$.

For this implementation, I defined functions that turn binary strings to decimals and vice versa. This will facilitate the general creation of lookup tables associated with the rules.

In [1]:

%matplotlib inline
%config InlineBackend.figure_format = 'svg'
import numpy as np
import matplotlib.pyplot as plt
from **future** import division, print_function
from numba import jit
import seaborn as sns

@jit
def binary_str(num):
res = []
for i in range(7,-1,-1):
if num//(2**i)==1:
res.append(1)
num -= 2**i
else:
res.append(0)
return res

@jit
def binary_dec(arr):
dec = 0
for i in range(len(arr)):
dec += arr[-1-i]\*2\*\*i
return dec

I then initialized an empty array, and initialized the first row a 50-50 percent chance of being 1 or 0 for each cell in the first row.

In [2]:

SIZE = 50

ECA = np.zeros((SIZE,SIZE))
ECA[0,:] = 1
ECA[0, SIZE//2] = 0

I then allowed the system to follow the rules (Rule 30 in particular) and wait for the resulting image…

In [3]:

for i in range(1,SIZE):
for j in range(SIZE):
pattern = [ECA[i-1,k%SIZE] for k in [j-1,j,j+1]]
index = int(binary_dec(pattern))
ECA[i, j] = binary_str(26)[7-index]

with sns.axes_style('white'):
plt.figure(figsize=(5,5))
plt.imshow(ECA, cmap='gray', interpolation='nearest')

Characterizing ECA Rules

Space: Entropy

Measure the Shannon Entropy \(H = \dfrac{1}{H*{max}} \sum*{i=0}^7 p*i\log{p_i}\) where $p_i$ is the frequency of all configurations resulting in the microstate $i$, where $H*{max}$ is the value of $H$ whenn all $p_i=\dfrac{1}{N}$ are equal.

$ \langle H \rangle$ is obtained after disregarding the transient period (roughly half of the cell length in space).

In [4]:

SIZE=128

def p_i(space_arr,N):
p_vals = np.zeros(N, dtype=np.bool)
states = np.array([binary_dec(space_arr[i-1:i+2]) for i in range(1,len(space_arr)-1)])
for i in range(N):
p_vals[i] = np.sum(states==i)
return p_vals/np.sum(p_vals)

N = 8
H_max = np.sum([1./N*np.log(1./N) for state_num in range(N)])
H = lambda timestep: np.sum([p*np.log(p+1e-10) for p in p_i(timestep,N)])/H_max

H_ave = lambda ECA: np.mean([H(timestep) for timestep in ECA[SIZE//2:,:]])

Time: Kinetic Energy

\(\langle K \rangle*t = \langle |s*{i+1}-s_i|^2\rangle_t\) This essentially counts the number of times states are flipped averaged across space and time.

In [5]:

def K(evolution):
SIZE = len(evolution)
return np.mean([np.mean((evolution[time]-evolution[time-1])\*\*2) for time in range(SIZE//2,SIZE)])

An interesting observation is when you generate all ECA rules, measure their respective $K$ and $H$ values, we will observe clustering for similarly classed rules.

In [6]:

def create_ECA(rule, SIZE):
ECA = np.zeros((SIZE,SIZE))
ECA[0,SIZE//2] = 1

# ECA[0,:] = 1\*(np.random.random(SIZE)>0.5)

    for i in range(1,SIZE):
        for j in range(SIZE):
            pattern = [ECA[i-1,k%SIZE] for k in [j-1,j,j+1]]
            index = int(binary_dec(pattern))
            ECA[i, j] = binary_str(rule)[7-index]
    return ECA

print(SIZE)
SIZE = 128
%time all_ECA = [create_ECA(rule, SIZE) for rule in range(SIZE)]

128
Wall time: 46.3 s

In [7]:

CA1 = [0, 4, 16, 32, 36, 48, 54, 60, 62]
CA2 = [8, 24, 40, 56, 58]
CA3 = [2, 10, 12, 14, 18, 22, 26, 28, 30, 34, 38, 42, 44, 46, 50]
CA4 = [52, 110]
colors = []
for i in range(256):
if i in CA1:
colors.append('m')
elif i in CA2:
colors.append(u'g')
elif i in CA3:
colors.append(u'y')
elif i in CA4:
colors.append(u'b')
else:
colors.append(u'None')

In [8]:

%time K_list = [K(ECA) for ECA in all_ECA]
%time H_list = [H_ave(ECA) for ECA in all_ECA]

Wall time: 130 ms
Wall time: 1.46 s

In [9]:

plt.scatter(K_list,H_list, alpha=0.5, c=colors, s=20, linewidths=0)
plt.ylabel('$H$')
plt.xlabel('$K$')

<matplotlib.text.Text at 0xd2d9fd0>

Clustering of ECA

Some form of clustering can be observed using the Entropy and Kinetic Energy descriptors. However, this clustering cannot be traced back to the types of ECA rules.

Consider the non-linear equation \begin{equation} N_{t+1} = N_t\exp{[r(1-N_t/K)]}, \end{equation} where $r$ and $K$ are the growth rate and carrying capacity, respectively. This is a simple model for the population growth of a single species.

We can plot this equation for different values of $r$.

In [1]:

%matplotlib inline
%config InlineBackend.figure_format = 'svg'
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

def subplotaxis_off(ax): # Turn off axis lines and ticks of the big subplot
ax.spines['top'].set_color('none')
ax.spines['bottom'].set_color('none')
ax.spines['left'].set_color('none')
ax.spines['right'].set_color('none')
ax.tick_params(labelcolor='w', top='off', bottom='off', left='off', right='off')

N_0 = 1
K = 1000
eq1 = lambda r, N_0, K: N_0*np.exp(r*(1-N_0/K))

def population_growth(r, timesteps, N_0=1, K=1000):
N = [N_0]
for time in timesteps[1:]:
N.append(eq1(r, N[-1], K))
return np.array(N)

timesteps = np.arange(0, 30)
r_list = [1.8, 2.3, 2.6]
population = [population_growth(r, timesteps) for r in r_list]

fig, axes = plt.subplots(3, figsize=(3,3), sharex=True, dpi=200)
with sns.axes_style("whitegrid"):
ax = fig.add_subplot(111, zorder=-1)
ax.grid(False)
for i, r in enumerate(r_list):
axes[i].plot(timesteps, population[i]/K)
axes[i].locator_params(axis='y', nbins=2)
axes[i].set_ylim((0,3))
axes[i].text(0.5, 0.99,r'$r=%.2f$' % r, horizontalalignment='center',
verticalalignment='top', transform = axes[i].transAxes)
ax.set_ylabel(r"$N/K$")
ax.set_xlabel(r"time, $t$")
subplotaxis_off(ax)

In the above examples, we observe stable equilibrium ($r=1.8$), and stable cycles with 2 points ($r=2.3$) or 4 points ($r=2.6$). Although not explicitly shown, this stable behavior can be seen for any initial value of $N_0$ and $K$.

On the other hand, setting $r=3.3$, we will observe that the behavior of the population depends on the initial value $N_0/K$. The code to produce the plots is as follows:

In [2]:

timesteps = np.arange(0, 30)
r_list = [3.3, 3.3, 3.3]
NK_ratio = [0.075, 1.5, 0.02]
population = [population_growth(r_list[i], timesteps, N_0=K\*NK) for i, NK in enumerate(NK_ratio)]

fig, axes = plt.subplots(3, figsize=(3,3), sharex=True, dpi=200)
with sns.axes_style("whitegrid"):
ax = fig.add_subplot(111, zorder=-1)
ax.grid(False)
for i, r in enumerate(r_list):
axes[i].plot(timesteps, population[i]/K)
axes[i].locator_params(axis='y', nbins=3)
axes[i].set_ylim(0,3.8)
axes[i].text(0.5, 0.99,r'$r=%.2f$' % r, horizontalalignment='center',
verticalalignment='top', transform = axes[i].transAxes)
ax.set_ylabel(r"$N/K$")
ax.set_xlabel(r"time, $t$")
subplotaxis_off(ax)

The behavior seen above heavily depends on the initial seed value of $N_0$, and we can see that although the 3 graphs have the same $r$ value, no two graphs exhibit the same bahavior. This strong dependence on the inital values indicates that the value of the growth rate $r$ is in the chaotic regime.

It is interesting to see that the equation we are modelling appears as a very simple and deterministic population growth model, and yes it exhibits deterministic, as well a arbitrarily dynamic behavior for large enough values of the growth rate ($r>2.692$).

Two competing species

We can also model the competition of two species using two coupled deterministic equations similar to the single species case,

In [3]:

N1_0 = 10
N2_0 = 10
K_1 = 1000
K_2 = 1000
alpha = np.array([[1,0.5], [0.5,1]])
species1 = lambda r_1, N_1, N_2, K_1=K_1, K_2=K_2: N_1*np.exp(r_1*(K_2 - alpha[0,0]*N_1 - alpha[0,1]*N_2)/K_2)
species2 = lambda r_2, N_1, N_2, K_1=K_1, K_2=K_2: N_2*np.exp(r_2*(K_2 - alpha[1,0]*N_1 - alpha[1,1]*N_2)/K_2)

def competition_growth(r_1, r_2, timesteps, N1_0=N1_0, N2_0=N2_0, K_1=K_1, K_2=K_2):
N = np.zeros([len(timesteps), 2])
N[0,:] = [N1_0, N2_0]
for i, time in enumerate(timesteps):
if i>0:
N[i,:] = [species1(r_1, N[i-1,0], N[i-1,1]), species2(r_2, N[i-1,0], N[i-1,1])]
return N

timesteps = np.arange(0, 30)
r_list = [1.1, 1.5, 2.5, 4]
population = [competition_growth(r, 2\*r, timesteps)[:,0] for r in r_list]

fig, axes = plt.subplots(len(r_list), figsize=(3,4), sharex=True, dpi=200)
with sns.axes_style("whitegrid"):
ax = fig.add_subplot(111, zorder=-1)
ax.grid(False)
for i, r in enumerate(r_list):
axes[i].plot(timesteps, population[i]/K)
axes[i].locator_params(axis='y', nbins=2)
axes[i].set_ylim(0,np.max(population[i]/K)+0.5)
axes[i].text(0.5, 0.99,r'$r=%.2f$' % r, horizontalalignment='center',
verticalalignment='top', transform = axes[i].transAxes)
ax.set_ylabel(r"$N_1/K$")
ax.set_xlabel(r"time, $t$")
subplotaxis_off(ax)

Tracking based on Image Segmentation

We can start things of by tracking a ping-pong ball undergoing projectile motion to calculate the acceleration due to gravity. We simply converted the video to a series of images using ffmpeg as a decoder. Then we took a set of images which corresponds to one projectile motion.

We select the ROI by color, just crop the pingpong ball. And then use Image segmentation using histogram back projection.

Now we have a collection of points which represent the ball. We can improve the quality of our collection of points using morphological operations to clean it as well. We can get the centroid using this following snippet of code:

IsCalculated = CreateFeatureStruct(%f);
IsCalculated.Centroid = %t;
C = AnalyzeBlobs(im, IsCalculated);
Centroid = C(1).Centroid;
y = [y Centroid(2)];
x = [x Centroid(1)];

Doing this for the rest of the images, we will get a collection of x and y positions, which we can plot using excel. Of course, these x and y positions need to be scaled properly, using

yscale = .05/20 // meters to pixels
xscale = .05/14 // meters to pixels
Y = -yscale*(y-240);
X = xscale*(x);

If we fit a second order polynomial to the y positions and time, we should get

The term $-4.8871$ is actually $-\frac{g}{2}$. Hence, we have $g=9.77\,\frac{m}{s^2}$.

Tracking Based on Image Difference

Vehicles will not always have the same color on the road. The method of image segmentation is good, but we need something more robust if we want to be able to make a generic car velocity estimator just by video processing. A possible alternative method is using image difference. We need an image of just the background.

Then we take the difference of every frame with this frame. To binarize the difference , we set a threshold value, which will determine if pixels in the difference will be black or white.

We’ll get another blob, from which we can get its position using centroids. We can get the following graph below:

The term $-4.8871$ is actually $-\frac{g}{2}$. Hence, we have $g=9.78\,\frac{m}{s^2}$. This is a similar result as with using image segmentation by color. We are now ready to track car videos.

Tracking cars from a video

A video frame would look like the image shown above. Since we are working with a video captured by a different person, we do not have a background that we can readily use as a reference. Instead, we utilize the difference of subsequent frames, and hope to find the outlines of the vehicles on the road. However, as shown in the next two figures, this is not as straightforward as the previous methodology of subtracting a reference background.

Dilation and erosion was performed by hand on a graphing paper. The initial shapes and structuring elements are outlined in red, while the results are outlined in blue. In the structuring element, we first chose a pixel to use as the “origin”. We reflect the image about this origin.

For Dilation, we move the element around the shape, such that the origin remains in the shape. The combined outlines of the structuring element as it moves through the shape is the Dilation.

Dilation

For Erosion, we move the element around the shape, such that the element remains in the shape. The shape traced out by the origin is the Erosion of the shape due to the element.

Erosion

In general, we can say that Dilation makes the shape bigger, while Erosion makes the shape smaller.

Computer Simulation

Dilation

Erosion

I performed Dilation and Erosion in Scilab using the IPD toolbox. The results I obtained were consistent with my hand drawn Dilations and Erosions.

Cancer Cell Detection

The image file Circles.jpg was first binarized by setting a threshold value. The binarized image was then cleaned of small white dots using OpenImage, and holes in big circles were filled using CloseImage. The result is shown below (as a whole image, not the subimages). The image was segmented into subimages so that better thresholding can be applied on the different areas of the image, which may not be uniformly illuminated.

From this, we can measure the best estimate for “normal cells”. We used SearchBlobs first, and then filtered the results to remove obviously large and small cells by area. Small cells would be less than 300 in area. Big cells will be above 600. After this filtering we get

mean = 492
stdev = 49.7

From the mean and standard deviation, we used the same process to binarize Circles with cancer.jpg. Using SearchBlobs, we removed blobs which fit the best estimates for a normal cell. This leaves us with our abnormal cells, shown in the image below.