Final Year Project Report

Metaphors In Finance
Michael Noonan

A thesis submitted in part fullment of the degree of BA/BSc (hons) in Computer Science Supervisor: Prof. Mark Keane Moderator: Dr. Liam Murphy

UCD School of Computer Science and Informatics College of Engineering Mathematical and Physical Sciences University College Dublin
May 5, 2011

Table of Contents

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 1.2 2 Project Specication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 3 3 3 4 4 5 6 6 7 8 11 13 13 14 15 18 19 21 24 25 25 26

Project Description and Report Structure . . . . . . . . . . . . . . . . 2.1 2.2 Project Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Report Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Background Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 3.2 3.3 3.4 Power Laws, The Pareto Principle and Graphs . . . . . . . . . . . . . . . .

Gerow and Keane Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stock Market Bubbles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Herding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Network Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 4.2 Random Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scale-Free Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5 6

Design and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . Evaluation and Comparison of Models . . . . . . . . . . . . . . . . . . . 6.1 6.2 6.3 Mean Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Power-Law Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Robustness Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . 7.1 7.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Page 1 of 28

Abstract

The Aim of this project is to give an insight into the language patterns observed in a corpus of over 17,000 nancial articles, to further research why these patterns arise, to examine their signicance and to develop specic software that models the data in a network like fashion. By evaluating and comparing the dierent models it is found that the the real world nancialdata model is Barabasi-like in structure. In other words, Barabasi-like properties such as preferential attachment and scale invariance occur in this and other real world networks. The random model on the other hand is dierent and follows a dierent structure. This project involves background research into power laws, random and scale-free networks and the area of cognomics - the study of how people act in large group situations typically through the use of language corpora. The software developed will include implementation of four dierent network models: A Network model of the actual structure of the data gleaned from the nancial corpus. A Random (Erdos-Renyi) Network. Two versions of a Barabasi Network. Upon completion of the software development and comparative analysis, conclusions are made based on the new knowledge acquired throughout the project.

Page 2 of 28

Chapter 1: Introduction

1.1

Project Specication

General Information: Gerow & Keane (2010) have shown that power-law analyses of the language used by nancial reporters across a corpus of over 17,000 articles (from the Financial Times, New York Times, and BBC) can predict the 2007 stock market bubble and crash; in simple terms, systematic changes in the language used in articles reect the emergence of an homogenous and positive (unrealistic) view of the market at the time even amongst seasoned commentators (see also Gerow, 2010). This work built up a large corpus of articles that can be analyzed further. This project will examine whether the observed network for this empirical data corresponds to theoretical predictions from either random-network models or scale-free networks. Mandatory: Review existing work on corpus-based approaches to metaphor. Review existing work on network models. Develop random network model, a Barabasi, scale-free model and a network model of the observed data. Develop empirical comparisons of these three models (statistical, qualitative). Draw conclusions on what model best corresponds to the known data. Discretionary: Develop novel methods for comparing dierent network models. Exceptional: Develop graphical interface for comparing models.

1.2

Acknowledgments

I would like to thank my project supervisor and mentor Professor Mark Keane for all his help during the past academic year. I would also like to thank Prof. Keanes former MSc student Aaron Gerow.

Page 3 of 28

Chapter 2: Project Description and Report Structure

2.1

Project Description

The aim of my nal year project is to build upon the research done by project supervisor Prof. Mark Keane and his former MSc student Aaron Gerow. In their paper entitled The voice of the herd: Power-law regularities in newspaper articles predict market bubbles, Gerow & Keane (2010) showed that a power-law analysis of the language used by nancial commentaries could predict stock market bubbles. These regularities in the form of language agreement were looked at in two dierent ways. One was Verb-convergence where the same Verbs were used and the second was Noun-convergence where the same Nouns were used. Verbs and Nouns are the building blocks of sentences and thus help identify common phrases used. As explained in the paper: Our basic hypothesis is that the language used by nancial commentators during an economic bubble will manifest emergent structure as it converges weekto-week on the same positive view of the stock market This convergence of Verbs and Nouns can be partly attributed to what is known as Herd mentality. This is where people such as nancial commentators become inuenced by the group or majority viewpoint and become less inclined to contribute their own independent views. As a result, a common style is seen and a common language structure can be observed. Herd mentality is studied in many dierent areas such as Psychology, Sociology and Economics and will be discussed in detail later on in this report. My research gives a further insight into why these language patterns exist and how they can be used to some degree in informing us about peoples thought processes, as viewed through the lens of their language usage. The software development aspect implements the network models described above while the comparison of the models shows how Barabasi properties exist within the real world nancial model. This is why the comparison was carried out; to determine whether the data from the nancial corpus was closely related to a Barabasi network or a random network. This new type of language analysis is growing, a lot of the worlds biggest companies have started researching these language patterns in the hope of gaining a greater understanding of customer trends. Google for example often performs various types of language analysis on its search queries. Some are so powerful that as a result they can predict annual car sales to a fairly accurate degree based on monthly searches (Choi & Varian 2009).

Page 4 of 28

2.2

Report Structure

The rst Chapter is merely an introduction to the project laying out the ocial project specication and acknowledging anyone who has helped with the project along the way. The second chapter gives a more detailed description of the project and what is to be achieved and why. The next section is all about background research, this chapter has numerous subsections devoted to the dierent areas in which research was carried out. The fourth chapter focuses on the dierent network models that were built outlining the approach taken and any problems encountered. The Design and Implementation chapter goes into more detail on the technical side of the software development, illustrating code examples and design issues. The penultimate section focuses on evaluation and comparison of the network models. Graphs will be used to compare the dierent results obtained from running the software. The nal chapter is about conclusions that can be drawn from the projects outcome along with any future work.

Page 5 of 28

Chapter 3: Background Research

Background research was a vital part of this project. Power-laws and the Pareto principle are key properties of scale-free networks. Learning about the work already carried out by others was also very important. Finally, the inner workings of the stock-market were researched along with the inuences that make people invest. From this projects point of view, Herding was of great interest and is discussed along with the others in this section.

3.1

Power Laws, The Pareto Principle and Graphs

Power-laws have been used for decades to analyze the relationship between dierent quantities. At rst they were thought of as rare but nowadays a power-law explosion has revealed that the phenomenon can be observed in many walks of life. They exist in social networks, in nature, in sport, almost everywhere. Zipfs law was one of the earliest reported power-law distributions. The law is named after linguist George Kingsley Zipf after he carried out a power-law analysis of the popular book Moby Dick (Zipf, 1932). What Zipf noticed was that a very small number of words were used frequently whilst a huge number of words were only appearing a fraction of the time. The frequency distribution followed a power-law of the this form (The power-laws derived from my own curves follow this form too): y = Cxa (3.1)

Poer-laws can be explained in more detail by looking at what is known as the Pareto principle (Koch, 1999). Named after Italian economist Vilfredo Pareto, this principle also known as the 80/20 rule is often used as short hand for expressing power laws. For example, the 80/20 rule can be used to express the distribution of wealth around the world. 80 percent of the overall wealth is held by 20 percent of the population. i.e. there are very few people who are super rich while the vast majority of the people are moderately wealthy. This is just like Zipfs Moby Dick analysis whereby a huge percentage of the words used came from a small set of common words such as the, and, at etc. I came up with one particular example that helped me understand power-laws and the Pareto principle. The premier league is the top football league in England, it consists of 20 teams from all over the country. Power-laws and Paretos principle can be seen in many dierent ways when looking at the premier league. Since its inception 20% of the teams have accounted for 100 percent of the trophies, in other words only 4 teams have won the premier league while the other 16 teams, or 80% have not. One reason is due to the wealth of these top teams. When a team wins the league they earn more money and are therefore more likely to win the league again. The rich get richer while the poor get poorer. This demonstrates a key aspect of power-laws i.e. the frequency of an event (e.g. winning the premier league trophy) varies as a power of some related attribute (e.g. the wealth of the league). So as the overall wealth of the league increases, the % distributed to the top 4 teams is far greater than the % distributed to the other 16. This demonstrates a key property of a Barabasi network namely preferential attachment meaning that the teams included in the
Page 6 of 28

top 20% have a bias in their favor towards winning the league again. Just like Google is more likely to link to a newly created webpage, teams in the top 4 of the premier league are more likely to win more trophies. This example shows that power-laws exist all around us and are quite easy to observe. Power-laws are being used today in new ways. With the evolution of the World Wide Web and the growth of social networks such as facebook, power-law distributions are being used to give an insight into the structural attributes of these networks by looking at the distribution between nodes and links. This is where graph theory comes in. Graphs can be created from almost anything provided there are nodes and links (Keane, 2010). Power-law analysis can be carried out on these graphs by looking at the degree distributions of the various nodes. For example, when a node such as a web page is linked to another page its degree increases by 1. As will be shown later on in this chapter, Random and Scale-free networks have dierent structures and dierent degree distributions.

3.2

Gerow and Keane Research

The rst bit of background research to be carried out revolved around the paper The voice of the Herd: Power-law regularities in newspaper articles predict market bubbles (Gerow & Keane, 2010). In this paper it was shown that a power-law analysis of a corpus of 17,000+ nancial articles could predict the 2007 stock market bubble. The corpus contained articles from some of the most inuential nancial news sources in the world such as The Financial Times and the New York Times collected over a period of four years. The data was collected via computerized web-searches and the Verbs and Nouns extracted via a parser. Each article was shallow-parsed to extract verb- and noun-phrases, using a parser that handles recursive phrase structure The power-law analysis showed that there was an uneven distribution of verbs and nouns throughout the articles. This bias became more pronounced as time grew closer to the 2007 bubble period. This uneven distribution reected an agreement in language used by the nancial commentators. As was mentioned before, this agreement was looked at primarily in terms of verb convergence (similar verbs being used) and noun convergence (similar nouns being used). As the 2007 stock market bubble grew closer, the exponents of the power laws which represent word frequency distributions trended upwards indicating a higher agreement in language. As the bubble started to burst the opposite was witnessed and a downward trend was seen. Graphs which visibly show such trends are displayed in section 3.3 below. Equation 3.2 shows a simple 8-week windowed, geometric mean of a where a refers to that weeks particular power-law value. The model for a given week looks like so: 8 weeki = ai3 ...ai ...ai+4 + 2 8 ai4 ...ai ...ai+3 (3.2)

It should be noted that 19 common verbs (or skywords) were excluded due to the fact they would skew the accuracy of any results obtained. Examples of such words were are be and
Page 7 of 28

was. Sketch engine was used to rank common lemmas with their most common arguments. The result of this process was 36 .csv les each named after the verb it represents. Within each le there are two columns, one for arguments (Arg) and another for frequency (Freq.). See Figure 3.1 below which gives an example of what one of the .csv les looks like. This paper forms the backbone of my research and I am using the same .csv les to build the network models.

Figure 3.1: Example .csv le

3.3

Stock Market Bubbles

Asset price bubble refers to the discrepancy between the real value and actual listing of a share, while bubble in the literary economists interpretation is a broader economic phenomenon where the continuous rise of share prices is fueled by the investors expectations of further increase (Komromi, 2006)

Be it company shares or commodities like gold, everything traded on the stock market has a value. This value is continuously monitored throughout the day and changes all the time. As can be seen from the quotation above, bubbles can be looked at in two ways. A mathematical way, in which a stocks current value is monitored relative to its real value and a speculative way, where bubbles are formed by the investors future expectations. A bubble can form when the value of a stock goes far and above the regular trading price and steadily increases as more and more investors plough money into the stock. The law of supply and demand forms the foundations of the stock markets inner workings. When stocks are in demand investors are willing to pay a higher price, often this is because they see demand increasing in the future and thus further price increases. The ultimate aim of a seasoned investor is to buy prime stocks when the price is low and sell these stocks back when the peak price has been reached. Knowing how much and when stock prices are going to change can be a combination of both skill and luck. A stock market bubble will eventually burst, as a result, investors begin to sell their shares o (they are led to believe that prices will continue to fall dramatically) and again the law of supply and demand tells us that if

Page 8 of 28

all these investors rush to sell their shares as quickly as possible supply will outstrip demand and they will have to accept a lower price. There have been major economic bubbles in the past many of which have been responsible for signicant damage to both the local and global economy. In Ireland, the eects of the property bubble collapse can be seen all over the country. Another well known bubble that was further reaching was the dot com bubble. This occurred during the mid 90s and lasted for around ve years. Share prices in many self-proclaimed e-commerce companies rose dramatically during these years. As this was a speculative bubble, many investors were taken aback by how quickly stocks in these companies were rising and bought large amounts of shares anticipating further increases. While many made millions from these stocks by selling before the bubble burst, a huge number of people lost millions of dollars. What some economists consider to be a bubble others do not. A key reason for this is due to the dierent models that are used to measure and evaluate stock performance and company prots. When looking back throughout history many stock valuations have been skewed due to changes in how they are measured. For example, when the popular computer manufacturer DELL began utilizing Just-In-Time production where inventory and associated costs were signicantly cut, the valuation of DELL stock had to be measured in a new light. The fact that dierent models give dierent valuations makes it dicult to agree on bubble classication and prediction. There are many dierent views regarding nancial bubbles and when/how they can be detected. Former federal reserve chairman Alan Greenspan once famously spoke of how a bubble could only be detected after it had burst and the damage had been done. Others believe they can be predicted and to a fairly accurate degree. One such individual is Didier Sornette, the director of the nancial crisis observatory at the Swiss federal institute of technology in Zurich. Prof. Sornette has long shown an interest in trying to predict how and when bubbles will form and how and when they would burst. Sornette and Woodard, (2009) explain how bubbles form and burst: A crash occurs because the market has entered an unstable phase and any small disturbance or process may reveal the existence of instability He mentioned how long term events can cause a bubble to burst and a crash to occur. Rather than looking at events immediately preceding the crash he believes we should look further back at smaller changes that gradually grow to where they contribute towards a bubble collapse. Language patterns are an example of a subtle change that can develop over many years. Traditionally, many economists wouldnt place much importance on what they deemed irrelevant factors towards a bubble bursting. What this project shows is that a lot can be learned from patterns derived from our language usage. Specically how the language used by nancial commentators can inuence stock market behaviour. How the stock market and in particular the bubble phenomenon relate to my project is interesting. The investors as described above are constantly looking at various dierent stocks on the market. They buy and sell and repeat this process over and over again. Decisions on what stocks will make a good return often comes from the news and in particular nancial articles in popular newspapers. Nowadays with smartphone apps many investors can be updated with the latest stock news at the touch of a button. Due to the reliance many investors have on these articles their content plays an important role in shaping how investors think and act. The language used can motivate certain actions while discouraging others. When there
Page 9 of 28

is a high rate of language agreement the inuence these articles can have on prospective investors cannot be understated. What causes this agreement in language among even the most seasoned experts is what this project focuses on. Herding is one way in which these patterns can emerge and will be described in detail in the next section. While analyzing the corpus of articles, language patterns emerge over time. These patterns can then be compared to the actual performance of the stock market by looking at the key indices such as the Dow Jones (United States), and the Nikkei 225 (Japan). Figure 3.2. below shows the relationship between the Dow Jones index and the power-law distribution of verb frequencies (Keane & Gerow, 2010). As can be seen there is a direct relationship between the two. As the Dow Jones reaches its peak so too does the verb convergence in the language. Likewise, as the Index plummets down the power-law distribution also trends downwards. There is also an interesting relationship between more positive language being used around

Figure 3.2: Relationship between Dow Jones Index & Power-law verb distribution

the time of the 2007 stock market bubble and subsequently less positive language being used during the bursting of the bubble and the crash. Figure 3.3, shows the relationship between the Dow Jones index and the type of language (positive or negative) over the four year period (Keane & Gerow, 2010). It goes along with the trend seen in Figure 3.2. Before the Dow Jones index reached its peak level there was a clear increase in the amount of positive language being used. This positive language was represented phrases such as: Stocks rose again today and so on. As the index began to fall and reach a low point negative language was used far more frequently. The type of language being used before the bubble bursts was positive, promoting a healthy environment for which to invest. This language can reect an unrealistic and exaggerated view of the stocks health. The amount of negative language increased as the bubble began to burst, nancial commentators would have been using phrases like Share prices plummet drastically. This promoted an environment which was unfriendly to investors resulting in many share valuations declining.

Page 10 of 28

Figure 3.3: Relationship between Dow Jones index & language valiance

3.4

Herding

One of the key parts of my research was to delve further into the eects of social herding. As mentioned at the beginning of this report herding is a form of group behavior whereby individual opinions are sacriced for the wider accepted opinion or voice of the majority. For example, if a certain sector in the stock market was performing well and there was a positive view that there would be further price increases many commentators would continue to go along with this positive view, using similar verb and noun phrases, rather than expressing an opposing opinion. It was important to have a more detailed look into this phenomenon and evaluate its consequences. Sornette(2004) explains that herding can play a vital role in bubbles forming and bursting: To understand stock markets, one needs to consider the impact of positive feedbacks via possible technical as well as behavioral mechanisms such as imitation and herding When analyzing the eects of herding, it is helpful to take an approach which encompasses a psychological or sociological viewpoint as well as an economic one. Humans are ultimately social beings and are inuenced by their environment and those that live in it. Our decisions are often based on how others act and react, more so than we might believe. Herding can be observed in the shareholders circle and also the experts circle. In terms of investment, many prospective shareholders may not have a detailed knowledge of the stock market, they could be regular people trying to make a bit of extra money. As a result of this they look towards those in the know to try and get some advice. Sornette & Woodard, (2009) outlines: a larger and larger group of people seek to become rich without a real understanding of the process involved

Page 11 of 28

In terms of inuencing stock market investment there is essentially herding in two dierent groups. Herding in terms of language agreement among nancial commentaries and herding among actual investors. Many of these people go along with the group and will not try and form their own opinion. Why is this so? Often in social situations a person will conform with the group view as it seems logical at the time. Group pressure is also a factor with many individuals less inclined to express innovative ideas when a majority view exists. What is interesting in terms of this project is why experts like nancial commentators are subject to herding, surely these people with their vast knowledge of the stock market can form multiple diering views of an investment situation?. There is a psychological element to why this herd behavior may exist. Rook, (2006) presents a psychological viewpoint. economists unanimously explain herd behavior out of the sheer number of others that partake in an action. In contrast, Psychologists maintain that it often is not the actual number of a majority that causes such conformity under pressure, but a need for consensus Basically for many of these nancial commentators, the stock market demands a consensus of some sort, saying a certain stock will go down by 10 points or go up by 10 points are completely opposing views and rarely will both opinions be right on the day. When looking at bubble collapses the traditional view was to ignore some of the underlying behavior mechanisms and look towards more concrete facts such as exchange rates or company collapses. Ultimately the stock market is run by people, people who are inuenced by many factors including the opinions of others. As a result herding plays a crucial role in how nancial commentators act and thus how prospective investors react. It is also interesting to note that the majority view as regards a stocks value is often out in the open and not hidden from view. Many stock brokers and related websites give consensus ratings to let prospective investors know other investor trends. It is quite common for stock broker sites to have a what the experts are saying section immediately giving such content and inuence over the reader, be it another commentator or a potential investor.

Page 12 of 28

Chapter 4: The Network Models

The software development aspect of this project involved implementing 4 dierent network models. The rst was a representation of the nancial data from the corpus of articles. The second was a random network, while the third and fourth were variations of Barabasistyle networks. The third model demonstrated preferential attachment at a point after the standard model had been built which would single-out preferential attachment and show how certain nodes gain more links than others. The other version involved growth (the gaining of links) from scratch along with preferential attachment. In the end the random network should have a normal link distribution (each node should receive a roughly a mean share of links) the Barabasi model should exhibit scale-free properties such as a power-law distribution and scale invariance. Before implementation of these models could be done, research had to be carried out to learn more about their structure. By implementing these models a comparison could be carried out which would reveal the relationship between the real world nancial model and Barabasi & random networks. It was found out that the real world network is far more Barabasi-like in structure.

4.1

Random Networks

My early research into random networks led me to the ErdosRenyi model, a popular model for generating random graphs. The model was named after Paul Erdos and Alfrd Rnyi and is based on the assumption that nodes are selected to be linked based on some random process. The G(n, P) model states that a graph is constructed by linking nodes in a random fashion based on probability (Goldburt & Zhang, 2006). Each edge is included in the graph based on a probability element represented by the letter P. The parameter P is like a weighting function within the range 0 to 1. As P gets closer to 1 the model is more likely to include graphs with more edges and as it gets closer to 0 the opposite. Random Networks possess some unique properties. Due to the fact that nodes are linked via a random process there is no bias involved when building the graphs. As a result the degree distribution (number of inlinks to each node) is quite even across the network. In other words, nodes with a huge share of the overall inlinks or Hubs do not exist or are far less likely to exist than in a scale-free network. Balke and Siberski, (2007) gave a lecture at the University of Hannover where they spoke about the degree distribution of random graphs: Random graph models suggest that the degree of the vertices will not deviate from the average degree in the graph or at least not by a signicant gure. These properties will be illustrated later on in the report when a comparative analysis of the network models is carried out.

Page 13 of 28

4.2

Scale-Free Networks

The other type of network that was to be modeled was a scale-free network with preferential attachment. A scale-free network is also referred to as a power-law network, this is because any network where the degree distribution follows a power-law is deemed to be scale-free.This means is that even if the scale is multiplied by a factor, the ratio of highly connected nodes to regular-connected nodes stays roughly the same. Scale-free networks are typically very large networks containing millions or even billions of nodes. They grow constantly as can occur in many dierent areas such as nature, society and technology. The World Wide Web for example is a very well known scale-free network. The specic type of scale-free network that was of interest to this project was a Barabasi-style model with preferential attachment properties. Balke and Siberski, (2007) mention how in 1999 Albert-Lazlo Barabasi (after whom the model is named) crawled a small portion of the World Wide Web in order to discover its structural properties. At the outset, Barabasi and his team expected that the web was so large that it would be a random-style network. Their ndings were surprising. Barabasi & Reka (2009) mentioned how the World Wide Web is scale-free. Much like the Pareto Principle mentioned above they found that most web pages only contained a handful of links, more interestingly it was found that a small minority of web pages contained a huge amount of links and accounted for a huge percentage of total links crawled. They commented: a few highly connected pages are essentially holding the World Wide Web together They go on to say that the scale-free power law like qualities are due in part to what is known as preferential attachment. Preferential attachment is the process by which nodes in a network such as the web are more likely to be connected to already well established nodes with a high number of inlinks than a less well established node. For instance, if a new web page is created and added to the billions of web pages already on the internet, it is far more likely that the page will be linked to a site like Google with a huge hub-quality than an individual blog. In scale-free networks with preferential attachment it is often common that the rich get richer and the poor get poorer. That is, as the network gains more links overall, the well established nodes also grow more than the less established nodes gaining more links. Hubs (nodes with a huge % of links) are a key part of scale-free networks and this is in stark contrast to random networks where there are no hubs. While it is easy to dierentiate between Random networks such as those based on the Erdos Renyi model and scale-free networks based on the Barabasi-style model, it can be quite dicult to pick a better model between the two as both have advantages and disadvantages. When building any network it is important that they are robust and resistant to attack. In random networks if certain nodes are removed the whole network can crumble. In large scale-free networks such as the World Wide Web removing several nodes will likely have no impact at all unless a major Hub is damaged. This makes scale-free networks stronger in the face of random failure, however it makes them more vulnerable to malicious attack. If key hubs are strategically targeted then the consequences can be dramatic. When my network models were implemented I chose nodes at random and asked what would happen if hypothetically speaking these nodes were removed. The results are laid out in the Robustness section towards the end of the report.

Page 14 of 28

Chapter 5: Design and Implementation

Before the actual implementation of the network models could be done it was important that the right approach was taken. It was decided that implementing the dierent network models and evaluating and comparing them could lead to some interesting insights. The next issue was what programming language to use. Ruby was to be the language of choice as it was the most ecient language to use given the project set-up. During the early weeks of coding some time was taken to adjust to the Ruby programming language. After this initial slow-down implementation began. I found that the theory behind both the Random and Barabasi Models was laid out in a such a way that would make the code dicult to implement and inecient. It was decided that the nancial model was to be carried out in a straight forward manner, this was not a problem. For this model nodes had to be created based on the data contained in the .csv les. Verb nodes were created and linked with all the Noun nodes they were associated with. Likewise each Noun node was created and linked to the desired Verb. For each Verb the number of inlinks are counted and displayed on screen. Nouns that link to multiple verbs are also displayed via a check node noun similarity function. For the random model noun nodes would be created and linked to verb nodes using a random process. Essentially nouns and verbs were taken from the .csv les and placed into dierent arrays, one for verbs and one for nouns. Using a simple random number generator a random node was built. Below is some code examples which demonstrate how the above was achieved. @random_noun = noun.get_nouns(rand(nounlist.length)) @random_verb = verb.get_verbs(rand(verbs.length)) randomnode = Random_Node.new(@random_noun, "NOUN", @random_verb) In the above case, the variables random noun and random verb contain nouns and verbs respectfully which have been taken from the dierent arrays. These variables are then used to create a new Random Node object. There was a slight problem with the random model in that Prof. Keane and I was not sure if the random generator was random enough to be used. After some research into the topic It was found that it was the best possible solution. For the Barabasi Model implementation was slightly more time consuming as there was a few dierent ways in which it could be done. My rst attempt at implementation only demonstrated one characteristic of the network and that was preferential attachment. I created a special array where instances of every noun was created, within this array I had a list of each instance of a verb. This array was basically an adaptation of the array used in the random network, except in this case each instance of a verb was taken into account for preferential attachment purposes. Just like the building of the random model a random noun node was selected and linked to a verb from this special array. This demonstrated preferential attachment as the verb nodes with the most instances (inlinks) were more likely to be chosen and linked to. See code below. @random_noun = noun.get_nouns(rand(nounlist.length)) @pref_verb = noun.get_eachinstance_verbs(rand(eachinstance_verbs.length)) prefnode = Random_Node.new(@random_noun, "NOUN", @pref_verb)

Page 15 of 28

The problem with the above version of the Barabasi model is that while it satises the preferential attachment property of the network, it does not satisfy the other property, Growth. One of the key aspects of a scale-free network is that it grows. Nodes are added continuously and preferential attachment determines where these nodes go upon creation. This version of the network took longer to implement. I began by giving each vertex (verb node) an initial degree of 1, that is to say that each vertex was in the network. If the degree was 0 that would mean the node was completely disconnected and preferential attachment could not take place. The vertices were stored in an array. A random noun node was created and linked to a verb node from this array. Each time a given vertex was chosen by the process its inlink degree increased by 1 and another instance of the node was placed into the array and the cycle continued. This meant that as a verb was linked to more and more it had a greater chance of being selected. Below is a code sample illustrating the process for the verb Advance @random_noun = noun.get_nouns(rand(nounlist.length)) @verb = @initial_verbs[rand(@initial_verbs.length)] if @verb == verb.get_verbs(0) @advance_degree+=1 @start = Barabasi_Node.new(@random_noun, "NOUN", @verb, @advance_degree) @prefnodebucket << @start @initial_verbs << verb.get_verbs(0) end Once the the core implementation of each of the four models was in place some extra coding was done to help gather some statistics from the model generation. Research into the dierent network models showed that the number of inlinks or degree of each vertex would be crucial in illustrating how the models were structured and so these inlinks had to be counted for each verb in each model. Just like in the nancial data model noun similarity is checked for each model. It is worth noting that in the Node class there are actually three variants or instances of a Node object that can be created. Each are practically identical apart from one attribute. This was purposely done so that the data could be presented to the user in the best way. For instance, in the random network model the Random Node instance of the Node class is used, and looks like so: class Random_Node attr_accessor :name, :type, :linkedto def initialize(name, type, linkedto) @name = name #Its name i.e. string (advance for verb or sector for noun) @type = type #whether the node is a verb or a noun @linkedto = linkedto #describes what the node is linked to end On the other hand, for the Barabasi model, the Barabasi Instance is used. This object is the exact same as the random node object except it has a degree attribute: class Barabasi_Node attr_accessor :name, :type, :linkedto, :degree
Page 16 of 28

def initialize(name, type, linkedto, degree) @name = name #Its name i.e. string (advance for verb or sector for noun) @type = type #whether the node is a verb or a noun @linkedto = linkedto #describes what the node is linked to @degree = degree #inlinks end

Page 17 of 28

Chapter 6: Evaluation and Comparison of Models

A key part of the project was establishing which similarities and dierences existed between the dierent models that were implemented. It was also important to analyze whether these models performed as they were supposed to according to the research carried out. In the end conclusions could be drawn which would give further information about the structure of the nancial data model and how it works. There were dierent ways in which the models could be compared, the key attribute in most of the cases was going to be the degree of the vertices (Verb Nodes). The three main comparative studies done were: Mean Analysis of the top 10 Verbs Power-Law analysis Robustness test Before each of these studies could be carried out it was important to establish the environment under which such tests would be run. I implemented my code using the TextMate general purpose editor on a MacBook Pro and this same environment was originally to be used to obtain model results. In the beginning, just to get testing underway I generated small models of 1000 noun nodes being added to the network. This was sucient to ensure the models were performing in a suitable fashion but it was soon realized that this small scale was not representative of the average degree of the nancial data model and thus had to be adjusted. To nd out how big the nancial data model was all the inlinks were added up. In the end it was shown that 112,758 inlinks were present in the network and so to generate a model of comparable size 112,758 noun nodes would be used in my own models. The next issue was how many runs would be needed to get a accurate representation of any data obtained. My Laptop performed best using 250 as the run count. In order to cut down on program running time nodes were created but not displayed to the screen and results were obtained as usual by adding up inlinks and dividing by 250 to get an average run. The TextMate editor was used instead of the interactive ruby shell.

Page 18 of 28

6.1

Mean Analysis

For the mean analysis, after the 250 runs were nished the top 10 verbs from each model were chosen. The idea was to nd out how many of the 112,758 nodes were distributed between these top 10 verbs thus giving an insight into any bias that might exist in the models. The results were as follows. In the random model no bias was found, 28 percent of the nodes were represented by the top 10 as shown in Figure 6.1 below. The top 10 verbs were : Retreat, Elevate, Rise, Slip, Strong, Ease, Worsen, Rally, Advance, Slide

Figure 6.1: Random Model

For the scale-free Barabsi models, there was indeed a bias in both cases. In version 1 the bias was slightly more pronounced with the top 10 verbs accounting for a huge 77 percent of all inlinks. This is very close to the 80-20 rule. See Figure 6.2 below. The top 10 verbs were : Fall, Rise, Strong, Gain, Weak, Lose, Drop, Rally, Climb, Increase

Figure 6.2: Barabasi Ver. 1 Model

Page 19 of 28

For the second version (Fig. 6.3), which satises both key properties of a Barabasi scale-free network, the results showed a big dierence in noun allocation with 66 percent of all nouns being linked to the top 10 verbs. The top 10 verbs were : Stable, Rally, Plunge, Weak, Unstable, Surge, Hard, Retreat, Float, Lose

Figure 6.3: Barabasi Ver. 2 Model

To show how inlinks are distributed as nodes are added, more runs were done to single out the biased distribution in the Barabasi models. These runs were separate from the ones above so the top 5 verbs arent the same. Figure 6.4 below shows the top 5 verbs and their inlink allocation over the rst 10 runs under random and Barabasi ver. 2 conditions. As can be seen, in the random model the top 5 verbs were allocated roughly the same amount of nouns on each run. There were small uctuations whereby the order of the top 5 changed. In the Barabasi ver. 2 model the opposite was seen. On the rst run there was a clear hierarchy formed. This order was maintained throughout the 10 runs. The verb which acts as the biggest hub was linked to more nouns than the second biggest, with the second biggest getting more links than the third and so on. This shows clearly both growth and preferential attachment at work. Once a hub emerges it is far more likely to gain more and more links as the network grows.

Figure 6.4: Distribution of links with growth

Page 20 of 28

The Barabasi networks follow a power-law distribution and this is shown in greater detail later on. A key property of a Barabasi network is that it is scale-free. Scale-invariance is also a property of power-laws and means that as scale changes the ratio of hub nodes to non-hub nodes remains constant. Figure 6.5 below demonstrates this. For one set of runs 112,758 nodes were added and the top verb was linked to 13.8% of all links. When the scale was increased by a factor of 10 and 1,127,580 nodes were added, the top verb got 13.9%. This shows that while the number of inlinks varied as a result of an increase in size the % of inlinks did not. To consider it in terms of the World Wide Web, If the number of web pages doubled overnight, Google would receive many more links but its overall share of links would stay constant.

Figure 6.5: Scale-Invariance

Figure. 6.6 below gives a good representation of all three models side by side. It shows the number of mean number of inlinks for the top 10 verbs across all the models. The random model has a normal distribution while both Barabasi models demonstrate a power-law and biased distribution.

Figure 6.6: Top 10 Verbs inlink Histogram

6.2

Power-Law Analysis

Another key aspect of the comparative analysis was to nd out whether the network models followed a power-law distribution. Of course from the background research that was carried the desired outcome was known. For the random model a power-law analysis should prove negative. In the case of the scale-free Barabasi models the distributions should follow a power-law. In order to carry out the power-law analysis the average inlink for each of the 36 verbs was noted down, ordered, and put into a spreadsheet. A curve was then generated along with a
Page 21 of 28

subsequent equation which represents the curve. Along with an equation for the curves I also have an R-squared value. The closer this value is to 1 the better the data ts the curve. A typical power-law curve exhibits Pareto Principle behavior in that it conforms to the 80-20 rule. A small portion of events will have a high distribution while the majority will have a much lower distribution. In the case of the random model no pronounced curve existed, this was known due to the gures all staying extremely close to the average. In the gure below Barabasi Version 2 is compared with the random model. As can be seen, each of the 36 verbs in the random model have a very even distribution with the most frequent and least frequent verbs only having a dierence of 16 inlinks. This is in stark contrast to Barabasi Version 2 whereby the curve follows a power-law. The most linked to verb contains over 15 percent of the entire allocation. The most linked to verb has a huge number of inlinks compared to that of the least linked to verb, a dierence of 17,411 links. This is over a thousand times the dierence seen in the random model which illustrates a huge bias.

Figure 6.7: Barabasi Ver. 2 versus Random Model

The rst version of the Barabasi model possesses a curve which is almost identical to that of the nancial data model. The gures below show the similar curves. My thoughts on why this is so along with any further conclusions will be laid out in the nal chapter of the report. Please note, there is a small error in the legend in Figure 6.8., the blue line represents the Barabasi Ver. 1 Model and not the nancial Model. In summary, a power-law analysis of the three applicable models (all but the random model) yielded the following power-law equations: Financial Data Model y = e67183 x1.509 Barabasi Version 1 Model y = e67391 x1.511 Barabasi Version 2 Model y = e64667 x1.417 (6.3) (6.2) (6.1)

Page 22 of 28

Figure 6.8: Barabasi Ver. 1 power-law distribution

Figure 6.9: Financial Data Model power-law distribution

Page 23 of 28

6.3

Robustness Test

In the background research section a little was discussed about the dierences between random style networks and scale-free type networks. Many researchers have wondered which model is stronger and least susceptible to damage. Scale-free networks such as the World Wide Web are thought of as being reasonably well secure in terms of major damage due to the fact that there are far less hubs than standard nodes. On the other hand random networks can suer major damage when nodes are removed from the network due to the fact they are all of fairly equal importance. This was tested out on my models once the mean analysis and power-law analysis were complete. To test the strength of the networks I assumed that the models vertices were in-fact web pages and noun nodes were other web pages that could be linked to it. I used the Random model and the second version of the Barabasi model. Once the models had been generated with the desired amount of nodes added (112,758) I randomly selected roughly 10 percent of the vertices to be removed from the network, this amounted to four vertices. In the case of the random network model, due to the fact each vertex or web-page had the a similar amount of inlinks all four could prove crucial in holding the network together and thus major damage could be caused. In terms of the scale-free network, out of the four vertices removed not one was considered a hub and together all four only made up 5 percent of the total links. If this sort of situation was to happen in the real world to the World Wide Web barely anyone would be aected. It is important to note however that in some cases (very rare) a hub was selected at random. This would be the equivalent to googles servers being shut down. A huge amount of webtrac would be aected as a result of such a huge hub being removed from the network. While this particular test does not exactly tie in with the language pattern element of the project I felt it was useful to carry out to provide some data based on some background research that was done into random and scale-free network structure.

Page 24 of 28

Chapter 7: Conclusions and Future Work

Once sucient research had been carried out along with the implementation of the network models, conclusions could be drawn from the results that were obtained. One of the main goals of this project as mentioned in the abstract was to nd out whether the underlying structure of the nancial data model was more closely linked to that of the random Erdos Renyi model or the scale-free Barabasi type. The nancial data model demonstrated a similar power-law distribution to the Barabasi models and diered greatly from the random model structure as seen below.

7.1

Conclusions

In the nancial model there are signicant verbs that gain a huge allocation of the nouns. It is interesting to observe that in the nancial data the verbs Fall (27,285 inlinks) and Rise (23,647 inlinks) are the top two verbs. Together they account for almost 50 percent of total links, this is signicant as they both provide good, clear representations of the two dierent stages of a bubble. Stocks rise before and during a bubble and Stocks fall as the bubble bursts and a crash may occur. The fact that phrases containing these two verbs occur a lot may mean that nancial commentators value them more, perhaps they see these verbs as a way of getting across their message to everyone while using phrases containing less known verbs such as exacerbate (285 inlinks) far less often. In terms of the random model it is completely random and thus no bias towards particular verbs exist. The Barabasi Version 1 model singles out preferential attachment and provides extremely similar results to the nancial model. As nodes are created, hubs such as Fall and Rise grab a huge amount of these nouns. In the Barabasi Ver. 1 model the Verb order is exactly the same as in the real-world nancial model. This further backs up the facts that in Barabasi models the order is hard to disrupt due to the eects of preferential attachment. When this is put in terms of nancial commentators and investment in the stock market.
Page 25 of 28

It can be thought of as similar to new articles being written by someone who already has read hundreds if not thousands of nancial articles by other commentators. Knowingly or unknowingly the most common verbs from the corpus of articles read are re-used over and over again. The bias within preferential attachment is essentially the herd mentality within nancial writing. This herd mentality is then converted into actions on the stock exchange. If investors are being subjected to these common verb phrases over and over again, then they are far more likely to act positively or negatively depending on how positive or negative these phrases are. For example, if I am a person wanting to invest in some stocks yet know little about the stock market or what is likely to give a good return, it is quite logical to assume I will turn to the so-called experts that I believe will give me good advice. If this advice is biased intentionally or unintentionally it will likely cause me to act in a certain way. In the second version of the Barabasi model where the network is grown from almost scratch, the bias isnt as signicant. When putting the second version of the Barabasi model into terms associated with language patterns in the stock market I like to think of the growth from scratch as the origination of new ideas, and the formation of hubs as a representation of those ideas that catch on. If certain nancial commentators through their research originate new opinions about certain stocks and express these positive or negative opinions with appropriate verb and noun phrases, then these opinions may be picked up by other nancial commentators who have read the article and re-used via quotations or similar phrases. These phrases and the use of Verbs and Nouns contained within them can snowball into hubs whereas other ideas proposed will not garner much publicity and thus be used less and less. Often the ideas that catch on may be inaccurate but eye-catching, many commentators may contain such phrases as a means to attract attention or a in gamble to predict a value of a stock in the hope they are proved correct. Due to the fact a bubble can be viewed as an inaccurate representation of a stocks true value many commonly used phrases and verbs coming up to a bubbles peak are extremely positive. Before the property bubble collapsed verbs like Rise, Strong Jump etc were commonly used. This meant a lot of investors viewed the stock as healthy and purchased some driving the price up and up. The fact that the Barabasi models are scale-free and contain a very similar power-law distribution as the nancial model would allow the assumption to be made that the data contained within the nancial model is scale-free also and that as the models grow and grow a similar 80-20 rule will continue to apply irrespective of size. So in other words, if the corpus of articles was tripled in size to nearly 60,000+ a similar type of result would be observed.

7.2

Future Work

In terms of future work I believe there is a lot more to be discovered in this area. Power-laws exist all around us in everyday life. Many dierent types of power-laws could be discovered and tested. I found that the relationship between power-law distributions of events and how people act to be of utmost interest and importance and I feel such an area could be explored further. It is obviously making some waves as many of the top companies in the world are carrying out power-law pattern analysis to learn more about their consumers and their competitors.

Page 26 of 28

As regards further research into peoples actions on the nancial side, Prof. Keane and Mr. Gerow carried out some research into language valency i.e whether the verb-noun phrases were positive, negative, or neutral. I feel that this is an area that I should have looked into more but didnt manage time suciently for it to be carried out correctly. It would be interesting to see how much of an impact language valiance can have on potential investors in the stock market. All of the articles contained within the corpus came from reputable nancial news sources like the Financial Times and so forth. It would be interesting to observe language patterns present in less formal sources such as blog posts or even comments on the articles themselves. The fact that many of these comments and blogs would be written by regular people might give a further insight into herd mentality in the minds of regular investors. Nowadays, with the availability of nancial articles on news apps on smartphones many more people are reading these articles and in many dierent languages. It would also be of interest to nd out how people who dont speak english are likely to invest after reading articles translated into their rst language. i.e. is some of the eect lost in translation?. All this research could contradict or conrm ndings from this paper. There is much work to be carried out on whether certain graphs are random in nature or scale-free. The fact that Barabasi thought the World Wide Web was going to exhibit random properties yet ultimately found out it was one of the best examples of a scale-free network shows that while we may assume certain structures are one or the other, that may not be the case when subjected to detailed research. Finally, I was not able to complete the exceptional requirement of developing a GUI for comparing the models. This could also be attempted at a later stage.

Page 27 of 28

Bibliography

[1] Adamic, L.A. and Huberman, B. (2002) Zipfs law and the Internet, Glottometrics 3, 2002,143-150 [2] Balke, W-T and Siberski, W. (2007) Random Graphs, Small-Worlds, and Scale-Free Networks, [Lecture notes], L3S Research Center, University of Hannover [3] Barabasi A-L. and Reka, A. (1999) Emergence of Scaling in Random Networks, Department of Physics, University of Notre-Dame, Notre-Dame, IN 46556 [4] Barabasi A-L. and Bonabeau, E. (2003) Scale-Free Networks, Sci. Amer. 288, Issue 5, 60 [5] Choi, H. and Varian, H. (2009) Predicting the Present with Goggle Trends [6] Gerow, A.M. and Keane, M.T. (2010) The Voice of the Herd: Power-law Regularities in Newspaper Articles Predict Market Bubbles. Unpublished Manuscript [7] Gerow, A.M. (2010) Metaphor in Financial Reporting [8] Goldburt, D. and Zhang, J. (2006) Large Graphs, [Lecture notes], CS485 Lecture 01 [9] Keane, M.T. (2010) From Graphs to Google, [Lecture notes] Computer Science in Practice Module, University College Dublin [10] Koch, R. (1999) The 80/20 Principle: The Secret to Achieving More with Less, Crown Business [11] Komromi, G. (2006) Anatomy of Stock Market Bubbles, First Edition, The ICFAI University Press [12] Odintsova, N. and Rish, I. (2004) Fault Diagnosis in Scale-Free versus Random Networks, IBM Technical Report [13] Ottaviani, M. and Srensen, P. (1999) Herd Behavior and Investment: Comment, Forthcoming, American Economic Review [14] Rook, L. (2006) An Economic Psychological Approach to Herd Behavior, Journal of Economic Issues Vol. 40 No.1, March 2006 [15] Sornette, D. and Woodard, R. (2009) Financial Bubbles, Real Estate bubbles, Derivative Bubbles, and the Financial and Economic Crisis [16] Sornette, D. (2004) Why Stock Markets Crash: Critical Events in Complex Financial Systems, Princeton University Press [17] Zipf, G.K. (1932) Selected Studies of the Principle of Relative Frequency in Language. Cambridge, MA. Harvard University Press

Page 28 of 28