Skip to main content

Authors in a Markov matrix Part 2 (3) Experimental results: Which author do people find most inspiring?


In this article, I will explain about our implementation and how the adjacency matrix looks like.

Implementation

We implemented the following programs:

  • Link_Vector_Extractor: Generate the author vector from the data.
  • Graph_Extractor: Generate the adjacency matrix from the data and the author vector.
  • Page_Rank: Compute page rank.
  • Remapper: Re-map the author vector according to the page rank result.

Our experiment is done with the computer CPU: Intel(R) Core(TM) DUO CPU P8400, 2 Cores, OS: 64bit Linux 3.2.0.32, Kubuntu 12.04. We used the following software: Python 2.7.3, Beautiful Soup 4.0.2, matlab R2006a, octave 3.2.4.

Adjacency matrix

Figures 2, 3, 4 show the adjacency matrices. In these Figures, blue points represent the connection.
Figure 2: Adjacency matrices. Top to bottom: German authors in de. wikipedia.org, en.wikipedia.org, ja.wikipedia.org.

Figure 3: Adjacency matrices. Top to bottom: English authors in de. wikipedia.org, en.wikipedia.org, ja.wikipedia.org.
Figure 4: Adjacency matrices. Top to bottom: Japanese authors in de. wikipedia.org, en.wikipedia.org, ja.wikipedia.org.
In Figure 2, German author, en.wikipedia.org has a regular pattern.  We haven't check what exactly causes this, however, we think this is the same problem of the template bias (will be discussed later) (Note 1). The German author in en.wikipedia.org has another peculiarity that we can see the higher number of links compare to other Wikipedia data shown in Table 2. We count the number of links is how many links are connected to the author vector entry, not all the links in the page since these are the connection in the adjacency matrix. For example, we didn't count the links to other language pages, self links, author links that were not on the author vector. Therefore, if the page has links to non-author person, we didn't count these links.
Table 2: Matrix size, non zero elements, and the average number of links between authors. Wiki ``en'' means en.wikipedia.org.

We removed rank sink nodes according to the PageRank algorithm [1]. We removed links related rank sink node pages, therefore we also removed so called dangling links (2.7 in [2]), since the links refereeing single sink rank node are the dangling links.  Additionally, we removed pages that has only outgoing links. In the original PageRank paper [2], nodes that have only outgoing links are not mentioned, though this doesn't make sense in the PageRank concept. However, it is easy to imagine the reason why the paper didn't mentioned about them. Since the original paper is for the web, it is hard to determine a node is not linked from any pages. To determine this, all the pages are considered. But, this is practically impossible on the web, therefore it is natural that the original paper didn't consider this outgoing link only nodes. However, our author vector has a limited size and we know all valid pages, therefore, we can remove these nodes. The link normalization value of PageRank calculation depends on whether considering these links or not. This means, it may affect the absolute value of the PageRank. However we are not interested in the absolute value of the PageRank. We are rather interested in the relative value, which author's influence is larger. The PageRank paper also mentioned this normalization has not large effect since they are also interested in the relative influence.

The matrix size is reduced by this operation, the result size is shown in Table 3.
Table 3: Adjacency matrices: original size, reduced size, and its rank.

Finally, I will show you who is the most important author in the next article (in a Wikipedia sense).

Note 1: When I wrote this article in my blog, my friend, Joerg M. pointed out this possibility (2012-12-29). Thanks.
Labels:

Comments

Post a Comment

Popular posts from this blog

Gauss's quote for positive, negative, and imaginary number

Recently I watched the following great videos about imaginary numbers by Welch Labs. https://youtu.be/T647CGsuOVU?list=PLiaHhY2iBX9g6KIvZ_703G3KJXapKkNaF I like this article about naming of math by Kalid Azad. https://betterexplained.com/articles/learning-tip-idea-name/ Both articles mentioned about Gauss, who suggested to use other names of positive, negative, and imaginary numbers. Gauss wrote these names are wrong and that is one of the reason people didn't get why negative times negative is positive, or, pure positive imaginary times pure positive imaginary is negative real number. I made a few videos about explaining why -1 * -1 = +1, too. Explanation: why -1 * -1 = +1 by pattern https://youtu.be/uD7JRdAzKP8 Explanation: why -1 * -1 = +1 by climbing a mountain https://youtu.be/uD7JRdAzKP8 But actually Gauss's insight is much powerful. The original is in the Gauß, Werke, Bd. 2, S. 178 . Hätte man +1, -1, √-1) nicht positiv, negative, imaginäre (oder gar um...
Post a Comment
Read more

Tezuka Osamu's Black Jack, "Shrinking"

I like several novel authors. My first favorite author is probably Teduka, Osamu. I still love him. The list grows by adding Hoshi, Shinichi, Agatha Christie, Hermann Hesse, and so forth. My first favorite article of Tezuka was Atom as most of the (boy's) Tezuka fans did. But my favorite is Black Jack. I try to summarize one story, it is still quite vivid in my memory. I first read this story when I was 13 - 15 years old. I re-read it at least several times since Black Jack is composed of many short episodes. The title should be "ちぢむ (SHRINKING)" or it might be "縮む(Shrinking)". (It is not so convenient to translate this to English, since English does not have a system to say the exact same word in several ways. So I just simulate it with capital letters.) Black Jack is a genius surgeon, but he does not have the license. In short, his medical activity is illegal. His skill is top level in the world, but, the fee is also out-of-law expensive. In the story ...
Post a Comment
Read more

My solution of Google drive hang up at "One moment please"

Today I installed Google drive to my Windows 7 environment to share files with my Linux machines. After sign in, the application window said "processing," then it hanged up. There was a button "you must enable javascript". I pushed it, then "One moment please..." after 5 minutes, I exited the program tried it again. It seems some security setting causes this problem. My solution: set  https://accounts.google.com  as a trusted site. Procedure: Open the control panel Go to network and control Go to Internet Options Open Security Tab Click Trusted sites Click the "site" button copy & paste  https://accounts.google.com  to "Add this website to the zone" and click Add button Now it worked for me. But if I removed this site, it still works. That puzzled me a bit...
25 comments
Read more