Update: Graph has now been made interactive.
I was originally put onto network visualisation by Simon Raper by his fantastic post graphing the history of philosophy. I’ve learned a lot in the last week and decided to be ambitious. I wanted to see what the entire network would look like – with everyone on Wikipedia. Well, everyone with an infobox containing ‘influences’ and/or ‘influenced by’. For those unfamiliar to this work please see his post first – even if it is just for the pictures!
For those new to these types of graphs: the node size represents the number of connections. To create the following graph I used a database version of Wikipedia to extract all the people with known influences. I then scaled the nodes by their degree of influence. The bigger the node, the bigger influence that person had on the rest of the network. Nietzsche, Kant, Hegel, Hemingway, Shakespeare, Plato, Aristotle, Kafka, and Lovecraft all appear as the largest nodes. Around these nodes, cluster other personalities who are similarly related thinkers/authors. I used a module to highlight communities in different colours which revealed sub-networks within the total structure. You’ll notice common themes amongst similarly coloured authors.
Method
- First I queried Snorql and retrieved every person who had a registered ‘influence’ or registered ‘influenced by’ value (restricted to people only so if they were influenced by ‘anime’, they were excluded).
- I then decoded these using a neat little URL decoder and imported them into Microsoft Excel for further processing (removing things like ‘(Musician)’ and other similar syntax).
- ‘Influenced By’ entries were also included by reversing the order of influence. Duplicates found in the ‘influences’ list were then removed. This ensured a more complete dataset.
- I then exported these as a csv and imported into Gephi and proceeded as usual. Fruchterman-Reingold algorithm followed by Force Atlas 2. I then identified communities using ‘Modularity’ and edited the rest in Preview. Due to the size, I’ve had to zoom up and take snapshots on regions of interest.
- The csv file containing all of the data can be obtained here so you can make your own maps.
Caveats
- The graph is obviously biased towards Western ideologies and culture – the people entering in the information are after all primarily English speakers. It must be said: There are a great many influential people missing from the graph.
- The graph is created from the datasets of dbpedia and so is intrinsically incomplete. By exactly how much? Well, I need to run a few more tests. Many human endeavours are sadly missing.
- This work is just trying to demonstrate that by combining the power of new open-source tools with the vast quantity of the information on the Internet, one can create useful and informative networks.
- The community identification was done using an in-built module of Gephi so I apologise if you disagree with some of the groupings.
- What does the word influence actually mean? Material? Ideological? I for one don’t know the motivations behind the connections shown — I am simply using the relations entered into Wikipedia by its contributors. Please make those you share this graph with aware of this crucial point.
The Future
- I would like to compare this network to the Indiana Philosophy Ontology Project and the database at Freebase.
- I already have designed a poster version and if there is sufficient interest, I can make this available. Update: Posters are now available! Version 1 & Version 2. Now VERSION 3 (Poster Version)
- Explore other algorithms on faster PCs. I am limited by the processing power of my desktop and had to resort using Force Atlas 2 in Gephi and good timing. On a faster machine, the other force algorithms might bring out the richness of the network in a more aesthetic manner but this is what will have to do for now.
The Network
I restricted the network to only include nodes which have 4 or more connections otherwise my computer crashes trying to render the full network. It also allows you to read the names of people who you probably are more interested in. Doing this however does remove many people from the network. Apart from this one selection, nothing else is altered.
There are a few main communities (roughly):
- Red – 19th/20th century philosophers
- Green – antiquity & enlightenment philosophers
- Pink – enlightenment authors
- Yellow – 19th/20th century authors
- Orange – fiction author
- Purple – comedians
These can be broken down in to further categories but to avoid flame-wars I’m going to avoid breaking the network down much further.
I get a real kick by starting at one node and travelling down the connections to a distantly related someone else. People in philosophy influenced fantasy writers who influenced comedians. It shows one thing above all: the evolution of ideas is a non-linear process. We too, are somewhere in this web, albeit at a smaller scale. We too, are the sum of many.
In Gephi, it is far more interactive because you can highlight nodes and every node that one is connected to becomes highlighted. You can search any name you wish and it becomes highlighted within the network. The list goes on.
Now for some pretty pictures. Click to enlarge as some text might be hard to read at the resolution set below. For best viewing, click here for the ability to dynamically zoom about the graph.
The Graph Of Ideas
Highlights include:
Philosophers of antiquity influencing 18th, 19th and 20th century philosophers.
Philosophers influencing authors.
The dominant figures of the landscape.
Philosophers of the ancient world.
The comedians. George Carlin & Richard Pryor are the dominant influences.
The fantasy/fiction writers.
19th and 20th century authors
Their influences come from an earlier generation. No clear boundaries here however.
We now come to the most influential people of the landscape (largest nodes). Keep in mind the various biases involved here. The philosophy biographies on Wikipedia tend to have more detailed content (and info boxes) and so will naturally be larger nodes in the network.
The big hitters.
An earlier generation of thinkers. Interestingly the bottom section of the network is essentially chronological – starting from the philosophers of antiquity (far left, green) and ending in the 20th century philosophers (far right, red).
The earlier generation of philosophers.
A patchwork of ideas.
Leave any questions or comments you might have in the comments section below. I’d love to hear of any suggestions for future networks.
UPDATE: As requested, there are poster versions now available: Version 1 & Version 2.
UPDATE: I have also created a graph which includes upstream connections between thinkers. Check it out here.
This is fantastic Griff. The literature part is amazing. I’ll put a link from my post to yours. By the way I did do exactly what you described for the influenced by part to get a fuller list (i.e. extract both, invert one and dedupe) I was just a bit lazy in the write up!. So glad some one has done it!
Simon
Thanks Simon! This is really fascinating stuff and I wouldn’t have ever done anything if I hadn’t read your original post. Thanks for the link. I also made a correction so people know you did the right thing!
I wonder if this is a scale-free network, and also what the distribution of the lobby index (see e.g. http://arxiv.org/pdf/0809.0514) is like.
Also: are there any hidden influencers? Nodes with a high lobby index but relatively low degree?
Anyone care to look into that?
That’s really great, Simon. Do you mind sharing the data so that I could try my hand on visualizing them using different layouts in Gephi?
fantastic; is there any way for you to share a higher-resolution version of the whole graph so that we can explore it ? It would be great to print out on a big size too…
Yep! I’ll be making it available soon. I’m travelling at the moment but will put it up when I return home.
great ! looking forward to that..
I’ve made a poster edition. Please check it out. http://www.redbubble.com/people/griffsgraphs/works/9068422-the-graph-of-ideas?p=poster
I might have missed this but what is the basis of the layout. Could it be arranged to show the most influential place in the world for example?
There is no basis per se. Their positions are determined by two force algorithms. You gave me an idea though. Stay tuned.
Agreed. This would be cool. I could also imagine adding directionality to the edges either based on date of birth or who dbpedia says influenced who. It would be interesting to see separately who were the generators and integrators of great thought.
I imagine directionality of the edges, location, and date of birth/death data would let you do a lot of cool things a few off the top:
- Identifying “most influential place” (as thegracefulcyclists mentioned)
- Tracking the centroid of thought generation over time
- Separating the great thought generators and integrators
- Identifying “downstream impact” of the early great thinkers (i.e. George Carlin is rightly measured by the breadth of his influence, but Galileo’s contribution is better measured about the incredible fruitfulness of his discoveries than the number of peers he immediately touched).
Great suggestions. I am working on a new version which has some of what you suggest.
nice work man! fascinating structures! now, about making this available in higher resolution: are you actually going to do this? or is this all about selling posters at redbubble? please don’t try to capitalize this. after all simonraper figured it out and shared it with everybody. so it would be nice if you did too! thanks for being social. felix
I will make a high resolution graph available. Some friends and family wanted to buy a poster so I made one just for them. I’ll make the full data and graphic available soon! Information should be free – the option to buy is there as exactly that – an option. I’ll let you know when it is available if you like. Thanks for the comment.
Hi Felix,
I have made the csv file available for public use. See the link in the text. The post version shows a high resolution version but I can make a better version if you would like. Let me know.
Thanks.
Griff
Thank you so much. I think it will enable important discussions, not only on the lines of influence, but also on the perception of the recorders of influence (I think someone else also commented on that).
The first thing I did was to compute the pagerank of the nodes, and the results surprised me initially – it shows Feyerabend, Acamben and Lakatos to be the most “important” philosophers (in terms of the algorithm that finds the most “important” web sites and the most “important” scholars according to publication citations). But what essentially shows is the understandably uneven efforts of those editing the ontologies – not their fault, most of the time.
I think we need to introduce some computation where influence is credited backwards somehow, to those who influenced many.
thanks a lot! sorry if i sounded a bit harsh. no offense intended. i totally appreciate your effort and i would love to see even more people pick up this idea. best. felix
Griff, lots of romanian in the midle of the graph. Care to explain the reason?
Thanks
My only guess is that Romanian thinkers influence one another. The same can be said of Arabic scholars and authors and other clusters. Given the isolation of certain schools of thought in history, it makes sense that thinkers of similar cultural backgrounds are located close to one another.
The graph and the idea behind it are astounding.
Can’t help noticing some dodgy philosopher classifications though. Bertrand Russel, Whitehead, Popper and Wittgenstein are Antiquity \ Enlightenment? Even Turing and Noam Chomsky??
Amazing work, looking forward to exploring it more.
Yes, they are very general classifications and wrong in some way. You highlight the main point though, everything is connected; everyone feeds off another.
I appreciate the effort, but you should really change the title. It’s misleading and obviously not what it appears to be. Sites like Gizmodo apparently can’t be bothered to read the asterisk notes and claim it’s the “World’s” ideas. But thanks for showing what could be done.
Do you have some suggestions as to what it should be changed to? Do you mean the blog post or the chart? I believe the emphasis is on the reader to properly understand what is going on. I’ll make an edit to remind people to read the caveats. Clearly my disclaimer and ample notification isn’t working…
This is a fantastic achievement and a great start for further exploration! I think many folks will be eagerly awaiting the full data graphic (can’t imagine how large that will be!). I am curious how you distinguished between individuals and groups (I saw a Monty Python node in there)
The full graph would be billions of people and require several gigapixels to properly view! “influence” is highly subjective and it could be quite misleading to make a proper map with everyone. Humans naturally adopt answers which *feel* right but rarely are. All sorts of biases are involved in this sort of stuff so one has to be careful! I didn’t select anything really… Monty Python has an infobox with influences and so that was scooped up. I did no manual selection of any of the data. Thanks for comment!
I’m curious about the graph & absence of some of the heavyweights. I didn’t find Luther or Athanasius or Augustine or Calvin, and I wondered if it’s a result of your filter that left them abandoned of any category or whether it’s a lack of reference articles in Wikipedia.
I had to remove people with less than 3 connections otherwise the graph was too big to handle. If you find someone on Wikipedia with influences then check the number of connections before looking.
Very cool. I think it would be very interesting to instead the color dimension to represent the date that each person was born. Either a two-color gradient or a rainbow – red for the first born and violet for the most recently born. This would accentuate the pioneers of each community who maybe aren’t as connected as their more contemporary counterparts. I imagine that the early philosophers just below and left of center would also stand out as a concentration of early contributors.
Yes, there is a whole another dimension to the graph which is largely unexplored. In my current works in progress I am trying to bring out this hidden information to make the graph a little more enlightening. Stay tuned!
Why doesn’t Socrates have a big node next to Plato and Aristotle?
That was one of my first thoughts. There are a number of nodes which probably should be larger by any definition of ‘influence’ (and some smaller too). In terms of the data, I am just plotting what Wikipedia provided me with not an objective graph (whatever that means). I’m working on something at the moment which you might be more pleased with – stay tuned.
In any case, perhaps one could argue that because we get all of our understanding of Socrates from Plato (and a few others) that he should in fact be larger anyway? I may be clasping at straws here though. I think in philosophical circles Plato is regarded to have had a bigger influence on the history of ideas than Socrates but this is just an aggregate opinion pulled out of my head from a few readings over the years (e.g. Whitehead stated ‘everything is a footnote to Plato’ and Bertrand Russell somewhat agreed but believe his influence to be negative). Who knows the answer to such things? Not me.
Great site too Matthew, I’ll have a read of some of your pieces when I get time. Thanks for stopping by.
Thanks Griff. I’m no expert on Socrates, but it’s just surprising to me that Wikipedia’s contributors have not judged many people (if anyone) to have been influenced by Socrates, given that he’s one of the few philosophers many people can name off the top of their heads. He even influenced the excellent Bill and Ted.
Haha yes you’re quite right though apart from his name, can the average person state anything he did? Plato’s allegory of the cave and republic of philosopher kings are more famous as ideas than any idea Socrates came up with, despite the fact people know his name. It is a problem with these “thoughts” of things… What does influence actually mean?
Good question. Are you using the Socratic method on me?
Haha, checkmate.
Nice idea and realization. It is even hard to imagine how one can further continue the work on graph, e.g. by adding the interactivity, decsription of influence and ideas to the edges… Anyway, one always want to have the whole picture and you gave it to us.
Also it is not clear from the pictures, do you have a “dynamical model”, where the dots are developed in time turning into the net? If yes, is there any “videos”?
This chart is pure awesomeness – it has a lot of potential for further functionality – e.g. a search function would be awesome to locate a POI; also a 3d-chart would be even cooler, one where u could highlight connection like in the “Collusion” add-on for firefox (https://addons.mozilla.org/de/firefox/addon/collusion/?src=api); Some filters would be useful too – e.g. time period, influence weighting, categories etc…
I am somewhat flabbergasted that heavyweights like Isaac Asimov, Poul Anderson and Arthur C. Clarke seem to have so low impact on the chart… all in all the connections and their weighting appear kinda arbitrary…
Anyways keep up the amazing work !!!
where are the women?
They are there but still under represented sadly. This is both a result of having male dominated history and a limited data set. I did what I could using the data available.
Fabulous , #loveyourwork , shared and followed
Thank you. It´s great. Now the outstanding importance of my favorite philosopher Nietzsche ist made clear.
I think this is a nice piece of work. But (although I’m both fan of visualizations and Nietzsche) I think the significance of it is rather low, unfortunately. The problem is best described when looking at Luther. He may not been cited very often in an academic sense but as translator of the latin bible into German and “creator” of the Lutheran theology he surely is one of the biggest influences ever (just read Marx or Max Weber’s The Protestant Ethic and the Spirit of Capitalism). As useful as your quantitative approach is it might not explain whose ideas are the most influential or show the “value” of a citation but show what Wikipedia authors found as citations when searching on Google.
That is why the graph is such an important start though. It clearly indicates the type of information that cannot be found within Wikipedia such as the weight of the edges and the way in which the individual was influence.
Fabulous works! Methodological question: do you use English Wikipedia only, or all versions of Wikipedia? English one is the most important, but some articles can be more developped in some languages. For example, in literature graph, french writers same less represented for me than english ones. (Article in French Wiki are more complete on french topics.) I don’t critic your work, wich is very impressive, I just want to understand where your data come from. Merci!
I only queried the English Wikipedia. Linking all of the Wikipedia’s together would take a bit more work because you would have to remove names which appear twice since a single person might have two differently spelled names. Perhaps this is something you could do!
Good work, but I do think you should change the title. Yes, I am aware that this has been discussed before. But it is important, as it detracts from the magnitude of the work. It doesn’t graph every idea in history. In fact, it graphs no ideas whatsoever. I would want to order a poster, but I can’t hang it on my wall as is.
It depicts some influential thinkers and their interrelations. Thus, I would call it “A depiction of the most influential thinkers and the relations between them” or – more succinctly – “Influential thinkers and the relations between them”.
Congrats on graphing this.
I can make a version just for you and send you the link. Would “Thinkers & Their Influences” be OK? Let me know and I’ll send you the link. White background I presume?
I think that would be awesome. Not just for me
Do you just want a plain white background?
Do you want the accompanying text in the bottom right?
I’ll leave the title in the post but offer this 3rd version to the poster.
Hi Pascal, Does the Graph Of Thinkers suffice? Let me know if this is better.
I think white background plus legend (text in the bottom right) is a winning combination.
Will order as soon as it becomes available.
Thanks!
can you specify the license of these images – we might feature it in Wikpedia’s research newsletter if it is CC-SA thanks
It is under CC BY-NC-SA 3.0. Hopefully this suits your needs.
Hi ! Just to inform you that we published a french translation of Simon Raper’s post with mention of your very interesting job: http://pegasusdata.com/2012/08/05/analyse-de-reseau-modeliser-lhistoire-de-la-philosophie/
Best regards.
Would like to see one tracing connections between historians if it’s possible…
I’ll see what I can do for you Matt.
This of mine will, I beleive, interest you.
Note tat it is a free PDF download.
http://humbox.ac.uk/3682/
Wonderful Work !
Its not my first time to pay a visit this site, i am visiting this site dailly and get pleasant
data from here all the time.
After I originally left a comment I seem to have clicked the -Notify me when new comments are added- checkbox and from now on every
time a comment is added I recieve four emails with the same comment.
Is there a way you are able to remove me from that service?
Thanks a lot!
Within your email there should be a stop notifications link which will allow you to deactivate it.
Hello! I just wanted to ask if you ever have any trouble with hackers?
My last blog (wordpress) was hacked and I ended up losing months of hard work due to no backup.
Do you have any methods to prevent hackers?