Uni Social Network Project progress

Using Perl to create graph and analyse it

2009-07-03T12:32:00.000-07:00

Perl is a brilliant language! it is simple, very straight forward, you can get started in no time!

Only two days of quick rush through of some basics of Perl, I managed to find and use some module code from CPAN. I started to parse an XML, then use the data built a graph and ready to do some advanced graph calculation and analysis!

Today, I experimented on parsing a small portion of an XML file (5 papers with 10 authors), then successfully created a graph out of it. With the methods already provided by the Graph module, I can easily query the average path length of the graph and so on! It turned out my small graph has an average path length of 1.54!

I am really looking forward to try this on a bigger XML and do more interesting analysis.

Learning Perl

2009-07-01T12:26:00.000-07:00

Another hot day here in Southampton.

I had a project meeting with Dr Carr today to report my progress and given some suggestion.

My big plan is sound: Gather dataset of University staff into some data structure -> analyse the data (How to analyse them requires more reading), generate some diagrams.

From the way Dr Carr suggested, a scripting language like Perl, python would be more suitable than a programming language like C in this context, so I had a look at them this afternoon. A quick compare made me to stick to Perl: It is a good text manipulator language(I always wanted to learn one), it has more resource for learning and more active from the impression I get from the search results.

Started to follow a tutorial this afternoon, wrote a couple of scripts, used several variable types: hash, scalar, array, and filehandler. Everything goes smoothly, I should be able to start using it in a day or two.

Once I get a hand of Perl, I should start to practice analysing and manipulating small amount of people exported from the ECS eprints.

Getting data for my project

2009-06-29T10:58:00.000-07:00

As from my last post, in order to start a social network analysis project, the first step is to get hold of some data so that I can generate diagrams and so on.

Good news is that all the person description data for the ECS members are all in readily usable RDF form, I can easily use a sparql frontend (at http://southampton.rkbexplorer.com/sparql/)to query and get out the description about a person. These descriptions include person's name, group he belongs to and some provides interests.

Bad news is all other schools do not have these information readily available.

Next up, I need to think about:
* What programming language to use for processing and visulise the data
* Think about in what form I want the data in for :
-generating graph using graphviz
-to do some processing on the data, (how many hops an author is linked to anthoer)

Question to ask Dr Carr:
*How can I get the author cloud and author graph working for University ECS

University, school, people, data

2009-06-25T12:25:00.000-07:00

After a bit of reading around, in order to carry out any research and analysis, I need a set of data that I can work on. In this particular case, it would be the university staff's names, roles and interests.

So today, I carried out a survey on the university structure, school and staff, found out where are those informations located, and start thinking the ways to extract those information automaticly.

Southampton university splits into three faculties -- Enginnering Science and Mathematics; Law, Arts and Social Science.; Medicine Health and Life Science, each have a few schools in there. Each school have their own web site, in which they tend to publish all the staff and what they do. Since these web page are developped independently, the layout and the information contained varies greatly. At this point, I have to say, the semantic web technology -- the linked and annotated data is so useful, I immediately know how to extract those from our ECS site, which is fully annotated.

I think, from what I explored today, except the ECS site, the fastest way to extract those people data from each web site is by type them out, how sad is this. But I'll keep an open mind in how to obtain these data.

Project direction amendment

2009-06-23T11:48:00.000-07:00

From the moment I get this project's subject, I wasn't fully comprehend the purpose and the object of this project.

After the meeting with my supervisor Dr Carr, I think I am beginning to see where this project could go and what are the areas I should spend my time on.

First of all, forget about most of my eprints efforts and my programming plan, this is not a programming project, there is no big software package to deliver at the end. The focus should be on the understanding of the subject.

There are two possible threads I can follow and develop on for this particular project:

Apply the general social network analysis (e.g. small world experiment, and its follow ups), into a university, academic network, does the "hub", "small world" still holds? How are the links form across the schools?
The weak tie argument: It is the weak ties that compressed the Internet into a such small network system, it is the weak ties spread out information. What is the role of the weak ties (eg. cross discipline relationships between lectures) in supporting the research in a university? How important is it to encourage interdisciplinary research, which allows these cross discipline relationships to form?

I will first tackle the 1st thread, so my tasks before next meeting :

Find out and read about the researches, experiments carried out in the area of analysing social networks, what's their data source? How do they conducted the experiment? What did they found out? Shortlist several activities that I can potentially do in this project and the problems and challenges that I need help from my supervisor.
update my project plan to reflect this change.

Installing the eprints software

2009-06-22T07:45:00.001-07:00

Last Friday, I tried to install the eprints software on one of my computers.

My first attempt was to use the live cd provided by the official website, it is suggested by the developper as the fastest way to get started. I created a virtual machine (used virtualbox) and booted from the live cd. Accoding to the guide, there supposed to have a shortcut on the desktop to initiate the installation, but there is not, nor anywhere else. A web page was opened after boot, greeting that I am successfully installed eprints. But I am sure nothing was installed on to the harddrive yet. After a while of explorering the live system, and with limited documentation, I decide to install eprints software inside a running linux system instead.

I have a virtual machine with kubuntu 8.04 running, downloaded the eprints3.31 deb package file from the eprints site and executed it. It started to download dependency packages from the web, totally 68. Unfortunately, it claimed not be able to download one of the package. I re-tried many times with no luck. I decided to install using the apt-get.

Following the guide, I added the eprints source and started to download the packages. This time, it successfully finished installing with no error. But just as I was following the next bit of guide to get started with the eprints, the guide became ambiguous and inaccurate again - the eprint user was not found.

I need to spend more time on this another time.

Some investigation into the subject

2009-06-18T05:30:00.000-07:00

Yesterday, I had a chance to look at the current development in the area of social networking. I conclude the following:

One way of doing a project in social networking can be build a software to extract the network relations between people in a repository/database, hence use those extracted information to facilitate expert finding, visualisation of the repository and the academic network.

In doing so, there are many challenges: find and extracting a researcher's profile from a web page, researcher name disambiguation, topic modelling

There are some one off development for the social network extraction from a particular database, ArnetMiner(www.arnetminer.com) is an example.

In the scope of this project, I can develop an extension to the eprints repository providing similar functionality to arnetminer.

Master Project topic decided

2009-06-17T05:38:00.000-07:00

The project was officially started from this Monday.

Tuesday afternoon, we gather together with our supervisor and discussed the potential projects that our web science-interested students can do.

10 of us got given 9 topics to choose from, here's the ones I am interested in:

Twitter investigation
University social networking
Life Guide data visualisation
Google Wave understanding

Since everyone wants to do the interested ones, there were many overlaps in the choices. I have to give up my top choice to go for social networking one. Hot projects were split into small ones to accommodate the demand.

My choice -- university social network, had split into three projects. One doing an interface for different age group, one utilising the external social network sources to form a university social network, I am using the internal resource, including the university eprint, to form the social network.

There is a project description and project plan due this Friday.