Sentiment analysis tools for humanities researchers
by Fiona Tweedie
Identifying and analysing sentiment in social media comments quickly became a valuable tool for marketers. What better way to gauge the popularity of products, from soft drinks to TV shows, than the conversation among users on Twitter, FaceBook, YouTube… It could be the exception to the rule, “Don’t read the comments”. But are there other uses for this sort of analysis. What could you learn by analysing all of the Tweets tagged #ausvotes (or even #sausage) during the last federal election?
This map shows where different terms have been used in hateful Tweets across the US. Image from http://users.humboldt.edu/mstephens/hate/hate_map.html
Sentiment analysis uses natural language processing (NLP) to identify the attitude being expressed in text. We’ve looked at a number of sentiment analysis tools to get a sense of the strengths and weaknesses of using automated tools to try to undertake this analysis. Sarcasm tends to be really confusing to sentiment analysis tools, and it’s important that researchers are able to edit the lists of terms in use to improve accuracy for their dataset - for instance to reflect in jokes or idioms used by a particular group.
Tools we looked at
At the most basic end of the spectrum are tools like Sentiment140, which allow you to search a term or hashtag in Twitter and return an overall proportion of positive and negative tweets. It’s easy to use, but you can’t train it and you can’t BYO data. It also wasn’t very accurate when tested.
More complex tools are able to identify and categorise key words, visualise relationships between participants in a conversation, such as who replies to whom or who is most retweeted, and can be trained to provide greater accuracy.
Netlytic, for instance, enables keyword extraction and visualisation and has some network visualisation, but this isn’t strong. For instance, it didn’t readily identify Twitter handles as individuals during testing, especially if they didn’t resemble a person’s name. It was quite easy to edit the sentiment terms used and import datasets and to conduct real-time searches of Twitter.
Positive emotion words visualised by Netlytic
EtcML has a wide range of classification tools available for use and the ability to train them to your dataset. It doesn’t seem to offer the range of options around keyword identification that Netlytic does, although you can create custom classifiers which will be sensitive to topic-specific vocabulary, such as ‘bland’ or ‘soggy’ in restaurant reviews.
Sentistrength assesses both the sentiment and the strength of sentiment expressed in short texts. The emotion words that it identifies and the strength attached to them can be edited. It is also capable with identifying topics and keywords and assessing the sentiment attached to each.
From analysis to insight
Sentiment analysis offers researchers a tool to extract information from a large collection of texts, but the data that it generates will need to be tested and interpreted. In some contexts, such as comment on a panel discussion, participants may be simply reflecting on what was said, which will yield large numbers of neutral comments without giving a clear sense of how much they are enjoying the discussion or the event.
