Sentiment Analysis: The Only Guide You’ll Ever Need

The use and popularity of sentiment analysis have been on a sharp rise for the past four years, and it is something you are going to start hearing more about; especially in the marketing and SEO sectors. As digital communication continues to expand, people are becoming more interested in learning more about the way companies are viewed by others, as well as how they communicate with us. Sentiment analysis allows you to delve right into the world of tone and context, but it’s not without its faults. In this guide, the only one you’ll need, everything is fully explained with handy examples.

Screenshot 2019 02 06 at 18.46.53

What is Sentiment Analysis?

The simplest definition of sentiment analysis is the process of analysing pieces of text to determine their emotional tone; so, you can tell if they have a positive, negative, or neutral attitude towards the topic they are writing about. However, there is more to the concept of sentiment analysis.

It is also known by the name opinion mining, and is often found within a natural language processor. While it has been in development for a little while now, the increased amount of public and private data being shared on the web means that it is becoming something that more and more people are interested in learning more about; especially in relation to companies and politicians.

The Depth of Sentiment Analysis

When we talk about the depth of sentiment analysis, it also refers to the scope. There are actually three levels of scope that this form of analysis can be applied to:

–        Document Level. This takes a look at the sentiment within an entire document.  

–        Sentence Level. This looks at the sentiment of a single sentence.

–        Sub-Sentence Level. This looks at the sentiment of sub-expressions within a sentence.

After looking at the scope, it’s good to delve into the different types of sentiment analysis, allowing further exploration of just how deep this actually goes. Most systems will focus on polarity, which is whether the emotion is positive, negative, or neutral. However, there are others that can detect specific emotions as well as intentions (interested or uninterested).

Here are some of the top forms of sentiment analysis:

–        Fine-Grained. This focuses on the traditional polarity results but gives you a little more detail. So, instead of just having positive, neutral, and negative, you can also add very positive and very negative to the mix for results with extra detail.

–        Emotion Detection. This system usually uses a complex machine learning algorithm to tell you the specific emotion behind the text, and it can even be used alongside polarity. However, it doesn’t take into account that everyone expresses themselves differently and may misinterpret sarcasm.

–        Aspect-Based. This uses the polarity system, but instead of looking at the overall emotional tone, it focuses on the emotions relating to a specific feature, or aspect, of the product. So, if someone said that Crème Eggs taste awful now because of the new recipe, the analysis would focus on the negativity associated with the new taste.

–        Intent. This is all about looking at what people do with the text they write as opposed to what they are saying. It helps machines to differentiate between a complaint and a request for help, so that the emotion behind the text is not misinterpreted, or that the margin is at least a lot smaller.

–        Multilingual. This can be very difficult, but it is possible, and resultantly it is often useful for companies that have an international presence. Some of it may require you to code it yourself, but most of the resources are readily available online for you to download and implement.

How Does it all Work?

Sentiment analysis is all about using a combination of methods and algorithms in order to keep things flowing smoothly and track mentions. You can create your own program if you are really savvy in the field, but if not, there is software out there that you can download and purchase for your own use. Brand24 actually have a really good and reliable software that is quite inexpensive when compared to others out there. It’s good to know how things work though, and this section takes you through the basics.

There are three main classifications for the methods and algorithms that make up sentiment analysis, and they are as follows:

–        Rule-Based. This is all about sentiment analysis working with a set of rules that have been created manually and without an automatic system.

–        Automatic. This relies on the work of machine learning techniques that process the data and then learn from it to create accurate readings.

–        Hybrid. This combines the above systems to create one that, essentially, has the best of both worlds. In many ways, it is the best option of the three.

With the basics explained in the short descriptions above, here is a little more information on how each of the three work, starting with the rule-based classification. This tends to use a type of scripting language that is able to define polarity and subjectivity within a written opinion and to do this a clearly defined set of rules needs to be written into the text.

While it can work very well, it is also quite a basic system because it is not usually able to comprehend the way in which words are combined. While there is the option to create more advanced processes, it can be difficult to maintain because there are going to be manual updates for things like new expressions and vocabulary.

The automatic methods don’t need manually created algorithms, and instead, rely on machine learning to determine the polarity of the text in question. The model needs training, and it will take time because it has to be fed information so that it can learn how to correctly determine tone and emotion, but the maintenance and overall implementation tends to be less work than doing it all manually. Obviously, there is a little room for error because machines are not currently able to fully comprehend the range and depth of human emotion, but on the whole,  it is quite an accurate and reliable system to use.

Why Would You Use it?

There is evidence to suggest that 80% of data in the world is not structured or organised, which is a shockingly high percentage. A great deal of this comes from text-based data like emails, social media posts, support tickets, and chats. These can be really tricky to sort through and organise correctly, as well as incredibly time-consuming and expensive to analyse. This is where sentiment analysis systems come in because they are there to help you make sense of everything and process them, making things easier and much more efficient.

As a result, it has a good number of advantages to it:

–        Scalability. It allows for data to be processed quickly and in a cost-effective manner, which saves you a lot of time that can be spent focusing on other aspects of your business.

–        Real-Time Analysis. All of the information appears in real-time, which means that you can assess the views of customers live, and are also able to handle situations, like a PR crisis, quickly and effectively.

–        Consistent Criteria. We all read things differently, and we only really agree on the sentiment behind text around 60% of the time. An algorithm like this one means that the tone is no longer seen in a subjective manner and a more accurate reading can be taken.

In addition to this handy information, here are a few reasons why people use sentiment analysis for their business:

–        They can identify negative mentions and comments, allowing them to assess the situation and see if they are able to make improvements to the way their company is run.

–        They can track customer reactions to changes in products, as well as try to stop social media crises from occurring.

–        They can track users that are incredibly happy, showing them where they are going right, but also giving them the chance to find potential brand ambassadors for further promotion of their products.

Issues with Sentiment Analysis

Of course, sentiment analysis is not without its challenges, and there are a few key ones that it faces. As the technology continues to evolve, the hope is that these will effectively be eliminated, but for now, they remain a bit of an obstacle.

#1 Context

When sentiment analysis starts scanning text, it is not able to take into account the context of the words or expressions being used. Unless the context in question is specifically mentioned as well, machines are not able to learn about it either. This can change the polarity of the result when the scan is finished. It is possible to reduce the margin for error with context, but it requires a lot of work.

#2 Sarcasm

This is hard for many humans to detect, let alone a complex algorithm, so it is no surprise that sarcasm (and irony) are difficult for sentiment analysis systems. Again, it is something that can cause results to show a different polarity to the one intended by the author, which can skew the overall readings slightly. There is no text cue to help a machine learn about sarcasm because it is a more complex emotion and tone that they are currently unable to read or comprehend.

#3 Comparisons

This tends to refer to comparing a new product to an old one or one from a competitor, and it is another area where the algorithm can become confused. It’s another case of context, and it can be quite hard to read. Some comparisons do not need any contextual cues in order to be read, but others do, and unless you have done a lot of pre-processing, you might need to look through some of them manually.

#4 Emojis

There are two types of emoji; Western and Eastern. Both forms are used frequently across social media and can be difficult for sentiment analysis software to read. There is a way around this though, and it requires all emojis to be pre-processed so that the algorithms can take them into account when it is reading through the text. It’s a lot of work, but it can help to improve the accuracy of the readings.

#5 The Subjectivity of Neutrality

The reason this one can be complicated is because the definition of neutral text is very subjective, and down to what individuals determine it to be. However, to make things easier, it is good to look at a general definition of the term. Here are some things you can expect to see in a neutral mention or comment:

–        Desires and wishes: things a customer would like to see implemented in the future. Not comparisons though, as these tend to be positive or negative.

–        Irrelevant information.

–        Objective texts that do not contain explicit opinions on the topic or product.

The Accuracy of Sentiment Analysis

This is probably the most important question because the accuracy of sentiment analysis means a lot. Yes, there are some issues and challenges with this technology, but even humans have a hard time with context and tone a lot of the time; especially in text form as opposed to the spoken word. You will find that sentiment analysis is accurate around 70-80% of the time, and while this is not perfect, it is an incredibly high percentage, and one that warrants a seal of approval in terms of reliability. It can make a huge difference to your business, and while the need for manual work will be needed from time to time, it’s really useful to see the general opinions and feelings held by your customers. Is sentiment analysis worth it? Absolutely.

Examples of Sentiment Analysis: Brands, Politics, and People

This final section is both a great source of examples taken from Brand24 (social and web mentions tool), but also quite a bit of fun to go through. We have gathered data for companies like RyanAir, as well as political movements like Brexit, and even the general opinion on Theresa May. Scrolling through here will provide you with the perfect opportunity to see Sentiment Analysis in action, including some of the challenges that were mentioned earlier on in this piece.

Screenshot Time!

You might remember that last year Ryanair had thousands of delayed and cancelled flights, costing customers thousands of pounds and leading to some pretty nasty comments on social media. There is a lot of debate as to whether the airline handled the situation well, but since then the delays have continued, and so has negative feedback on social media. It does make me wonder if they actually use Sentiment Analysis tools.

Screenshot 2019 02 06 at 18.54.52

Screenshot 2019 02 06 at 18.55.46

Screenshot 2019 02 06 at 18.56.47

Screenshot 2019 02 06 at 18.57.50

Screenshot 2019 02 06 at 18.59.14

Screenshot 2019 02 06 at 19.02.06

Sentiment Analysis

Theresa May vs Ryanair

Data back from 2018 Dec

This comparison is just a little fun really, and it’s an interesting graph to look at. You can read a little more about why we chose May and Brexit below.

Screenshot 2019 02 06 at 19.04.01

Screenshot 2019 02 06 at 19.05.35

Screenshot 2019 02 06 at 19.07.04

Sentiment Analysis Results for Theresa May: Last 30 days

So, why would I choose Theresa May and Brexit for sentiment analysis? Both of these are issues that people are deeply divided on, and the reactions on social media continue to enforce this.

You can see in the pie charts above that opinion is still very much nearly equally divided as to whether they feel positively about both May and Brexit. In the UK, these are massive topics, and they aren’t ones that are taken lightly either. After all, they will completely change the country; and whether it will be for better or worse is still hotly debated.

Below: Number of web and social mentions. The red line is the number of negative mentions.

Screenshot 2019 02 06 at 19.10.17

SEMRush Sentiment Analysis

Semrush sentiment analysis. Negative mentions are manually reviewed. I chose Semrush because it is a market leader in SEO, and there are generally very positive things said about it. Of course, every business has those that dislike it or feel it could be better, but how many of the mentions are actually negative? Sometimes, a manual review is required, and this is something we have looked into.

Screenshot 2019 02 06 at 19.12.55

Below, you will find a few examples of the challenges sentiment analysis has; showing the need for manual review of the results at times. These tweets aren’t actually saying negative things about the company, but because they contain one or more words that are processed by the algorithm as being negative, they are processed as such. By manually reviewing this, you can see how many of the negative mentions are actually legitimate.

Screenshot 2019 02 06 at 19.14.18

Screenshot 2019 02 06 at 19.15.31

The same goes for Ahrefs, as this is another situation where sentiment should be manually reviewed. If it is not, as you can see below, the data can be misleading.

Screenshot 2019 02 06 at 19.17.31

Another example, this time for Craig Campbell ( Not Me )

These tweets aren’t related to me, Craig Campbell, but instead a politician by the same name. Again, a manual check is often required for accurate results. The reason we chose to look at Craig is that this kind of situation can occur frequently; where a company leader has the same name as someone else, and so the mentions get mixed up.

Screenshot 2019 02 06 at 19.21.06

Just for fun, some sentiment analysis for comments about Brexit. After all, who doesn’t like to have a good laugh when facing the end of their country and world as we know it?

Screenshot 2019 02 06 at 19.22.52

To Conclude

After all this, hopefully, you have learned a lot about sentiment analysis and its uses within the business sector (and even outside of it). It is able to help you determine the number of negative mentions a brand or person has, show you how customers react to change, and even let you know where the happy customers are. While it can be a great tool for getting to know your audience (as well as seeing how competitors are doing), there are a few faults.

The main issue is that sentiment analysis it’s not always accurate at detecting the emotion behind the words, and sometimes a positive message can be interpreted as a negative one because it contains a “bad” word. Of course, as development continues, there is hope that it will be able to overcome these small challenges, and even with them, it remains a pretty revolutionary system that’s quick and easy to use.

Again special thanks to Brand24 for retrieving amazing data. You can read my review about Brand24 tool here.

This article was provided by Milosz Krasinski.

Need help with your digital marketing or SEO campaign? Find out how we can help by getting in touch today!


seo profile image

Craig Campbell

I am a Glasgow based SEO expert who has been doing SEO for 18 years.

  • social media icon
  • social media icon
  • social media icon

Online Courses