Sentiment analysis is the technique used for understanding people’s emotions and feelings, with the help of machine learning, regarding a particular product or service. Sentiment analysis models require a high volume of a specific dataset.
One of the most challenging aspects of creating and training a model is acquiring the right volume and type of sentiment analysis dataset. At upGrad, we have compiled a list of ten accessible datasets that can help you get started with your project on sentiment analysis.
Sentiment Analysis Datasets
1. Stanford Sentiment Treebank
The first dataset for sentiment analysis we would like to share is the Stanford Sentiment Treebank. The dataset contains user sentiment from Rotten Tomatoes, a great movie review website.
It contains over 10,000 pieces of data from HTML files of the website containing user reviews. The sentiments are rated on a linear scale between 1 to 25. One is the most negative, whereas 25 is the most positive sentiment. The dataset is free to download, and you can find it on the Stanford website.
2. IMDB Movie Reviews Dataset
The second dataset on our list is the IMDB Movie Reviews dataset. It has 25,000 user reviews from IMDB. The dataset is classified binary and also contains additional unlabelled data that can be used for training and testing purposes.
The dataset is available to download from Kaggle or Stanford website, labeled ‘Large Movie Review Dataset. If you’re looking for an IMDB user reviews dataset for sentiment analysis, there are plenty of options available. You can choose one according to your purpose and use.
3. Paper Reviews Data Set
The Paper Reviews dataset contains reviews mostly in Spanish and English from a conference on computing. It has a total of 405 instances (N), which is evaluated with a 5-point scale. The evaluation done is as follows:
- -2: very negative
- -1: negative
- 0: neutral
- 1: positive
- 2: very positive
The sentiment score expresses the user’s opinion about the paper. The dataset can be useful in predicting the opinion of academic paper reviews. The dataset is available for download from the University of California website.
Learn Artificial Intelligence Course from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.
4. Twitter US Airline Sentiment
The Twitter US Airline Sentiment dataset, as the name suggests, contains tweets of user experience related to significant US airlines. The dataset includes tweets since February 2015 and is classified as positive, negative, or neutral.
The dataset contains information such as the Twitter user ID, airline name, date and time of the tweet, and the airlines’ negative experiences. The dataset is available for download from Kaggle.
The Sentiment140 dataset for sentiment analysis is used to analyze user responses to different products, brands, or topics through user tweets on the social media platform Twitter. The dataset was collected using the Twitter API and contained around 1,60,000 tweets. The data is sorted into six fields;
- The polarity of the tweet (0 = negative, 2 = neutral, 4 = positive)
- The ID of the tweet
- The date of the tweet
- The query
- The Twitter user
- The textual data contained in the tweet
The dataset can be downloaded from the Sentiment140’s or Stanford’s website. The dataset is useful for brand management, polling, and purchase planning purposes.
6. Opin-Rank Review Dataset
The Opin-Rank review dataset for sentiment analysis contains user reviews, around 3,00,000, about cars and hotels. The dataset comprises user reviews collected from websites such as Edmunds (cars), and TripAdvisor (hotels).
The majority of the dataset contains full reviews from TripAdvisor, approx 2,59,000. Edmunds user reviews stand at approx 42,230. There are comprehensive reviews of hotels in 10 different cities from across the globe, such as Dubai, Chicago, Las Vegas, and Delhi, to name a few. The data fields include the date, review title, and the full review.
Similarly, there are car reviews from Edmund of car models from the year 2007 – 2009. The review data includes the date, author names, favorites, and the full report. The dataset is available to download from the GitHub website.
7. Amazon Product Data
The Amazon product data is a subset of a much larger dataset for sentiment analysis of amazon products. The superset contains a 142.8 million Amazon review dataset. This subset was made available by Stanford professor Julian McAuley.
It provides user reviews from May 1996 to July 2014 for products listed across various categories on Amazon. There is an updated version (2018 edition) available for download. It contains 233.1 million user reviews from May 1996 to Oct 2018.
The old dataset can be downloaded from the University of San Diego website, whereas the new dataset can be found on GitHub. Both datasets contain data points such as ratings, price, product description, and helpful votes, to name a few. The new dataset contains additional data such as technical details and similar product tables.
8. WordStat Sentiment Dictionary
The WordStat Sentiment Dictionary dataset for sentiment analysis was designed by integrating positive and negative words from the Harvard IV dictionary, the Regressive Imagery Dictionary, and the Linguistic and Word Count dictionary. It contains about 15,000 words of data combined.
The dataset takes into account negations to classify user sentiment either as positive or negative. The dataset is available for the public for download. However, you cannot use it for commercial purposes without authorization. You can download the latest version of the dataset from Provalisresearch’s website.
Also Read: Top ML Dataset Project Ideas
9. Sentiment Lexicons For 81 Languages
As the name suggests, the Sentiment Lexicon for 81 languages contains contextual data from Afrikaans to English to Yiddish, for a total of 81 words. The data includes positive as well as negative lexicons for the number mentioned above of languages. The dataset is useful for analysts and data scientists working on Natural Language Processing projects such as chatbots.
10. Bag of Words Meets Bag of Popcorns
The last but not least dataset for sentiment analysis is ‘bag of words meets the bag of popcorns.’ As you may have guessed, this dataset is also related to user sentiment of movies. It consists of 50,000 IMDB reviews. The dataset uses the binary classification for user sentiment. If the IMDB rating is less than 5 for a particular movie, the sentiment score is 0. Similarly, if the rating is greater than or equal to 7, the sentiment score is 1. You can download the dataset from Kaggle.
We hope this blog covering ten diverse datasets for sentiment analysis helped you. If you’re further interested in learning about sentiment analysis and the technologies associated, such as artificial intelligence and machine learning, you can check our Executive PG Programme in Machine Learning & AI course.
What dataset is suitable for sentiment analysis?
Sentiment analysis can be done on both consumer facing or product based datasets. A consumer facing dataset would capture a consumer mindset about events or situations, products or brands with regards to general satisfaction, or even how a consumer feels about a recent event. For example, a dataset from a consumer feedback site that allows you to take a survey and review a product or service. There are many datasets available for sentiment analysis. Some of those include Twitter Sentiment Analysis, Bing Sentiment Dataset, Movie Review Sentiment Classification, IMDb Sentiment Classification, etc.
What are the common challenges with which sentiment analysis deals?
Sentiment analysis is based on opinion mining, a domain that requires the use of linguistic, statistical and machine learning methods. People have different opinions, but they often don't voice their views due to social pressures, fear and lack of time. Sentiment analysis can be a solution, but it provides only an approximate sentiment score. Using sentiment analysis to do sentiment mining is challenging, because we need to explain why a certain text is negative or positive, and not just one number. This is why these methods rarely work very well.
How can you increase the accuracy of a sentiment analysis?
To increase the accuracy of a sentiment analysis, you have to define a sentiment lexicon which is going to help you in recognizing the sentiment of the sentence. Sentiment lexicons allow you to develop some sort of dictionary which contains all the relevant words in the sentence and also the sentiment score associated with it. To acquire a sentiment lexicon, you can use Twitter API to get the tweets. Then you can use Natural Language Processing to find the sentiment of the sentence. You can also use NER to extract the sentiment.