It is often very difficult for AI researchers to gather social media data for machine learning. Luckily, one free and accessible source of SNS data is Twitter.
Numerous educational organizations, research teams, and independent researchers have scraped tweets from Twitter and made the data available for public use.
From sentiment analysis models to content moderation models and other NLP use cases, Twitter data can be used to train various machine learning algorithms.
Below is a list of some of the best open Twitter datasets for machine learning.
Best Twitter Datasets for Natural Language Processing and Machine learning
A dataset containing tweets about the large tech company, Apple. The tweets in this dataset were compiled using tweets containing the hashtag #AAPL, the reference @apple, and others. The tweets were then divided into positive, negative, or neutral sentiments.
This dataset for machine learning consists of 10,000 tweets which include the hashtag #AvengersEndgame.
This dataset contains 150,000 tweets mentioning Charlottesville or containing the #Charlottesville hashtag.
The Credibility Corpus in French and English was created to analyze information credibility and detect misinformation and rumors. The dataset is comprised of both French and English tweets about rumors.
This dataset is a large corpus of tweets and replies to and from customer service support lines on Twitter.
The Every Donald Trump Tweet dataset is a compilation of every tweet the president has ever posted. The data was later moved to the TrumpTwitterArchive, but can still be accessed.
From FollowtheHashtag, this dataset is a collection of 200,000 geolocated tweets from Tokyo.
Also from FollowtheHashtag, this dataset is a collection of 200,000 geolocated tweets from the United States of America.
The tweets collected for this dataset capture audience reactions for each episode by collecting Game of Thrones related tweets after each episode of season 8 was released.
This is a simple social media dataset comprised of pre-processed tweets for sentiment analysis. The tweets have been organized into positive, neutral, and negative categories.
During an investigation into Russia’s influence on the 2016 US election, Twitter deleted 200,000 Russian troll tweets. This Twitter dataset includes details on both the individual tweets and accounts from which they were posted.
Sentiment 140 is a tool for discovering the overall sentiment for a brand, topic, or product on Twitter. The company has also made their training data available for download on their site.
A simple dataset for sentiment analysis, the SMILE Twitter Emoticon Dataset contains 3,085 tweets each expressing a different emotion: anger, disgust, happiness, surprise, and sadness.
From the SNAP library database at Stanford University, this dataset contains 476 million tweets from 20 million users over a 7-month period.
This Twitter dataset is composed of over 52,000 tweets from the 20 most-followed Twitter profiles. For this dataset retweets were not collected.
The Twitter US Airline Sentiment Dataset contains tweets about major US airlines classified into the following categories: positive, neutral, and negative.
Twitter Friends is a dataset for machine learning which contains user information. The dataset contains the following information: avatar, follower count, friends count, account name, user ID, accounts the user is following, user’s language, last post info, hashtags used by the user, ID of user’s last tweet.
This Twitter dataset contains 5234 news events from Twitter, as well as the tweets talking about those news events.
A Twitter dataset composed of 20,000 rows, Twitter User Data includes the following information: user name, random tweet, account profile, image, and location information.
Including over 10,000 tweets, this dataset was created to build classifiers that identify the language of tweets. Each tweet is annotated as English, non-English, includes code switching, language ambiguity, or automatically generated. The tweets came from 130 countries.
This is a dataset consisting of over 150 million tweets related to COVID-19, beginning from March 11th, 2020. The tweets crawled are of all languages, with English, Spanish, and French being the most prevalent.
Can’t find the data you’re looking for? From linguistic annotation to text classification, translation corpus data, and more, Lionbridge provides a wide array of AI training data services.
Lead image via MORAN on Unsplash