Welcome back Cyber warriors đŞ. Today, Twitter!
đ¤ On September 7, 2011, Twitter announced that it had 100 million active users logging in at least once a month and 50 million active users every day. On March 31, 2014, Twitter announced that there were 255 million monthly active users (MAUs) and 198 million mobile MAUs đ¤.As of Q1 2019, Twitter had more than 330 million monthly active users đââď¸.
Today we will using a tool Twint, to scrape info on anyone we want on twitter, all we will need is their username đ and we can access their entire twitter history without using twitter or needing to have an account whatsoever đš
Twint utilizes Twitterâs search operators to let you scrape Tweets from specific users đ˛, scrape Tweets relating to certain topics, hashtags & trends, or sort out sensitive information from Tweets like e-mail and phone numbers đľď¸ââď¸ . I find this very useful, and you can get really creative with it too. Twint also makes special queries to Twitter allowing you to also scrape a Twitter user's followers, Tweets a user has liked, and who they follow without any authentication, API, Selenium, or browser emulation đĽˇ
Some of the benefits of using Twint vs Twitter API đ¤:
Can fetch almost all Tweets (Twitter API limits to last 3200 Tweets only);
Fast initial setup;
Can be used anonymously and without Twitter sign up;
No rate limitations.
So Cyber Morans, fire up your Kali Linux boxes and lets get cracking đŞ
Installation đ
Open a new terminal on your linux machine and run:
git clone --depth=1 https://github.com/twintproject/twint.gitafter the download in complete; cd twint to get into the folder then,
pip3 install . -r requirements.txt
or
pip3 install twintCLI Use: Basic Examples and Combos đŚ
đtwint -u username - Scrape all the Tweets of a user (doesn't include retweets but includes replies).
đtwint -u username -s pineapple - Scrape all Tweets from the user's timeline containing pineapple.
đtwint -s pineapple - Collect every Tweet containing pineapple from everyone's Tweets.
đtwint -u username --year 2014 - Collect Tweets that were tweeted before 2014.
đtwint -u username --since "2015-12-20 20:30:15" - Collect Tweets that were tweeted since 2015-12-20 20:30:15.
đtwint -u username --since 2015-12-20 - Collect Tweets that were tweeted since 2015-12-20 00:00:00.
đtwint -u username -o file.txt - Scrape Tweets and save to file.txt.
đtwint -u username -o file.csv --csv - Scrape Tweets and save as a csv file.
đtwint -u username --email --phone - Show Tweets that might have phone numbers or email addresses.
đtwint -s "NelsonHavi" --verified - Display Tweets by verified users that Tweeted about Nelson Havi.
.
đtwint -g="48.880048,2.385939,1km" -o file.csv --csv - Scrape Tweets from a radius of 1km around a place in Paris and export them to a csv file.
đtwint -u username -es localhost:9200 - Output Tweets to Elasticsearch
đtwint -u username -o file.json --json - Scrape Tweets and save as a json file.
đtwint -u username --database tweets.db - Save Tweets to a SQLite database.
đtwint -u username --followers - Scrape a Twitter user's followers.
đtwint -u username --following - Scrape who a Twitter user follows.
đtwint -u username --favorites - Collect all the Tweets a user has favorited (gathers ~3200 tweet).
đtwint -u username --following --user-full - Collect full user information a person follows
đtwint -u username --timeline - Use an effective method to gather Tweets from a user's profile (Gathers ~3200 Tweets, including retweets & replies).
đtwint -u username --retweets - Use a quick method to gather the last 900 Tweets (that includes retweets) from a user's profile.
đtwint -u username --resume resume_file.txt - Resume a search starting from the last saved scroll-id.
đtwint -u noneprivacy --csv --output none.csv --lang en --translate --translate-dest it --limit 100 - get 100 english tweets and translate them to italian.
Twitter can shadow-ban accounts, which means that their tweets will not be available via search. To solve this, pass --profile-full if you are using Twint via CLI or, if are using Twint as module, add config.Profile_full = True. Please note that this process will be quite slow.
Twint can be imported as a library on a python script to fully automate this process. Lemme show you.
still on that terminal, type nano twintexample.py and hit enter;
This opens a text editor (nano) where we will write some python đ code đ
when you hit enter đ
Now lets write some python đ code; We will be searching a web developer and tech memelord on twitter kirinyetbrian and we will be looking for all his tweets with the term 'dev' in them.
PS: đĽ You can use any twitter username you want as long as you have permission. This is because this tool will scrape tweets from private and none private users as well those that have blocked you đĽ
We will import twint and use variable 'c', assign that to the twint.Config() function (It's in the twint lib we imported first) so to assign the script's search parameters (as args) and then call the function at the end. dont forget the (c) - these are the function's arguments.
ctrl + x to exit nano
y then enter to save script
Now lets run it đ python3 Twintexample1.pyđ;
Unfortunately âď¸, i cannot show the results here since I DONT have Brian's permission. However, run it yourself, use it on other users and even enhance the script to do more than that. You can get creative with this tool like using it to track insightment of violence or continue to monitor people who have blocked or are private.
I personally use it to look for events using a key word like 'hatupangwingwi' to track UDA campaigns across twitter (purely for oversight) or search key phrases to uncover malicious campaigns like fake forex exchange promotions, trace and track misinformation campaigns etc. Twint's power is your imagination. Try more and have fun with other đ scripts like;
This outputs the results into a .csv file đ
This translates the first 100 tweets from English to Italian đ
Conclusion
Twint can be used to scrape information on twitter regardless their privacy settings. It can be useful to relentlessly pursue some malicious activities, by law enforcement đŽââď¸ to try and gather information, hold hate-speech accountable, trace the origins đľď¸ââď¸ and spread of misinformation ...etc.
Thank you for your time, Like and leave a comment/review and as always. I will see you in the next one but till then, stay awesome! đ đŞ
