# twitter-scraper

**Repository Path**: Yang-Leon/twitter-scraper

## Basic Information

- **Project Name**: twitter-scraper
- **Description**: Scrape the Twitter Frontend API without authentication.
- **Primary Language**: Python
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2021-10-15
- **Last Updated**: 2023-04-27

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Twitter Scraper

![GitHub](https://img.shields.io/github/license/bisguzar/twitter-scraper) ![GitHub contributors](https://img.shields.io/github/contributors/bisguzar/twitter-scraper) ![code size](https://img.shields.io/github/languages/code-size/bisguzar/twitter-scraper) ![maintain status](https://img.shields.io/maintenance/yes/2020)

[🇰🇷 Read Korean Version](https://github.com/bisguzar/twitter-scraper/blob/master/twitter_scraper/__init__.py)

Twitter's API is annoying to work with, and has lots of limitations — luckily their frontend (JavaScript) has it's own API, which I reverse–engineered. No API rate limits. No restrictions. Extremely fast.

You can use this library to get the text of any user's Tweets trivially.

## Prerequisites

Before you begin, ensure you have met the following requirements:

* Internet Connection
* Python 3.6+

## Installing twitter-scraper

If you want to use latest version, install from source. To install twitter-scraper from source, follow these steps:

Linux and macOS:
```bash
git clone https://github.com/bisguzar/twitter-scraper.git
cd twitter-scraper
sudo python3 setup.py install 
```

Also, you can install with PyPI.

```bash
pip3 install twitter_scraper
```

## Using twitter_scraper

Just import **twitter-scraper** and call functions!


### → function **get_tweets(query: str [, pages: int])** -> dictionary
You can get tweets of profile or parse tweets from hashtag, **get_tweets** takes username or hashtag on first parameter as string and how much pages you want to scan on second parameter as integer. 

#### Keep in mind:
* First parameter need to start with #, number sign, if you want to get tweets from hashtag.
* **pages** parameter is optional.

```python
Python 3.7.3 (default, Mar 26 2019, 21:43:19) 
[GCC 8.2.1 20181127] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from twitter_scraper import get_tweets
>>> 
>>> for tweet in get_tweets('twitter', pages=1):
...     print(tweet['text'])
... 
spooky vibe check
…
```

It returns a dictionary for each tweet. Keys of the dictionary;

| Key       | Type       | Description                                                      |
|-----------|------------|------------------------------------------------------------------|
| tweetId   | string     | Tweet's identifier, visit twitter.com/USERNAME/ID to view tweet. |
| isRetweet | boolean    | True if it is a retweet, False othercase                         |
| time      | datetime   | Published date of tweet                                          |
| text      | string     | Content of tweet                                                 |
| replies   | integer    | Replies count of tweet                                           |
| retweets  | integer    | Retweet count of tweet                                           |
| likes     | integer    | Like count of tweet                                              |
| entries   | dictionary | Has hashtags, videos, photos, urls keys. Each one's value is list|

### → function **get_trends()** -> list
You can get the Trends of your area simply by calling `get_trends()`. It will return a list of strings.

```python
Python 3.7.3 (default, Mar 26 2019, 21:43:19) 
[GCC 8.2.1 20181127] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from twitter_scraper import get_trends
>>> get_trends()
['#WHUTOT', '#ARSSOU', 'West Ham', '#AtalantaJuve', '#バビロニア', '#おっさんずラブinthasky', 'Southampton', 'Valverde', '#MMKGabAndMax', '#23NParoNacional']
```

### → class **Profile(username: str)** -> class instance
You can get personal information of a profile, like birthday and biography if exists and public. This class takes username parameter. And returns itself. Access informations with class variables.


```python
Python 3.7.3 (default, Mar 26 2019, 21:43:19) 
[GCC 8.2.1 20181127] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from twitter_scraper import Profile
>>> profile = Profile('bugraisguzar')
>>> 
>>> vars(profile)
{'username': 'bugraisguzar', 'location': 'Kocaeli, Türkiye', 'birthday': None, 'profile_photo': 'https://pbs.twimg.com/profile_images/1116760468633288715/9prl254I_400x400.png', 'name': 'Buğra İşgüzar', 'biography': ('geliştirici', []), 'website': 'bisguzar.com', 'followers_count': 432, 'likes_count': 2468, 'following_count': 240, 'tweets_count': 749}
>>> 
>>> profile.location
'Kocaeli, Türkiye'
>>> profile.name
'Buğra İşgüzar'
>>> profile.username
'bugraisguzar'
```


## Contributing to twitter-scraper
To contribute to twitter-scraper, follow these steps:

1. Fork this repository.
2. Create a branch with clear name: `git checkout -b <branch_name>`.
3. Make your changes and commit them: `git commit -m '<commit_message>'`
4. Push to the original branch: `git push origin <project_name>/<location>`
5. Create the pull request.

Alternatively see the GitHub documentation on [creating a pull request](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request).

## Contributors

Thanks to the following people who have contributed to this project:

* @kennethreitz (author)
* @bisguzar (maintainer)
* @lionking6792



## Contact
If you want to contact me you can reach me at [@bugraisguzar](https://twitter.com/bugraisguzar).


## License
This project uses the following license: [MIT](https://github.com/bisguzar/twitter-scraper/blob/master/LICENSE).