Counting Discord Channels Using Selenium29 Dec 2017
For my research project, I wanted to get a better sense of how many Discord servers exist, considering there was no official number out there.
So, I searched on Google for a list of Discord servers and found a few websites.
I started out with finding a way to scrape the first website that was listed when I searched for Discord servers’ list: https://discord.me/servers/1
There wasn’t an official number of Discord servers listed so I investigated how the website looked like. It was fairly simple with buttons “Next” and “Previous” that helped navigate between different pages of Discord servers.
Using Inspect Element, I identified the element associated with the titles of the channels as well as identified the element associated with the “next” button. Then, using Selenium, I constructed a loop which clicked on the next button, waited for 3 seconds, and added all the servers to the list:
# The path works because I have moved the "chromedriver" file in /usr/local/bin browser = webdriver.Chrome(executable_path='/usr/bin/chromedriver') listofChannels= #https://discord.me/servers/1 browser.get('https://discord.me/servers/1') a = 0 #Use infinite loop since the total number of pages is unknown. #At least in jupyter notebooks, it'll stop when it can't find the "Next" button on the last page) while True: names = browser.find_elements_by_class_name("server-name") for name in names: listofChannels.append(name.text) moveforward = browser.find_elements_by_xpath("//*[contains(text(), 'Next')]") moveforward.click() print a a+=1 time.sleep(3)
Then, I used the Pandas library to make sure there were no duplicate servers:
#Find Duplicates using Pandas library cleanList = pd.unique(listofChannels).tolist()
Finally, to tally up all of the servers, I just used Python’s len() function:
count = len(cleanList) print count
In the end, I counted 13,040 servers listed on the discord.me site.