Many data analysis, SEO exports require the Google search results links for their data analysis. If they want to copy more results, It is very hard to copy results from the browser. The Python programming language is mainly used in the web scraping and data science community. It has modules and tools that we can use in our own projects.
This tutorial will collect and parse Google search results and write the information we have gathered to a CSV file.
Prerequisites
If you don’t have Python on your machine, download and install Python latest version on your machine.
Download Link: Download Python | Python.org
After the successful installation, we need to install googlesearch and pandas packages.
Open your command prompt/terminal and run the below command.
1. Install google search
pip install googlesearch-python
2. Install pandas
pip install pandas
3. Create a new python file and copy/ paste the below script.
from googlesearch import search
import csv
import pandas as pd
import re
from datetime import datetime
print('================================================')
print('GOOGLE SEARCH')
print('================================================')
# Get user search keyword.
searchKeyWord = input('Enter your search keyword: ')
# Get Total number of records.
totalNoOfRecords = input('How many records you need to save in CSV? ')
print('Please wait. Your request is being processed. \n')
resultLinks = []
# Search the keyword in google.
results = search(searchKeyWord, num_results=int(totalNoOfRecords))
#results = '\n'.join([str(elem) for elem in results])
# Convert result into data frame.
df = pd.DataFrame(results)
now = datetime.now()
outputFileName = 'SearchOutput' + now.strftime("%d-%m-%Y") + '.csv'
# Write output in CSV format.
df.to_csv(outputFileName,mode='a', encoding='utf-8', index=False , header=False)
print('================================================\n')
print('Your data has been processed successfully!!. Please check the output in the below file!!\n')
print(outputFileName + '\n')
print('================================================\n')
Terminal Output
CSV Output
Conclusion
We have scraped the Google search links and stored the links that we gathered within a CSV file. Hope it will be very useful for you and save your golden hours.