Skip to main content

Writing Import.io Data to a Database using Python

This article will go over how to write Import.io data to a MySQL database. This method can be altered by changing the SQLAlchemy URI and database connector. You can read more about SQLAlchemy here.

 Dependencies

pip3 install requests pandas mysql-connector

 

  1. Go to the Integrate tab for your Import.io extractor
  2. Copy the CSV from the last successful run endpoint
  3. Replace the endpoint into the below with the CSV endpoint
import requests  
import pandas as pd  
import mysql.connector  
from sqlalchemy import create_engine  
  
API_ENDPOINT = ''  

session = requests.Session()  
response = session.get(url=API_ENDPOINT).content  
  
df = pd.read_csv(io.StringIO(response.decode('utf-8-sig')))  
  
print(df.head())  
  
engine = create_engine('mysql+mysqlconnector://os.environ['MYSQL_USER']:os.environ['MYSQL_PASSWORD']@os.environ['MYSQL_HOST_IP']:os.environ['MYSQL_PORT']/sandbox', echo=False)  
  
# Writing Dataframe to Mysql and replacing table if it already exists  
df.to_sql(name='Importio', con=engine, if_exists = 'replace', index=False)