My Project : Spotify-end-to-end-de-project

# NOTE : Watch The Data With Darshil Python DE Course Project Section Video for More Understanding of this Blog

 My Git-Hub: Link.

# Code Of Project => Available in the Git HUB Repos

Darshil Bhai Git Hub  : Link

# Introduction : 

In this project we will build complete ETL pipeline using the spotify API on AWS.

The Pipeline will retrieve data from the spotify API, transform it to desired format, and load it into AWS data store.

The Pipeline will 

1. Retrieve data from the spotify API, 
2. Transform it to desired format, and 
3. Load it into AWS data store.

# Services Used

1. S3 (SIMPLE STORAGE SERVICE) ~ Amazon
2. AWS Lambda
3. Cloud Watch
4. Glue Crawler
5. Data Catlog
6. Amazon Athena

# About DataSet/API

This API contents information about music artist,albums and songs This site --->>> Spotify API.

# Install Packages

pip install numpy
pip install spotify
pip install pandas

Most Important

# Project Execution Flow

Extract data from API -> Lambda Trigger(every 1 Hour) -> Run Extract code -> Store Raw Data -> Trigger Transforn Function -> Transform data and load it using Athena