Flight Delays Exercise: Extract/Transform/Load, Transform raw data into semi-structured format
Transform raw data (available as a CSV) into a semi-structured format for consumption using Spark SQL and the Spark Dataframes API providing a dataset covering for 2012 and 2013.
Needed components:
- Raw flights data from DOT website