Spark

• Spark Introduction
  • Features of Apache Spark
  • Spark Built on Hadoop
  • Components of Spark
  • References
• Spark RDD
  • Resilient Distributed Datasets
  • Iterative Operations on MapReduce
  • Interactive Operations on MapReduce
  • Iterative Operations on Spark RDD
  • Interactive Operations on Spark RDD
  • References
• Spark Installation
  • Verifying Java Installation
  • Downloading Scala
  • Installing Scala
  • Downloading Apache Spark
  • Installing Spark
  • Verifying the Spark Installation
  • References
• Spark SQL Introduction
  • Features of Spark SQL
  • Spark SQL Architecture
  • References
• Spark SQL DataFrames
  • SQLContext
  • DataFrame Operations
  • Running SQL Queries Programmatically
  • Inferring the Schema using Reflection
    • Example
    • Start the Spark Shell
    • Create SQLContext
    • Import SQL Functions
    • Create Case Class
    • Create RDD and Apply Transformations
    • Store the DataFrame Data in a Table
    • Select Query on DataFrame
    • Where Clause SQL Query on DataFrame
    • Fetch ID values from agefilter DataFrame using column index
  • Programmatically Specifying the Schema
    • Example
    • Start the Spark Shell
    • Create SQLContext
    • Read Input from Text File
    • Create an Encoded Schema in a String Format
    • Import Respective APIs
    • Generate Schema
    • Apply Transformation for Reading Data from Text File
    • Apply RowRDD in Row Data based on Schema
    • Store DataFrame Data into Table
    • Select Query on DataFrame
  • References
• Spark SQL Data Sources
  • Spark SQL JSON Datasets
  • Spark SQL Hive Tables
    • Start the Spark Shell
    • Create SQLContext Object
    • Create Table using HiveQL
    • Load Data into Table using HiveQL
    • Select Fields from the Table
  • Spark SQL Parquet Files
    • Open Spark Shell
    • Create SQLContext Object
    • Read Input from Text File
    • Store the DataFrame into the Table
    • Select Query on DataFrame
  • References