Spark Labs

Welcome to Spark labs bundle.

Viewing the files:

Markdown files are plain text files and can be viewed in any text editor.

For best viewing experience we recommend the following setup

Chrome

• Get Chrome browser from : https://www.google.com/chrome/browser/desktop/

• Install Markdown Preview Plus Plugin for Chrome.
  Go to this url in Chrome & click add to Chrome.
  https://chrome.google.com/webstore/detail/markdown-preview-plus/febilkbfcbhebfnokafefeacimjdckgl?hl=en-US
  If this link does not work, search for markdown-preview-plus in https://chrome.google.com/webstore

• Once installed, go to 'Window --> Extensions' menu of Chrome ; Find 'Markdown Preview Plus' plugin ; set 'Allow access to file urls'

• Open any *.md file using Chrome (File --> Open)

Editors

We recommend using a 'programmer's editor' to view these files. These editors have integrated file browser and allows navigating through files quickly.

• For Windows : Sublime, NodePad++, Textpad

• For Mac : Sublime, TextWrangler

• For Linux : Sublime, GEdit, Vim

---

Labs

1. Scala Primer (01-scala)

   • 1.1 - Scala shell
   • [1.2 - Functions]
   • 1.3 - File IO
   • 1.4 - Higher Order Functions
   • 1.5 - Vending Machine class
   • 1.6 - Unit testing with SPECS2

2. Spark Intro (02-intro)

   • 2.1 - Install and run Spark
   • 2.2 - Spark Shell

3. RDDs (03-rdd)

   • 3.1 - RDD basics
   • 3.2 - Working with multiple RDDs
   • 3.3 - Key Value Pair RDDs
   • 3.4 - Map Reduce
   • 3.5 - Clickstream processing
   • 3.6 - Caching

4. API (04-api)

   • 4.1 - Submit first application
   • 4.2 - Mapreduce using API

5. Spark SQL (05-sql)

   • 5.1 - Dataframes
   • 5.2 - SQL
   • 5.3 - Data formats

6. MLLib (06-mllib)

   • 6.1 - Kmeans
   • 6.2 - Recommendations
   • 6.3 - Classification

7. GraphX (07-graphx)

   • 7.1 - Influencers (Twitter)
   • 7.2 - Shortest path (in LinkedIn)

8. Streaming (08-streaming)

   • 8.1 - Streaming over TCP
   • 8.2 - Windowed Count
   • 8.3 - Kafka Streaming

9. Operations (09-ops)

   • 9.1 - Cluster setup