Apache Spark Analytics

Video Game Sales Data Analysis

Bhavesh

Project idea – The idea behind this project is to analysis Video Game Sales and Dive into data on popular video games using the following dimensions such as Year, Platform, Publisher and Genre

Problem Statement or Business Problem

Visualizes sales & platform data on video games that sold more than 100k copies.
Dive into data on popular video games using the following dimensions:

  • Year
  • Platform
  • Publisher
  • Genre

Attribute Information or Dataset Details:​

  • rank: integer (nullable = true)
  • name: string (nullable = true)
  • platform: string (nullable = true)
  • year: string (nullable = true)
  • genre: string (nullable = true)
  • publisher: string (nullable = true)
  • na_sales: double (nullable = true)
  • eu_sales: double (nullable = true)
  • jp_sales: double (nullable = true)
  • other_sales: double (nullable = true)
  • global_sales: double (nullable = true)

Technology Used

  1. Apache Spark
  2. Spark SQL
  3. Scala
  4. DataFrame-based API
  5. Databricks Notebook

Introduction

Welcome to this project on Video Game Sales Data Analysis in Apache Spark Analytics using Databricks platform community edition server which allows you to execute your spark code, free of cost on their server just by registering through email id.

In this project, we explore Apache Spark on the Databricks platform.

I am a firm believer that the best way to learn is by doing. That’s why I haven’t included any purely theoretical lectures in this tutorial: you will learn everything on the way and be able to put it into practice straight away. Seeing the way each feature works will help you learn Apache Spark thoroughly by heart.

We’re going to look at how to set up a Spark Cluster and get started with that. And we’ll look at how we can then use that Spark Cluster to take data coming into that Spark Cluster, a process that data, and analyze the data in Databricks platform. That’s pretty much what we’re going to learn in this tutorial.

In this project, we will be performing Medical Appointment Data Analysis

We will learn:

  • Preparing the Data for Processing.
  • Basics flow of data in Apache Spark, loading data, and working with data, this course shows you how Apache Spark is perfect for a Data Analysis job.
  • Learn the basics of Databricks notebook by enrolling in Free Community Edition Server

The goal is to provide you with practical tools that will be beneficial for you in the future. While doing that, you’ll develop a model with a real use opportunity.
I am really excited you are here, I hope you are going to follow all the way to the end of the Project. It is fairly straight forward fairly easy to follow through the article we will show you step by step each line of code & we will explain what it does and why we are doing it.

Free Account creation in Databricks

Click here

Creating a Spark Cluster

Click here

Basics about Databricks notebook

Click here

Loading Data into Databricks Environment

Click here

Download Data

Click here

Load Data in Dataframe

%scala

val Video_Game_Sales = spark.read.option(“inferSchema”,”true”).option(“header”, “true”).csv(“/FileStore/tables/video_game_sales.csv”)

display(Video_Game_Sales)

Print Schema of Dataframe

%scala

Video_Game_Sales.printSchema()

root

|– rank: integer (nullable = true)

|– name: string (nullable = true)

|– platform: string (nullable = true)

|– year: string (nullable = true)

|– genre: string (nullable = true)

|– publisher: string (nullable = true)

|– na_sales: double (nullable = true)

|– eu_sales: double (nullable = true)

|– jp_sales: double (nullable = true)

|– other_sales: double (nullable = true)

|– global_sales: double (nullable = true)

Creating Temporary View

%scala

Video_Game_Sales.createOrReplaceTempView(“Video_Game_Sales”)

Exploratory Data Analysis or EDA

Rise & Fall of Video Game Consoles

Top 10 Games (by Global Sales)

Top 25 Platforms (by Global Sales)

Top 10 Consoles (By Global Sales)

Popular Genres Across Platforms

Top 10 Games: Proportion of Sales in Markets

%sql

select name,
sum(na_sales) as North_America,
sum(eu_sales) as Europe,
sum(jp_sales) as Japan,
sum(other_sales) as Other_Sales,
sum(global_sales) as Global_Sales
from Video_Game_Sales
group by name
order by global_sales desc
Limit 10;

Games per Genre

Games per Genre over time

Total Sales per Market (Grouped by Genre)

%sql

select genre,
sum(na_sales) as North_America,
sum(eu_sales) as Europe,
sum(jp_sales) as Japan,
sum(other_sales) as Other_Sales,
sum(global_sales) as Global_Sales
from Video_Game_Sales
group by genre
order by global_sales desc

By Bhavesh