Bhavesh

74 Posts
Installing Apache Flume on Ubuntu

Installing Apache Flume on Ubuntu

System Requirements:Java Runtime Environment - Java 1.8 or laterMemory - Sufficient memory for configurations used by sources, channels or sinksDisk Space - Sufficient disk space for configurations used by channels or sinksDirectory Permissions - Read/Write permissions for directories used by agentThe first step is to create a folder Flume:Make flume directory in /home/dataengineer/mkdir flumecd flume We need to go to the website  https://flume.apache.org/ and click on download. A new webpage will get open click on  apache-flume-1.11.0-bin.tar.gz A new webpage will get open https://www.apache.org/dyn/closer.lua/flume/1.11.0/apache-flume-1.11.0-bin.tar.gz  and copy the link shown to you. Type the below commandwget https://dlcdn.apache.org/flume/1.11.0/apache-flume-1.11.0-bin.tar.gz You will be able to…
Read More
MySQL client and Server Installation

MySQL client and Server Installation

Step 1: Update/Upgrade Package Repositorysudo apt updatesudo apt upgradeStep 2: Install MySQLsudo apt install mysql-serverWhen asked if you want to continue with the installation, answer Y and hit ENTER.Note: If you only want to connect to a remote MySQL server instead of hosting a database on your machine, install only the MySQL Client by running:sudo apt install mysql-client Step 3: Check if MySQL Service Is Runningsudo systemctl status mysql Step 4: Log in to MySQL Serversudo mysql -u root
Read More
Installing Apache Sqoop on Ubuntu

Installing Apache Sqoop on Ubuntu

Step 1) Create a Sqoop directory by using the command mkdir sqoop so that we can download Apache Sqoop.Step 2) Download the stable version of Apache Sqoop (ie Apache Sqoop 1.4.7 in the year 2022) Website URL https://archive.apache.org/dist/sqoop/1.4.7/wget https://archive.apache.org/dist/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gzStep 3) Unzip the downloaded file using the tar commandtar -xvzf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gzStep 4) Edit the .bashrc file by using the commandnano .bashrcStep 5) Enter the following commands below in bashrc file and save itexport SQOOP_HOME="/home/dataengineer/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0"export PATH=$PATH:$SQOOP_HOME/bin Step 6) Execute the below command on the command prompt so bashrc gets activated.source ~/.bashrcStep 7) Check the installed sqoop version using the below commandsqoop version…
Read More
Vehicle Sales Report – Data Analysis

Vehicle Sales Report – Data Analysis

Project idea – The idea behind this project is to analysis and generate Vehicle Sales Report generation and Dive into data on popular vehicles using the following dimensions such as Total Revenue, Total Products Sold, Quarterly Revenue, Total Items Sold (By Product Line), Quarterly Revenue (By Product Line), and Overall Sales (By Product Line) Problem Statement or Business Problem Visualizes Vehicle sales data and generate a report out of it, Dive into data on the vehicle using the following dimensions:Total RevenueTotal Products SoldQuarterly RevenueTotal Items Sold (By Product Line)Quarterly Revenue (By Product Line)Overall Sales (By Product Line)Proportion of Monthly Revenue…
Read More
Video Game Sales Data Analysis

Video Game Sales Data Analysis

Project idea – The idea behind this project is to analysis Video Game Sales and Dive into data on popular video games using the following dimensions such as Year, Platform, Publisher and Genre Problem Statement or Business Problem Visualizes sales & platform data on video games that sold more than 100k copies. Dive into data on popular video games using the following dimensions: Year Platform Publisher Genre Attribute Information or Dataset Details:​ rank: integer (nullable = true) name: string (nullable = true) platform: string (nullable = true) year: string (nullable = true) genre: string (nullable = true) publisher: string (nullable…
Read More
SLACK Data Analysis

SLACK Data Analysis

Project idea – The idea behind this Analysis project is to analysis Slack usage. Problem Statement or Business Problem Slack is a messaging program designed specifically for the office, but has also been adopted for personal use. Slack including persistent chat rooms (channels) organized by topic, private groups, and direct messaging. In addition to these online communication features, Slack integrates with other software. In this tutorial we will try to analyze usage of slack software. Technology Used Apache Spark Spark SQL Scala DataFrame-based API Databricks Notebook Introduction Welcome to this project on Slack Data Analysis in Apache Spark Analytics using…
Read More
Life Expectancy Prediction using Machine Learning – Part 1

Life Expectancy Prediction using Machine Learning – Part 1

Project idea – The idea behind this ML project is to build a model for Life Expectancy and Statistical Analysis on factors influencing Life Expectancy Problem Statement or Business Problem Although there have been lot of studies undertaken in the past on factors affecting life expectancy considering demographic variables, income composition and mortality rates. It was found that affect of immunization and human development index was not taken into account in the past. Also, some of the past research was done considering linear regression based on data set of one year for all the countries. Hence, this gives motivation to…
Read More
Life Expectancy Prediction using Machine Learning – Part 2

Life Expectancy Prediction using Machine Learning – Part 2

Scatter Plot (Life_Expectancy VS Adult_Mortality) Scatter Plot (Life_Expectancy VS Infant_Deaths) Scatter Plot (Life_Expectancy VS Alcohol) Scatter Plot (Life_Expectancy VS Percentage_Expenditure) Scatter Plot (Life_Expectancy VS Hepatitis_B) Scatter Plot (Life_Expectancy VS Under_Five_Deaths) Scatter Plot (Life_Expectancy VS Polio) Scatter Plot (Life_Expectancy VS Total_Expenditure) Scatter Plot (Life_Expectancy VS Diphtheria) Scatter Plot (Life_Expectancy VS HIV_AIDS) Scatter Plot (Life_Expectancy VS GDP) Scatter Plot (Life_Expectancy VS Population) Scatter Plot (Life_Expectancy VS Thinness_1_19_years) Scatter Plot (Life_Expectancy VS Thinness_5_9_years) Scatter Plot (Life_Expectancy VS Income_Composition_of_Resources) Scatter Plot (Life_Expectancy VS Schooling) Scatter Plot (Schooling VS Adult_Mortality) Scatter Plot (Schooling VS Income_Composition_of_Resources) Scatter Plot (Adult_Mortality VS Income_Composition_of_Resources) Collecting all String Columns into…
Read More
Medical Appointment Data Analysis

Medical Appointment Data Analysis

Project idea – The idea behind this Analysis project is to analysis a person makes a doctor's appointment, receives all the instructions, and no-show. Who to blame? Problem Statement or Business Problem Problem A person makes a doctor's appointment, receives all the instructions, and no-show. Who to blame? In this tutorial we will try to analyze why would some patient not show up for his medical appointment and whether there are reasons for that using the data we have. We will try to find some correlation between the different attributes we have and whether the patient shows up or not.…
Read More
Predicting Possible Loan Default Using Machine Learning

Predicting Possible Loan Default Using Machine Learning

Project idea – The idea behind this ML project is to build a model for a Loan Prediction Based on Customer Behavior and determine the risk factor. Problem Statement or Business Problem About CompanyWonderful Dream Housing Finance company deals in all home loans. this ML project is to build a model for a Loan Prediction Based on Customer BehaviorProblemCompany wants to automate the loan risk factor based on customer detail behavior. A loan default occurs when a borrower takes money from a bank and does not repay the loan. Details are Income, Age, Experience, Married/Single, House_Ownership, Car Ownership, Profession, City,…
Read More