Blog - Page 3 of 8 - Projects Based Learning

Practice Apache Superset without Installing

Apache Superset is a modern data exploration and visualization platform. Superset is fast, lightweight, intuitive, and loaded with options that make it easy for users of all skill sets to explore and visualize their data, from simple line charts to highly detailed geospatial charts.PresetPreset Cloud is a fully hosted, hassle free cloud service for Apache Superset™. Get started for free today!www.preset.ioWe should start with Starter Plan Hassle free Superset in the cloud, best for small teams.Free: Forever, for up to 5 usersFeatures:Unlimited dashboards and chartsNo-code chart builderCollaborative SQL editorOver 40 visualization typesChart and dashboard cachehttps://youtu.be/49ItnEXsN7M

Install Apache Spark On Ubuntu

Bigdata HadoopApache Spark

With this tutorial, we will learn the complete process to install Apache Spark 3.2.0 on Ubuntu 20. Prerequisite: Spark runs on Java 8/11, Scala 2.12, Python 3.6+ and R 3.5+. Python 3.6 support is deprecated as of Spark 3.2.0. Java 8 prior to version 8u201 support is deprecated as of Spark 3.2.0. For the Scala API, Spark 3.2.0 uses Scala 2.12. You will need to use a compatible Scala version (2.12.x). Steps for Installing Apache Spark Step 1 - Create a directory for example $mkdir /home/bigdata/apachespark Step 2 - Move to Apache Spark directory $cd /home/bigdata/apachespark Step 3 - Download…

Marketing Analytics Part 1

Apache Spark AnalyticsApache Spark, Apache Spark Analytics, Data Visualization, Databricks, Scala, Spark SQL

Marketing analytics consists of both qualitative and quantitative, structured and unstructured data used to drive strategic decisions in relation to brand and revenue outcomes. Overall goalYou're a marketing analyst and you've been told by the Chief Marketing Officer that recent marketing campaigns have not been as effective as they were expected to be. You need to analyze the data set to understand this problem and propose data-driven solutions.Section 01: Exploratory Data Analysis Are there any null values or outliers? How will you wrangle/handle them?Are there any variables that warrant transformations?Are there any useful variables that you can engineer with the given data?Do…

Marketing Analytics Part 2

Apache Spark AnalyticsApache Spark, Apache Spark Analytics, Data Visualization, Databricks, Scala, Spark SQL

Are there any useful variables that you can engineer with the given data?Review a list of the feature names below, from which we can engineer:The total number of dependents in the home ('Dependents') can be engineered from the sum of 'Kidhome' and 'Teenhome'The year of becoming a customer ('Year_Customer') can be engineered from 'Dt_Customer'The total amount spent ('TotalMnt') can be engineered from the sum of all features containing the keyword 'Mnt'The total purchases ('TotalPurchases') can be engineered from the sum of all features containing the keyword 'Purchases' The total number of campaigns accepted ('TotalCampaignsAcc') can be engineered from the sum of…

Marketing Analytics Part 3

Apache Spark AnalyticsApache Spark, Apache Spark Analytics, Data Visualization, Databricks, Scala, Spark SQL

NumStorePurchases VS MntGoldProds MntFishProducts Distribution Campaign 1 Campaign 2 Campaign 3 Campaign 4 Campaign 5 Section 03: Data Visualization Products VS Amount Spent Purchases Conclusion Recall the overall goal: You're a marketing analyst and you've been told by the Chief Marketing Officer that recent marketing campaigns have not been as effective as they were expected to be. You need to analyze the data set to understand this problem and propose data-driven solutions...Summary of actionable findings to improve advertising campaign success:Advertising campaign acceptance is positively correlated with income and negatively correlated with having kids/teensSuggested action: Create two streams of targeted advertising campaigns,…

Apache Hive Installation Steps on Ubuntu

Bigdata HadoopApache Hadoop, Apache Hive, Bigdata, Hadoop, Linux

With this tutorial, we will learn the complete process to install Apache Hive 3.1.2 on Ubuntu 20.The Apache Hive data warehouse software facilitates reading, writing, and managing large datasets residing in distributed storage using SQL. Structure can be projected onto data already in storage. A command line tool and JDBC driver are provided to connect users to Hive.Steps for Installing Hadoop on UbuntuStep 1 - Create a directory for example $mkdir /home/bigdata/apachehive Step 2 - Move to hadoop directory $cd /home/bigdata/apachehive Step 3 - Download Apache Hive (Link will change with respect to country so please get the download link from…

Installing Apache Pig on Ubuntu

Bigdata HadoopApache Pig, Linux

Download a recent stable release from one of the Apache Download website https://pig.apache.org/releases.htmlClick on Download A new Page will get open (https://www.apache.org/dyn/closer.cgi/pig) Click on the link as marked in the below image A new page will get Open Click on Latest Folder so the new page will get open Download the file as shown in the image We have downloaded the file in directory /home/dataengineer/apachepig/ Unzip the file using the below commandtar -xvzf pig-0.17.0.tar.gzAdd /pig-n.n.n/bin to your path. Use export (bash,sh,ksh) or setenv (tcsh,csh). For example:$ export PATH=/<my-path-to-pig>/pig-n.n.n/bin:$PATHExecuting Pig Help Command

Apache Hadoop 3.3.1 Installation Steps on Ubuntu (Part 1)

Bigdata HadoopApache Hadoop, Bigdata, Hadoop, Linux

With this tutorial, we will learn the complete process to install Hadoop 3.3.1 on Ubuntu 20.Supported Java VersionsApache Hadoop 3.3 and upper supports Java 8 and Java 11 (runtime only)Please compile Hadoop with Java 8. Compiling Hadoop with Java 11 is not supported: HADOOP-16795 - Java 11 compile support OPENApache Hadoop from 3.0.x to 3.2.x now supports only Java 8Apache Hadoop from 2.7.x to 2.10.x support both Java 7 and 8Required software for Linux include: Java must be installed. Recommended Java versions are described at HadoopJavaVersions. ssh must be installed and sshd must be running to use the Hadoop scripts that…

Apache Hadoop 3.3.1 Installation Steps on Ubuntu (Part 2)

Bigdata HadoopApache Hadoop, Bigdata, Hadoop, Linux

Use the following property in the respective filesFile: nano etc/hadoop/core-site.xml: <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://localhost:9000</value> </property> </configuration> File: nano etc/hadoop/hdfs-site.xml <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> </configuration> File: nano etc/hadoop/mapred-site.xml <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>mapreduce.application.classpath</name> <value> $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value> </property> </configuration> File: nano etc/hadoop/yarn-site.xml <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.env-whitelist</name> <value> JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE, HADOOP_YARN_HOME,HADOOP_HOME,PATH,LANG,TZ,HADOOP_MAPRED_HOME*</value> </property> </configuration> Now check that you can ssh to the localhost without…

Installing Apache Superset on Ubuntu (Linux) Machine

UncategorizedApache Superset, Linux

Installing Superset from Scratch In Ubuntu 20.04 the following command will ensure that the required dependencies are installed: sudo apt-get install build-essential libssl-dev libffi-dev python3-dev python3-pip libsasl2-dev libldap2-dev Python Virtual EnvironmentWe highly recommend installing Superset inside of a virtual environment. pip install virtualenv You can create and activate a virtual environment using: # virtualenv is shipped in Python 3.6+ as venv instead of pyvenv. # See https://docs.python.org/3.6/library/venv.html python3 -m venv venv . venv/bin/activate Installing and Initializing SupersetFirst, start by installing apache-superset: pip install apache-superset Then, you need to initialize the database: superset db upgrade Finish installing by running through the…

Tags