Hadoop Installation on Windows-10 Natively | Light & Fast
Hello Guys,
In this video i have explained in detail, how you can Setup a single node hadoop cluster on you native windows 10 PC.
Many of us are very comfortable with Windows environment, or may be do not have heavy resources on a computer or a laptop, this setup will help you to setup a Hadoop environment, Natively on you windows 10 computer, which will be very light to use, and there will be no longer any need to run those heavy Hadoop Virtual machines.
Jump Timings
=================
Why Windows? : 00:16
Benefits of custom setup : 00:40
Host laptop configuration : 01:50
Check JDK : 02:40
Environment Variable Setup: 03:59
Hadoop Installation: 05:20
Config Files Setup : 07:15
Winutils usage and setup : 12:55
Testing the environment : 14:55
Checking URLs: 16:25
Important links (mentioned in the Video)
=========================================
Hadoop Download : https://hadoop.apache.org/releases.html
Winutils Download: https://github.com/selfgrowth/apache-...
YARN UI (Post setup): http://localhost:8088/cluster
NameNode UI(Post setup) :http://localhost:9870
Property Tags (mentioned in the Video) Replace () with angle brackets in below tags
=======================================
core-site.xml
------------------
(configuration)
(property)
(name)fs.defaultFS(/name)
(value)hdfs://localhost:9000(/value)
(/property)
(/configuration)
mapred-site.xml
---------------------
(configuration)
(property)
(name)mapreduce.framework.name(/name)
(value)yarn(/value)
(/property)
(/configuration)
hadoop-env.cmd
--------------
replace %JAVA_HOME% with the path of the java
C\java\jdk1.8xx (without bin)
hdfs-site.xml
--------------------
(configuration)
(property)
(name)dfs.replication(/name)
(value)1(/value)
(/property)
(property)
(name)dfs.namenode.name.dir(/name)
(value)xxxxxx\data\namenode(/value)
(/property)
(property)
(name)dfs.datanode.data.dir(/name)
(value) xxxxxdata\datanode(/value)
(/property)
(/configuration)
yarn-site.xml
------------------
(configuration)
(property)
(name)yarn.nodemanager.aux-services(/name)
(value)mapreduce_shuffle(/value)
(/property)
(property)
(name)yarn.nodemanager.auxservices.mapreduce.shuffle.class(/name)
(value)org.apache.hadoop.mapred.ShuffleHandler(/value)
(/property)
(/configuration)
Note:
In case you notice any error like
JAVA_HOME not correctly setup up - While using/starting hadoop ,
replace your "C:\Program Files\xxx" to "C:\Progra~1\xxx" in your JAVA_HOME variable. (the space might cause this issue.
