- Download MySql connector jar and place it in Hive library. Setup configuration files for local Hive Setup HDFS for storing Hive data Starting Hive Each Step is described in detail below Validating hadoop: Hadoop version can be checked using below command. $ hadoop version Output: Hadoop 3.0.3.
- Release 2.7.7 available. This is a maintenance release of Apache Hadoop 2.7. It addresses CVE-2018-8009.
At the end of this eight-step process, we will be able to have a local Hadoop instance on our laptop for tests so that we can practice with it.
![Hadoop download for mac Hadoop download for mac](/uploads/1/2/4/3/124384562/393042507.png)
Here is what I learned last week about Hadoop installation: Hadoop sounds like a really big thing with a complex installation process, lots of clusters, hundreds of machines, terabytes (if not petabytes) of data, etc. But actually, you can download a simple JAR and run Hadoop with HDFS on your laptop for practice. It's very easy!
Let's download Hadoop, run it on our local laptop without too much clutter, then run a sample job on it. At the end of this eight-step process, we want to be able to have a local Hadoop instance on our laptop for tests so that we can practice with it.
Our plan:
- Set up
JAVA_HOME
(Hadoop is built on Java). - Download Hadoop tar.gz.
- Extract Hadoop tar.gz.
- Set up Hadoop configuration.
- Start and format HDFS.
- Upload files to HDFS.
- Run a Hadoop job on these uploaded files.
- Get back and print results!
![Download hadoop Download hadoop](/uploads/1/2/4/3/124384562/796797904.png)
Sounds like a plan!
1. Set Up JAVA_HOME
As we said, Hadoop is built, on Java so we need JAVA_HOME
set up.
2. Download Hadoop tar.gz
Next, we download Hadoop!
3. Extract Hadoop tar.gz
Now that we have tar.gz on our laptop, let's extract it.
4. Set Up HDFS
Now, let's configure HDFS on our laptop:
The configuration should be: