Setup Guide for

HDP Administration 1

Revision 1.1
Hortonworks University
Overview
The Hortonworks Training Course that you are viewing is taught using a virtual machine
(VM) for the lab environment. Before running the labs in the course, you must complete
the steps in this document. Here is an overview of the steps you will perform:

Download the virtual machine file. The file is very large (~ 8GB) and may take
several hours to download, depending on your Internet connection.

OPTIONAL - You should use a download-manager to download this file

successfully. It assures a successful download of a such a big file. (i.e. igetter -
http://www.igetter.net/downloads.html)

You must unzip this file using 7Zip on Windows platform. (http://www.7-
zip.org/download.html)

You must install VMWare Player to open the VM file on Windows. On Mac, you
can download a free trial of VMWare Fusion at
http://www.vmware.com/products/fusion/.

OPTIONAL - You can also use Oracle Virtualbox, instead of VMWare. Please see
additional steps on the last page for the instruction on Using existing VM on
Oracle Virtualbox.

It is possible that you will need to configure the BIOS on your Windows machine
to enable virtualization.

THE FOLLOWING SETUP INSTRUCTIONS MUST BE COMPLETED AT LEAST 24

HOURS PRIOR TO THE START OF THE CLASS!

IMPORTANT: It is critical for the success of the training that prior to the start
of the class each students computer has the VM file downloaded, imported
into VMWare, and verified that the virtual machine starts on the students
computer and a MapReduce can execute successfully.

IMPORTANT: If any of the following steps fail, or if you have any issues or questions,
please send an email to [email protected].

2 Copyright 2015, Hortonworks, Inc.

System Requirements
Please be Advised:
The downloadable VM for this course is very large (over 5GB) and may take over 2
hours to download based on your Internet speed. In addition, you must have at
least 80GB of hard drive space to successfully install and run the labs.

The following is the recommended minimal system requirements:

Reasonably powerful x86/amd64 hardware. Recent Intel (4th generation i5/i7 or
above preferred) or equivalent AMD processor should do.

RAM: At least 12 GB

Available Hard-disk: at least 80 GB

64-bit OS (Windows 7, Windows 8 and Mac OSX)

VMWare Player for Windows (or VMWare Fusion for Mac)

Virtualization must be enabled on BIOS for Windows based machine

o Procedures to turn on virtualization settings in your computer's BIOS vary
depending on the BIOS manufacturer. Check the information that came
with your computer or go to the computer manufacturer's website.
o Alternatively, to find out whether your computers CPU is capable of
hardware-assisted virtualization, you can download and run the
Hardware-Assisted Virtualization Detection Tool
(http://www.microsoft.com/en-us/download/details.aspx?id=592)
provided by Microsoft.

Copyright 2015, Hortonworks, Inc. 3

Install the Classroom Virtual Machine

Objective: Import the VMWare image into VMWare Player and start the
virtual machine successfully.

Before You Begin: Make sure your hardware meets the necessary
requirements.

Perform the following steps at least one day prior to the start of the course:

Step 1: Install VMWare Player

1.1. On Windows, download the latest 64-bit version of VMWare Player:

http://www.vmware.com/support/download-player.html

On a Mac, download a free trial of VMWare Fusion at

http://www.vmware.com/products/fusion/.

1.2. Run the downloaded installation file.

Step 2: Download the VM

2.1. Using one of the following links, download the VM file. This file is over 5GB
in size and may take more than 2 hours to download, depending on your
internet connection:

US - https://s3.amazonaws.com/vm-us-east/op_management/HDP2.3-Hadoop-
Administration-1.zip

Europe - https://s3-eu-west-1.amazonaws.com/vm-
europe/op_management/HDP2.3-Hadoop-Administration-1.zip

Singpore - https://s3-ap-southeast-1.amazonaws.com/vm-
asia/op_management/HDP2.3-Hadoop-Administration-1.zip

Sydney - https://s3-ap-southeast-2.amazonaws.com/vm-
asia/op_management/HDP2.3-Hadoop-Administration-1.zip

4 Copyright 2015, Hortonworks, Inc.

 2.2. Unzip the file you just downloaded. Please make sure to use 7zip utility for
Windows platform.

Step 3: Open the VM File

3.1. Start the VMWare Player application and select the option to Open a Virtual
Machine:

3.2. Browse to and open the unzipped VM file

3.3. It will take several minutes for the file to be imported into VMWare
Player/Fusion:

3.4. When the import is complete, the VM will show up in the VMWare
Player/Fusion library.

Step 4: Modify the VMs Memory

4.1. Go to VM settings and set the Memory to a size reasonable for your
hardware. For example, if your machine has 12 GB of RAM, then set the base
memory to 10GB. If your machine has 16GB of RAM, then you can increase the
base memory to 12GB.

Step 5: Start the VM

5.1. Click the Play virtual machine link to start the VM.

5.2. If following windows pops up then click on Dont Upgrade & I copied it

Copyright 2015, Hortonworks, Inc. 5

 5.3. Wait for the VM to start - it may take several minutes. When the VM has
started successfully, you should see the following screen:

5.4. Double click on Terminal icon:

5.5. Wait for up to 5 minutes before connecting to the hadoop cluster:

6 Copyright 2015, Hortonworks, Inc.

 # ssh node1

RESULT: You should be connected to the classroom VM running on VMWare

Fusion/Player.

Copyright 2015, Hortonworks, Inc. 7

OPTIONAL Using existing VM on Oracle Virtualbox

Step 1: Download VirtualBox from this

locaiton: https://www.virtualbox.org/wiki/Downloads

Step 2: Verify that your machine/laptop meets the System Requirements

Step 3: Navigate into the unzipped folder that was just extracted, and look for a file with
file extenstion as .vmdk. This file should be a few GB large. You will use this in
later steps.

Step 4: Open VirtualBox

Step 5: Create a New Machine and with an appropriate name e.g. HDP2.3-Pig & Hive
Rev6 OR what ever is relevant for the course, to help you identify the VM later.

Step 6: Select the Type to be Linux

Step 7: Version to be Red Hat 64 bit and Click Continue

Step 8: Select Memory to be at least 10 GB (recommended to be 10-12 GB if your

machine has the capacity. Remember this is 8GB within your total physical RAM;
its the memory allocation of the VM). Click Continue

Step 9: Select Use an existing virtual hard drive file. Heres where youll select
the .vmdk file from Step 3 above.

Step 10: Click Create and wait for few minutes to complete the process.

Step 11: Once the VM is ready to use, go to Settings -> system -> Processor; and make
sure there are 4 CPUs. Save your changes.

Step 12: Start the newly created VM and test the steps to validate your hadoop cluster.

IMPORTANT: If any of these steps failed, or if you have any issues or questions, please
contact to your instructor OR send an email to [email protected].

8 Copyright 2015, Hortonworks, Inc.