Hortonworks Data Platform Sandbox on AWS

1. Boot an Amazon Linux instance with at least 16GB of RAM
2. Execute the following. Some steps will take a while

## install docker
sudo yum update
sudo yum install docker

## fix docker for importing the sandbox image larger than 10GB

docker daemon --storage-opt dm.basesize=20G

## start docker
sudo service docker start

## confirm docker is working
docker info

## download Hortonwork Sandbox docker image with s3cmd 

wget http://ufpr.dl.sourceforge.net/project/s3tools/s3cmd/1.6.1/s3cmd-1.6.1.tar.gz
tar xzf s3cmd-1.6.1.tar.gz

cd s3cmd-1.6.1
sudo python setup.py install

s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3
Access Key: xxxxxxxxxxxxxxxxxxxxxx
Secret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: xxxxxxxxxx
Path to GPG program [/usr/bin/gpg]:

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP and can't be used if you're behind a proxy
Use HTTPS protocol [No]: Yes

New settings:
  Access Key: xxxxxxxxxxxxxxxxxxxxxx
  Secret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  Encryption password: xxxxxxxxxx
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: True
  HTTP Proxy server name:
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)

Save settings? [y/N] y
Configuration saved to '/root/.s3cfg' 

s3cmd la

s3cmd get s3://hortonwork-sandbox/HDP_2.5_docker.tar.gz

## load docker image
sudo docker load -i HDP_2.5_docker.tar.gz 

sudo docker info 

## confirm image is available
docker images

## download the Hortonwork sandbox startup script

wget https://raw.githubusercontent.com/hortonworks/data-tutorials/1f3893c64bbf5ffeae4f1a5cbf1bd667dcea6b06/tutorials/hdp/hdp-2.6/sandbox-deployment-and-install-guide/assets/start-sandbox-hdp.sh

ls

## modify the sandbox image name in the bash script

sudo vi start-sandbox-hdp.sh

## replace all words "sandbox-hdp" in the sandbox startup bash script with "sandbox"

sudo sh start-sandbox

How to access the sandbox?

http://aws-public-IP:8080/


Services Provided By the Sandbox












Service
URL
Sandbox Welcome Page
http://host:8888
Ambari Dashboard
http://host:8080
ManageAmbari
http://host:8080/views/ADMIN_VIEW/2.4.0.0/INSTANCE/#/
Hive User View
http://host:8080/#/main/views/HIVE/1.5.0/AUTO_HIVE_INSTANCE
Pig User View
http://host:8080/#/main/views/PIG/1.0.0/Pig_INSTANCE
File User View
http://host:8080/#/main/views/FILES/1.0.0/AUTO_FILES_INSTANCE
Apache Zeppelin View
http://host:9995
Hadoop Configuration
http://host:50070/dfshealth.html http://host:50070/explorer.html


The following Table Contains Login Credentials:









Service
User
Password
Ambari, OS
admin
refer to step 2.1
Ambari, OS
maria_dev
maria_dev
Ambari, OS
raj_ops
raj_ops
Ambari, OS
holger_gov
holger_gov
Ambari, OS
amy_ds
amy_ds


Please go to Section 3 to know more about these users.

https://hortonworks.com/hadoop-tutorial/learning-the-ropes-of-the-hortonworks-sandbox/
https://hortonworks.com/tutorials/


Minimize your AWS costs, you can create a snapshot from the volume. Each time you want to launch the sandbox again, you can create the volume from the snapshot and attach it to the EC2, make sure it is attach to the same root device, e.g. /dev/xvda.


To restart the docker image:


sudo docker info


sudo docker images


sudo docker start sandbox


sudo sh start-sandbox-hdp.sh


then you can access to the Hortonwork Ambari login screen by typing the following in your browser:


http://aws-public-IP:8080/


https://hortonworks.com/hadoop-tutorial/learning-the-ropes-of-the-hortonworks-sandbox/