Introduction

Apache Ambari project is aimed at making Hadoop management simpler by developing software for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari provides an intuitive, easy-to-use Hadoop management web UI backed by its RESTful APIs.

Ambari enables System Administrators to:

• Provision a Hadoop Cluster
• Manage a Hadoop Cluster
• Monitor a Hadoop Cluster

Ambari is officially confirmed to work on X86_64 platform with JDK7 on Ubuntu 12/14, CentOS 5/6, SuSE 11, while version for JDK8/Ubuntu16 is still in development.

The challenges for AArch64 porting are:

• Node.js dependencies: Ambari uses old version of node.js (v0.10.44) while AArch64 support only comes to v4.x and later.
• Phantomjs dependencies: Ambari uses phantomjs 1.9.8, while AArch64 support only comes to v2.1.0 and later.
• Jars dependencies: some Java projects which referenced by Ambari have no AArch64 support. 

Build

Sources

• Upstream: https://github.com/apache/ambari.git
• Release-2.2.0 is used in this Wiki

Setup Environment

• Ubuntu 16.04 64bit for AArch64, or CentOS7 64bit for AArch64
• Oracle JDK 1.8.0_111-b14

Pre-Requisites

Dependencies

Build Steps

Install Pre-requisities

For Ubuntu:

sudo apt install python python-dev rpm yum build-essential libfreetype6 libfreetype6-dev fontconfig fontconfig-config libfontconfig1-dev libssl-dev openssl findbugs -y

For CentOS7

sudo yum groupinstall "Development Tools"
sudo yum install python python-devel openssl-devel openssl openssl-libs freetype freetype-devel fontconfig-devel fontconfig gcc gcc-c++ make build autoconf automake cppunit-devel cmake bzip2 rpm-build

Setup maven

The maven comes along with Ubuntu 16.04 is v3.3.9. Ambari requires maven to be 3.0.5. To build v3.0.5, do as:

wget https://archive.apache.org/dist/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
tar xvf apache-maven-3.0.5-bin.tar.gz
cd apache-maven-3.0.5/bin
export PATH=$PWD:$PATH

Make sure the version of Maven is 3.0.5 when the following command is issued.

mvn --version

Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2016-11-02 01:10:51+0000)
Maven home: /opt/maven
Java version: 1.8.0_101, vendor: Oracle Corporation
Java home: /opt/jdk/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-45-generic", arch: "aarch64", family: "unix"

Setup python tools

• For python 2.6, download 
  
  

  wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086
  sudo sh setuptools-0.6c11-py2.6.egg

• For python 2.7, download 
  
  

  wget https://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg#md5=fe1f997bc722265116870bc7919059ea
  sudo sh setuptools-0.6c11-py2.7.egg

Setup nodejs/npm

Nodejs and npm come along with Ubuntu 16.04 are: nodejs@v4.2.6, npm@3.5.2.

For Ubuntu, nodejs/npm can be installed by:

sudo apt-get install -y nodejs npm
cd /usr/bin && sudo ln -s nodejs node
sudo npm install -g brunch@1.7.10

For CentOS7, nodejs/npm need to be built from source.

git clone https://github.com/nodejs/node.git
cd node
git checkout -b 4.2.6 v4.2.6
./configure --prefix=/usr && make -j8
sudo make install
sudo npm install -g brunch@1.7.10

The version of built out binaries are: nodejs@v4.2.6, npm@2.14.12.

As long as they are installed, pom.xml in ambari-admin needs to be changed to reflect these versions. The target nodejs/npm version are defined in "configuration" field of "frontend-maven-plugin".

Build PhantomJS

AArch64 is not supported in phantomjs v1.9.8. Attached patch needs to be applied.

git clone https://github.com/ariya/phantomjs.git
cd phantomjs
git checkout -b 1.9.8 1.9.8
git apply 0001-Patch-to-build-phantomjs-on-AArch64.patch
./build.sh --confirm --jobs 4

When the build is finished, create tar file for deployment

cd deploy
./package.sh

You can test phantomjs build by issuing:

./bin/phantomjs test/run-tests.js

Install phantomjs-1.9.8-linux-aarch64.tar.bz2 to the system and add phantomjs to $PATH. Check if phantomjs is properly installed by doing:

ent-user@ambari-dk:~$ phantomjs --version
1.9.8

Replace frontend-maven-plugin

Ambari uses fronend-maven-plugin@v0.0.16, which doesn't support AArch64. Do following to rebuild this plugin for AArch64.

git clone https://github.com/eirslett/frontend-maven-plugin.git
cd frontend-maven-plugin
git checkout -b 0.0.16 frontend-plugins-0.0.16
git apply frontend-maven.patch
mvn clean -DskipTests install

Replace leveldbjni

levedbjni is used in Ambari-metrics. It only provides x86/x86_64 version in maven repo. So AArch64 version of leveldbjni needs to be built and installed. Sometimes one leveldbjni test may fail, just ignore the failure.

wget http://pkgs.fedoraproject.org/repo/pkgs/snappy/snappy-1.0.5.tar.gz/4c0af044e654f5983f4acbf00d1ac236/snappy-1.0.5.tar.gz
tar -xf snappy-1.0.5.tar.gz
cd snappy-1.0.5
./configure --disable-shared --with-pic --host aarch64-unknown-linux --build aarch64-unknown-linux
make -j4
cd ..
git clone git://github.com/chirino/leveldb.git
git clone git://github.com/fusesource/leveldbjni.git
export SNAPPY_HOME=`cd snappy-1.0.5; pwd`
export LEVELDB_HOME=`cd leveldb; pwd`
export LEVELDBJNI_HOME=`cd leveldbjni; pwd`
cd leveldb
export LIBRARY_PATH=${SNAPPY_HOME}
export C_INCLUDE_PATH=${LIBRARY_PATH}
export CPLUS_INCLUDE_PATH=${LIBRARY_PATH}
git apply ../leveldbjni/leveldb.patch
wget https://raw.githubusercontent.com/google/leveldb/master/port/atomic_pointer.h -O port/atomic_pointer.h
make libleveldb.a

Note: If you face following error on CentOS7
error: #error This file requires compiler and library support for the 
ISO C++ 2011 standard. This support is currently experimental, and must 
be enabled with the -std=c++11 or -std=gnu++11 compiler options.

You have to edit Makefile and add CXXFLAGS +="-std=c++11"

cd ${LEVELDBJNI_HOME}
git checkout -b 1.8 leveldbjni-1.8
mvn clean install -P all -P linux64 -DskipTests=truebn

Build Ambari

To build Ambari, a certain version number should be provided. This version number IS 5-DIGITS, not "4-digits" mentioned on Ambari's Wiki Page. The last digit may vary but the first 3 digits should be same as Ambari source version. In our case this is 2.2.0. Patch is provided to make Ambari built on AArch64. 

git clone https://git-wip-us.apache.org/repos/asf/ambari.git
cd <ambari_root_dir>
git checkout release-2.2.0
git apply ambari_build_aarch64.patch
mvn versions:set -DnewVersion=2.2.0.0.1
pushd ambari-metrics
mvn versions:set -DnewVersion=2.2.0.0.1
popd
mvn -B clean install package rpm:rpm -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl -Drat.ignoreErrors=true

Note 1: Ambari 2.4.2 should also work, just checkout release-2.4.2 and set correct version string in above command lines. For building ambari-2.4.2, apply the attached patch ambari-build_aarch64_2.4.2.patch instead of ambari_build_aarch64.patch.

Note 2: Building Ambari in CentOS may require python2.6. You can install by following above "Setup python tools" section, or by creating a symbol link "sudo ln -s /usr/bin/python2.7 /usr/bin/python2.6".

Ambari Server will create following packages

• RPM will be created under AMBARI_DIR/ambari-server/target/rpm/ambari-server/RPMS/aarch64.

Ambari Agent will create following packages

• RPM will be created under AMBARI_DIR/ambari-agent/target/rpm/ambari-agent/RPMS/aarch64.

Ambari Metrics will create following packages

• RPM will be created under  AMBARI_DIR/ambari-metrics/ambari-metrics-timelineservice/target/rpm/ambari-metrics-collector/RPMS/noarch.

Patches

Running Ambari

Run Ambari Server

First, install Pre-Requisities

yum install postgresql

Then install the Ambari Server RPM.

yum install ambari-server/target/rpm/ambari-server/RPMS/aarch64/ambari-server-*.rpm

Initialize Ambari Server:

ambari-server setup

Start up Ambari Server:

ambari-server start

To access Ambari, go to: http://{ambari-server-hostname}:8080

The initial username/password is admin/admin.

Run Ambari Agent

Install the Ambari Agent RPM.

yum install ambari-agent/target/rpm/ambari-agent/RPMS/aarch64/ambari-agent-2.2.0.0-1.aarch64.rpm

Then edit the location of Ambari Server in /etc/ambari-agent/conf/ambari-agent.ini by editing the hostname line.

Start Ambari Agent:

sudo ambari-agent start