Introduction
...
Apache Ambari project is aimed at making Hadoop management simpler by developing software for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari provides an intuitive, easy-to-use Hadoop management web UI backed by its RESTful APIs.
...
- Provision a Hadoop Cluster
- Manage a Hadoop Cluster
- Monitor a Hadoop Cluster
Ambari is officially confirmed to work on X86_64 platform with JDK7 on Ubuntu 12/14, CentOS 5/6, SuSE 11, while version for JDK8/Ubuntu16 is still in development.
The challenges for AArch64 porting are:
...
On this collaborate page we try to explain how to build and install the Ambari on CentOS 7.4 and Debian 9.0
Build
...
Sources
- Upstream: https://github.com/apache/ambari.git
- Release-2.46.2 1 is used in this Wiki
Setup Environment
- Ubuntu 16Debian 9.04 0 64bit for AArch64, or CentOS7 CentOS-7.4 64bit for AArch64Oracle JDK 1.8.0_111-b14
- jdk8u-server-release-1804
Pre-Requisites
Oracle JDK 1.8.0_111-b14jdk8u-server-release-1804
Dependencies
maven@v3.05.53, nodejs@v4.2.6, npm@3 npm@2.514.212, brunch@1.7.10, phantomjs@1phantomjs@2.91.81, python>=2.6, python-dev, rpm, yum, g++
Build Steps
Install Pre-requisities
For UbuntuDebian 9.0:
Code Block |
---|
|
sudo apt install git python python-dev rpm yum build-essential libfreetype6 libfreetype6-dev fontconfig fontconfig-config libfontconfig1-dev libssl-dev openssl findbugs -y |
...
Code Block |
---|
|
sudo yum groupinstall "Development Tools"
sudo yum install git python python-devel openssl-devel openssl openssl-libs freetype freetype-devel fontconfig-devel fontconfig gcc gcc-c++ make build autoconf automake cppunit-devel cmake bzip2 rpm-build |
Setup maven
The maven comes along with Ubuntu 16.04 is v3.To setup maven 3.95. Ambari requires maven to be 3.0.5. To build v3.0.5, do as:
Code Block |
---|
|
wget httpshttp://archivewww-eu.apache.org/dist/maven/maven-3/3.05.53/binaries/apache-maven-3.05.53-bin.tar.gz
tar xvf apache-maven-3.05.53-bin.tar.gz
cd apache-maven-3.05.53/bin
export PATH=$PWD:$PATH |
Make sure the version of Maven is 3.05.5 3 when the following command is issued.
...
Code Block |
---|
|
Apache Maven 3.05.53 (r01de14724cdef164cd33c7c8c2fe155faf9602da3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 20162018-1102-02 0124T19:1049:51+000005Z)
Maven home: /opthome/centos/maven/apache-maven-3.5.3
Java version: 1.8.0_101-release, vendor: Oracle Corporation
Java home: /home/centos/opt/jdkjdk8u/jdk8u-server-release-1804/jre
Default locale: en_USIN, platform encoding: UTF-8
OS name: "linux", version: "4.412.0-45-generic1.1.aarch64", arch: "aarch64", family: "unix" |
For python 2.6, download
Code Block |
---|
|
wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086
sudo sh setuptools-0.6c11-py2.6.egg |
For python 2.7, download
Code Block |
---|
|
wget https://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg#md5=fe1f997bc722265116870bc7919059ea
sudo sh setuptools-0.6c11-py2.7.egg
The python 2.6 did't work for me, hence I have just created a softlink of v2.6 for v2.7 python
$ sudo ln -s /usr/bin/python2.7 /usr/bin/python2.6 |
Setup nodejs/npm
Nodejs and npm come with different versions along with Ubuntu 16.04 are: nodejs@v4.2.6, npm@3.5.2./Debian
For Ubuntu/Debian, nodejs/npm can be installed by:
Code Block |
---|
|
sudo apt-get install -y nodejs npm
cd /usr/bin && sudo ln -s nodejs node
sudo npm install -g brunch@1.7.10
Note that if you are using Debian 9 stretch then please follow the below steps https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions
sudo apt-get install curl
curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
sudo apt-get install -y nodejs
cd /usr/bin && sudo ln -s nodejs node
sudo npm install -g brunch@1.7.10 |
...
As long as they are installed, pom.xml in ambari-admin needs to be changed to reflect these versions. The target nodejs/npm version are defined in "configuration" field of "frontend-maven-plugin".
Build PhantomJS
The following steps explain about AArch64 is not supported in phantomjs v1v2.91.8. Attached patch needs to be applied.1. Note that you have to install all the dependency packages before you proceed further. Refer collaborate page for PhantomJS
Code Block |
---|
|
git clone https://github.com/ariya/phantomjs.git
cd phantomjs
git checkout -b v2.1.91 2.8 1.9.8
git apply 0001-Patch-to-build-phantomjs-on-AArch64.patch
1
./build.shpy -c -confirm --jobs 4j $(getconf _NPROCESSORS_ONLN) |
When the build is finished, create tar file for deployment
...
Code Block |
---|
|
./bin/phantomjs test/run-tests.js |
Install phantomjs-2.1.9.81-linux-aarch64.tar.bz2
to the system and add phantomjs to $PATH. Check if phantomjs is properly installed by doing:
Code Block |
---|
|
ent-user@ambari-dk:~$ phantomjs --version
2.1.9.81 |
Replace frontend-maven-plugin
Ambari uses fronend-maven-plugin@v0.0.16, which doesn't support AArch64. Do following to rebuild this plugin for AArch64.
Code Block |
---|
|
git clone https://github.com/eirslett/frontend-maven-plugin.git
cd frontend-maven-plugin
git checkout -b 0.0.16 frontend-plugins-0.0.16
git apply frontend-maven.patch
mvn clean -DskipTests install |
Replace leveldbjni
levedbjni is used in Ambari-metrics. It only provides x86/x86_64 version in maven repo. So AArch64 version of leveldbjni needs to be built and installed.
Code Block |
---|
|
wget http://pkgs.fedoraproject.org/repo/pkgs/snappy/snappy-1.0.5.tar.gz/4c0af044e654f5983f4acbf00d1ac236/snappy-1.0.5.tar.gz
tar -xf snappy-1.0.5.tar.gz
cd snappy-1.0.5
./configure --disable-shared --with-pic --host aarch64-unknown-linux --build aarch64-unknown-linux
make -j4
cd ..
git clone git://github.com/chirino/leveldb.git
git clone git://github.com/fusesource/leveldbjni.git
export SNAPPY_HOME=`cd snappy-1.0.5; pwd`
export LEVELDB_HOME=`cd leveldb; pwd`
export LEVELDBJNI_HOME=`cd leveldbjni; pwd`
cd leveldb
export LIBRARY_PATH=${SNAPPY_HOME}
export C_INCLUDE_PATH=${LIBRARY_PATH}
export CPLUS_INCLUDE_PATH=${LIBRARY_PATH}
git apply ../leveldbjni/leveldb.patch
wget https://raw.githubusercontent.com/google/leveldb/master/port/atomic_pointer.h -O port/atomic_pointer.h
make libleveldb.a
cd ${LEVELDBJNI_HOME}
git checkout -b 1.8 leveldbjni-1.8
mvn clean install -P all -P linux64 -DskipTests=true |
Build Ambari
To build Ambari, a certain version number should be provided. This version number IS 5-DIGITS, not "4-digits" mentioned on Ambari's Wiki Page. The last digit may vary but the first 3 digits should be same as Ambari source/release version. In our case this is 2.26.01. Patch is provided to make Ambari built on AArch64. Apply all the patches before you are going for the build. You can directly clone and build my AMBARI git repository - https://git.linaro.org/people/naresh.bhat/apache/ambari.git
Code Block |
---|
|
git clone https://gitgithub.linaro.orgcom/legapache/bigdata/ambari.git
cd ambari
git checkout release-2.6.1
Download and apply following patches
git am 0001-ambari-build-aarch64-2.4-aarch.6.1.patch
git am 0002-ambari-metrics-grafana-Add-jdeb-support.patch
git am 0003-ambari-funtest-Add-jdeb-support.patch
git am 0004-ambari-logsearch-Add-jdeb-support.patch
git am 0005-ambari-Add-jdeb-arm64-support.patch
mvn versions:set -DnewVersion=2.46.21.0.0
pushd ambari-metrics
mvn versions:set -DnewVersion=2.46.21.0.0
popd
On CentOS 7.4 to generate rpm's you can issue below command.
mvn -B clean install package rpm:rpm -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl -Drat.ignoreErrors=true
On Debian 9 to generate Debian packages you can issue below command.
mvn -B clean install jdeb:jdeb -DnewVersion=2.6.1.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.ignoreErrors=true
|
Ambari Server will create following packages
...
- RPM will be created under
AMBARI_DIR
/ambari-metrics/ambari-metrics-timelineservice/target/rpm/ambari-metrics-collector/RPMS/noarch.
Patches
View file |
---|
name | 0001-Patch-to-build-phantomjs-on-AArch640004-ambari-logsearch-Add-jdeb-support.patch |
---|
height | 250 |
---|
|
View file |
---|
name | 0003-ambari-funtest-Add-jdeb-support.patch |
---|
height | 250 |
---|
|
View file |
---|
name | 0002-ambari-metrics-grafana-Add-jdeb-support.patch |
---|
height | 250 |
---|
|
View file |
---|
name | 0005-ambari-Add-jdeb-arm64-support.patch |
---|
height | 150250 |
---|
|
View file |
---|
name | frontend-maven0001-ambari-build-aarch64-2.6.1.patch |
---|
height | 150250 |
---|
|
Running Ambari
Run Ambari Server
First, install Pre-Requisities
Code Block |
---|
|
sudo yum install postgresql
sudo yum install postgresql-server |
Then install the Ambari Server RPM.
Code Block |
---|
|
sudo yum install ambari-server/target/rpm/ambari-server/RPMS/aarch64/ambari-server-*.rpm |
Initialize Ambari Server:
Code Block |
---|
|
sudo ambari-server setup |
Start up Ambari Server:
Code Block |
---|
|
sudo ambari-server start |
To access Ambari, go to: http:
The initial username/password is admin/admin.
Run Ambari Agent
Install the Ambari Agent RPM.
Code Block |
---|
|
sudo yum install ambari-agent/target/rpm/ambari-agent/RPMS/aarch64/ambari-agent-2.4.2.0-0.aarch64.rpm |
...