Building ELK (ElasticSearch, LogStash and Kibana) on Aarch64

Owned by Ganesh Raju (Deactivated)
Last updated: Feb 17, 2019Version comment

Introduction

 

Build

Sources

Environment

Pre-Requisites

Docker, git

Dependencies

Install Prerequisites

# install prerequisites
echo "deb http://http.debian.net/debian jessie-backports main" | tee /etc/apt/sources.list.d/backports.list
sudo apt-get update -y
sudo apt-get install -y -t jessie-backports openjdk-8-jdk git build-essential rubygems texinfo locales-all automake autoconf libtool curl zip unzip rpm texinfo locales-all maven ant tar wget python2.7 python-requests

wget https://services.gradle.org/distributions/gradle-3.5.1-bin.zip -O /tmp/gradle-3.5.1-bin.zip
cd ${HOME} && unzip /tmp/gradle-3.5.1-bin.zip && rm /tmp/gradle-3.5.1-bin.zip
ln -s gradle-3.5.1 gradle

# install docker engine. The easiest way to install docker is to use get.docker.com script. Download and run get.docker.com script:
curl -sSL https://get.docker.com/ | sh

Build Steps

Following scripts are expected to run in a debian-8 container.

Building ElasticSearch

Debian

# setup environments
export LANG="en_US.UTF-8"
export PATH=${HOME}/gradle/bin:$PATH
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64
export JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF8"

# setup JAVA_HOME
cd /usr/lib/jvm/java-8-openjdk-*
export JAVA_HOME=${PWD}

# clone the ElasticSearch v5.4.1
git clone --depth 1 https://git.linaro.org/leg/bigdata/elasticsearch.git -b v5.4.1 ${WORKSPACE}/elasticsearch
cd ${WORKSPACE}/elasticsearch

# okay everything is in place
gradle assemble -Dbuild.snapshot=false

Building Logstash

Debian

# install prerequisites

sudo gem install rake
sudo gem install bundler

# setup environments
export RELEASE=1

# clone the Logstash definitions
git clone --depth 1 https://git.linaro.org/leg/bigdata/logstash.git -b v5.4.1 ${WORKSPACE}/logstash
cd ${WORKSPACE}/logstash

# okay everything is in place
rake bootstrap
rake plugin:install-default

rake artifact:deb

Building Kibana

Debian

# install prerequisites

sudo gem install fpm -v 1.5.0
sudo gem install pleaserun -v 0.0.24
sudo ln -s /usr/bin/python2.7 /usr/bin/python

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash
source ${HOME}/.bashrc

# clone the Kibana definitions
git clone --depth 1 https://git.linaro.org/leg/bigdata/kibana.git -b v5.4.1 ${WORKSPACE}/kibana


curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash
source ${HOME}/.profile
			
#Install the version of node.js listed in the .node-version file (this can be easily automated with tools such as nvm and avn)
nvm install $(cat ${WORKSPACE}/kibana/.node-version)


cd ${WORKSPACE}/kibana
			
#Install npm dependencies
npm install
npm rebuild node-sass
			
npm run build -- --deb --release


mkdir -p out
cp -a ${WORKSPACE}/kibana/target/kibana-*-arm64.deb* out/

Patches

Elasticsearch

Enable ARM64 support in SecComp

Logstash

Support JRuby 9K for Logstash v5.4.1

Kibana

Support packaging archive on arm64

Support create deb/rpm packages for arm64

Issues and Resolutions


JIRA Reference:


Built Artifacts

Linaro snapshots