Building and Installing Tensorflow on AArch64
This post concentrates on building Tensorflow on AArch64.
About Tensorflow
TensorFlow is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. TensorFlow is one of the best libraries to implement deep learning. TensorFlow is a software library for numerical computation of mathematical expressional, using data flow graphs.
It’s best to run TensorFlow on machines equipped with GPUs
A common workflow of TensorFlow (And this is common for any supervised machine learning platform) is like this:
- Training cluster reads from input dataset, uses algorithms to build a data model. For example, see Google’s cat recognition learning.
- Serving cluster serves the data model, so that any client can connect to the serving cluster to predict new samples, for example, ask “Is this a cat?”
- Training cluster can periodically update data model when new data available, so clients can get prediction results from the up-to-date model.
Tensorflow models can be found here.
Main Use Cases of TensorFlow
Voice/Sound Recognition
Text Based Applications
Image Recognition
Time Series
Video Detection
Prerequisites
- Java 8 (e.g. openjdk-8-jre) installed
Python
- Pip
- NumPy
python-virtualenv
- Bazel 0.5.4
Build Instructions
Install OpenJDK 8
$ wget http://openjdk.linaro.org/releases/jdk8u-server-release-1708.tar.xz $ tar xvf jdk8u-server-release-1708.tar.xz $ cd jdk8u-server-release-1708 $ export JAVA_HOME=$PWD $ cd jre/lib/security/ $ rm cacerts (for CentOS 7) $ ln --symbolic /etc/pki/java/cacerts . (for Debian Jessie) $ ln --symbolic /etc/ssl/certs/java/cacerts . $ cd jdk8u-server-release-1708/bin $ export PATH=$PWD:$PATH
Install Python dependencies
To install TensorFlow, you must install the following packages:
numpy
, which is a numerical processing package that TensorFlow requires.dev
, which enables adding extensions to Python.pip
, which enables you to install and manage certain Python packages.wheel
, which enables you to manage Python compressed packages in the wheel (.whl) format.
$ sudo apt update $ sudo apt install python-pip $ pip install --upgrade pip
Install NumPy
$ python -m pip install --user numpy
Install other dependencies
$ sudo apt-get install python-numpy python-dev python-pip python-wheel python-virtualenv
Build Bazel
# Pre-requisites for Bazel $ sudo apt-get install pkg-config zip g++ zlib1g-dev unzip $ wget https://github.com/bazelbuild/bazel/releases/download/0.5.4/bazel-0.5.4-dist.zip $ mkdir bazel-0.5.4 $ unzip bazel-0.5.4-dist.zip -d bazel-0.5.4 $ cd bazel-0.5.4
Modify bazel source file to make it compatible with aarch64.
diff --git a/scripts/bootstrap/buildenv.sh b/scripts/bootstrap/buildenv.sh index 502f2c1..a2ab4dc 100755 --- a/scripts/bootstrap/buildenv.sh +++ b/scripts/bootstrap/buildenv.sh @@ -40,7 +40,7 @@ PLATFORM="$(uname -s | tr 'A-Z' 'a-z')" MACHINE_TYPE="$(uname -m)" MACHINE_IS_64BIT='no' -if [ "${MACHINE_TYPE}" = 'amd64' -o "${MACHINE_TYPE}" = 'x86_64' -o "${MACHINE_TYPE}" = 's390x' ]; then +if [ "${MACHINE_TYPE}" = 'amd64' -o "${MACHINE_TYPE}" = 'x86_64' -o "${MACHINE_TYPE}" = 's390x' -o "${MACHINE_TYPE}" = 'aarch64' ]; then MACHINE_IS_64BIT='yes' fi diff --git a/src/main/java/com/google/devtools/build/lib/util/CPU.java b/src/main/java/com/google/devtools/build/lib/util/CPU.java index 7a85c29..e5f3eae 100755 --- a/src/main/java/com/google/devtools/build/lib/util/CPU.java +++ b/src/main/java/com/google/devtools/build/lib/util/CPU.java @@ -26,6 +26,7 @@ public enum CPU { X86_64("x86_64", ImmutableSet.of("amd64", "x86_64", "x64")), PPC("ppc", ImmutableSet.of("ppc", "ppc64", "ppc64le")), ARM("arm", ImmutableSet.of("arm", "armv7l")), + AARCH64("aarch64", ImmutableSet.of("aarch64")), S390X("s390x", ImmutableSet.of("s390x", "s390")), UNKNOWN("unknown", ImmutableSet.<String>of()); diff --git a/third_party/BUILD b/third_party/BUILD index 9cd2fac..f1cd14c 100755 --- a/third_party/BUILD +++ b/third_party/BUILD @@ -583,6 +583,11 @@ config_setting( ) config_setting( + name = "aarch64", + values = {"host_cpu": "aarch64"}, +) + +config_setting( name = "freebsd", values = {"host_cpu": "freebsd"}, )
Compile Bazel
$ ./compile.sh #Copy bazel to $PATH $ sudo cp output/bazel /usr/local/bin/
Build Tensorflow
$ git clone https://github.com/tensorflow/tensorflow.git $ ./configure # Please specify the location of python. [Default is /usr/bin/python]: # Please specify optimization flags to use during compilation [Default is -march=native]: # Do you wish to use jemalloc as the malloc implementation? (Linux only) [Y/n] jemalloc enabled on Linux # Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] No Google Cloud Platform support will be enabled for TensorFlow # Do you wish to build TensorFlow with Hadoop File System support? [y/N] No Hadoop File System support will be enabled for TensorFlow # Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N] No XLA support will be enabled for TensorFlow Found possible Python library paths: /usr/local/lib/python2.7/dist-packages /usr/lib/python2.7/dist-packages # Please input the desired Python library path to use. Default is [/usr/local/lib/python2.7/dist-packages] Using python library path: /usr/local/lib/python2.7/dist-packages # Do you wish to build TensorFlow with OpenCL support? [y/N] No OpenCL support will be enabled for TensorFlow # Do you wish to build TensorFlow with CUDA support? [y/N] No CUDA support will be enabled for TensorFlow Configuration finished Extracting Bazel installation... ..................... INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes. ..................... INFO: Downloading http://pkgs.fedoraproject.org/repo/pkgs/gmock/gmock-1.7.0.zip/073b984d8798ea1594f5e44d85b20d66/gmock-1.7.0.zip: 297,424 bytes
$ bazel build -c opt --copt="-funsafe-math-optimizations" --copt="-ftree-vectorize" --copt="-fomit-frame-pointer" --verbose_failures tensorflow/tools/pip_package:build_pip_package #Build pip package $ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
Install Tensorflow
$ sudo pip install /tmp/tensorflow_pkg/tensorflow-1.5.0rc0-cp27-cp27mu-linux_aarch64.whl
Create a virtualenv environment
Create a virtualenv environment by issuing one of the following commands:
$ sudo mkdir /tensorflow $ sudo virtualenv --system-site-packages /tensorflow
Activate the virtualenv environment
Activate the virtualenv environment by issuing one of the following commands:
$ source /tensorflow/bin/activate bash
The preceding source command should change your prompt to the following:
(tensorflow)$
Running Tensorflow on Hadoop
https://www.tensorflow.org/deploy/hadoop