...

Bigtop builds components in containers based on target target ${OS}-${ARCH} docker images.

At present Bigtop supports those $those ${OS}-${ARCH} combinations  combinations offcially

centos-6  debian-8          OS: debian-9   fedora-25-ppc64le  ubuntu-1416.04   centos-7  fedora-26     ubuntu-16.04          ubuntu-16.04-ppc64le
centos-7  debian-8-aarch64  fedora-25  opensuse-42.1opensuse-42.3
ARHC: x86_64   ppc64le   aarch64       ubuntu-14.04-ppc64le  ubuntu-16.04-aarch64

Before going to build stage, target OS/arch combination docker images, bigtop/puppet and  and bigtop/slaves should  should be created. If not, please refer Create build images

Build

Brief

Bigtop uses gradle to build and package components.

Build script for each components can be found at <BIGTOP_SRC_TOP>/bigtop-packages/src/common/<package>/do-component-build.

Debian packaging configurations and scripts are located at <BIGTOPat <BIGTOP_SRC_TOP>/bigtop-packages/src/deb/<component>.

RPM packaging configurations and scripts are located at <BIGTOPat <BIGTOP_SRC_TOP>/bigtop-packages/src/rpm/<component>.

There are several operations taks for a certain component: clean, pkg, deb, rpm.

• clean: clean up build 
• pkg: create tarball package
• deb: create deb package
• rpm: create rpm package

Here is an example:

./gradlew hadoop-clean                # clean existing hadoop build outputs
./gradlew hadoop-pkg                  # build and create hadoop release tarball
./gradlew hadoop-deb                  # build and create hadoop release deb package. Note: pkg will also be created
./gradlew hadoop-rpm                  # build and create hadoop release rpm package. Note: pkg will also be created

Following cmd can be used to get detail gradle tasks info:

./gradlew tasks

Build components

The recommend  build process is to use docker container from target OS-ARCH image to build specified components.

Here is an example to build hbase for ubuntun-16.04/aarch64 platform ( BIGTOP_SRC_TOP is the top of cloned bigtop source dir):

$ cd ${BIGTOP_SRC_TOP}
$ docker images                         # check if required bigtop/slaves:trunk-ubuntun-16.04-aarch64 is present
REPOSITORY          TAG                          IMAGE ID            CREATED             SIZE
bigtop/slaves       trunk-ubuntu-16.04-aarch64   eb812af63515        7 hours ago         3.039 GB
bigtop/puppet       ubuntu-16.04-aarch64         82ef2c36b6e6        7 hours ago         311.5 MB
aarch64/ubuntu      16.04                        a7d1ddc47ced        12 weeks ago        109.7 MB
$ docker run --rm -v `pwd`:/ws bigtop/slaves:trunk-ubuntu-16.04-aarch64 bash -l -c 'cd /ws ; ./gradlew hbase-deb'

Then bigtop will fetch source code tarball from specified URL, applying patches, created patched source tarball/deb/srpm, build and package.

If everything goes well, target package can be found at BIGTOP_SRC_TOP/output/<package>

$ ls output/hbase
hbase_1.1.9-1_all.deb        hbase_1.1.9-1.debian.tar.xz   hbase_1.1.9.orig.tar.gz       hbase-regionserver_1.1.9-1_all.deb
hbase_1.1.9-1_arm64.build    hbase_1.1.9-1.dsc             hbase-doc_1.1.9-1_all.deb     hbase-rest_1.1.9-1_all.deb
hbase_1.1.9-1_arm64.changes  hbase_1.1.9-1_source.changes  hbase-master_1.1.9-1_all.deb  hbase-thrift_1.1.9-1_all.deb