Skip to end of banner
Go to start of banner

Build components

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Prepare

Bigtop builds components in containers based on target ${OS}-${ARCH} docker images.

At present Bigtop supports those ${OS}-${ARCH} combinations offcially

centos-6  debian-8          debian-9   fedora-25-ppc64le  ubuntu-14.04          ubuntu-16.04          ubuntu-16.04-ppc64le
centos-7  debian-8-aarch64  fedora-25  opensuse-42.1      ubuntu-14.04-ppc64le  ubuntu-16.04-aarch64

Before going to build stage, target OS/arch combination docker images, bigtop/puppet and bigtop/slaves should be created.

Build

Brief

Bigtop uses gradle to build and package components.

Build script for each components can be found at <BIGTOP_SRC_TOP>/bigtop-packages/src/common/<package>/do-component-build.

Debian packaging configurations and scripts are located at <BIGTOP_SRC_TOP>/bigtop-packages/src/deb/<component>.

RPM packaging configurations and scripts are located at <BIGTOP_SRC_TOP>/bigtop-packages/src/rpm/<component>.

There are several taks for a certain component: clean, pkg, deb, rpm.

  • clean: clean up build 
  • pkg: create tarball package
  • deb: create deb package
  • rpm: create rpm package

Here is example:

./gradlew hadoop-clean                # clean existing hadoop build outputs
./gradlew hadoop-pkg                  # build and create hadoop release tarball
./gradlew hadoop-deb                  # build and create hadoop release deb package. Note: pkg will also be created
./gradlew hadoop-rpm                  # build and create hadoop release rpm package. Note: pkg will also be created

Following cmd can be used to get detail gradle tasks info:

./gradlew tasks

Build components

The recommend  build process is to use docker container from target OS-ARCH image to build specified components.

Here is example for build hbase for ubuntun-16.04/aarch64 platform ( BIGTOP_SRC_TOP is thetop of cloned bigtop source dir):

$ cd BIGTOP_SRC_TOP
$ docker images                         # check if required bigtop/slaves:trunk-ubuntun-16.04-aarch64 is present
REPOSITORY          TAG                          IMAGE ID            CREATED             SIZE
bigtop/slaves       trunk-ubuntu-16.04-aarch64   eb812af63515        7 hours ago         3.039 GB
bigtop/puppet       ubuntu-16.04-aarch64         82ef2c36b6e6        7 hours ago         311.5 MB
aarch64/ubuntu      16.04                        a7d1ddc47ced        12 weeks ago        109.7 MB
$ docker run --rm -v `pwd`:/ws bigtop/slaves:trunk-ubuntu-16.04-aarch64 bash -l -c 'cd /ws ; ./gradlew hbase-deb'

Then bigtop will fetch source code tarball from specified URL, applying patches, created patched source tarball/deb/srpm, build and package.

If everything goes well, target package can be found at BIGTOP_SRC_TOP/output/<package>

$ ls output/hbase
hbase_1.1.9-1_all.deb        hbase_1.1.9-1.debian.tar.xz   hbase_1.1.9.orig.tar.gz       hbase-regionserver_1.1.9-1_all.deb
hbase_1.1.9-1_arm64.build    hbase_1.1.9-1.dsc             hbase-doc_1.1.9-1_all.deb     hbase-rest_1.1.9-1_all.deb
hbase_1.1.9-1_arm64.changes  hbase_1.1.9-1_source.changes  hbase-master_1.1.9-1_all.deb  hbase-thrift_1.1.9-1_all.deb
  • No labels