| Table of Contents |
|---|
Prepare
Bigtop builds components in containers based on target ${OS}-${ARCH} docker images.
At present Bigtop supports those ${OS}-${ARCH} combinations offcially
| Code Block | ||
|---|---|---|
| ||
OS: debian-9 ubuntu-16.04 centos-7 fedora-26 opensuse-42.3
ARHC: x86_64 ppc64le aarch64 |
Before going to build stage, target OS/arch combination docker images, bigtop/puppet and bigtop/slaves should be created. If not, please refer Create build images
Build
Brief
Bigtop uses gradle to build and package components.
Build script for each components can be found at <BIGTOP_SRC_TOP>/bigtop-packages/src/common/<package>/do-component-build.
Debian packaging configurations and scripts are located at <BIGTOP_SRC_TOP>/bigtop-packages/src/deb/<component>.
RPM packaging configurations and scripts are located at <BIGTOP_SRC_TOP>/bigtop-packages/src/rpm/<component>.
There are several taks for a certain component: clean, pkg, deb, rpm.
- clean: clean up build
- pkg: create tarball package
- deb: create deb package
- rpm: create rpm package
Here is an example:
| Code Block | ||
|---|---|---|
| ||
./gradlew hadoop-clean # clean existing hadoop build outputs
./gradlew hadoop-pkg # build and create hadoop release tarball
./gradlew hadoop-deb # build and create hadoop release deb package. Note: pkg will also be created
./gradlew hadoop-rpm # build and create hadoop release rpm package. Note: pkg will also be created |
Following cmd can be used to get detail gradle tasks info:
| Code Block | ||
|---|---|---|
| ||
./gradlew tasks |
Build components
The recommend build process is to use docker container from target OS-ARCH image to build specified components.
Here is an example to build hbase for ubuntun-16.04/aarch64 platform ( BIGTOP_SRC_TOP is the top of cloned bigtop source dir):
| Code Block | ||
|---|---|---|
| ||
$ cd ${BIGTOP_SRC_TOP}
$ docker images # check if required bigtop/slaves:trunk-ubuntun-16.04-aarch64 is present
REPOSITORY TAG IMAGE ID CREATED SIZE
bigtop/slaves trunk-ubuntu-16.04-aarch64 eb812af63515 7 hours ago 3.039 GB
bigtop/puppet ubuntu-16.04-aarch64 82ef2c36b6e6 7 hours ago 311.5 MB
aarch64/ubuntu 16.04 a7d1ddc47ced 12 weeks ago 109.7 MB
$ docker run --rm -v `pwd`:/ws bigtop/slaves:trunk-ubuntu-16.04-aarch64 bash -l -c 'cd /ws ; ./gradlew hbase-deb' |
Then bigtop will fetch source code tarball from specified URL, applying patches, created patched source tarball/deb/srpm, build and package.
If everything goes well, target package can be found at BIGTOP_SRC_TOP/output/<package>
| Code Block | ||
|---|---|---|
| ||
$ ls output/hbase
hbase_1.1.9-1_all.deb hbase_1.1.9-1.debian.tar.xz hbase_1.1.9.orig.tar.gz hbase-regionserver_1.1.9-1_all.deb
hbase_1.1.9-1_arm64.build hbase_1.1.9-1.dsc hbase-doc_1.1.9-1_all.deb hbase-rest_1.1.9-1_all.deb
hbase_1.1.9-1_arm64.changes hbase_1.1.9-1_source.changes hbase-master_1.1.9-1_all.deb hbase-thrift_1.1.9-1_all.deb |