Smoke Tests
Setup repo
Bigtop release repo doesn't have arm64 support. To execute smoke test it is required to setup local repo with packages we build out.
Code Block |
---|
|
docker run -v `pwd`:/ws bigtop/slaves:erp17.08-debian-8-aarch64 bash -l -c 'cd /ws ; ./gradlew apt' |
Create test configuration
Bigtop smoke test uses yaml to configure test repo/environment/components. A typical configuration yaml can be found at <BIGTOP_SRC_TOP>/provisioner/docker/erp17.08-debian-8-aarch64
Generally following properties should be customized for certain smoke test:
- docker image
- repo url
- distro type
- components to install
- components to test
Following is configuration yaml for Hive:
Code Block |
---|
|
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
docker:
memory_limit: "4g"
image: "bigtop/puppet:debian-8-aarch64"
repo: "http://bigtop-repos.s3.amazonaws.com/releases/1.2.0/debian/8-aarch64/aarch64"
distro: debian
components: [hdfs, yarn, mapreduce, hive]
enable_local_repo: true
smoke_test_components: [hive]
jdk: "openjdk-8-jdk" |
Execute test
It is quite straight forward to execute smoke in bigtop
Code Block |
---|
|
cd provisioner/docker
./docker-hadoop.sh -C <smoke_test_cfg_yaml> -c <node_count> -s -d |
Before running smoke test, it is suggested to run environment check first
Code Block |
---|
|
./docker-hadoop.sh -E |
For detailed usage, just issue: ./docker-hadoop.sh
Code Block |
---|
|
usage: docker-hadoop.sh [-C file ] args
-C file Use alternate file for config.yaml
commands:
-c NUM_INSTANCES, --create NUM_INSTANCES Create a Docker based Bigtop Hadoop cluster
-d, --destroy Destroy the cluster
-e, --exec INSTANCE_NO|INSTANCE_NAME Execute command on a specific instance. Instance can be specified by name or number.
For example: docker-hadoop.sh --exec 1 bash
docker-hadoop.sh --exec docker_bigtop_1 bash
-E, --env-check Check whether required tools has been installed
-l, --list List out container status for the cluster
-p, --provision Deploy configuration changes
-s, --smoke-tests Run Bigtop smoke tests
-h, --help |