Versions Compared

Old Version 11

changes.mady.by.user Former user

Saved on

compared with

New Version 12

changes.mady.by.user Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Pre-requisites

  • OpenJDK8
  • Zookeeper
  • git
  • maven@v3.3.9

Install OpenJDK

$ sudo apt-get install openjdk-8-jdk

...

Edit zoo.cfg and the below


$ vi zoo.cfg


dataDir=/var/lib/zookeeper
server.1=node1:2888:3888
server.2=node2:2888:3888
server.3=node3:2888:3888


Now, do the below changes in log4.properties file as follows.

$ vi log4j.properties


zookeeper.log.dir=/var/log/zookeeper 
zookeeper.tracelog.dir=/var/log/zookeeper 
log4j.rootLogger=INFO, CONSOLE, ROLLINGFILE

...

$ /home/zookeeper/zookeeper-3.4.13/bin/zkServer.sh status


Zookeeper Service Status Check.

In Zookeeper ensemble If you have 3 nodes, out of them, one will be in leader mode and other two will be in follower mode. You can check the status by running the following commands. If you have just one then it will be standalone.

...

Install Pre-requities for Build

$ sudo apt-get install git


Hooking up upstream Maven 3.3.9 (for Debian Jessie only)

...

Clone and Build Apache Drill

$ git clone https://gitbox.apache.org/repos/asf/drill.git

$ cd drill

$ git branch v1.15.0 origin/1.15.0

$ git checkout v1.15.0

$ mvn clean package -DskipTests

After successful compilation.

$ cd distribution/target/apache-drill-1.15.0/apache-drill-1.15.0

Edit your computer /etc/hosts file and make sure that the loopback is commented. e.g.

#127.0.0.1 localhost
#127.0.1.1 ubuntu

replace with your host IP-address

<IP-address> ubuntu
<IP-address> localhost

Because in distributed mode the loopback IP 127.0.1.1 cannot be binded reference https://stackoverflow.com/questions/40506221/how-to-start-drillbit-locally-in-distributed-mode

Next you need to edit the conf/drill-override.conf and change the zookeeper cluster ID e.g. as below

drill.exec:

{ cluster-id: "1", zk.connect: "<IP-address>:2181" }

Now you can run the drillbit and watchout the log. To play more with drillbit you can refer drill-override-example.conf file.

apache-drill-1.15.0$ ./bin/drillbit.sh help
Usage: drillbit.sh [--config|--site <site-dir>] (start|stop|status|restart|run|graceful_stop) [args]

In one of the terminal switch on the logs with the tail command

apache-drill-1.15.0$ tail -f log/drillbit.log

apache-drill-1.15.0$ ./bin/drillbit.sh start
Starting drillbit, logging to /mnt/nvme0n1p3/Projects/Apache-Components-Build/drill/distribution/target/apache-drill-1.15.0/apache-drill-1.15.0/log/drillbit.out
apache-drill-1.15.0$ ./bin/drillbit.sh status
drillbit is running.
apache-drill-1.15.0$ ./bin/drillbit.sh graceful_stop
Stopping drillbit
...

You can either stop or do a graceful stop. We can repeat the same steps on more than one machines (nodes).

I could able to run the Drill and access the http://IP-Address:8047 and run a sample querry in distributed mode. So In order to do in a distributed mode. I just need to do a similar setup on multiple machines (nodes). Reference - https://drill.apache.org/docs/starting-the-web-ui/

If you have CentOS distribution you need to bit careful because of firewall.  I have executed the following extra steps on CentOS 7

If you are using the CentOS 7   you should be little careful because the connection errors may be caused because of the firewall issues. I have used below set of commands to disable the firewall.

[centos@centos ~]$ sudo systemctl stop firewalld
[centos@centos ~]$
[centos@centos ~]$ sudo firewall-cmd --zone=public --add-port=2181/udp --add-port=2181/tcp --permanent
[sudo] password for centos:
success
[centos@centos ~]$
[centos@centos ~]$ sudo firewall-cmd --reload
success
[centos@centos ~]$
[centos@centos ~]$ zkServer.sh restart
ZooKeeper JMX enabled by default
Using config: /home/centos/zookeeper-3.4.12/bin/../conf/zoo.cfg
ZooKeeper JMX enabled by default
Using config: /home/centos/zookeeper-3.4.12/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
ZooKeeper JMX enabled by default
Using config: /home/centos/zookeeper-3.4.12/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[centos@centos ~]$

REFERENCE:

https://stackoverflow.com/questions/13316776/zookeeper-connection-error

https://www.tutorialspoint.com/zookeeper/index.htm

https://blog.redbranch.net/2018/04/19/zookeeper-install-on-centos-7/

https://drill.apache.org/docs/distributed-mode-prerequisites/