Build and Test instructions of BigData Apache components

$ wget http://openjdk.linaro.org/releases/jdk8u-server-release-1708.tar.xz 
$ tar xvf jdk8u-server-release-1708.tar.xz 
$ cd jdk8u-server-release-1708 
$ export JAVA_HOME=$PWD 
$ cd jre/lib/security/ 
$ rm cacerts 

(for CentOS 7) $ ln --symbolic /etc/pki/java/cacerts . 
(for Debian Jessie) $ ln --symbolic /etc/ssl/certs/java/cacerts .  


$ cd jdk8u-server-release-1708/bin 
$ export PATH=$PWD:$PATH  

$ wget http://www.us.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
$ tar -xvf apache-maven-3.3.9-bin.tar.gz
$ cd apache-maven-3.3.9/bin 
$ export PATH=$PWD:$PATH  

$ git clone https://github.com/apache/flink.git
$ cd flink
$ mvn clean package -DskipTests  

$ git clone https://github.com/apache/kafka.git
$ cd kafka
$ ./gradlew jar 

$ git clone https://github.com/apache/oozie.git
$ cd oozie
$ mvn clean package -DskipTests

$ cd minitest
$ mvn clean test 

$ git clone https://github.com/apache/nifi.git
$ cd nifi
$ mvn clean package -DskipTests  

Testing:
-------

nareshbhat@debian:~/nifi$ cd nifi-assembly/

nareshbhat@debian:~/nifi/nifi-assembly$ ls -lhd target/
drwxr-xr-x 5 nareshbhat nareshbhat 4.0K Feb 16 15:37 target/

nareshbhat@debian:~/nifi/nifi-assembly$ ls -lhd target/nifi*
drwxr-xr-x 3 nareshbhat nareshbhat 4.0K Feb 16 15:36 target/nifi-1.6.0-SNAPSHOT-bin
rw-rr- 1 nareshbhat nareshbhat 1.2G Feb 16 15:38 target/nifi-1.6.0-SNAPSHOT-bin.tar.gz
rw-rr- 1 nareshbhat nareshbhat 1.2G Feb 16 15:37 target/nifi-1.6.0-SNAPSHOT-bin.zip

nareshbhat@debian:~/nifi/nifi-assembly$ mkdir ~/example-nifi-deploy

nareshbhat@debian:~/nifi/nifi-assembly$ tar xzf target/nifi-*-bin.tar.gz -C ~/example-nifi-deploy

nareshbhat@debian:~/nifi/nifi-assembly$ ls -lh ~/example-nifi-deploy/
total 4.0K 
drwxr-xr-x 6 nareshbhat nareshbhat 4.0K Feb 16 15:49 nifi-1.6.0-SNAPSHOT

nareshbhat@debian:~/nifi/nifi-assembly$ cd ~/example-nifi-deploy/nifi-*

nareshbhat@debian:~/example-nifi-deploy/nifi-1.6.0-SNAPSHOT$ ./bin/nifi.sh start
Java home: /home/nareshbhat/Apache-packages-compilation/JDK/jdk8u-server-release-1708
NiFi home: /home/nareshbhat/example-nifi-deploy/nifi-1.6.0-SNAPSHOT
Bootstrap Config File: /home/nareshbhat/example-nifi-deploy/nifi-1.6.0-SNAPSHOT/conf/bootstrap.conf
nareshbhat@debian:~/example-nifi-deploy/nifi-1.6.0-SNAPSHOT$

I could able to see the NiFi screen shot using web browser (Attached).  I did't use the https://nifi.apache.org/docs/nifi-docs/html/getting-started.html guide to very extended level.  But I just make sure that the basic things works fine.

nareshbhat@debian:~/example-nifi-deploy/nifi-1.6.0-SNAPSHOT$ ./bin/nifi.sh stop
Java home: /home/nareshbhat/Apache-packages-compilation/JDK/jdk8u-server-release-1708
NiFi home: /home/nareshbhat/example-nifi-deploy/nifi-1.6.0-SNAPSHOT
Bootstrap Config File: /home/nareshbhat/example-nifi-deploy/nifi-1.6.0-SNAPSHOT/conf/bootstrap.conf
2018-02-16 16:02:54,540 INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi has accepted the Shutdown Command and is shutting down now
2018-02-16 16:02:54,575 INFO [main] org.apache.nifi.bootstrap.Command Waiting for Apache NiFi to finish shutting down...
2018-02-16 16:02:56,602 INFO [main] org.apache.nifi.bootstrap.Command NiFi has finished shutting down.
nareshbhat@debian:~/example-nifi-deploy/nifi-1.6.0-SNAPSHOT$

Install dependency package

$ sudo apt-get install libevent-dev
$ sudo apt-get update
$ sudo apt-get install mysql-server php5-mysql php5 php5-memcached

Clone and build memcached

$ git clone https://github.com/memcached/memcached.git
$ cd memcached
$ ./autogen.sh
$ ./configure
$ make all

You can run the testapp

$ ./testapp

You can check memcached version and run 

$ ./memcached -V
memcached 1.5.4

I have started the memcached service on on console with below command

nareshbhat@debian:~/memcached$ memcached start

and test it on another console

nareshbhat@debian:~$ sudo netstat -tupan | grep 11211
tcp 0 0 0.0.0.0:11211 0.0.0.0:* LISTEN 20423/memcached 
tcp6 0 0 :::11211 :::* LISTEN 20423/memcached 
udp 0 0 0.0.0.0:11211 0.0.0.0:* 20423/memcached 
udp6 0 0 :::11211 :::* 20423/memcached
nareshbhat@debian:~$

To verify memcached is recognized by the web server:

1. Create a phpinfo.php file in the web server’s docroot:

$ sudo nano /var/www/html/phpinfo.php

=========phpinfo.php start============
<?php
// Show all information, defaults to INFO_ALL
phpinfo();
=========phpinfo.php end============

2. Go to that page in your web browser.
For example, http://<IP-address>/phpinfo.php

3. Make sure memcached displays all the information on web browser

I have also executed database_test.php, cache_test.php

$ git clone https://github.com/apache/cassandra.git
$ cd cassandra
$ git tag
$ git checkout cassandra-3.11.1
$ ant clean
$ ant

After successful build you can run the cassandra using command

$ ./bin/cassandra -f

On another terminal I run the below example

$ bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.1-SNAPSHOT | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
cqlsh> CREATE SCHEMA schema1
   ... WITH replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
cqlsh> USE schema1;
cqlsh:schema1> CREATE TABLE users (
           ...                  user_id varchar PRIMARY KEY,
           ...                  first varchar,
           ...                  last varchar,
           ...                  age int
           ...                );
cqlsh:schema1> INSERT INTO users (user_id, first, last, age)
           ...                VALUES ('jsmith', 'John', 'Smith', 42);
cqlsh:schema1> SELECT * FROM users;

 user_id | age | first | last
---------+-----+-------+-------
  jsmith |  42 |  John | Smith

(1 rows)
cqlsh:schema1>