Current Plan

Coming soon…

Work in Progress

Coming soon…

Plan of Record

Coming soon…

Health Checks

Coming soon…

Portal

Documentations

ERP 

• ERP 16.12: Installing Hadoop 2.7.2, Spark 2.0 and Hive 2.0.1

• ERP 17.08 Building ELK (ElasticSearch, LogStash and Kibana) on Aarch64

• ERP 17.08 - Build, Configure, Install and Test Hadoop, Spark, Hive, Hbase in containerized environment

• Resources

  • ERP 17.08 deb packages

Big Data Components

1. Apache Bigtop

   • Building Hadoop 2.7.2, Spark 2.0, Hive 2.0.1 using Apache BigTop

   • Building BigTop using Docker container

   • Bigtop Sandbox with Hadoop, Spark, Hive, HBase for Aarch64

   • Bigtop Smoke Tests

   • Resources

     • Bigtop 1.2 Components

     • Bigtop Docker images

     • Upstream github

     • CI Loop

     • v1.2 Release

     • Bigtop AArch64 patches

     • Bigtop Project Ideas and Roadmap

2. Big Data Core Components

   • Apache Hadoop

     • Setup, Configure and Install ODPi Hadoop

     • Building and Running Apache Hadoop

     • Building Hadoop 2.7.2, Spark 2.0, Hive 2.0.1 using Apache Bigtop

     • Apache Hadoop Build, Install, Configure and Run Guide

     • Building, Running, Configuring and Profiling Apache Hadoop

     • Apache Hadoop Tuning Notes

     • Apache Hadoop Map Reduce Notes

     • OpenJDK javac Nullpointerexception building Hadoop

     • Patch 1/3 Introduce the HyperCrc32C Checksum class

     • Patch 2/3 libhadoop: CRC: ARM NEON Support

     • Patch 3/3 ModifyCRC to target NEON routine

     • hadoop-lca13

3. Big Data Operations

   • Apache Ambari

     • Build Apache Ambari on AArch64

     • Build Apache Ambari on AArch64 (Copy)

     • Install, Setup and Configure Apache Ambari

     • JVM Null pointer error in javac when building Apache Ambari

   • Apache Zookeeper

     • Enablement of Apache Zookeeper on AArch64

   • Apache Oozie

   • Apache Falcon

   • Ganglia

4. Big Data Streaming Tools

   • Apache Spark

     • Building Hadoop 2.7.2, Spark 2.0, Hive 2.0.1 using Apache Bigtop

     • Spark 2.0 - Build, configure and Installation steps using Apache Bigtop

     • Profiling Spark on AArch64

     • Building Apache Spark 1.6 on AArch64

     • Building Apache Spark 2.0

     • Spark Dependency Libraries

   • Apache Flink

   • Apache Beam

   • Apache TezJIRA Inprogress

   • Apache Flume

   • Apache Storm

   • Apache Alluxio

   • Apache Kafka

     • Apache Kafka Streams

   • Apache NiFi

     • Difference between NiFi and MiNiFi

   • Apache MiNiFi

5. Big Data Data warehousing and Database Tools

   • Apache Hive

     • Building Apache Hive

   • Apache HBase

     • Enablement of Apache HBase on AArch64

   • Apache Cassandra

   • Postrgres

   • Memcached

   • MySQL

   • Redis

   • Apache Drill

   • Apache Sqoop

6. Big Data Data Governance and Security

   • Apache Ranger

   • Apache Knox

   • Apache Atlas

   • Apache Sentry

7. Big Data File Formats

   • Apache Parquet

   • Apache Avro

8. Big Data Datascience Notebooks

   • Apache Jupyter

   • Apache Zeppelin

9. Big Data Analytics

   • ELK - ElasticSearch, Logstash and Kibana

     • ERP 17.08 Building ELK (ElasticSearch, LogStash and Kibana) on AArch64

     • ELK Setup and Run on AArch64

   • Apache Arrow

   • Apache Kudu

10. Big Data ML - Machine Learning

    • H20

      • H20 Install and Run Guide

      • H20 Scaling Study

      • Sparkling Water - Spark + H20

11. Big Data component dependencies

    • NETLIB-JAVA AArch64 Natives Support

Tests 

• Smoke Tests

  • BigTop smoke tests 

• Integration Tests

• ODPi Spec Tests

Benchmarking

• TeraSort

• Building, Running, Configuring and Profiling Apache Hadoop

• Spark Bench

• TPC-H

• TPCxHS

• Apache Bench

• BigBench

• Building and Running HiBench on AArch64 Platform

• HiveTestBench

• HiBench

  • Spark profiling on Arm64

Build and Port

• Build Apache Ambari on AArch64

• HBase Enablement on AArch64

• Apache Flink on AArch64

• Zookeeper Enablement on AArch64

• NETLIB-JAVA AArch64 Natives Support

• Apache Ambari Install, Setup and Configuration

Machine Learning

• Building and Installing Tensorflow on AArch64

Misc

• Onboarding info - Welcome to Team

• Big Data team work summary v1.0

• CRC32 vs Non-CRC32 Study

• Configuring archiva with Tomcat

Bigtop

Blogs/Presentations

State of Big Data on Aarch64 - Apache Bigtop

Big Data benchmarking

Himalayan Odyssey

Big Data Roadmap

Strategic Engineering

Big Data and OpenJDK Strategic Engineering - 2018

Big Data and OpenJDK Strategic Engineering - 2017

Recently updated