.Big Data Team Space v1.0

About Us

Engineers

IRC Channel:  #linaro-bigdata

Mailing List: leg-bigdata@linaro.org 

Portal

Documentations

ERP 

• ERP 16.12: Installing Hadoop 2.7.2, Spark 2.0 and Hive 2.0.1

• ERP 17.08 Building ELK (ElasticSearch, LogStash and Kibana) on Aarch64

• ERP 17.08 - Build, Configure, Install and Test Hadoop, Spark, Hive, Hbase in containerized environment

• Resources

  • ERP 17.08 deb packages

Big Data Components

1. Apache Bigtop

   • Building Hadoop 2.7.2, Spark 2.0, Hive 2.0.1 using Apache BigTop

   • Building BigTop using Docker container

   • Bigtop Sandbox with Hadoop, Spark, Hive, HBase for Aarch64

   • Bigtop Smoke Tests

   • Resources

     • Bigtop 1.2 Components

     • Bigtop Docker images

     • Upstream github

     • CI Loop

     • v1.2 Release

     • Bigtop AArch64 patches

     • Bigtop Project Ideas and Roadmap

2. ODPi

   • Building ODPi BigTop packages

   • Setup, Configure and Install ODPi Hadoop

   • ODPi Hadoop Cluster Setup Guide

   • ODPi BigTop Hadoop Installation, setup and running, ODPi BigTop Hadoop Installation, setup and running

3. Big Data Core Components

   • Apache Hadoop

     • Setup, Configure and Install ODPi Hadoop

     • Building and Running Apache Hadoop

     • Building Hadoop 2.7.2, Spark 2.0, Hive 2.0.1 using Apache Bigtop

     • Apache Hadoop Build, Install, Configure and Run Guide

     • Building, Running, Configuring and Profiling Apache Hadoop

     • Apache Hadoop Tuning Notes

     • Apache Hadoop Map Reduce Notes

     • OpenJDK javac Nullpointerexception building Hadoop

     • Patch 1/3 Introduce the HyperCrc32C Checksum class

     • Patch 2/3 libhadoop: CRC: ARM NEON Support

     • Patch 3/3 ModifyCRC to target NEON routine

     • hadoop-lca13

4. Big Data Operations

   • Apache Ambari

     • Build Apache Ambari on AArch64

     • Build Apache Ambari on AArch64 (Copy)

     • Install, Setup and Configure Apache Ambari

     • JVM Null pointer error in javac when building Apache Ambari

   • Apache Zookeeper

     • Enablement of Apache Zookeeper on AArch64

   • Apache Oozie

   • Apache Falcon

   • Ganglia

5. Big Data Streaming Tools

   • Apache Spark

     • Building Hadoop 2.7.2, Spark 2.0, Hive 2.0.1 using Apache Bigtop

     • Spark 2.0 - Build, configure and Installation steps using Apache Bigtop

     • Profiling Spark on AArch64

     • Building Apache Spark 1.6 on AArch64

     • Building Apache Spark 2.0

     • Spark Dependency Libraries

   • Apache Flink

   • Apache Beam

   • Apache Tez

   • Apache Flume

   • Apache Storm

   • Apache Alluxio

   • Apache Kafka

     • Apache Kafka Streams

   • Apache NiFi

     • Difference between NiFi and MiNiFi

   • Apache MiNiFi

6. Big Data Data warehousing and Database Tools

   • Apache Hive

     • Building Apache Hive

   • Apache HBase

     • Enablement of Apache HBase on AArch64

   • Apache Cassandra

   • Postrgres

   • Memcached

   • MySQL

   • Redis

   • Apache Drill

   • Apache Sqoop

7. Big Data Data Governance and Security

   • Apache Ranger

   • Apache Knox

   • Apache Atlas

   • Apache Sentry

8. Big Data File Formats

   • Apache Parquet

   • Apache Avro

9. Big Data Datascience Notebooks

   • Apache Jupyter

   • Apache Zeppelin

10. Big Data Analytics

    • ELK - ElasticSearch, Logstash and Kibana

      • ERP 17.08 Building ELK (ElasticSearch, LogStash and Kibana) on AArch64

      • ELK Setup and Run on AArch64

    • Apache Arrow

    • Apache Kudu

11. Big Data ML - Machine Learning

    • H20

      • H20 Install and Run Guide

      • H20 Scaling Study

      • Sparkling Water - Spark + H20

12. Big Data component dependencies

    • NETLIB-JAVA AArch64 Natives Support