The aim of this project is to make AArch64 a first class citizen in the Big Data, Analytics and Data Science community (e.g., Hadoop, Spark, etc.). Big Data and Data Science technologies are vital and have become mature with various production implementations. Linaro drives engineering activities and ARMv8 builds. for Apache Ambari, BigTop, Spark and Hadoop.
Edit the macro below and add the appropriate project in the JQL query
Edit the macro below and add the appropriate project in the JQL query
Edit the macro below and add the appropriate project in the JQL query
Building BigTop using Docker container
Building Hadoop 2.7.2, Spark 2.0, Hive 2.0.1 using Apache BigTop
[https://wiki.linaro.org/LEG/Engineering/BigData/ODPi BigTop Hadoop Installation, setup and running]
[https://wiki.linaro.org/LEG/Engineering/BigData/ODPiHadoopMultinodeClusterSetup]
[https://wiki.linaro.org/LEG/Engineering/BigData/ODPi_Setup_Config_Run]
[https://wiki.linaro.org/LEG/Engineering/BigData/ODPi_Setup_Config_Run/ODPi_BigTop_Building]
ERP 16.12: Installing Hadoop 2.7.2, Spark 2.0 and Hive 2.0.1
Bigtop Sandbox with Hadoop, Spark, Hive, HBase for Aarch64
Build Apache Ambari on AArch64
Build and Install Apache Ambari V2.6.1 on AArch64
[https://wiki.linaro.org/LEG/Engineering/BigData/horrible-jvm-debug-case-for-ed-ambari]
Apache Ambari Install, Setup and Configuration
[https://wiki.linaro.org/LEG/Engineering/BigData/HadoopBuildInstallAndRunGuide]
[https://wiki.linaro.org/LEG/Engineering/BigData/hadoopbuildrun]
Spark 2.0 - Build, configure and Installation steps using Apache BigTop
[https://wiki.linaro.org/LEG/Engineering/BigData/SparkDependencyLibraries]
[https://wiki.linaro.org/LEG/Engineering/BigData/Building_Spark_1_6]
[https://wiki.linaro.org/LEG/Engineering/BigData/Building_Spark_With_BigTop]
[https://wiki.linaro.org/LEG/Engineering/BigData/Spark]
[https://wiki.linaro.org/LEG/Engineering/BigData/Hive]
Zookeeper Enablement on AAarch64
Building ELK (ElasticSearch, LogStash and Kibana) on Aarch64
[https://wiki.linaro.org/LEG/Engineering/BigData/H2OInstallAndRunGuide]
[https://wiki.linaro.org/LEG/Engineering/BigData/H2OScalingStudy]
[https://wiki.linaro.org/LEG/Engineering/BigData/SparklingWaterGuide]
[http://openjdk.linaro.org/hadoop-terasort-benchmark-results]
[https://wiki.linaro.org/LEG/Engineering/BigData/hadoopconfig]
[https://wiki.linaro.org/Internal/People/SteveCapper/hadoop-lca13]
[https://wiki.linaro.org/LEG/Engineering/BigData/ApacheBench]
HiBench [wiki] [Collaborate]
[https://wiki.linaro.org/LEG/Engineering/BigData/BigBench]
[https://wiki.linaro.org/LEG/Engineering/BigData/HiveTestBench]
[https://wiki.linaro.org/LEG/Engineering/BigData/SparkBench]
[https://wiki.linaro.org/LEG/Engineering/BigData/HadoopTuningGuide]
NETLIB-JAVA AArch64 Natives support
[https://wiki.linaro.org/LEG/Engineering/BigData/MapReduceNotes ]
[https://wiki.linaro.org/LEG/Engineering/BigData/horrible-jvm-debug-case-for-ed-hadoop]
[[https://wiki.linaro.org/LEG/Engineering/BigData/hadoopbuildrun?action=AttachFile&do=view&target=0001-Introduce-the-HyperCrc32C-Checksum-class.patch| PATCH 1/3 Introduce the HyperCrc32C Checksum class]]
[[https://wiki.linaro.org/LEG/Engineering/BigData/hadoopbuildrun?action=AttachFile&do=view&target=0002-libhadoop-CRC-ARM-NEON-support.patch| PATCH 2/3 libhadoop: CRC: ARM NEON support]]
[[https://wiki.linaro.org/LEG/Engineering/BigData/hadoopbuildrun?action=AttachFile&do=view&target=0003-Modify-HyperCRC-to-target-NEON-routine.patch| PATCH 3/3 Modify HyperCRC to target NEON routine]]
[https://wiki.linaro.org/LEG/Engineering/BigData/Feasibility]
[https://wiki.linaro.org/LEG/Engineering/BigData/CRC32vsNonCRC32Study]
[https://wiki.linaro.org/LEG/Engineering/BigData/configuring-archiva-with-tomcat7]
[https://wiki.linaro.org/LEG/Engineering/BigData/horrible-jvm-debug-case-for-ed-hadoop]
Project Meetings
Project Contacts
Ganesh Raju (Project Lead) | ganesh.raju@linaro.org
Yuqi Gu (Assignee, ARM) | yuqi.gu@linaro.org
Mailing Address: leg-bigdata@linaro.org
Blogs / Presentations:
Bigtop v3.0 With the Upgraded Mpack: New Era of Big Data Distribution
Apache Bigtop v1.5 and Wikimedia: Empower BigData in the real world
Demo - Smart City Use Case with Hadoop, Spark, H2O and Sparkling Water