Tcmalloc 64K page size support on Arm64

Description

TCMalloc is Google's customized implementation of C's malloc() and C++'s operator new used for memory allocation within our C and C++ code. TCMalloc is a fast, multi-threaded malloc implementation.

https://github.com/google/tcmalloc contains the repo for tcmalloc which has been open sourced at 2020. In 2015, Google also has opened source a gperftools repo, which contains the tcmalloc tools. Since “Profiling a Warehouse-Scale Computer” (Kanev 2015), we have invested in improving application productivity via optimizations to the implementation (per-CPU caches, sized delete, fast/slow path improvements, hugepage-aware backend).

There are some storage project such as Ceph are using the gperftools as tcmalloc function, but actually this repo is a little un-active and the new TCmalloc is coming. Also the old gperftools has issue with 64K page size allocation: https://github.com/gperftools/gperftools/pull/1269

This task, we will build and test the tcmalloc and help some upstream to bump their dependencies to the new tcmalloc

Activity

Kevin Zhao 
September 22, 2021 at 2:27 PM

tcmalloc:Patch is submitted to tcmalloc upstream

the patch is submitted, but need to review and merge

https://github.com/gperftools/gperftools/pull/1269

Kevin Zhao 
September 8, 2021 at 4:42 AM

According to discussion and investigation with upstream, we still need to stick to gperftools to realize the tcmalloc function.

Kevin Zhao 
September 1, 2021 at 8:56 AM

https://github.com/google/tcmalloc/issues/2 contains discussion when tcmalloc open source last year. It contains some new features, but with some obvious disadvantages: https://github.com/gperftools/gperftools/issues/1169#issuecomment-585321782

“Thing is google tcmalloc seems to be more like exposing of internal code, than attempt to offer useful malloc that will meet diverse needs and will last. A bunch of bits amputated. No intention as far as I know to offer stable ABI. No intention to support wider range of OSes. “

Also, gperftools has been widely used for a long time, and upstream does not have much incentive to replace as there are some limitations. Google/tcmalloc does not have released packages also, while gperftools does.

We should continue consider doing changes in gperf tcmalloc.

Project like Istio has used google/tcmalloc but due to lack of heap profiler : https://github.com/istio/istio/issues/28233

Won't Do

Details

Assignee

Reporter

Original estimate

Time tracking

No time logged4w remaining

Sprint

Priority

Checklist

Sentry

Created August 26, 2021 at 5:11 AM
Updated April 12, 2023 at 1:09 AM
Resolved April 12, 2023 at 1:09 AM