OpenZFS Checksum optimization on Arm64 - fletcher4

Description

Checksums and Their Use in ZFS

End-to-end checksums are a key feature of ZFS and an important differentiator for ZFS over other RAID implementations and filesystems. Advantages of end-to-end checksums include:

  • detects data corruption upon reading from media

  • blocks that are detected as corrupt are automatically repaired if possible, by using the RAID protection in suitably configured pools, or redundant copies (see the zfs copies property)

  • periodic scrubs can check data to detect and repair latent media degradation (bit rot) and corruption from other sources

  • checksums on ZFS replication streams, zfs send and zfs receive, ensure the data received is not corrupted by intervening storage or transport mechanisms

Checksum Algorithms

The checksum algorithms in ZFS can be changed for datasets (filesystems or volumes). The checksum algorithm used for each block is stored in the block pointer (metadata). The block checksum is calculated when the block is written, so changing the algorithm only affects writes occurring after the change.

Activity

Kevin Zhao 
February 19, 2024 at 1:44 AM

Still not merged

Kevin Zhao 
July 10, 2023 at 2:20 PM

Wait for merge

Kevin Zhao 
January 5, 2023 at 2:57 AM

Got approved by one reviewer

Kevin Zhao 
November 23, 2022 at 9:00 AM

Kevin Zhao 
November 8, 2022 at 3:49 AM

Update the scores gotten from Arm Neoverse N2 VM(16C 64G )

The original one:

implementation native scalar 4746352227 superscalar 6141900717 superscalar4 8200328458 aarch64_neon 6906022985 fastest superscalar4

The Neon with Prefetch L1:

implementation native scalar 4752052388 superscalar 6146310868 superscalar4 8232842617 aarch64_neon 8637126932 fastest aarch64_neon

The Neon with rewritten superscalar4:

implementation native scalar 4736703971 superscalar 6146491009 superscalar4 8224078783 aarch64_neon 8723473052 fastest aarch64_neon

Details

Assignee

Reporter

Original estimate

Time tracking

No time logged3w remaining

Priority

Checklist

Sentry

Created October 13, 2022 at 1:44 AM
Updated February 19, 2024 at 1:44 AM