CephFS Update
0 likes287 views
The document summarizes updates to CephFS in the Pacific release, including improvements to usability, performance, ecosystem integration, multi-site capabilities, and quality. Key updates include MultiFS now being stable, MDS autoscaling, cephfs-top for performance monitoring, scheduled snapshots, NFS gateway support, feature bits for compatibility checking, and improved testing coverage. Performance improvements include ephemeral pinning, capability management optimizations, and asynchronous operations. Multi-site replication between clusters is now possible with snapshot-based mirroring.
![6
● Feature bit support for turning on/off required file system features
○ Clients not supporting features will be rejected
● Multiple MDS FS scrub (online integrity check)
● Kernel client (and mount.ceph) support for msgr2[.1]
○ Kernel mount option -o ms_mode=crc|secure|prefer-crc|prefer-secure
● Support for recovering mounts from blocklisting
○ Kernel reconnects with -o recover_session=clean
○ ceph-fuse reconnects with --client_reconnect_stale=1; page cache should be disabled
● Improved test coverage (doubled test matrix, 2500 -> 5000)
CEPHFS ROBUSTNESS](https://image.slidesharecdn.com/publiccephmonthcephfsupdate-210621233701/85/CephFS-Update-6-320.jpg)
![[OpenInfra Days Korea 2018] Day 2 - CEPH 운영자를 위한 Object Storage Performance T...](https://cdn.slidesharecdn.com/ss_thumbnails/openinfradayobjectstorageperformancefinal2-180704062033-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Similar to CephFS Update (20)
Ad
Recently uploaded (20)
Ad
CephFS Update
- 1. 1 Ceph Month: CephFS Update 2021.06.21 Patrick Donnelly
- 2. 2 What’s new in Pacific! https://ceph.io/community/new-in-pacific-cephfs-updates/
- 4. 4 ● MultiFS is marked stable! ○ Automated file system creation: use ‘ceph fs volume create NAME’ ○ MDS automatically deployed with cephadm/rook ● MDS autoscaler (start/stop MDS based on file system max_mds, standby count) ● cephfs-top (preview) ○ See client sessions and performance of the file system ● Continued improvements to cephfs-shell ● Scheduled snapshots via new snap_schedule mgr module. ● First class NFS gateway support ○ active/active configurations + HA (haproxy + cephadm || rook) ○ automatically deployed via the Ceph orchestrator (Rook and cephadm) ● MDS-side encrypted file support (kernel-side development on-going): some patches still TBB CEPHFS USABILITY
- 6. 6 ● Feature bit support for turning on/off required file system features ○ Clients not supporting features will be rejected ● Multiple MDS FS scrub (online integrity check) ● Kernel client (and mount.ceph) support for msgr2[.1] ○ Kernel mount option -o ms_mode=crc|secure|prefer-crc|prefer-secure ● Support for recovering mounts from blocklisting ○ Kernel reconnects with -o recover_session=clean ○ ceph-fuse reconnects with --client_reconnect_stale=1; page cache should be disabled ● Improved test coverage (doubled test matrix, 2500 -> 5000) CEPHFS ROBUSTNESS
- 8. 8 ● Ephemeral pinning (policy based subtree pinning) ○ Distributed pins automatically shard sub-directories (think: /home) ○ Random pins shard descendent directories probabilistically ● Improved capability/cache management by MDS for large clusters ○ Cap recall defaults improved based on larger production clusters (CERN) ○ Capability acquisition throttling for some client workloads ● Asynchronous unlink/create (partial MDS support since Octopus) ○ Miscellaneous fixes and added testing ○ Kernel v5.7 and downstream in RHEL 8 / CentOS 8 (Stream) ○ libcephfs/ceph-fuse support in-progress: https://github.com/ceph/ceph/pull/38964 CEPHFS PERFORMANCE
- 10. 10 ● Replication targets (remote clusters) configured on any directory ● New cephfs-mirror daemon to migrate data ○ Managed by Rook or cephadm ● Snapshot-based ○ When snapshot is created on source cluster, it is replicated to remote cluster ● Initial implementation in Pacific ○ Single daemon ○ Inefficient incremental updates ○ Improvements will be backported CEPHFS: SNAPSHOT-BASED MIRRORING
- 12. 12 CSI / OPENSTACK MANILA ● RWX/ROX -- CephFS via mgr/volumes ○ New PV snapshot stabilization ■ Limits in place for snapshots on subvolumes. ○ New authorization API support (Manila) ○ New ephemeral pinning for volumes
- 13. 13 Windows client: cephfs-dokan ● ceph-dokan.exe -c c:ceph.conf -l x ● Developer preview ● QA integration TBD
- 14. 14 What’s coming in Quincy!
- 15. 15 ● Async metadata operations ○ Async rmdir/mkdir + link/rename (same dir) ○ Support in libcephfs TBM: https://github.com/ceph/ceph/pull/38964 ● Ceph-fuse performance ○ Take advantage of recent libfuse changes ● libcephfs (ceph-fuse/nfs-ganesha) ○ Break up big client lock ● fscache support in kernel client ● recursive unlink RPC (.trash directory) Performance
- 16. 16 ● MDS Rolling Upgrades (Planned) ○ per-MDS compat to simplify upgrades: https://github.com/ceph/ceph/pull/40511 ● libcephfssqlite ○ See libcephsqlite: https://ceph.io/community/new-in-pacific-sql-on-ceph/ ● mds_memory_target Usability
- 17. 17 ● HA / active-active (likely backported to Pacific) ● Automatic periodic snapshot + sync to remote cluster ○ Arbitrary source tree, destination in remote cluster ● Discussing more sophisticated models ○ Bidirectional, loosely/eventually consistent sync ○ Simple conflict resolution behavior? CephFS multi-site replication
- 18. 18 Questions?