KB 754061 - BeeGFS Metadata Check Before Large File Ingest

KB 754061 - BeeGFS Metadata Check Before Large File Ingest

1. Purpose

To validate metadata and inode capacity before ingesting a very large number of small files into a BeeGFS filesystem.


Step 1 - Check Metadata Inodes

Run beegfs-df on any BeeGFS client
This checks if the nodes have enough Inode capacity for ingesting large number of files and this command will also shows the available capacity for both Storage and Metadata targets.

For example:

beegfs-df


Quote

METADATA SERVERS:

TargetID   Cap. Pool        Total         Free    %      ITotal       IFree    %

========   =========        =====         ====    =      ======       =====    =

     101      normal    5093.9GiB    5093.7GiB 100%     3561.9M     3561.9M 100%

     201      normal    5093.9GiB    5093.7GiB 100%     3561.9M     3561.9M 100%

     301      normal    5093.9GiB    5093.7GiB 100%     3561.9M     3561.9M 100%

     401      normal    5093.9GiB    5093.7GiB 100%     3561.9M     3561.9M 100%

     501      normal    5093.9GiB    5093.7GiB 100%     3561.9M     3561.9M 100%

     601      normal    5093.9GiB    5093.7GiB 100%     3561.9M     3561.9M 100%

     701      normal    5093.9GiB    5093.7GiB 100%     3561.9M     3561.9M 100%

     801      normal    5093.9GiB    5093.7GiB 100%     3561.9M     3561.9M 100%

STORAGE TARGETS:

TargetID   Cap. Pool        Total         Free    %      ITotal       IFree    %

========   =========        =====         ====    =      ======       =====    =

     101      normal   13728.8GiB   13633.0GiB  99%     1439.8M     1439.8M 100%

     102      normal   13728.8GiB   13633.0GiB  99%     1439.8M     1439.8M 100%

The IFree column shows the Inodes available in each metadata target
​


Step 2. Check Client Mount

Run df -i /mnt/beegfs to confirm the mount is healthy and not inode-constrained.


Step 3 - Identify Metadata Target for Ingest Directory

Run beegfs-ctl --getentryinfo </path/to/ingest/directory> to check what metadata server is attributed to their directory
Note the metadata target ID and all files created under this directory will hit the same metadata server.

For example:

Quote

# beegfs-ctl --getentryinfo /mnt/beegfs/

Entry type: directory

EntryID: root

Metadata node: meta_06_tgt_0601 [ID: 601]

Stripe pattern details:

+ Type: RAID0

+ Chunksize: 512K

+ Number of storage targets: desired: 4

+ Storage Pool: 1 (Default


Step 4 - Validation Checklist

Before proceeding with the ingest, confirm:

  • Metadata targets show sufficient IFree

  • BeeGFS client mount has available inodes

  • Ingest directories are spread across metadata servers

  • Expected file count is documented