Managing OpenSM within NVIDIA UFM and getting OpenSM Logs from UFM

Managing OpenSM within NVIDIA UFM and getting OpenSM Logs from UFM

Overview

In NVIDIA UFM (Unified Fabric Manager), OpenSM — the InfiniBand Subnet Manager — is managed internally by UFM rather than as a standalone systemd service. This allows UFM to control the IB fabric, collect telemetry, and maintain HA configurations without requiring manual OpenSM service management.

How UFM Handles OpenSM

  • OpenSM is launched directly from the UFM installation path:

    /opt/ufm/opensm/sbin/opensm
  • The configuration file used by UFM is located at:

    /opt/ufm/files/conf/opensm/opensm.conf
  • UFM monitors OpenSM, ensuring it runs on the designated primary node and manages failover in HA configurations.

  • Telemetry and diagnostic processes (like ibdiagnet) are also invoked by UFM to monitor port health, link utilization, and performance counters.

Note: On UFM-managed systems, OpenSM does not have a systemd service. Attempting systemctl status opensm will return "Unit opensm.service could not be found."

OpenSM Logs in UFM

All OpenSM-related logs and dumps are stored under the UFM log directory:

/opt/ufm/files/log/

Relevant files include:

  • opensm.log — main OpenSM runtime log

  • opensm-statistics.dump — fabric statistics dump

  • opensm-smdb.dump — Subnet Manager database snapshot

  • opensm-activity.dump — recorded activity

  • opensm-virtualization.dump — virtualization-specific info

Other diagnostic logs and telemetry outputs may also contain OpenSM-related data.

Copying OpenSM Logs to Your Local Computer

1. Create a local folder in the home directory

mkdir -p ~/ufm_opensm_logs

2. Copy all OpenSM logs and directories

cp -r /opt/ufm/files/log/opensm* ~/ufm_opensm_logs/

3. Verify the copied files

ls -lh ~/ufm_opensm_logs/

4. Transfer logs to your local machine using SCP

From your local computer, run:

scp -r root@<UFM_NODE_IP>:~/ufm_opensm_logs/ /local/path/to/save/

Replace:

  • <UFM_NODE_IP> with the UFM node’s IP (e.g., T10PHGXUFMST02)

  • /local/path/to/save/ with the local directory where you want to store the logs

Optional: For faster transfer, compress the folder first:

tar -czvf ~/ufm_opensm_logs.tar.gz -C ~ ufm_opensm_logs scp root@<UFM_NODE_IP>:~/ufm_opensm_logs.tar.gz /local/path/

Summary

  • OpenSM in UFM is managed by UFM; no standalone systemd service exists.

  • Logs and dumps are centralized under /opt/ufm/files/log/.

  • Copying logs to your home directory and using scp allows easy transfer to your workstation for analysis or archival.

    • Related Articles

    • How to Change Default IB Interface in NVIDIA UFM (e.g., from ib0 to ib1)

      By default, NVIDIA UFM binds to the ib0 InfiniBand interface during startup. If ib0 is down or unavailable, UFM will fail to start. In such cases, you may need to change the default interface to ib1 (or any other available IB interface). This article ...
    • Fix: UFM Service Not Starting on UFM Appliance UFM-MUA9502H

      Note: This appliance UFM-MUA9502H is EOL as of 2022. Pre-requisites Ensure the following conditions are met before attempting to start the UFM service: Ensure the license is attached. Verify bond0 is configured with eth0 and eth1. Command: show ...
    • How to Collect Logs from NVIDIA UFM (UFM System Dump)

      Purpose This article explains how to collect diagnostic logs from NVIDIA Unified Fabric Manager (UFM) using the web-based GUI. The UFM system dump is typically required by NVIDIA Support for troubleshooting fabric health, host visibility, alerts, and ...
    • How to Set NVIDIA GPU Power Limit on Ubuntu using nvidia-smi

      Managing the power consumption of your NVIDIA GPU on Ubuntu is a practical way to optimize energy efficiency, especially for workloads that do not require full GPU performance. This guide explains how to check and set power limits for your NVIDIA GPU ...
    • How to Attach/Install License on UFM Appliance

      Get the License You can get a 60-day free evaluation license by applying via this link: NVIDIA License Portal Licenses can be purchased via the NVIDIA Sales Team. Licenses are calculated based on the number of HCA Adapters managed by the UFM. ...