v9.1.0b_releasenotes_digest_edition_v4.0
v9.1.0b_releasenotes_digest_edition_v4.0
0b
Fabric OS v9.1.0b Release Notes Digest
Version 4.0
Broadcom FOS-910b-RN
June 5, 2023
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Copyright © 2023 Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries.
For more information, go to www.broadcom.com. All trademarks, trade names, service marks, and logos referenced herein
belong to their respective companies.
Broadcom reserves the right to make changes without further notice to any products or data herein to improve reliability,
function, or design. Information furnished by Broadcom is believed to be accurate and reliable. However, Broadcom does
not assume any liability arising out of the application or use of this information, nor the application or use of any product or
circuit described herein, neither does it convey any license under its patent rights nor the rights of others.
The product described by this document may contain open source software covered by the GNU General Public License
or other open source license agreements. To find out which open source software is included in Brocade products or to
view the licensing terms applicable to the open source software, please download the open source attribution disclosure
document in the Broadcom Support Portal. If you do not have a support account or are unable to log in, please contact
your support provider for this information.
Use of all versions of Brocade’s Fabric OS is subject to the terms and conditions of the Brocade Fabric Operating System
and Feature Licenses and License Keys End User License Agreement, effective October 1, 2019, as amended by
Brocade from time to time. It is the user’s responsibility to understand and comply with the terms of the EULA. By
downloading, installing, using, posting, distributing or otherwise making available FOS, you agree to be bound on an
ongoing basis by the EULA as updated by Brocade from time to time.
Broadcom FOS-910b-RN
2
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Table of Contents
Broadcom FOS-910b-RN
3
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Revision History................................................................................................................................ 67
Broadcom FOS-910b-RN
4
Fabric OS v9.1.0b Document_Type Fabric OS v9.1.0b Release Notes Digest
Chapter 1: Preface
If you purchased Brocade product support from a Broadcom OEM/solution provider, contact your OEM/solution provider
for all your product support needs.
• OEM/solution providers are trained and certified by Broadcom to support Brocade products.
• Broadcom provides backline support for issues that cannot be resolved by the OEM/solution provider.
• Brocade Supplemental Support augments your existing OEM support contract, providing direct access to Brocade
expertise. For more information on this option, contact Broadcom or your OEM.
For questions regarding service levels and response times, contact your OEM/solution provider.
To expedite your call, have the following information immediately available:
General Information:
Technical support contract number, if applicable.
Switch model.
Switch operating system version.
Error numbers and messages received.
supportSave command output and associated files.
For dual-CP platforms running Fabric OS® 6.2 and above, the supportSave command gathers information from both
CPs and any AP blades installed in the chassis.
Detailed description of the problem, including the switch or fabric behavior immediately following the problem and any
specific questions.
Description of any troubleshooting steps already performed and the results.
Serial console and telnet session logs.
Syslog message logs.
Broadcom FOS-910b-RN
5
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FT00X0054E9
White papers and data sheets are available at www.broadcom.com. Product documentation for all supported releases is
available at myBroadcom to registered users. Registered users can also find release notes at myBroadcom.
Broadcom FOS-910b-RN
6
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
The following sections outline how to locate and download Brocade product manuals and release notes from Broadcom
and myBroadcom. Although the illustrations show Fibre Channel and Fabric OS (FOS), they work for all Brocade products
and operating systems.
– Click the Product Search box, select FIBRE CHANNEL NETWORKING, and select a product from the product
list.
The list of documents and software available for the product displays.
4. Clear the Software check box to list only the documents and release notes.
Quality is our first concern and we have made every effort to ensure the accuracy and completeness of this document. If
you find an error, omission or think that a topic needs further development, we want to hear from you. You can provide
feedback by sending an email to [email protected]. Provide the publication title, publication number,
and as much detail as possible, including the topic heading and page number, as well as your suggestions for
improvement.
Broadcom FOS-910b-RN
8
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Chapter 3: Overview
The Fabric OS v9.1.0b release is a patch release based on FOS v9.1.0. This release includes all fixes from
Fabric OS v9.1.0 through Fabric OS v9.1.0b. All hardware platforms and features supported in FOS v9.1.0 are also
supported in FOS v9.1.0b.
Fabric OS v9.1.0b includes hardware, software enhancements, and defect fixes.
NOTE The FOS v9.1.0a patch release was created for internal qualification only and not released.
Broadcom FOS-910b-RN
9
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
4.1 Hardware
The following section lists hardware introduced with this release.
4.1.1 Platforms
FOS v9.1.0b adds support for the existing Brocade X6 director platform with new Switch Types 165.5 (X6-4) and 166.5
(X6-8).
4.1.2 Enhancements
The FPGA Upgrade for CR64 to support Gen 7 Fibre Channel 2KM QSFP (XBR-000476) for ICLs on Brocade X7(CR64)
is enhanced in FOS v9.1.0b. With FOS v9.1.0b the FPGA Upgrade can be performed non-disruptively by upgrading one
CR64 at a time and waiting for the first upgraded CR64 to come online before upgrading the other CR64.
FOS-836970—After a Brocade G730 power cycle, on rare occurrences, the console may display random characters
from the serial console port. If that occurs, the console will be locked for about 7 minutes, then timeout.
After this timeout, the user will be able to login. The Ethernet management port is unaffected during a serial console
timeout, and logins via any protocol such as SSH and telnet can continue.
FOS-837563—Brocade G630 switch (Switch type = 184) may experience sudden reboot - resets.
FOS-837280—Boot over SAN device cannot boot up after upgrading firmware on 32G FC switches.
FOS-837583—SNMP daemon leaks memory and causes switch to hafailover/hareboot/panic when switch runs out of
memory.
FOS-836031—Switch panic after FDMI daemon terminated.
FOS-826227—Devices in default allaccess zone cannot communicate to each other across LISLs in FICON
environment on all platforms.
FOS-834530—Switch panics during adding aliases to zone configuration.
FOS-838223—Devices in 64G switch with default allaccess zone cannot communicate to each other in FICON
environment.
FOS-836458—User may encounter a series of RTWR errors following fastboot of one of the core switch in cores-
edges topology 2021/12/02-14:05:11 (PST), [RTWR-1003], 1260, SLOT 2 | FID 3, INFO, Core-X7-
8_Upgraded_130057_LS3, essd15: RTWR retry 64 to domain 17, iu_data 31000000.
Broadcom FOS-910b-RN
10
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
5.1 Hardware
The following section lists new hardware introduced with FOS v9.1.0.
NOTE When deploying the Gen 7 Fibre Channel 2KM QSFP (XBR-00476) for ICLs on a Brocade X7, you
must upgrade the Field Programmable Gate Array (FPGA) on each CR64 blade. In FOS 9.1.0, the
FPGA upgrade is disruptive since the chassis must be disabled during the FPGA upgrade and
subsequently powered off and on.
In FOS v9.1.0b the FPGA upgrade can be performed non disruptively
For more information, see FPGA Upgrade (under Important Notes).
Broadcom FOS-910b-RN
11
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
For details of each specific deprecated feature see Deprecated Software Features, the deprecated features are in the
following categories:
Zoning (select functionality)
FICON (select functionality)
In FOS v9.0.x the root account is disabled by default and a new default maintenance account and role was introduced
to enhance the support and troubleshooting functionality.
In FOS v9.1.0 access to the root account is removed and the account cannot be activated (references to root may
persist in FOS and should be ignored).
Instead, the maintenance account is to be used to troubleshoot and run diagnostics commands when necessary. These
diagnostics commands are in addition to the FOS commands available to the admin accounts. These commands are not
meant to be used for day-to-day administration of Brocade switches.
The maintenance account is disabled by default. An account with admin authorization can enable the maintenance
account from CLI. The default password for this account is “password”, upon first login with the maintenance account
password change is mandated.
The maximum number of parallel sessions at any point in time (including local or remote access) for the maintenance
account is increased from 2 to 4.
Brocade TruFOS Certificates are recognized and supported beginning with FOS v9.0.0 and ensure the switch is running
authentic FOS with full support and security features.
FOS v9.1.x enforces the Trusted FOS (TruFOS) Certificate for the following platforms:
X7
G730
X6
G630
With the enforcement of a valid TruFOS Certificate, the applicable switch platforms (Chassis and Enterprise) must have a
valid certificate to upgrade to FOS v9.1.x (or later).
The Gen 7 platforms (X7 and G730) ship with valid (3 year) TruFOS Certificates.
The Gen 6 platforms (X6 and G630) which were shipped with FOS 8.x must install a TruFOS Certificate at FOS v9.0.x
prior to upgrading to FOS v9.1.x.
You can request TruFOS Certificates for entitled platforms on Brocade CSP and through the OEM Assist Site.
Broadcom FOS-910b-RN
12
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Once upgraded to FOS v9.1.x, the TruFOS Certificate must be valid to perform any FOS firmware updates including both
upgrade and downgrade.
With FOS v9.1.x a MAPS alert is triggered when a TruFOS Certificate is expiring within 60 days. The alert is repeated
weekly until the certificate is renewed.
In case an expiring TruFOS Certificate is not replaced, MAPS alerts on the date of expiry and a provides reminder alert
every 30 days after expiration.
NOTE Apart from prohibiting FOS firmware updates there are no other restrictions due to an expired
TruFOS Certificate.
With FOS v9.1.x, authentication of NTP servers is supported. The enhancement provides support for authentication of
NTP servers based on symmetric keys.
The NTP service will run only on the principal/primary switch in a fabric, using one or more NTP servers depending on
configuration, as the switch’s clock source.
NTP symmetric authentication is not supported in AG mode.
From Fabric OS v9.1.x Restrictive Time Adjustment (RTA) is active, meaning the system is monitored and protected from
disproportionate time changes, consequently the date cannot be changed beyond ±7 days using the date command. The
support provider for a switch should be contacted in the event that a system requires a time change of more than seven
days.
Broadcom FOS-910b-RN
13
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
With FOS v9.1.x the firmwaredownload completed status message will be displayed only after the download
completes on DP/blades.
The output below shows the difference between FOS v9.1.0 and FOS v9.0.0 in the message displayed upon completion
when queried with the command firmwaredownloadstatus.
FOS v9.1.0:
firmwaredownload command has completed successfully.
Use firmwareshow to verify the firmware versions.
Broadcom FOS-910b-RN
14
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS v9.0.x:
firmwaredownload command has completed successfully on CP(s).
DP(s) upgrade may still be in progress.
Use firmwareshow to verify the firmware versions.
The following FOS Infrastructure commands have REST enhancements in FOS v9.1.0.
For more information see the REST API documentation.
REST support is enhanced to include SCC, DCC, and FCS policy features.
Fabric Security: SCC, DCC, FCS policy features
Firmware cleaninstall
IPadm
Ethif
Configupload/download
Factory reset
RON –show
Usbstorage: -enable, -disable, -list, -remove
Firmware management commands: Firmwaresync, firmwareshow --history, firmwarecheck --
enable, firmwarecheck --disable
NOTE All the URI’s are executed only from the active CP since the REST infrastructure does not support
standby CP interface.
In FOS v9.1.0 the Management Port Tracker feature is a mechanism to scan the network stack periodically to get the
information of the network connections at the switch. It saves the connection details such as the local IP address/port
created at the switch, name of the process that created the socket, the remote IP address/port of the connection, the time
at which connection is first seen at switch and time at which connection state changed last or closed.
With FOS v9.1.0 a standardized format for messages associated with Port ID is included in RASLOG, AUDITLOG and
SYSLOG. There is no change to the existing message string which will include existing notation of port id.
Example:
2020/08/24-22:37:36:016582 (UTC), [C5-1001], 150/30, CHASSIS | PORT 0/11, ERROR, swd77,
S0,P11(Bp49) user_idx:11 [PID 0x020b00] faulted due to invalid SFP, conflicting speed or
laser fault. Check if the SFP or port speed is valid for the configuration.
Broadcom FOS-910b-RN
15
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
The ‘configure’ and ‘configureChassis’ CLIs allow a user to make configuration changes in a user-interactive way that
involves input of configuration setting parameters when prompted.
With FOS v9.1.0 a non-interactive method of changing these configurations is available using the ‘configure --
query/--set/--show/--default’ family of CLIs.
The command syntax is displayed below:
5.4.2.3.1 Zone Server: Get More Information (GMI) Common Transport (CT) Commands
In FOS v9.1.0, support was added to the Zone Server to support the Get More Information (GMI) Common Transport (CT)
command. GMI support is supported for the following Zone Server commands:
Get Active Peer Zone (GAPZ)
Get Zone Set (GZS)
Get Active Zone Set (GAZS)
Get Alias List (GAL)
Get Alias Member List (GAM)
Get Alias Name (GAN)
FOS v9.1.0 includes major feature enhancements to Flow Vision, IO Insight and VM Insight.
Supporting monitoring of IT, ITL / ITN / VMIT / VMITL / VMITN flows at 5-minute or 6-hour granularity.
FOS v9.1.0 also supports increased scale for Gen 7 platforms, new functionality, and new IO Insight metrics. This section
describes the changes in FOS v9.1.0 to Flow Vision, IO Insight, and VM Insight.
FOS v9.1.0 supports Flow Vision and IO Insight on Gen 6 and Gen 7 platforms. New VMID+ is available on Gen 7
platforms. VMID+ provides VM Insight for monitoring at the VM level with no requirement for storage array support,
whereas VM Insight on Gen 6 platforms requires storage array support.
The table below documents the platform support for Flow Vision, IO Insight and VM Insight in FOS v9.1.0.
Table 1 FOS v9.1.0 Flow Vision and IO Insight Platform Support Matrix
Platform Blade Flow Vision IO Insight VM Insight
Brocade G630 N/A Supported Supported VMID
Brocade G620 N/A Supported Supported VMID
Brocade G610 N/A Supported Not supported Not Supported
Brocade 7810 N/A Supported Not supported Not Supported
Broadcom FOS-910b-RN
16
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Brocade X7-4 and X7-8 FC-32-48 FC32-64 FC32-X7-48 Supported Supported VMID
With FOS v9.1.0, Flow vision scalability is increased for Gen 7 platforms. Shown in Table 2 below are the maximum scale
numbers per platform across all logical switches when VF is enabled (reserving Flow Vision capacity for one or more
specific logical switches is not supported). Because scalability is hardware dependent, the X7 chassis max scale is not
achievable when populated with FC32 port blades.
Brocade Gen 6 platforms allocate Virtual Channels (VCs) to isolate traffic flows that are going through a single physical
port. Each traffic flow by default uses one of the 4 medium priority VCs. The VC selection is based on port address (PID)
and is static.
Traffic Optimizer was introduced in FOS v9.0.0 on Brocade Gen 7 platforms to automatically optimize SAN traffic by
segregating traffic flows with different characteristics. This is done by grouping traffic flows with similar pre-defined
attributes, such as flow destination speed and priority, and place them in a dedicated Performance Group which consists
of an allocation of VCs. The FOS fabric provides optimal shared resource allocation for each Performance Group.
Traffic Optimizer supports the following features in FOS v9.0:
Destination speed-based traffic flow classification
Credit-stalled device isolation via Slow Drain Device Quarantine (SDDQ)
Compatibility with Gen 6 platforms connected to the same fabric
Compatibility with Adaptive Networking (QoS Zone and CSCTL) features
Broadcom FOS-910b-RN
17
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
In FOS v9.1.0 Traffic Optimizer includes the concept of policy profiles. The profile available in FOS v9.0 is named
Sys_TrafOpt_Version1.
With FOS v9.1.0 a new profile named Sys_TrafOpt_Version2 is added.
The new profile supports the following features:
Destination speed-based traffic flow classification and protocol based (SCSI/NVMe) traffic flow classification
Credit-stalled device isolation via Slow Drain Device Quarantine (SDDQ)
Oversubscribed device isolation (OS)
Compatibility with Gen 6 platforms connected to the same fabric
Compatibility with Adaptive Networking (QoS Zone and CSCTL) features
Since activation of the new Traffic Optimizer policy (Sys_TrafOpt_Version2) available with FOS v9.1.0 prohibits non-
disruptive downgrade from FOS v9.1.0 to FOS v9.0.x, the new policy is by default scheduled to activate 30 days after
non-disruptive upgrade to 9.1.0. This provides ample time for a non-disruptive downgrade from FOS v9.1.0 to FOS
v9.0.1x (for any reason) in a production environment.
Activation of the new policy is non-disruptive.
When activated it will be applied to devices which login after the new policy is active while existing flows continue to follow
Sys_TrafOpt_version1 until devices logout/login to the fabric.
A new administrative command is available to configure which policy to activate, manage the activation schedule or abort
scheduled activation.
The two profiles available in FOS v9.1.0 are listed in the table below:
TO Profile Name Description Recommend Use Cases
Sys_TrafOpt_Version1 TO profile which segregates flows based on device Applicable for "SCSI only" environments without
speed congestion events.
Sys_TrafOpt_Version2 TO profile which segregates flows based on device Applicable for multiprotocol (SCSI and NVMe)
speed, protocol and congestion events environments including congestion events.
The FOS CLI command trafopt is introduced in FOS v9.1.0 (corresponding REST API is also provided).
The command is used to display and select which of the available Traffic Optimizer profiles to activate as well as - if
needed - change the default activation schedule of the new policy (Sys_TrafOpt_Version2).
In addition, the command allows to display details of a Performance Group and display the flows associated with a given
Performance Group.
The syntax of the command is displayed below:
<trafopt operations>
--help /* Trafopt command help */
--show /* Display Performance Group Dashboard and available TO profile */
[-profile <profile_name>] /* Display information for performance group part of a
specified profile */
Broadcom FOS-910b-RN
18
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
The Performance Groups (PGs) supported in FOS v9.1.0 for each of the available profiles are listed in the table below.
These PGs provide speed-based classification and, with Profile_version2 protocol-based and (dynamically)
oversubscribed-based classifications, are available and are fully compatible with Gen 6 platforms and legacy QoS
features.
Sys_TrafOpt_Version1 Sys_TrafOpt_Version2
PG_4G_8G_16G PG_4G_8G_16G
PG_SCSI_32G PG_SCSI_32G
PG_SCSI_64G PG_SCSI_64G
PG_NVMe_32G
PG_NVMe_64G
PG_OVER_SUBSCRIPTION_4G_16G
PG_OVER_SUBSCRIPTION_32G
PG_OVER_SUBSCRIPTION_64G
PG_SDDQ_QOS_LOW_1 PG_SDDQ_QOS_LOW_1
PG_SDDQ_QOS_LOW_2 PG_SDDQ_QOS_LOW_2
PG_QOS_HIGH_1 PG_QOS_HIGH_1
PG_QOS_HIGH_2 PG_QOS_HIGH_2
PG_QOS_HIGH_3 PG_QOS_HIGH_3
PG_QOS_HIGH_4 PG_QOS_HIGH_4
PG_QOS_HIGH_5 PG_QOS_HIGH_5
Broadcom FOS-910b-RN
19
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Available in FOS v9.1.0, Sys_TrafOpt_Version_2 classifies flows based on the destination speed and protocol (for 32G
and 64G speeds) and places the flows into the respective Performance Group. This classification completely isolates
different flow speeds and protocols into different VCs allocated for each Performance Group to eliminate performance
interference of lower speed devices to higher speed devices and SCSI to NVMe when they connect to the same fabric.
Figure 1 below shows the effect of speed and protocol-based classification. The figure uses different colors to represent
different Performance Groups (not all possible flow combinations are shown).
Traffic Optimizer by default supports the SDDQ action on credit-stalled devices detected by Fabric Performance Impact
monitoring. SDDQ quarantines the flows associated with credit-stalled devices to Low QoS Performance Group.
Gen 6 platforms in FOS v9.1.0 continue to use the legacy PID-based VC allocation and maintain compatibility with Gen 7
platforms in the same fabric. If a traffic flow source port is on a Gen 7 platform, speed/protocol/oversubscribed based
classification is used until reaching a port on a Gen 6 platform. When that occurs, the flow will use PID-based VC
allocation for the subsequent flow path.
Broadcom FOS-910b-RN
20
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Gen 7 platforms maintain compatibility if the QoS feature is enabled. If QoS Low or High priority are defined, the traffic
flow will be assigned to one of the QoS Performance Groups defined in Table 3.
If CS_CTL is enabled on a port, traffic flows on that port retains the legacy CS_CTL based classification. The traffic flows
are not classified to any Performance Groups.
With the initial release of FTC daemon on FOS v9.0.0, Access Gateway (AG) support was limited to FPIN for events
detected within the fabric, but not for FPI events detected by MAPS on the AG, as well as hardware-level signalling (i.e.,
Exchange Diagnostics Capabilities or “EDC”) was not supported on AG with FOS v9.0.0.
With FOS v9.1.0, AG-mode switches gain the ability to drive FPINs for FPI detected locally on the AG, and also support
hardware-level congestion detection signalling for devices attached directly to the AG. AG devices can now also
automatically notify end devices of detected congestion points and type of congestion. Participating devices can
automatically mitigate the congestion condition based on the notification.
FOS v9.1.0 implements support for Fabric Traffic Controller on Access Gateway (AG) to support Fabric Congestions
Notification detected on the AG for connected devices.
The FTC module provides the following functionality:
Registers devices that are capable of receiving congestion notifications
Delivers Fabric Performance Impact Notifications (FPINs) to devices locally connected to the switch
Deliver FPINs to remote switches for delivery to devices connected to the remote switches
Process FPINs received from connected devices
NOTE Firmware downgrade from FOS v9.1.0 to FOS v9.0 will be blocked if the
portCfgCongestionSignal setting is enabled and EDC is registered and in use on any port when
a Gen 7 switch is configured as an Access Gateway.
Gen 7 platforms can detect congestion on egress ports and generate hardware-level ARB congestion signals to the
connected ports. The hardware-level signals are supported only on ports on Gen 7 platforms.
With FOS v9.1.0 hardware level signals are supported on Gen 7 switches in AG mode.
To receive the hardware-level congestion signal, a device needs to exchange with a switch port about the capabilities for
receiving signals. FTC supports the protocol negotiation using Exchange Diagnostics Capabilities (EDC) FC-ELS
command.
Software notifications are generated by FOS to notify devices. The notifications are FC-ELS commands called Fabric
Performance Impact Notifications (FPINs). FPINs are supported on both Gen 6 and Gen 7 platforms.
To receive FPINs, a device uses Register Diagnostics Functions (RDF) FC-ELS command to register the capability. When
congestion conditions are detected by software, FOS sends FPINs to the registered devices. The FPINs include the
following congestion descriptors to indicate the congestion condition.
Congestion Descriptor – Sent to a device that is on a congested port detected by Fabric Performance Impact (FPI)
monitoring.
Peer Congestion Descriptor – Sent to devices that are in the same zone with a device that is on a congested port.
Link Integrity Descriptor – Sent to devices on the port when MAPS detects the port with link integrity errors (CRCs and
ITWs). Also sent to peer devices in the same zone.
Delivery Descriptor – Sent to a device when an FCP or an FC-NVMe frame originated from the device has been
dropped by the switch. FOS Frame Viewer feature detects a frame drop and drives this FPIN descriptor.
Broadcom FOS-910b-RN
21
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Besides the FPIN descriptor frames, FTC will coalesce a list of FPINs descriptors that are for remotely connected devices.
The coalesced FPINs frames are sent as inter-switch FPIN frames to all supporting domains in a fabric. Inter-switch
FPINs are only sent to domains running FOS v9.0 or later.
Most rules affecting FPIN remain unchanged. In FOS v9.1.0 latency and oversubscription rules are enhanced with a FPI
monitoring interval of 10s. In FOS v9.0 the FPI monitoring interval is 60s.
The table below lists the Fibre Channel standard references that defined the Fabric Congestion Notification.
FC-FS-5 ANSI INCITS 545-20xx Rev 1.x, Fibre Channel - Framing and Primitive signal definitions
Signaling - 5
FC-LS-6 ANSI INCITS xxx-20xx Rev 0.x, Fibre Channel - Link Describe FPIN RDF and EDC ELS
Services - 6
FC-GS-8 ANSI INCITS 548-201x Rev 11.03, Fibre Channel - Generic Device-related information (e.g., N_Port Name)
Services - 8
FC-SW-8 ANSI INCITS xxx-20xx Rev 11.03, Fibre Channel - Switch TBD - T11 Work Group not yet started
Fabric - 8
In FOS v9.1.0 the command fabricNotification, is enhanced to be supported on switches in AG mode to display the
EDC and FPIN registration information maintenance by FTC. Only local device information is displayed by this command.
Two new CLI options have been added to display relevant F_Port and/or N_Port information on the AG:
fabricNotification –-show -fport [-all/<port index>]
fabricNotification –-show -nport [-all/<port index>]
5.4.2.6.5 portCfgCongestionSignal
In FOS v9.1.0 the CLI command, portCfgCongestionSignal is supported on switches in AG-mode to enable or
disable a Gen 7 platform port from hardware-level signals for congestion. By default, this configuration is enabled. For
switches in AG mode upgraded from FOS v9.0 the configuration must be enabled manually. Changing this configuration is
not disruptive. The following shows the syntax of this command.
portCfgCongestionSignal -enable <port_range>
portCfgCongestionSignal -disable <port_range>
Broadcom FOS-910b-RN
22
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Most FPINs are triggered by MAPS and FPI detection. With support for FPIN on switches in AG-mode, MAPS FPIN action
is now supported in AG-mode with FOS v9.1.0 to trigger FTC to generate FPINs upon violation of MAPS rules.
The following MAPS command is now supported in AG-mode:
mapsconfig --action FPIN
In FOS v9.1.0 there are no changes to the Frame Viewer command to enable and disable FPIN to be triggered by Frame
Viewer upon detection of dropped frames.
framelog –action [FPIN | fpin]: Enable FPIN delivery with Frame Viewer
frameLog -action [NONE | none]: Disable FPIN delivery with Frame Viewer
A new default N-Port group called ALL_N_PORTS is introduced in AG platforms. The MAPS default policies are
enhanced to include default port errors and performance metrics monitoring rules on ALL_N_PORTS group to monitor N-
Ports
MAPS is capable of detecting congestion on a port due to a credit-stall device, oversubscription or link-errors in 10
seconds compared to 1-minute in pre FOS v9.1 releases. Therefore, MAPS can trigger faster FPIN notifications to
devices or mitigating credit-stall devices by quarantining the device connected port before the congestion propagates into
the fabric.
The default FPI rules are configured with default Quiet Time of 15-minutes to reduce the number of alerts if a port remains
congested for a longer time.
In FOS v9.1.0 MAPS is enhanced to provide consistent monitoring of ports across state change and port type. This
enhancement applies to the following use cases:
State change stats monitoring for FC ports
SFP state
Broadcom FOS-910b-RN
23
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
MAPS monitors the error stats of FC ports using port groups and the groups are dynamic in nature.
These groups are as follows:
ALL_E_PORTS
ALL_F_PORTS
ALL_HOST_PORTS
ALL_TARGET_PORTS
ALL_OTHER_F_PORTS
Dynamic groups based on Node WWN
The above groups’ content is dynamic in nature and can change depending on the state of the ports.
For example, E-port 0 remains in the ALL_E_PORTS as long as it is online.
At this point the E-port threshold is not applied to the E-port 0 leading to unexpected behaviour.
To overcome this limitation of the MAPS in FOS prior to v9.1.0, members are not removed from the group until top of the
hour to ensure all rule alerts are completed.
With these enhancements, users can monitor state change consistently and SFP state can be consistently monitored
irrespective of port type.
In FOS v9.1.0 MAPS is enhanced to support FENCE and DECOM actions to DEV_LATENCY_IMPACT monitoring
system rule and the corresponding ROR rules.
Since DEV_LATENCY_IMPACT supports multiple disruptive actions, the following mutual exclusion is validated when
rules are configured.
For example, if both SDDQ and FENCE are configured the user would get an error message and the rule will not be
configured.
RASLOG NONE
SNMP NONE
EMAIL NONE
FENCE SDDQ,TOGGLE
DECOM SDDQ,TOGGLE
UNQUAR FENCE,DECOM
FMS NONE
FPIN NONE
RE_BALANCE NONE
Broadcom FOS-910b-RN
24
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
In FOS v9.1.0 MAPS any recovery operation from a user or automatically done by MAPS should result in restarting the
rule monitoring for the respective monitoring systems in MAPS. This will enable earliest detection of the issue if it was to
reoccur.
NOTE This is applicable when a user performs port enable after a FENCE action.
MAPS has an auto un-quarantine feature where it automatically clears a port from quarantine state if latency is not
observed on the port for a pre-configured period of time. After this un-quarantine action is performed MAPS will restart
monitoring on this port for latency and trigger the rule as soon as MAPS detects the issue again, irrespective of the
timebase of the rule.
Similarly, a user can issue sddquarantine --clear after investigating a port and fixing the latency issue. MAPS will
restart monitoring on this port for latency and trigger the rule as soon as it detects the issue again, irrespective of the
timebase of the rule.
In FOS v9.1.0 MAPS is enhanced to automatically monitor the IO stats for each flow and port. To do this, MAPS is
enhanced to add two new groups - sys_flow_monitor_scsi, and sys_flow_monitor_nvme, and following IO
metrics:
Sys_flow_monitor
Sys_flow_monitor_scsi
Sys_flow_monitor_nvme
MAPS is enhanced to monitor flows present in different logical switches. With this feature, MAPS learns the flows in all
logical switches and monitors the added groups, sys_flow_monitor_scsi and sys_flow_monitor_nvme which are present in
all logical switches.
Broadcom FOS-910b-RN
25
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
In FOS v9.1.0 MAPS is enhanced to support IO timebase to monitor the flows in the hardware. This monitoring is
performed for each IO and is performed in the hardware. This generates the violation stats and the stats being monitored
for each port using WR_1stXFER_RDY_VIOL, WR_1stXFER_RDY_VIOL, WR_STATUS_TIME_VIOL,
RD_STATUS_TIME_VIOL monitoring systems.
MAPS is enhanced to monitor the frame drops due to routing errors which can happen due to the following two reasons:
Frames that are unroutable
Frames with unreachable destinations
This enhancement enables ‘bad route’ detection using MAPS rule violations. Routing problems are immediately identified
as they occur (MAPS rule violations) rather than having to work backwards through reported symptoms to find an issue.
NOTE Monitoring is only in effect when the fabric is stable in order to ignore the errors during fabric
reconfiguration when routing table updates happen.
Certificate monitoring is enhanced to cover all certificate types. MAPS RASLOG related to Security certificate monitoring
is enhanced to provide additional information pertaining to the certificate (such as Serial-ID).
The following is the list of certificates monitored by MAPS.
HTTPS
SYSLOG
LDAP
RADIUS
FCAP
ASC
KAFKA
MAPS is enhanced with a default Global Quiet Time (GQT) of 24 hours (default) on new platforms to ease adoption of
MAPS.
This change is only applicable to new platforms, with Brocade G730 as the only currently supported platform.
Global Quiet Time (GQT) is supported in MAPS since FOS v9.0.0. When GQT is configured, MAPS subsequent alerts for
an already triggered rule are not sent until the quiet time duration is expired, in this case 24h.
Broadcom FOS-910b-RN
26
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
In FOS v9.1.0 MAPS RASLOG messages are improved with a new “Quiet Time” field to inform users that subsequent
notifications are suppressed.
Example of the new quiet time RASLOG MAPS message:
2021/05/18-18:38:30 (GMT), [MAPS-1001], 621, FID 128 | PORT 0/0, CRITICAL, sw0, port0, E-
Port 0, Condition=ALL_PORTS(DEV_LATENCY_IMPACT/NONE==IO_FRAME_LOSS), Current
Value:[DEV_LATENCY_IMPACT, IO_FRAME_LOSS, (1 C3TX Timeouts) ],
RuleName=defALL_PORTS_IO_FRAME_LOSS, Dashboard Category=Fabric Performance Impact, Quiet
Time=15 min.
In FOS v9.1.0, FCR provides a CLI configuration to enable the migration mode on an FCR switch which allows the user to
proceed with decommission actions leading to removal of the FCR switch from the backbone fabric without traffic
disruption.
When an FCR switch is enabled for migration, the traffic originating from Edge fabric and destined towards remote Edge
fabrics should get migrated on alternative IFLs in the Edge fabric going to other FCRs in the backbone. Once the desired
traffic migration is achieved, the user can perform actions to decommission the older FCR switch where there is no longer
traffic.
The migration mode feature will apply only to FCR Edge-to-Edge (E2E) topologies having shared edge fabrics - this
means edge fabrics having IFL(s) to more than one FCR in the backbone fabric to reach the same translate domain.
The scope of the migration mode configuration is not applicable to the entire FCR backbone. The migration mode setting
applies only to the FCR switch (identified for decommission) in which the mode was enabled.
All the FCR switches in the backbone fabric should be running FOS v9.1.0 or above for the Migration Mode feature to be
supported. The Migration Mode feature will be disabled by default. The migration mode configuration will be lost on an
upgrade or downgrade.
Support for Traffic Isolation (TI) and Frame Redirection (RD) zones is fully deprecated in FOS v9.1.0. In the previous
release (FOS v9.0.x) phase 1 deprecation was initiated and consisted of graceful removal (i.e. removing
create/add/remove CLI support, warning via RASLOG when TI/RD zones are imported via merge/HA/remote-creation).
Phase 2 is effectuated in FOS v9.1.0 with full deprecation consisting of the following rules and requirements:
Firmware upgrade to FOS v9.1.0 will not be permitted when TI/RD zones exist in the zone database.
The TI and RD options under the ‘zone’ CLI are deprecated.
TI/RD zones will not be allowed to be imported into a switch running FOS v9.1.0.
Broadcom FOS-910b-RN
27
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Support for Broadcast zones is fully deprecated in FOS v9.1.0. In the previous release (FOS v9.0.x) phase 1 deprecation
was initiated and consisted of graceful removal (i.e. removing create/add/remove CLI support, warning via RASLOG when
Broadcast zones are imported via merge/HA/remote-creation). Phase 2 is effectuated in FOS v9.1.0 and consist of full
deprecation consisting of the following rules and requirements:
Firmware upgrade to FOS v9.1.0 will not be permitted when a Broadcast zone exists in the zone database.
Broadcast zone creation is not allowed
Broadcast zones will not be allowed to be imported into a switch running FOS v9.1.0.
The following two FICON features have been discontinued by IBM in 2019 and are no longer supported in FOS v9.1.0:
PDCM (Prohibit Dynamic Connectivity Mask)
FAF (File Access Facility)
The following CLI commands for users with admin role privilege have been modified, for details see the Brocade Fabric
OS 9.1.0 Command Reference Guide:
aaconfig fcrconfigure lscfg portdporttest
ag fcredgeshow mapsconfig portinjecterror
apploginhistory fcrrouterportcost mapsdb portshow
Broadcom FOS-910b-RN
29
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Broadcom FOS-910b-RN
30
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
NOTE On Brocade G620, G630, Brocade X6-8, and Brocade X6-4 platforms, this license enables the use of
IO Insight capability. The license itself is identified as “Fabric Vision and IO Insight” on these
platforms.
FICON Management Server – Also known as CUP (Control Unit Port), this license enables host control of switches in
mainframe environments.
Integrated Routing – This license allows any Fibre Channel port in a Brocade X7-4, X7-8, G720, G730 and G620 to be
configured as an EX_Port supporting Fibre Channel Routing (FCR).
Integrated Routing Ports on Demand – This license allows any Fibre Channel port in a Brocade 7810, G630, X6-8, or
X6-4 to be configured as an EX_Port supporting Fibre Channel Routing. The maximum number of EX_Ports supported
per platform is provided in the license.
ICL POD License – This license activates ICL ports on X6 or X7 platform core blades. An ICL license must be installed
on the director platforms at both ends of the ICL connection.
On the Brocade X6-8:
The first ICL POD license enables 8 UltraScale ICL QSFP ports on each core blade of the X6-8 director, which are
QSFP port numbers 0-3 and 8-11. The second ICL POD license enables all UltraScale ICL QSFP ports on each core
blade of the director.
On the Brocade X6-4:
On the X6-4, the first ICL POD license enables 4 UltraScale ICL QSFP ports on each core blade of the director, which
are QSFP port numbers 0, 1, 4, and 5. The second ICL POD license enables all UltraScale ICL QSFP ports on each
core blade of the director.
Broadcom FOS-910b-RN
31
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
The following licenses are available in Fabric OS v9.1.0 as either universal temporary or regular temporary licenses:
Fabric (E_Port)
Extended Fabric
Trunking
Integrated Routing
Integrated Routing Ports on Demand
FICON Management Server (CUP)
Fabric Vision
Extension Upgrade
NOTE Temporary licenses for features available on a per-slot basis enables the feature for all slots in the
chassis.
Temporary and universal temporary licenses have durations and expiration dates established in the licenses themselves.
FOS will accept up to two temporary licenses and a single universal license on a unit. Universal temporary license keys
can be installed only once on a particular switch, but they can be applied to as many switches as desired. Temporary use
duration (the length of time for which the feature will be enabled on a switch) is provided with the license key. All universal
temporary license keys have an expiration date after which the license can no longer be installed on any unit.
Temporary or universal temporary licenses for Extension Upgrade do not enable additional ports on 7810.
Broadcom FOS-910b-RN
32
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
SX6 Gen 6 Extension blade Supported. Up to a maximum of four blades of this type.
SX6 Gen 6 Extension blade Supported. Up to a maximum of four blades of this type.
Broadcom FOS-910b-RN
33
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Broadcom FOS-910b-RN
34
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
8.1.1.1 Loading FOS PSDs via Web Tools or FOS Command Line
Installing a FOS PSD image on a switch is performed in the same manner as using a traditional full FOS image. If a FOS
PSD image is loaded on an incorrect switch model (for example, attempting to load a FOS PSD image for a Gen 6 entry
level switch on a Gen 6 Director), the following error message displays:
Cannot download the requested firmware because the firmware doesn't support this
platform. Please enter another firmware.
Brocade SANnav™ Management Portal v2.1.1 or earlier does not support FOS PSD images. However, FOS PSD
images are supported with SANnav v2.1.1.3 and later releases. SANnav v2.1.1.3 and later can both host and install
FOS PSD images onto Brocade switches.
v9.1.0b_EMB.tar.gz Fabric OS v9.1.0b for Linux to install on G648 and MXG610S platforms
v9.1.0b_EMB.zip Fabric OS v9.1.0b for Windows to install on G648 and MXG610S platforms
Broadcom FOS-910b-RN
35
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
v9.1.0b_G6G7_DIR.tar.gz Fabric OS v9.1.0b for Linux to install on X6-8, X6-4, X7-8 and X7-4 platforms
v9.1.0b_G6G7_DIR.zip Fabric OS v9.1.0b for Windows to install on X6-8, X6-4, X7-8 and X7-4 platforms
The image files for each respective platform can be downloaded from https://www.broadcom.com/mybroadcom, except for
YANG files which are available on https://www.broadcom.com.
NOTE For Brocade X6, G630, X7 and G730 a valid Trusted FOS (TruFOS) Certificate is required to upgrade
to FOS v9.1.x For details on how to obtain a TruFOS Certificate, see Trusted FOS Certificate
(TruFOS) enablement.
Broadcom FOS-910b-RN
36
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Table 4 Gen 6 and Gen 7 Platforms and Supported Firmware Downgrade Versions from Fabric OS v9.1.x
Platforms Fabric OS v9.1.x Fabric OS v9.0.x Fabric OS v8.2.x
Brocade G610 (Switch Type 170.4 or higher) Supported Supported Not Supported
(Fabric OS v9.0.1b and
later)
Brocade G620 (Switch Type 162) Supported Supported Supported
Brocade X6-4 (Switch Type 165.5) Supported Not Supported Not Supported
(Fabric OS v9.1.0b
and later)
Brocade X6-8 (Switch Type 166.5) Supported Not Supported Not Supported
(Fabric OS v9.1.0b
and later)
Broadcom FOS-910b-RN
37
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
When upgrading a Gen 7 director or switch (X7-4, X7-8, G730, G720) to FOS v9.1.x, it is recommended to upgrade to
FOS v9.1.1c. For additional information see TSB 2023-289-A.
During a firmware downgrade from v9.1.0 to a pre-v9.1.0 build, a transitory SEC-3089 error message may be
displayed after firmwaredownload completes, but before a reboot is done.
Refer to the Brocade Fabric OS Software Upgrade User Guide for detailed instructions on non-disruptive and disruptive
upgrade procedures.
Broadcom FOS-910b-RN
38
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
This chapter contains information that you should consider before you use this Fabric OS release.
9.1 Scalability
All scalability limits are subject to change. Limits may be increased once further testing has been completed, even after
the release of this version of the Fabric OS software. For current scalability limits for Fabric OS software, refer to the
Brocade SAN Scalability Guidelines for Brocade Fabric OS 9.X document.
In FOS v9.1.0 the primary scalability changes are increased Flow Vision support, see the section Flow Vision, IO Insight,
and VM Insights enhancements for more detail.
9.2 Compatibility/Interoperability
This section describes important compatibility and interoperability across Brocade products.
NOTE Web Tools will always show English language irrespective of Browser or Operating System language
setting.
Broadcom FOS-910b-RN
39
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
For the latest support and posting status of all release of Brocade Fabric OS, refer to the Brocade Software: Software
Release Support and Posting Matrices published to the Brocade Product End-of-Life web page
https://www.broadcom.com/support/fibre-channel-networking/eol.
Broadcom FOS-910b-RN
40
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
NOTE Fabric OS v9.1.x support SNMPv1 and SNMPv3, SNMPv2 deprecated in Fabric OS v9.0.1a, SNMP
walk using SNMPv2 is blocked in Fabric OS v9.1.x.
Broadcom FOS-910b-RN
41
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Enabling the VMID+ configuration on F_Ports connected to encryption-supported third-party devices is not supported.
YANG is a standard data modelling language that defines the data sent over the FOS REST API. Each FOS REST API
module is defined in a YANG module file with a .yang name extension. To download the Brocade FOS-specific YANG
files from the Broadcom website, perform the following steps:
1. Go to https://www.broadcom.com/products/fibre-channel-networking/software/fabric-operating-system/
2. Select Downloads
3. The YANG files can be located under the Yang Modules
4. Unzip or untar the Fabric OS package file; the yang.tar.gz file contains the collection of YANG module files that
this FOS release version supports. Untar the yang.tar.gz file to obtain individual YANG module files.
Alternatively, the YANG modules for a specific FOS version can be downloaded from
https://github.com/brocade/yang.
Broadcom FOS-910b-RN
42
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Emulex 16G Emulex 32G Emulex Gen 7 QLogic 16G QLogic 32G
Adapter Adapter Adapter Adapter Adapter
D_Port tests will fail between a port with a 64G optic on a switch or director operating with FOS v9.0.1b and a port on a
G720, X7, G620 (switchType 183), or G630 (switchType 184) operating with FOS v9.0.0x. Any of these platforms
operating with FOS v9.0.0x should be upgraded to FOS v9.0.1a or later prior to running D_Port tests to a 64G optic.
If Diagnostic POST is enabled, supportSave should not be started until the POST tests are completed after a switch
or director boots up. Starting supportSave collection when POST tests are still running can result in unpredictable
behaviour.
9.3.6 DWDM
For best performance and resiliency when deploying native FC ISLs over DWDM, best practice is to deploy distinct
ISLs over DWDM with in-order delivery (iodset) configured on the switches.
Trunking over DWDM is not generally recommended due to the risk of out-of-order frame delivery. Trunking relies on
deterministic deskew values across all trunked links to provide in-order delivery as well as FC primitives for trunk
formation. These deskew values cannot be guaranteed with DWDM equipment in the path.
Use of trunking over DWDM links should only be done when validated and supported by the DWDM vendor.
With Gen 7 switches the permitted deskew (variance in latency due to difference in cable length) is less than with Gen
5 or Gen 6 switches to ensure high performance across trunks at all supported speeds, including 16G and 32G.
2 Adapter firmware or driver versions that are later than the ones listed in the table may not work.
Broadcom FOS-910b-RN
43
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
9.3.8 Extension
Extension between a Brocade 7810 or SX6 running FOS v9.x and a Brocade 7840 is supported only if the 7840 is running
FOS 8.2.1 or later. The following table documents the combinations.
9.3.9 FCoE
The following topologies for FCoE on the FC32-64 are not supported with FOS 9.1.x:
Cisco UCS server directly connected to the FC32-64 without a Fabric Interconnect module.
Cisco UCS server with a Fabric Interconnect module connected to the FC32-64 via a Nexus 5000 series switch in
between. Neither running FCoE NPV mode nor L2 switching mode on the Nexus 5000 is supported.
FCoE devices are supported in edge-to-edge fabric topology. They are not supported in edge-to-backbone fabric
topology over FCR configurations.
9.3.10 FC-NVMe
FC-NVMe is supported in edge-to-edge fabric topology with device type information (e.g. Initiator or Target) over FCR
configurations.
FC-NVMe is supported in edge-to-backbone fabric topology without device type information over FCR configurations.
Broadcom FOS-910b-RN
44
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
To upgrade the FPGA on the CR64 blades perform the following steps:
1. Perform the following command to verify current FPGA code level fpgaupgrade --latest
2. Verify the current FPGA code level is lower than 0x01.0a for the CR64 blade slots
• Slot 7 and 8 on X7-8
• Slot 5 and 6 on X7-4
After verification proceed to the next step.
3. Verify both CR64 blades are online with the command slotshow.
4. Prepare for upgrade of the FPGA on the first CR64 blade with the command portdecom <ICL port> -qsfp
perform this for all connected E-ports (ICL ports) on the CR64 blade.
5. Disable the first CR64 blade on which the ICL ports were decommissioned in the previous step portdisable -
s <core blade slot #>.
6. Upgrade the FPGA on the first CR64 blade with the command fpgaupgrade -s <core blade slot #>
a. Respond Yes to automatically power-off and power-on the blade.
i. Do you want to power-off and power-on the slot # automatically, after
FPGA and/or CPLD upgrade (y/[n])?:
b. In case you respond No to automatically power-off and power-on the blade perform these steps manually.
i. slotpoweroff <core blade slot #>
ii. slotpoweron <core blade slot #>
7. Verify the FPGA on the first CR64 blade is upgraded with the command fpgaupgrade –latest.
a. Verify the FPGA code level is 0x01.0a.
8. Enable the first CR64 blade with the command portenable -s <core blade slot #> (as needed).
9. Persistently enable all ICL ports on the CR64 blade (which were disabled in step 5 prior to the upgrade)
portcfgpersistentenable <ICL port>.
Repeat this step for all connected E-ports (ICL ports) on the CR64 blade.
10. Verify the ICL ports are online with the command switchshow.
11. Repeat steps 4 through 11 on the second CR64 blade.
The FPGA upgrade is now complete.
Broadcom FOS-910b-RN
45
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
To upgrade the FPGA on the CR64 blades perform the following steps:
1. Perform the following command to verify current FPGA code level fpgaupgrade –latest.
2. Verify the current FPGA code level is lower than 0x01.0a for the CR64 blade slots.
• Slot 7 and 8 on X7-8
• Slot 4 and 5 on X7-4
After verification proceed to the next step.
3. Disable the chassis with the command chassisdisable.
4. Upgrade the FPGA on the first CR64 with the command fpgaupgrade -s <core blade slot #>.
5. Upgrade the FPGA on the second CR64 with the command fpgaupgrade -s <core blade slot #>.
6. Power off and then power on chassis (reboot is not sufficient).
7. Confirm the FPGA code level after upgrade with the command fpgaupgrade –latest.
8. Verify the FPGA code level is 0x01.0a.
9.3.14 Security
The security enhancements in this section all apply to both FOS v9.0.x and FOS v9.1.x
FOS v9.0.x and v9.1.x require passwords for admin and user accounts to be changed from the default password
string “password”. In the following scenarios, default password may still be present in FOS v9.0.x and v9.1.x. It is
recommended to change the password in this scenario or at the next login prompt:
– A default password is used in an earlier FOS version (prior to v9.0.0). FOS is upgraded from the earlier FOS
version to FOS v9.x.
– A default password is used in an earlier FOS version on active CP. The standby CP runs FOS v9.x and becomes
active due to HA failover.
– A default password is used in an earlier FOS version. Password is distributed from the earlier FOS version to
FOS v9.x.
It is recommended to reconfigure shared secrets for F_Port authentication between Access Gateway and switch
before firmware upgrade to FOS v9.x. The shared secrets should be configured as given in the following table.
Access Gateway FOS Version Edge Switch FOS Version Shared Secret Configuration
Pre-v9.0.0 9.0.0 or later AG local secret = Switch local secret
AG peer secret = Switch peer secret
9.0.0 or later 9.0.0 or later AG local secret = Switch peer secret
AG peer secret = Switch local secret
It is recommended to reconfigure shared secrets for F_Port authentication between HBAs and a switch before the
switch is upgraded to FOS v9.0.0 or later. Without reconfiguration, shared secrets configured in earlier FOS versions
will fail F_Port authentication when a device port resets. The shared secrets should be configured as given in the
following table.
FOS Version Shared Secret Configuration
FOS v9.x does not support F_Port authentication to Marvell QLogic BR series (Former Brocade Product Line) HBAs
as these HBAs only support legacy Brocade F_Port authentication. For these devices to connect to FOS v9.x, F_Port
authentication must be disabled.
FOS v9.x does not support F_Port trunking when F_Port authentication is enabled
Broadcom FOS-910b-RN
46
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
9.3.15 Zoning
When performing configdownload with a file that contains unsorted zone membership, any unsorted members will be
automatically sorted in the system when configdownload completes. As a result, when a switch is later re-enabled,
port segmentation may occur due to adjacent switches having the same zones with unsorted membership lists. Users
can recover from segmentation by executing cfgDisable, cfgClear, and cfgSave operations in order to clear the zoning
database from the switch that just performed ‘configdownload’. After segmented ISL ports are re-enabled, zone merge
can proceed.
NOTE These steps should ONLY be performed if the zone database is the same on the configdownload
switch as it is on the rest of the fabric.
Broadcom FOS-910b-RN
47
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
9.3.17 Miscellaneous
After a power supply unit is removed from a Brocade G620, the historyshow command may miss the entries for this
FRU removal or insertion event. In addition, the RASLog error message EM-1028 may be logged when the power
supply is removed. This condition can be corrected by power-cycling the switch.
After running offline diagnostics mode 1 on QSFP ports, a Brocade G620 must be rebooted before operational use.
After running offline diagnostics with portledtest, portloopbacktest, or turboramtest commands on
FOS v9.x, Brocade G630 with swtichType 184 must be rebooted before operational use.
All links in an ICL QSFP connection on a Brocade X6 Director must be configured to the same speed using the
portcfgspeed command from one of the following supported speeds: 16Gb/s, 32Gb/s, or ASN. To connect an ICL
from an X6 with a 4x32GFC breakout optic (P/N 57-1000351-01) or a 4x16G FC optic to a 4x16G FC optic in a DCX®
8510, the X6 port’s speed must be set to 16Gb/s.
Brocade G630 LEDs illuminate amber and green during power-up.
The CLI command option snmpconfig -set accesscontrol is planned to be deprecated in the next major
release.
When replacing a FC32-64 blade with a FC32-48 blade, flexport and FCoE configurations should be removed before
the FC32-64 blade is removed.
Enhanced checks are performed on optics during firmware upgrade to FOS v9.0.0 or later. Firmware download is
blocked if unsupported optics are discovered. The scanning of the optics takes a few minutes to complete. The amount
of time it takes is dependent on the number of ports on a switch. On a fully loaded eight (8) slot director, it can take up
to five (5) minutes to complete. In addition, ports with optics that fail the enhanced checks in FOS v9.x will not be able
to come online due to the optics as invalid module.
Brocade G620 with switchType 183 and G630 with switchType 184 do not support the following legacy optical
modules:
– 16G SWL (HAA1, HAA2 serial number)
– 16G LWL (HDA1, HDA2, HDA3 serial number)
– 32G QSFP SWL (ZTA serial number)
The following examples show the sfpShow CLI outputs with the serial numbers of the legacy optical module
sfpshow <port> -f
...
Serial No: HAA11213107BTY2
...
sfpshow <port> -f
....
Serial No: HDA318014000DN1
....
sfpshow <port> -f
....
Serial No: ZTA11517000001K
All user ports in a Gen 7 ICL QSFP port must be assigned to the same logical switch when Virtual Fabric is configured.
Port 0 of the ICL QSFP must be enabled first before port 1, port 2, and port 3 within the same QSFP to be enabled. If
port 0 of the Gen 7 ICL QSFP becomes offline, port 1, port 2, and port 3 of the QSFP will become offline as a result.
All user ports in a Gen 7, 2KM ICL QSFP port must be assigned to the same logical switch when Virtual Fabric is
configured. Port 3 of the ICL QSFP must be enabled first before port 0, port 1, and port 2 within the same QSFP to be
enabled. If port 3 of the Gen 7, 2KM ICL QSFP becomes offline, port 0, port 1, and port 2 of the QSFP will become
offline as a result.
The output of CLI command sfpShow or any other interfaces to retrieve information from Gen 7 SWL QSFP (Part
Number 57-1000490) and LWL QSFP (Part Number 57-1000491) does not match the Part Numbers on the media
sticker labels. The output shows Gen 6 Part Number (57-1000351 for SWL or 57-1000480 for LWL). This does not
affect operation of the optics.
Broadcom FOS-910b-RN
48
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
When a fabric with FOS v9.x is connected to a fabric with pre-FOS v9.0.0, RASLOG message FABR-1001 is
generated as shown in the following example. This is an expected message. There is no impact on the ISL
functionality.
[FABR-1001], 35, FID 128, WARNING,, port 62, incompatible VC count
FOS v9.x has disabled directory listing in CLI shell. As a result, entering <tab><tab> key does not list all CLIs
available. Users can enter help command to list the commands. The shell tab completion by entering the first letter
followed by <tab> key is supported.
The FCR support of “Long Distance Fabric” mode conflict cannot coexist with long distance port configuration. If long
distance mode (LD, LS, or LE) is enabled on the EX_Port and the EX_Port detected Backbone Fabric's "Long
distance fabric" configuration is different from the connected Edge Fabric's "Long distance fabric" configuration, then
the EX_Port will be disabled.
If “Long Distance Fabric” is enabled on a switch via the configure command, it is recommended to upgrade the
switch from FOS v8.2.x directly to FOS v9.0.0a or later. If the “Long Distance Fabric” configuration is enabled on an
E_Port or EX_Port, firmware upgrade or downgrade to FOS v9.0.0 will effectively cause the “Long Distance Fabric”
configuration to be disabled.
If an HTTPS certificate is installed on a switch in FOS v9.x, HTTP access is blocked by default as HTTPS access is
supported.
When portloopbacktest mode1 test runs on multiple Gen 7 ICL ports with multiple iterations, the test may fail.
The workaround is to run the test on one ICL port at a time with a reduced number of iterations.
Running long distance LE mode between any blades or switches among FC32-X7-48, FC64-48, or G720 with port
QoS mode enabled and vc_translation_link_init mode enabled may result in frame timeouts. The
workaround for this problem is to use LS or LD mode for long distance.
If downloading firmware on an unsupported platform, a write post to /rest/operations/show-status/message-id/20000
occurs and will incorrectly concatenate firmware download error messages. No recovery is needed, and this
behaviour will not cause any functional impact.
Brocade G620 with switchType 183, G630 with switchType 184, and FC32-X7-48 blades do not support 4G EX-Port.
Brocade G620 with switchType 183, G630 with switchType 184, and FC32-X7-48 blades support 4G E-Port
connected between any of the three listed only
When Connecting Brocade G730 with X7, G720, G620 switchType 183 or G630 switchType 184 these switches
should run FOS v9.0.1c or later.
When performing a configdownload operation this will not overwrite the existing MAPS "Custom RASLog mode"
feature configuration on the switch. For example, if custom raslog mode is disabled in the switch but it is enabled in
downloaded configuration, then the feature will remain disabled in switch and must be manually configured after the
configdownload operation is complete.
In case an NPIV flow is identified as SDDQ or Over Subscribed (and moved by Traffic Optimizer to an OS PG), the
flow movement may cause some frames to be delivered Out of Order (OOO). In general, open systems devices have
no issues when this happens.
In FOS v9.1.0b, CPX blades in X6 Switch Type 165.5 and 166.5 and X7 are displayed as CPX7 in slotshow
command output.
When upgrading from FOS v9.0.x to FOS v9.1.0b, the AG ports will be moved from ALL_HOST_PORTS group to
ALL_OTHER_F_PORTS group. Consequently, the MAPS thresholds for ALL_OTHER_F_PORTS will apply to these
ports in FOS v9.1.0b. The default thresholds for the groups ALL_HOST_PORTS and ALL_OTHER_F_PORTS are the
same and if these are not changed there is no impact. In case custom thresholds are used and these are configured
differently for the groups ALL_HOST_PORTS and ALL_OTHER_F_PORTS the thresholds (monitoring) for AG ports
are impacted accordingly.
In FOS v9.0.1b, v9.0.1c, v9.0.1d, 9.1.0 and 9.1.0b the following platforms incorrectly changed subtype:
Incorrect Subtype Shown in FOS v9.0.1c Correct Subtype
162.1 162.0
162.6 162.5
184.1 184.0
When upgrading a Gen 7 director or switch (X7-4, X7-8, G730, G720) to FOS v9.1.x, it is recommended to upgrade to
FOS v9.1.1c. For additional information see TSB 2023-289-A.
Broadcom FOS-910b-RN
49
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
This section lists the Common Vulnerabilities and Exposures (CVEs) that have been addressed. Each CVE is identified by
the CVE ID number. For the latest security vulnerabilities disclosures, please visit Brocade Security Advisories web page
at https://www.broadcom.com/support/fibre-channel-networking/security-advisories
There are no current CVEs in FOS v9.1.0 and v9.1.0b.
Broadcom FOS-910b-RN
50
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Defect ID Description
FOS-823675 On a 32G DWDM port, D_Port diagnostics fails on the spinfab throughput test and DWDM line flips fail.
FOS-826227 Devices in default allaccess zone cannot communicate to each other across LISLs in FICON
environment on all platform.
FOS-833202 SSH to active CP on X6-4 is failing when the switch has only DSA and ECDSA host keys.
FOS-835720 "/var/tmp/install: line 259: sync: command not found" message is seen multiple times during
firmwarecleaninstall
FOS-835940 The difference of the "cfgshow --transdiffsonly" output would be when a cfgadd CLI is executed, in
9.0.1x version: the output of the zoning cfg got changed will list all of existing non-changed zone
member with "-+" in front of all zone member. But with 8.2.x version, there is no "-+" in front of all
untouched zoning member.
FOS-836043 DCX8510 returning chassis S/N when being queried for brocade-chassis info via REST, when WWN 1
S/N was previously returned and used for entitlement.
FOS-836219 CLI "sfpshow -all" did not display complete output and the polling of smart SFP data stopped. It reported
an very old "Last poll time:"
FOS-836232 The user will observe that Tunnel with Preshared key will not come up after config download.
FOS-836247 Port IDs get dropped from Name Server database after HAfailover.
FOS-836265 During code upgrade from FOS v8.2.1x to FOS v8.2.3x, FOS cannot completely be brought up due to
cald core dumps. User observes the switch hanging.
FOS-836458 User may encounter a series of RTWR errors following fastboot of one of the core switch in cores-edges
topology 2021/12/02-14:05:11 (PST), [RTWR-1003], 1260, SLOT 2 | FID 3, INFO, Core-X7-
8_Upgraded_130057_LS3, essd15: RTWR retry 64 to domain 17, iu_data 31000000.
FOS-836970 After a Brocade G730 power cycle, the console may display random characters from the serial console
port. If that occurs, the console will be locked for about 7 minutes. Login will be prevented during this
time.
Broadcom FOS-910b-RN
51
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-836572 'snmpconfig' CLI returns error 'Failed to get snmp config info' due to SNMP service not restarting after
getting disrupted.
FOS-837280 Boot over SAN device cannot boot up after upgrading firmware on 32G FC switches.
FOS-837403 Local switch sends SCSI Inquiry with 0 bytes Allocation Length to attached storage devices.
FOS-837405 Flow vision reports the wrong direction of flows for the SCSI devices that don't register FC4 features.
FOS-837518 ESMd panic seen when issuing a `portcfgge lan --set -lan` command.
FOS-837563 Brocade G630 switch (Switch type = 184) may experience sudden reboot - resets.
FOS-837583 SNMP daemon leaks memory and causes switch to hafailover/hareboot/panic when switch runs out of
memory.
FOS-837678 When an initiator logs in to the fabric, the discovery of targets on a remote (different domain) G730 can
be delayed or fail.
FOS-838223 Devices in 64G switch with default allaccess zone cannot communicate to each other in FICON
environment.
FOS-838410 After upgrade to FOS 9.1.0, login may fail with the following message: Cannot make/remove an entry
for the specified session
Defect ID Description
FOS-808629 Incorrect warning message is seen during firmware update using Webtools and can alarm user.
FOS-810953 After completion of FIPS zeroize, the console session was left open.
FOS-812852 REST user-config container does not have a representation to indicate if the user account has default
password.
FOS-813847 WebTools Name Server view does not show WWN Company ID for OUI 34:80:0d
FOS-815401 WebEm Netstat Metric value does not match the CLI value
FOS-817207 Expiration warning does not display for maps rule for syslog switch certificate
FOS-817406 Blade does not initialize after a blade insert and raslog BL-5204 with reason=62.
FOS-817855 FFDC with "[SWCH-5026], 20296/0, SLOT 2 | FFDC | FID 128, WARNING, X7-4_74_56, Area
collision 0 3703360 -1 192, OID:0x42000000, switch_area.c, line: 531, comp:SWITCH_0" error on
Gen6+ chassis
FOS-818010 EDC and RDF registrations will complete successfully, however, the registered values are not retained
when viewed using the "fabricNotification" CLI.
Broadcom FOS-910b-RN
52
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-818632 Getting delayed response from the FOS firmwaredownload API if we enable FIPS on the switch
FOS-818641 Devices on a F-Port are no longer visible after the port is added to a F-Port trunk
FOS-818653 Application I/O latencies increase 1 to 2ms when running low I/O rates over an extension tunnel.
FOS-818655 Domain ID, SCC_POLICY, port names, and port configurations reverted to the previous settings in a
different environment.
FOS-819093 REST POST brocade-security/user-config allows different role-id to be assigned to same Logical
Fabric
FOS-819275 sys_flow_monitor dashboard reports more than the actual IO commands for an NVMe initiator after
login or rescan. The issue will be noticed only during initialization due to the monitoring of NVMe
admin commands.
FOS-819641 WebTools does not support adding DHCHAP secret for a switch that's currently not present in the
regular fabric.
FOS-819820 Encountered FCIP tunnels bounce between two Brocade-7810 switches during HA reboot.
FOS-819862 The new blade inserted into a chassis slot, that is of different type than the previous one, encounter
errors and failures.
FOS-819887 The frequency of the stats collection has to be aligned to the wall clock's 00mts or 30mts. For
example: 11:00, 11:30, 12:00, 12:30. But due to an issue in the handling of drift, the Brocade Support
Link (BSL) data collection may not happen in the wall-clock aligned time, but could be random. This
can impact the FA's data co-relation logic.
FOS-819902 IP filter rule deletion does not work when user performs add and delete operations at the same time.
FOS-819948 FCR config replay for lsan-enforce-tag is not working as per RFC5789 define.
FOS-819951 CLI command portledtest shows the test as passed but LEDs still continue to glow in any color.
FOS-820023 After a hafailover, blades are no longer seen by CP with following raslog: [PLAT-1000], 5841, SLOT 6
| FFDC | CHASSIS, CRITICAL, Brocade_DCX, fabPciFindBus No PCI bus found...
FOS-820163 VLAN and MTU Size is not displayed in FCIP circuit view
FOS-820198 Configdownload fails when there are 64 user defined roles in config file
FOS-820262 Firmware downgrade from FOS 9.0.0 to FOS 8.2.x is blocked with the following message even though
the entries in the table are not configured - "SNMPV1 has non default groupname. That is the first 3
Broadcom FOS-910b-RN
53
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
entries should have the group name as (rw) and the next 3 entries should have (ro). Please use
snmpconfig --set snmpv1 -index <1-3> -groupname rw and snmpconfig --set snmpv1 -index <4-6> -
groupname ro"
FOS-820297 After timezone / date change, the next day's Brocade Support Link (BSL) data collected will be
interpreted incorrectly.
FOS-820371 MAPS is showing the switch as marginal due to an expired trufos_cert but the license doesn't expire
until tomorrow
FOS-820410 The first 3 entries of groupname is changed to 'rw' from 'ro' after installing FOS v9.0.0
FOS-820442 QSFP port set to Auto Speed Negotiation (AN) at 16G speed on QSFP on G620 (switchType 183) or
G630 (switchType 184) will toggle after hareboot or after non-disruptive firmware migration to FOS
9.0.0.
FOS-820626 Process cald might restart during supportsave or firmwaredownload from FOS 9.0.0 through REST
interface in presence of SANnav application monitoring the same fabric.
FOS-820970 Management applications (BNA, SANnav, Webtools) cannot communicate with the switch, and BSL
data collection will also stop. Traffic is not impacted and operation of the switch via CLI is not
impacted.
FOS-821073 Customers may encounter: 1. Specific 3rd party devices flood the switch with GID_PN requests,
leading to delays and impact monitoring applications. 2. Maps publishes an alert about Tx Power of
SFP being 0.
FOS-821353 Weblinker memory leaks observed when RADIUS is configured as AAA authentication.
FOS-821545 Peer beacon status is not shown in switch ports page when its enabled
FOS-821746 Edge to edge routing was not working and frames dropped on E-port or EX-port.
FOS-822104 User may experience FCIP ingress traffic stopped due to no more internal credits available and
associated C3-1014/C3-1015 RASLOG messages on a 7840 platform or C4-1014/C4-1015 RASLOG
messages in a chassis with an SX6 blade. In the case of an SX6 blade, the blade will be faulted after
multiple attempts to recover internal credits.
FOS-822312 Performance degradation for FICON flows after FOS upgrade to FOS 8.2.1 or higher
FOS-822906 Interactive firmwaredownload presents the EULA but does not allow the user to accept it.
Broadcom FOS-910b-RN
54
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-823368 Flow's IO rate (number of IOs per second) and throughput (number of IOs bytes transferred per
second) is diluted for READ when the device has only WRITE traffic and vice versa.
FOS-823756 A 3rd party device is unsuccessful in moving from one switch port to another switch port.
FOS-823842 CLI 'seccertmgmt show -ca -server <application>' does not display multiple CA certificates imported
for an application.
FOS-824480 RAS message indicates verification failure of CA certificate on a switch with an expired FCAP switch
certificate.
FOS-824498 MAPS test email defaulted "From address" to [email protected] on some switches and
rejected by postmaster.
FOS-824516 ACL defined DB is cleared when the active policy is empty and 'distribute -p SCC|DCC|FCS -d
<domains>' CLI is executed for peer domains.
FOS-824545 "Unable to Process IO Module Request due to Internal Error" is returned by FC IOM when hostname
configuration for more than 30 characters is tried.
FOS-824778 Missing error handling when applying an invalid hex value to a 'mibcapability' configuration.
FOS-824781 Flows do not get learnt on X7-8 chassis, after flow monitoring is activated.
FOS-824782 Statistic counter for TELNET_POLICY ('secStatsShow' CLI) increments during SSH login failure.
FOS-825008 Observed error "Error in operation" in response to "extncfg --config -clear -slot #"
FOS-825192 SNMP OID "ifAlias" does not return a value for FCIP tunnels (VE port) when SNMP GET/WALK
operations are executed.
FOS-825197 Audit log "SEC-3021" displays "username" as NONE when authentication fails.
FOS-825202 Audit log is not generated when LDAP authorization fails in non-VF mode.
FOS-825365 Enhancement to provide periodic raslog warning messages on the impending End Of Support (EOS)
for the System and remove auditdump from EOS blocked commands.
FOS-825388 FCIP Tunnel up, but all I/O stops flowing over the tunnel and application timesout.
FOS-825390 User logout messages are not consistent when there are a large number of user accounts on the
setup.
FOS-825413 'fipscfg --zeroize' CLI does not clear the authentication and privacy keys for SNMPv3 users for
indexes 7 to 12.
FOS-825419 Switch allows import of FCAP certificate with WWN, in the Common Name field of certificate, different
from switch WWN.
Broadcom FOS-910b-RN
55
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-825673 Port is disabled with the reason of: Disabled (LD SFP is not supported in AG mode)
FOS-825731 Audit/Raslog message[SEC-3050] is not generated after using CLI 'sshutil rekeyinterval' on a switch.
FOS-825732 During warm recovery on an Access Gateway, user disabled F-ports get enabled.
FOS-825768 swSfpPoweronHrs is returned as zero for some ports, whereas CLI displays a non-zero value.
FOS-825769 When IPv6 is defined in access control list, that particular IP is blocked for SNMP GET/SET operation
FOS-825906 Chassis WWN shows all zeroes when embedded switch is in AG mode, running ag --show command.
FOS-826088 Switch paniced on long spinfab run - "ASSERT - Failed expression: VALID_PTR(ctl)"
FOS-826120 MAPS daemon terminates multiple times and does not fully recover after a policy containing 50 RoR
rules is activated.
FOS-826123 Flow vision displays metrics for an NVMe flow when no IO is present. The NVMe flow counts the
NVMe commands.
FOS-826171 TCL shows tunnel ID as -1 after modifying it for the target VE using ECM. Also IP-Extension traffic will
not flow over the tunnel.
FOS-826172 If a user role is created with upper and lower case characters, it is stored as lowercase. However,
when specifying a role under the ldapcfg command it does not do the same conversion.
FOS-826186 MAPS 'logicalgroup --show' CLI displays more number of flows than that is present on the switch
FOS-826243 The REST peak counters for fibrechannel-statistics seen as zero despite traffic flowing.
FOS-826747 A port blade failed and went missing from the "slotshow" CLI and switch monitor components such as
maps continue to report system status as healthy.
FOS-826921 Following messages are printed on the console: INIT: Switching to runlevel: 6 IP Address update to
trap sessions failed. IP Address update to trap sessions failed.
FOS-826987 'filterportshow' output is incomplete and switch panic with console output, "BUG: Bad page map in
process supportsave".
FOS-827149 "snmptraps --show" CLI command displays all traps even though only a sub-set of them are
supported.
FOS-827154 Enabling persistent PID on a switch can cause NPIV devices not to log in due to FDISC reject. In this
case the I-Series will do an immediate LOGO. Once this happens, we do not preserve that PID.
Broadcom FOS-910b-RN
56
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Instead, the same PID is handed out to the next device that logs in. It appears the host is
misbehaving, but the switch is not handling the situation correctly.
FOS-827227 SNMP swConnUnitFECUnCorrectedCounter OID shows more port entries than what is present on the
Chassis.
FOS-827306 Repeated XTUN-1997 RASLOGs generated in an Extension configuration through a 7810, 7840 or
SX6 blade that may or may not be associated with a VE port down event. The specific trigger is:
ttmh0933|tifa1243| timer [1] reset error
FOS-827342 Switch reports class-3-discards value as 4294967295 after clearing port stats.
FOS-827559 CP crash resulting from Oops: Kernel access of bad area, sig: 11
FOS-827718 Standby CP may remain at down level firmware version after a concurrent dual CP firmware upgrade
failure
FOS-827719 CORE and FFDC are generated on newly active CP after a dual CP firmwaredownload.
FOS-827734 In swDeviceStatusTrap, swFCPortSpecifier value shows -1/<port_index> instead of the actual port
number.
FOS-827757 Minimum quiet time for SFP monitoring system rules can be configured as 2 minutes for all the
platforms.
FOS-827771 MAPS fences a port due to a falsely read Class3 TX Timeout (C3TXTO) counter.
FOS-827887 4 tunnels between two directors doing PPRC / Global Mirror jobs have divergent traffic patterns when
they should be balanced.
FOS-827926 MAPS state change monitoring from dashboard does not work for AG.
FOS-827976 cold boot after ONMd crash when standby becomes active during hafailover process.
FOS-828412 SNMP-Walk mib-2 does not get most groups except for interfaces and if-mib.
FOS-828899 DP Panic after upgrading to FOS8.2.1 through FOS8.2.2d while running FICON XRC traffic over XRC
Emulation enabled FCIP Tunnel.
FOS-829031 After firmware upgrade to FOS9.0.1/9.0.1a, domain controller frames originated from an F port cannot
pass through ISLs. The impacted services include: • RDP Read Diagnostic Parameters (RDP). • Set
Bit-error Reporting Parameters (SBRP) • Read Link Error Status Block (RLS)
Broadcom FOS-910b-RN
57
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-829367 When displaying snmpv1 configuration - snmpconfig --show snmpv1, it does not return the header
part.
FOS-829423 FCIP DP Panic and DP Reset after switching an FCIP Circuit from wantool testing state back to active
FCIP Circuit
FOS-829441 "XTUN-1001 : memory allocation error" messages in RASLOG after using FCIP SLA or manual
WTOOL functionality.
FOS-829537 Following an hareboot, the agautomapbalance configuration is not the same as it was before the
hareboot.
FOS-829594 Issuing a switchenable when the switch is already enabled may lead to I/O disruption
FOS-830052 Port initialization interop issues with a certain SFP, when connecting to a specific 3rd party device.
This may result in some ports showing errors and some going into no_sync, port_Flt state.
FOS-830310 Firmwaredownload fails with a generic error-message: "Firmwaredownload sanity check failed. Please
contact the service provider if the issue persists."
FOS-830714 With flat zoning, user observed 400ms delay in ACK to a FLOGI during ports flipping test, and ports
getting throttled without come online.
FOS-830884 WebTool "Switch Administration -> Extended Fabric" tab are missing slots for X6 directors.
FOS-831329 End devices report errors after a link of an FEC-enabled trunk is bounced.
FOS-831688 Various operations fail after the switch runs out of disk space.
FOS-831727 Long distance E-port trunk does not come back up after port bounce.
FOS-831875 SNMP application may lose connection to switch momentarily during snmp walk of IPV6 address table
and occasionally the user may observe CP lost HA SYNC if the walk is performed during
hafailover/hareboot.
FOS-831881 snmpV2 MIBS does not show any output when VF ID is set in the context
FOS-832917 The X7 has two power supplies that are faulty, but MAPS did not trigger the expected rules to put the
switch into a MARGINAL state as expected and SANnav does not reflect a Degraded state.
Broadcom FOS-910b-RN
58
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-832960 Disruption on Access Gateways when F-port trunks change trunk master.
FOS-833550 CP may encounter kernel panic when the FCIP DP generates a large number of XTUN-1006 events
per second.
FOS-834163 Extension tunnel on 7810, 7840 or SX6 blade will not come online in a NAT (Network Address
Translation) enabled WAN environment.
FOS-834621 eHCL sequence encounters fatal FICON FD error, which results in VE recovery.
FOS-834868 firmwareactivation CLI fails with SULB-1043, after staging a firmware upgrade/downgrade:
2021/09/22-14:58:50:367354, [SULB-1043], 76469/35239, SLOT 1 | CHASSIS, INFO, ,
Firmwareactivate command failed.
FOS-834912 SANnav reports error: "Registration for telemetry profile 'xxx' has failed and Switch panic after cald
termination.
FOS-835201 After upgrading to FOS v9.0.0, observed that both CPs ping pong panic on X6 director every couple
minutes.
FOS-835791 Individual ports, that were previously disabled, are enabled while running CLI command
switchcfgpersistentenable
Defect ID Description
FOS-563298 The output of "flow --show sys_mon_analytics -srcdev <SID> -dstdev <DID>" CLI does not display the
metrics table for the specified single flow between the pair of source and destination devices.
FOS-594770 Alerts are not being posted in RAS for C3TXTO rule in MAPS
FOS-621420 Customer has noticed expiry date in the self signed certificate is not included the leap years.
FOS-651967 Under Port admin--> FC/Ethernet Ports, additional port Info for Ethernet ports are not shown.
FOS-653188 SCSI frametypes SCSI, SCSIGoodStatus, SCSITur and SCSXferrdy inadvertently works on NVMe
traffic.
FOS-654871 user may see ASL gets fenced during switch being toggled.
Broadcom FOS-910b-RN
59
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-655799 If 'flow --reset' isn't performed on a logical switch, before deleting the logical switch (FID), the old flows
(belonging to the deleted logical switch) will not be reset.
FOS-656035 Eth0 may falsely report link, or change speed to other than configured speed.
FOS-656972 Proxy FCoE devices are successfully imported into the FCR backbone but device type does not display
it as FCoE devices.
FOS-660470 New user with assigned user defined role may not work and encounter error "permissions denied"
FOS-800557 User authentication may fail with RADIUS configured as primary AAA server.
FOS-801128 Portperfshow command may display throughput greater than the port speed.
FOS-802896 Non disruptive firmware upgrade to FOS v8.2.1 may fail on a chassis with sys_mon_all_vms monitoring
flow.
FOS-807015 MAPS rules do not trigger on NVME flow for the IO fields IO_RD_BYTES & IO_WR_BYTES.
FOS-809191 Brocade 7810 extension switch's secondary NTP server does not go to active state after the active
NTP server goes down.
FOS-811246 Switch is not accessible via web interfaces - Webtools, BNA, or SANnav.
FOS-813825 SNMP "Failed to get snmp config info" when executing the snmpconfig CLI command.
FOS-816098 Raslog TS-1001, indicating NTP Query Failed, occurs daily or weekly.
FOS-816124 The raslog TS-1002 floods console indicating continuous toggling of active clock server between LOCL
and external
FOS-816201 Date and audit logs in auditdump output shows timezone as "Localtime" when timezone is set in old
format
FOS-817433 Error message "portPeerBeacon is not supported by the other end" with AG mode F-Ports.
FOS-818589 The throughput on a 10G trunk link shows more than 100% at full line rate.
FOS-818700 Incorrect yang statements in ipfilter configuration, user management and certificate management
FOS-818869 inet6 address family not supported in 'netstat' command. S8Gen7:FID128:admin> netstat -A inet6 -nr
Address family `inet6' not supported. S8Gen7:FID128:admin> netstat --help
FOS-819139 User is not able to use special character | (pipe) in username on configupload command.
FOS-819632 On firmware migration from 8.2.2 to 9.0.0 with 256 accounts, there will be 257 accounts due to
maintenance and SSH/telnet fails for last account
Broadcom FOS-910b-RN
60
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-819757 When default_fips template is applied with REST interface, the connection is closed to the switch from
the client and only some configuration is zeroized.
FOS-819966 sys_flow_monitor dashboard displays port speed as 0 for host and target of the monitored flow.
FOS-820016 sys_flow_monitor dashboard is not reporting statistics on active flows and the state of the flow is
deactivated after switchdisable and switchenable operation.
FOS-820095 Time out frames could be seen when Encryption/Compression enabled E_ports on FC32-X7-48 are
connected to different neighbors(i.e to different domains)
FOS-820261 NVMe frames are incorrectly tagged as SCSI frames in the flow mirror output, CLI command flow --
show [flow name] --feat mirror
FOS-820273 CLI configdownload failed for filter "snmp" with "Bad Syntax" and "Failed to Parse" errors.
FOS-820334 Weblinker termination observed when generating self-signed certificate for HTTPS
FOS-820361 Customer may encounter CP resets triggered by SysRq, along with a display of numerous "Help"
messages seen on the console.
FOS-820436 CLI "userconfig --change root -e yes" cannot manage the target account with error message of
"conflicting LF permission".
FOS-820627 On G610, ISL trunk cannot form or port stays in Gport after moving in/out of F-port trunk group.
FOS-820710 After code upgrade, the user defined flows are not showing up in the admin level command of "flow --
show";
FOS-820837 CLI "flow --show" no longer displays any of the customized flows and flow monitor deactivate fails with "
ioctl failed" message.
FOS-820856 High CPU load observed on switch once a connection via WebTools is made with https.
FOS-820884 BR6547 is not using credit value configured on peer switch using portcfgeportcredits command
FOS-821126 Device discovery issue in FCR Backbone to Edge fabric when the IFL is trunked.
FOS-821178 Switch experience kernel panic shortly after a SupportSave command is invoked.
FOS-821268 Observed a weblinker process termination when generating certificate for HTTPS and flash space
eventually went to over 90%.
Broadcom FOS-910b-RN
61
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-821444 Following an abrupt power failure on port blade, the switch continues to access the blade via PCI
causing subsequent system level failures such as CP reboot, hung CP, or a blade that cannot be
powered on.
FOS-821627 Message of the day (MOTD) is not displayed upon connecting to the switch.
FOS-822411 BNA Dashboard and historical performance graphs may report inaccurate data.
FOS-822511 After replacing MAC value in seccryptocfg, SSH and nmap show the value is not actually applied to
one of the two CPs.
FOS-822941 After a finite number of outstanding Fastwrite (FW) sequences FW processing will not generate a
transfer ready sequence, but rather will shuttle the exchange (pass through mode). This will result in
the pass through IOs experiencing higher IO response time when compared to the FW sequences.
The FW sequence IOs will complete in as little as 1 Round Trip Time (RTT), whereas the pass through
IOs will complete no sooner than 2 RTTs.
FOS-823035 Spurious power supply voltage and current readings on BR6520 or BR7840 can lead to EM-1229 and
EM-1230 log messages to be displayed and the switch could be shutdown due to perceived excessive
power consumption.
FOS-823433 In ISL between G720 or FC64-48 (64G optic) to a G630 (32G optic) may take longer time to converge
at 32G speed. This is applicable when the G630 switch is running pre-FOS 9.0.1
FOS-823765 Traffic disruption encountered when Encryption block errors occur due to errors, such as, "frames too
long".
FOS-824646 When AD legacy configuration for user authorization is set on the TACACS+ server and authorization
translates to default FID128.
FOS-824735 Password distribution ('distribute' CLI) to AG switch results in "Error: Operation failed. Unknown error
code - 9" message.
FOS-825189 For SNMP oid ifName and ifAlias, interface names/Alias follows different naming format for FC and GE
ports in respective output.
FOS-825690 The chassis management module does not show the FOS version.
FOS-826780 Add fence and decommission actions for the monitoring items TXP and RXP in the MAPS policy.
FOS-826804 Switch needs a manual reboot to recover from HA Out of Sync state after mdd daemon panic.
Observed raslog: [HAM-1013], , SLOT 1 | CHASSIS, CRITICAL, , Can't restart (md (pid=27115)):
System unready or LS trans in progress. Reboot/Failover manually if necessary.
Broadcom FOS-910b-RN
62
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-827834 Trap connUnitPortStatusChange has extra OID "connUnitPortName", and Trap connUnitEventTrap has
missing OID "connUnitEventId"
FOS-828208 Webtools hardware view does not show the Led status correctly for ports 50, 51 and 117 .
FOS-828392 Port Optics is not visible in Sannav with error: Fail to get SFP details.
FOS-829440 Firmware upgrade from FOS version below FOS-8.1.2k to higher FOS versions fail during the
installation of prom-t10xx-3.0.32-1.rpm package with the following error: Failed to write to prom
partition! error: execution of %post scriptlet from prom-t10xx-3.0.32-1 failed, exit status 1 Failed to
install Package prom-t10xx-3.0.32-1.ppc.rpm. Retrying :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Firmware download failed - Firmware download timed out. (0x23) Aborting firmwaredownload now.
FOS-829779 SANNAV fails to generate switch supportsave on switches running FOS v8.2.2 and above.
FOS-831297 FOS Version 9.0.1a and 9.0.1b does not show any entry in NameServer View in Webtools if only one
zone is present in fabric.
FOS-833133 When doing firmwarecleaninstall, the following will be displayed on the console: INSTALL26: ERROR -
Cannot get file - /sbin/create_release_file. Please verify the network is good and retry
firmwarecleaninstall after reboot INSTALL26: install FAILED with 0 warnings. FirmwareCleaninstall
failed, you can reboot to retry.
FOS-833935 Encountered Weblinker posting raslog RAS-1004 with VERIFY after run out of file descriptor.
FOS-834918 sfpshow CLI displays all zeros for txp, rxp etc smart data, and it triggers MAPs alert event.
FOS-835154 Dual CP reboot following some zoning changes performed via HTTP.
Defect ID Description
FOS-633099 Using the SSH syntax "ssh admin@<ip address> <cli command>" in LS specific context, generates the
output for the default switch.
FOS-806764 User sees the following error when executing the CLI command portshow: porttrunkarea is configured
for F-port trunks.'-i or -x' option not supported.
Broadcom FOS-910b-RN
63
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-810530 Zone merge slow performance and failure on that switch that has defzone all access defined. Along
with this behavior IPC drops RASLOGs events and/or termination of process nsd maybe seen.
FOS-823009 Sys_flow_monitor is not automatically activated after installing the Fabric vision license.
FOS-823675 On a 32G DWDM port, D_Port diagnostics fails on the spinfab throughput test and DWDM line flips fail.
FOS-823999 Flow monitor reports only SCSI stats for an IT flow with SCSI and NVMe exchanges across time.
FOS-825237 Occasional glitches reported in total byte count and total IO count when an external script pulls data at
the same time flow statistics are being updated in the backend.
FOS-826015 Device authentication policy mismatch between switch and HBA causes trunk master port to attain G-
port state while slave port remains F-port.
FOS-832033 The following BL-1061 message could be seen for GigE ports on 7810 when there is no mismatch in
SFP speed and port speed config. [BL-1061]ERROR, aw16, port speed(10G) and SFP speed(1G)
mismatch in port(ge2) slot number(0). Insert SFP matching port speed.
FOS-833202 SSH to active CP on X6-4 is failing when the switch has only DSA and ECDSA host keys.
FOS-833402 A user account, with a user defined role assigned with user management privileges for chassis but not
for current VF, will fail to exercise any user account management operations.
FOS-833753 1. Continuous console flooding of "query updated FCS dbs(s)" messages observed on primary FCS
switch after FCS policy distribution. 2. HA out of sync when primary FCS switch is a chassis platform.
FOS-833824 The "Max UDP conn exceeded" counter in the output of "lan-stats --global" remains always at 0 even
though the number of running UDP flows are more than the supported emulated UDP flows.
FOS-833982 REST PATCH on /brocade-security/sec-crypto-cfg incorrectly return 400 Bad Request on configuration
replay.
FOS-833984 REST PATCH on /brocade-security/user-config incorrectly return 400 Bad Request on configuration
replay.
FOS-834340 During core or port blade insertion/removal in a chassis, Web Tools will not be able to get data from
switch and it will become blank for 3-5 minutes. A browser refresh after this period will re-enable the
Web Tools functionality.
FOS-834530 The customer has an issue with their G720 when adding Aliases to the configuration - FOS v9.0.0b
Each time that an alias is added the "zoned" daemon crashes and the switch Panics: Fri Aug 6
15:31:53 2021 admin, FID 128, 172.28.151.49, alicreate
"test","50:06:0e:80:22:16:16:05"2021/08/06-15:32:00:017036, [KSWD-1002], 2218/196, FFDC |
CHASSIS, WARNING, BrocadeG720, Detected termination of process zoned:2655., hasm_swd.c, line:
225, comp:insmod, ltime:2021/08/06-15:32:00:0166362021/08/06-15:32:00:018470, [RAS-5001],
2219/0, CHASSIS, INFO, BrocadeG720, Message KSWD-1002 caused FFDC event, ffdcd.c, line: 1097,
comp:raslogd, ltime:2021/08/06-15:32:00:0179352021/08/06-15:32:00:018511, [RAS-5007], 2220/0,
CHASSIS, INFO, BrocadeG720, Skip FFDC rasod for KSWD-1002 (rasod is done by panic) , ffdcd.c,
Broadcom FOS-910b-RN
64
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-834539 Users will see "-rbash: clihistory: command not found" sometimes, while running "serviceshell --show"
FOS-835082 Import of HTTPS certificates results in continuous restart of weblinker. Enabling HTTPS fails.
FOS-835714 An incorrect error message is returned when the flows of an NPIV port are being quarantined.
FOS-835809 'Unmount USB Drive' option is displayed in WebEM after USB device is inserted. User would not be
able to use the USB device connected to the switch as there is no option to mount the device.
FOS-835940 The difference of the "cfgshow --transdiffsonly" output would be when a cfgadd CLI is executed, in
9.0.1x version: the output of the zoning cfg got changed will list all of existing non-changed zone
member with "-+" in front of all zone member. But with 8.2.x version, there is no "-+" in front of all
untouched zoning member.
FOS-835998 tsclockserver Active server shows as NONE on non-principal switches however working fine on
Principal switch with legacy mode disabled. Similarly, tsclockserver Active server shows as NONE
when set to LOCL in legacy mode disabled.
FOS-836024 The configdownload operation will not overwrite the existing MAPS "Custom RASLog mode" feature
configuration in switch. For example, if custom raslog mode is disabled in switch but it is enabled in
downloaded configuration, then the feature will remain disabled in switch.
FOS-836043 DCX8510 returning chassis S/N when being queried for brocade-chassis info via REST, when WWN 1
S/N was previously returned and used for entitlement.
FOS-836048 The CP will get stuck in hung state and no CLI's can be executed on the switch.
FOS-836077 On Standby CP, lines with '0's appear on console during hot code load.
Broadcom FOS-910b-RN
65
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
FOS-836219 CLI "sfpshow -all" did not display complete output and the polling of smart SFP data stopped. It reported
an very old "Last poll time:"
FOS-836265 During code upgrade from FOS v8.2.1x to FOS v8.2.3x, FOS cannot completely be brought up due to
cald core dumps. User observes the switch hanging.
FOS-836290 In Non-VF mode, "Configdownload - all" via REST has improper console log message 1) When switch
is enabled it says "configDownload complete: are downloaded". Expected message "configDownload
complete: All selected config parameters are downloaded" 2) When switch is disabled the command
fails. It says "Activating configDownload: Switch is disabled No parameters downloaded.
configDownload not permitted"
FOS-836313 The session which gets modified to HTTPS from HTTP based session, gets terminated when HTTP
protocol is disabled after HTTPS certificate generation.
FOS-836339 REST GET operation on brocade-security/sshutil-public-key fails with error "Named account does not
exist"
FOS-836369 D-Port test does not start on the secondary port links of a 53G BMF QSFP, when the test is initiated
using "portdporttest --start" command option.
FOS-836491 flow CLI does not report TIMEOUT count for an Initiator or Target device.
Broadcom FOS-910b-RN
66
Fabric OS v9.1.0b Fabric OS v9.1.0b Release Notes Digest
Revision History
Broadcom FOS-910b-RN
67