View Issue Details

IDProjectCategoryView StatusLast Update
0000083Rocky-Linux-8perftestpublic2022-03-28 16:18
ReporterUlrich Sibiller Assigned ToRelease Engineering  
PrioritynormalSeveritymajorReproducibilityalways
Status assignedResolutionopen 
Summary0000083: perftest compilation difference to RHEL8.5
Descriptionin the perftest package there are several utils, e.g. ib_write_bw. The rocky version offers different command line switches than the RHEL version:

Rocky 8.5:
# rpm -qi perftest
Name : perftest
Version : 4.5
Release : 1.el8
Architecture: x86_64
Install Date: Tue 30 Nov 2021 10:28:41 AM CET
Group : Unspecified
Size : 2096269
License : GPLv2 or BSD
Signature : RSA/SHA256, Wed 13 Oct 2021 12:17:43 AM CEST, Key ID 15af5dac6d745a60
Source RPM : perftest-4.5-1.el8.src.rpm
Build Date : Tue 12 Oct 2021 10:21:18 PM CEST
Build Host : ord1-prod-x86build001.svc.aws.rockylinux.org
Relocations : (not relocatable)
Packager : infrastructure@rockylinux.org
Vendor : Rocky
URL : https://github.com/linux-rdma/perftest
Summary : IB Performance Tests
Description :
Perftest is a collection of simple test programs designed to utilize
RDMA communications and provide performance numbers over those RDMA
connections. It does not work on normal TCP/IP networks, only on
RDMA networks.
# ib_write_bw --help | grep dis
      --disable_pcie_relaxed Disable PCIe relaxed ordering
#

RHEL:
# rpm -qi perftest
Name : perftest
Version : 4.5
Release : 1.el8
Architecture: x86_64
Install Date: Mon 28 Mar 2022 04:25:10 PM CEST
Group : Unspecified
Size : 2087805
License : GPLv2 or BSD
Signature : RSA/SHA256, Wed 19 May 2021 08:38:24 AM CEST, Key ID 199e2f91fd431d51
Source RPM : perftest-4.5-1.el8.src.rpm
Build Date : Wed 19 May 2021 01:28:41 AM CEST
Build Host : x86-037.build.eng.bos.redhat.com
Relocations : (not relocatable)
Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Vendor : Red Hat, Inc.
URL : https://github.com/linux-rdma/perftest
Summary : IB Performance Tests
Description :
Perftest is a collection of simple test programs designed to utilize
RDMA communications and provide performance numbers over those RDMA
connections. It does not work on normal TCP/IP networks, only on
RDMA networks.
# ib_write_bw --help | grep dis
#

-> RHEL does not know --disable_pcie_relaxed

As this command line switch is activated at compile time depending on the IBV_ACCESS_RELAXED_ORDERING flag being present in /usr/include/infiniband/verbs.h it seems that the Rocky compilation environment differs from the one of RHEL which is bad in terms of compatibility. (Note that the flag is also present on RHEL, but obviously not at compile time, it seems. So _maybe_ this is a RHEL bug, but as Rocky claims compatibility it should also be bug compatible...)

(I have not checked if this is valid for other switches in the perftest suite, too)
TagsNo tags attached.

Activities

Louis Abel

Louis Abel

2022-03-28 15:52

administrator   ~0000097

To preface this, we do not make changes to perftest (you can verify this by searching git.rockylinux.org/staging/patch and looking for perftest) and our build environment and packages generally match the base mock package set templates when you install mock on an EL or Fedora system and use it to build a source package. These packages are closely related to how CentOS does their builds in koji for stream and is likely close to what Red Hat does in their own brew system.

With that being said, I rebuilt the package using mock on rhel 8. This was the result.

# mock -r rhel-8-x86_64 perftest-4.4-8.el8.src.rpm

...

# rpm -qip perftest-4.4-8.el8.x86_64.rpm
Name : perftest
Version : 4.4
Release : 8.el8
Architecture: x86_64
Install Date: (not installed)
Group : Unspecified
Size : 2107709
License : GPLv2 or BSD
Signature : (none)
Source RPM : perftest-4.4-8.el8.src.rpm
Build Date : Mon 28 Mar 2022 03:23:20 PM UTC
Build Host : repocompare.rockylinux.org
Relocations : (not relocatable)
URL : https://github.com/linux-rdma/perftest
Summary : IB Performance Tests
Description :
Perftest is a collection of simple test programs designed to utilize
RDMA communications and provide performance numbers over those RDMA
connections. It does not work on normal TCP/IP networks, only on
RDMA networks.

...

# dnf install ./perftest-4.4-8.el8.x86_64.rpm -y

...

# ib_write_bw --help | grep dis
      --disable_pcie_relaxed Disable PCIe relaxed ordering

It is hard to argue whether this is a bug with RHEL or not because the same behavior you are seeing in Rocky can be easily replicated by rebuilding perftest on RHEL 8.5. What it seems to me (by looking at the build times on the CentOS koji https://koji.mbox.centos.org/koji/buildinfo?buildID=17571) is:

  * The package was built during the initial 8.5 development cycle (8.4 was in beta or close to release around this time)
  * The package was never rebuilt or updated to accommodate the package updates or changes in dependencies of perftest (which would dictate the appearance of this option)

It is true that we should remain completely and fully compatible, but this is something where it is unclear what has to be patched or changed to remove this option to match RHEL.

Setting to NEEDINFO.
Louis Abel

Louis Abel

2022-03-28 15:56

administrator   ~0000098

I should note too that while 4.4-8 was rebuilt, the same behavior occurs with the presented version 4.5-1 on this bug report.

# dnf install ./perftest-4.5-1.el8.x86_64.rpm
# ib_write_bw --help | grep dis
      --disable_pcie_relaxed Disable PCIe relaxed ordering

Which leads me to believe it has something to do with the version of the deps used at the time of 4.5-1's initial compile in RHEL.
Ulrich Sibiller

Ulrich Sibiller

2022-03-28 16:02

reporter   ~0000099

The question is who coul provide the required info...

I have checked the packages for pertest 4.5-1.el8 on rocky85, alma85, centos85 and rhel85 and found more:

The centos, alma and rhel versions are identical (featurewise and linkingwise) while the rockypackage is different.

Regarding linking:

# ldd ./ib_write_bw_rocky85
        linux-vdso.so.1 (0x00007ffe440dc000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f72eee70000)
        libmlx5.so.1 => /lib64/libmlx5.so.1 (0x00007f72eec1d000)
        libefa.so.1 => /lib64/libefa.so.1 (0x00007f72eea16000)
        librdmacm.so.1 => /lib64/librdmacm.so.1 (0x00007f72ee7fb000)
        libibverbs.so.1 => /lib64/libibverbs.so.1 (0x00007f72ee5db000)
        libpci.so.3 => /lib64/libpci.so.3 (0x00007f72ee3cd000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f72ee1ad000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f72edde8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f72ef424000)
        libnl-3.so.200 => /lib64/libnl-3.so.200 (0x00007f72edbc5000)
        libnl-route-3.so.200 => /lib64/libnl-route-3.so.200 (0x00007f72ed93f000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f72ed73b000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f72ed524000)

The libresolv.so.2 is not required by the centos, alma and rhel versions. Only the rocky version requires it.
Louis Abel

Louis Abel

2022-03-28 16:18

administrator   ~0000100

So does the rebuilt version as presented in the comments.

# rpm -qip perftest-4.5-1.el8.x86_64.rpm
Name : perftest
Version : 4.5
Release : 1.el8
Architecture: x86_64
Install Date: (not installed)
Group : Unspecified
Size : 2095501
License : GPLv2 or BSD
Signature : (none)
Source RPM : perftest-4.5-1.el8.src.rpm
Build Date : Mon 28 Mar 2022 03:54:47 PM UTC
Build Host : repocompare.rockylinux.org
Relocations : (not relocatable)
URL : https://github.com/linux-rdma/perftest
Summary : IB Performance Tests
Description :
Perftest is a collection of simple test programs designed to utilize
RDMA communications and provide performance numbers over those RDMA
connections. It does not work on normal TCP/IP networks, only on
RDMA networks.

# ldd /usr/bin/ib_write_bw
        linux-vdso.so.1 (0x00007ffe74b33000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f2ca2cb7000)
        libmlx5.so.1 => /lib64/libmlx5.so.1 (0x00007f2ca2a64000)
        libefa.so.1 => /lib64/libefa.so.1 (0x00007f2ca285d000)
        librdmacm.so.1 => /lib64/librdmacm.so.1 (0x00007f2ca2642000)
        libibverbs.so.1 => /lib64/libibverbs.so.1 (0x00007f2ca2422000)
        libpci.so.3 => /lib64/libpci.so.3 (0x00007f2ca2214000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f2ca1ff4000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f2ca1c2f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2ca326b000)
        libnl-3.so.200 => /lib64/libnl-3.so.200 (0x00007f2ca1a0c000)
        libnl-route-3.so.200 => /lib64/libnl-route-3.so.200 (0x00007f2ca1786000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f2ca1582000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f2ca136b000)

Which brings us back to my initial comment that the package was built during the initial 8.5 development and was never rebuilt or updated to accommodate updates or changes in deps.

Issue History

Date Modified Username Field Change