View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0007657Rocky-Linux-8rsyslogpublic2024-08-16 19:402024-08-21 08:42
ReporterAditya Desai Assigned ToLouis Abel  
PriorityhighSeveritycrashReproducibilitysometimes
Status needinfoResolutionopen 
Summary0007657: rsyslogd - segfaults while trying to stop process
DescriptionWhile running tests with tracking for any user space/kernel core dumps, rsyslogd segfault was seen when we try to stop the rsyslogd process.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1568 2024-07-30T19:43:49.055144+00:00 automated-tests systemd[1]: journald_log_capture.service: Succeeded.
1569 2024-07-30T19:43:49.055289+00:00 automated-tests systemd[1]: Stopped Service to capture journald-exclusive logs in a file after boot.
1570 2024-07-30T19:43:49.055345+00:00 automated-tests systemd[1]: Stopping Service to capture journald-exclusive logs in a file after boot...
1571 2024-07-30T19:43:49.055402+00:00 automated-tests systemd[1]: Stopping System Logging Service...
1572 2024-07-30T19:43:49.640844+00:00 automated-tests kernel: [ 683.407007] show_signal_msg: 26 callbacks suppressed
1573 2024-07-30T19:43:49.643498+00:00 automated-tests kernel: [ 683.407010] rsyslogd[10739]: segfault at 8 ip 000055bd2fc5cfab sp 00007ffeb2b405 f8 error 4 in rsyslogd[55bd2fc00000+a7000] likely on CPU 0 (core 0, socket 0)
1574 2024-07-30T19:43:49.643501+00:00 automated-tests kernel: [ 683.407019] Code: c0 01 48 39 d0 ba 00 00 00 00 48 0f 44 c2 48 89 87 30 02 00 00 31 c0 c3 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8b 87 48 02 00 00 <48> 8b 50 08 48 8b 00 48 89 16 48 89 87 48 02 00 00 31 c0 c3 90 f3
1575 2024-07-30T19:43:49.057461+00:00 automated-tests rsyslogd[10739]: [origin software="rsyslogd" swVersion="8.2102.0-15.el8" x-pid="10739" x-in fo="https://www.rsyslog.com"] exiting on signal 15.
 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Panic stack trace:

Not a huge log bundle, attaching it to Jira
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
16839 2024-07-30T19:43:49.057802+0000 automated-tests audisp-syslog[1049]: node=localhost.localdomain type=ANOM_ABEND msg=audit(1722368629.051:45 0): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:syslogd_t:s0 pid=10739 comm="rsyslogd" exe="/usr/sbin/rsyslogd" sig=11 res =1 AUID="unset" UID="root" GID="root"
16840 2024-07-30T19:43:49.059047+0000 automated-tests kernel: show_signal_msg: 26 callbacks suppressed
16841 2024-07-30T19:43:49.059121+0000 automated-tests kernel: rsyslogd[10739]: segfault at 8 ip 000055bd2fc5cfab sp 00007ffeb2b405f8 error 4 in r syslogd[55bd2fc00000+a7000] likely on CPU 0 (core 0, socket 0)
16842 2024-07-30T19:43:49.059146+0000 ants-automated-tests kernel: Code: c0 01 48 39 d0 ba 00 00 00 00 48 0f 44 c2 48 89 87 30 02 00 00 31 c0 c3 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8b 87 48 02 00 00 <48> 8b 50 08 48 8b 00 48 89 16 48 89 87 48 02 00 00 31 c0 c3 90 f3
16843 2024-07-30T19:43:49.145796+0000 automated-tests systemd[1]: Created slice system-systemd\x2dcoredump.slice.
16844 2024-07-30T19:43:49.163503+0000 automated-tests systemd[1]: Started Process Core Dump (PID 29815/UID 0).
16845 2024-07-30T19:43:49.163756+0000 automated-tests audisp-syslog[1049]: node=localhost.localdomain type=SERVICE_START msg=audit(1722368629.159 :451): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-coredump@0-29815-0 comm="systemd" exe="/usr /lib/systemd/systemd" hostname=? addr=? terminal=? res=success' UID="root" AUID="unset"
16846 2024-07-30T19:43:49.371319+0000 automated-tests systemd-coredump[29816]: Process 10739 (rsyslogd) of user 0 dumped core.
16847
16848 Stack trace of thread 10739:
16849 #0 0x000055bd2fc5cfab qDeqLinkedList (rsyslogd)
16850 #1 0x000055bd2fc5e9d4 queueDrain (rsyslogd)
16851 #2 0x000055bd2fc5ea5d qDestructLinkedList (rsyslogd)
16852 #3 0x000055bd2fc60d57 qqueueDestruct (rsyslogd)
16853 #4 0x000055bd2fc6b905 actionDestruct (rsyslogd)
16854 #5 0x000055bd2fc31f99 cnfstmtDestruct (rsyslogd)
16855 #6 0x000055bd2fc31e59 cnfstmtDestructLst (rsyslogd)
16856 #7 0x000055bd2fc63b4e doDestructCnfStmt (rsyslogd)
16857 #8 0x000055bd2fc4a061 llExecFunc (rsyslogd)
16858 #9 0x000055bd2fc63c8e destructAllActions (rsyslogd)
16859 #10 0x000055bd2fc16f25 main (rsyslogd)
16860 #11 0x00007fd487c3a7e5 __libc_start_main (libc.so.6)
16861 #12 0x000055bd2fc1725e _start (rsyslogd)
16862 2024-07-30T19:43:49.379528+0000 automated-tests systemd[1]: systemd-coredump@0-29815-0.service: Succeeded.
16863 2024-07-30T19:43:49.380615+0000 automated-tests systemd[1]: rsyslog.service: Main process exited, code=dumped, status=11/SEGV
16864 2024-07-30T19:43:49.382129+0000 automated-tests audisp-syslog[1049]: node=localhost.localdomain type=SERVICE_STOP msg=audit(1722368629.375: 452): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-coredump@0-29815-0 comm="systemd" exe="/usr/ lib/systemd/systemd" hostname=? addr=? terminal=? res=success' UID="root" AUID="unset"
16865 2024-07-30T19:43:49.380688+0000 automated-tests systemd[1]: rsyslog.service: Failed with result 'core-dump'.
16866 2024-07-30T19:43:49.382137+0000 automated-tests audisp-syslog[1049]: node=localhost.localdomain type=SERVICE_STOP msg=audit(1722368629.375: 453): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=rsyslog comm="systemd" exe="/usr/lib/systemd/systemd " hostname=? addr=? terminal=? res=failed' UID="root" AUID="unset"
 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
This was from rsyslog version
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[root@Macon01-4 ~]# yum info rsyslog
Last metadata expiration check: 3:09:27 ago on Thu 01 Aug 2024 04:57:20 PM UTC.
Installed Packages
Name : rsyslog
Version : 8.2102.0
Release : 15.el8
Architecture : x86_64
Size : 2.4 M
Source : rsyslog-8.2102.0-15.el8.src.rpm
Repository : @System
From repo : iso-unmodular
Summary : Enhanced system logging and kernel message trapping daemon
URL : http://www.rsyslog.com/
License : (GPLv3+ and ASL 2.0)
Description : Rsyslog is an enhanced, multi-threaded syslog daemon. It supports MySQL,
             : syslog/TCP, RFC 3195, permitted sender lists, filtering on any message part,
             : and fine grain output format control. It is compatible with stock sysklogd
             : and can be used as a drop-in replacement. Rsyslog is simple to set up, with
             : advanced features suitable for enterprise-class, encryption-protected syslog
             : relay chains.


[root@Macon01-4 ~]#
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Looks like this was fixed in the upstream in https://github.com/rsyslog/rsyslog/commit/74a49d3b63acf62a713f1b9e9e6672712607c77b#diff-6706842f0e8d067a33945987affabdaae4acbcfd6a7b5291efaca1d4718cbf5f
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The version from Rocky9 which is version 8.2310.0-4.el9
as shown in https://rockylinux.pkgs.org/9/rockylinux-appstream-x86_64/rsyslog-8.2310.0-4.el9.x86_64.rpm.html
contains the patch for this bug.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Would it be possible to backport this patch from Rocky 9 to Rocky 8??
Steps To ReproduceRepeatedly restart rsyslog
TagsNo tags attached.

Activities

Louis Abel

Louis Abel

2024-08-21 08:42

administrator   ~0008221

Thank you for the report. And apologies for any spam mail you got - We are currently fighting a spam issue.

>While running tests with tracking for any user space/kernel core dumps, rsyslogd segfault was seen when we try to stop the rsyslogd process.

Unfortunately, it's unclear what you are doing on the system. What are the actual "tests" you running and what are you doing on the system that ends up leading to a segfault on rsyslog? How is rsyslog configured? The logs provided is not enough to go on and is not enough to reproduce this issue.

Please provide more information as to what is being done (step by step) that leads to the segfault while stopping rsyslog. Please also include the rsyslog configuration if you have made any changes to it.

---

Note: As we aim to be compatible with RHEL, we don't backport patches into Rocky Linux. In the event that this is reproducible on our test systems, it will also need to be tested and verified on a RHEL 8 system. If it is reproducible on RHEL 8, then an issue will need to be opened at https://issues.redhat.com.

Issue History

Date Modified Username Field Change
2024-08-16 19:40 Aditya Desai New Issue
2024-08-21 08:42 Louis Abel Assigned To => Louis Abel
2024-08-21 08:42 Louis Abel Status new => needinfo
2024-08-21 08:42 Louis Abel Note Added: 0008221