View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0005611 | Rocky-Linux-9 | sssd | public | 2024-01-27 01:45 | 2024-12-02 08:41 |
Reporter | Dennis Brylow | Assigned To | Louis Abel | ||
Priority | urgent | Severity | major | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Platform | x86_64 | OS | Rocky | OS Version | 9.3 |
Fixed in Version | 9.5 | ||||
Summary | 0005611: sssd-2.9.1-4.el9_3.5 breaks authentication for all Active Directory users. | ||||
Description | Dozens of machines lost the ability to authenticate any Active Directory (AD) users with the automatic upgrade to sssd-2.9.1-4.el9_3.5 and its associated packages this week. Sshd reports in /var/log/secure: sshd[420278]: pam_sss(sshd:account): Access denied for user foo: 4 (System error) sshd[420278]: fatal: Access denied for user foo by PAM account configuration [preauth]. GUI does not authenticate. Console prints "System error", and then reprompts for login. Machines are still on the AD realm, and can pull info with 'getent passwd username'. Superuser can su to AD users and operate as normally. All local accounts appear to be functioning. But all AD accounts cannot authenticate, and the log messages are surprisingly terse and unhelpful. Issue can be fixed by forcing downgrade back to sssd-2.9.1-4.el9_3.1.x86_64.rpm and 17 associated sssd-*, libsss_* and libipa_hbac packages. | ||||
Steps To Reproduce | 1) Install Minimal Rocky-9.3. 2) Install latest versions of realmd, sssd and associated co-requisites. 3) Join machine to AD realm and configure sssd.conf file. 4) Enjoy total absence of working AD authentication for all of your remote and console users. | ||||
Additional Information | All prior versions of sssd have worked out-of-the-box in this Active Directory configuration since at least Rocky 8 and CentOS 7. Full list of downgraded packages to effect solution: libipa_hbac-2.9.1-4.el9_3.1.x86_64.rpm libsss_autofs-2.9.1-4.el9_3.1.x86_64.rpm libsss_certmap-2.9.1-4.el9_3.1.x86_64.rpm libsss_idmap-2.9.1-4.el9_3.1.x86_64.rpm libsss_nss_idmap-2.9.1-4.el9_3.1.x86_64.rpm libsss_sudo-2.9.1-4.el9_3.1.x86_64.rpm sssd-2.9.1-4.el9_3.1.x86_64.rpm sssd-ad-2.9.1-4.el9_3.1.x86_64.rpm sssd-client-2.9.1-4.el9_3.1.x86_64.rpm sssd-common-2.9.1-4.el9_3.1.x86_64.rpm sssd-common-pac-2.9.1-4.el9_3.1.x86_64.rpm sssd-ipa-2.9.1-4.el9_3.1.x86_64.rpm sssd-kcm-2.9.1-4.el9_3.1.x86_64.rpm sssd-krb5-2.9.1-4.el9_3.1.x86_64.rpm sssd-krb5-common-2.9.1-4.el9_3.1.x86_64.rpm sssd-ldap-2.9.1-4.el9_3.1.x86_64.rpm sssd-nfs-idmap-2.9.1-4.el9_3.1.x86_64.rpm sssd-proxy-2.9.1-4.el9_3.1.x86_64.rpm | ||||
Tags | No tags attached. | ||||
Thank you for the report. "4 (System error)" is an unhandled exception. This generally means that something is happening between SSSD and its auth provider (in this case AD) that it's not expecting and can't proceed. With that said, I have been unable to reproduce this issue with this configuration: * Windows Server 2022, Forest Level 2016, fully updated. * Rocky Linux 9.3, fully updated. Enrolled with a standard configuration setup. % dnf install realmd -y % rpm -q sssd-ad realm realmd-0.17.1-1.el9.x86_64 sssd-ad-2.9.1-4.el9_3.5.x86_64 % realm join ad.angelsofclockwork.net Password for Administrator: * Installing necessary packages: oddjob-mkhomedir oddjob % id label@AD.ANGELSOFCLOCKWORK.NET uid=588201103(label@ad.angelsofclockwork.net) gid=588200513(domain users@ad.angelsofclockwork.net) groups=588200513(domain users@ad.angelsofclockwork.net) % ssh label@ad.angelsofclockwork.net@localhost label@ad.angelsofclockwork.net@localhost's password: [label@ad.angelsofclockwork.net@rl9 ~]$ id uid=588201103(label@ad.angelsofclockwork.net) gid=588200513(domain users@ad.angelsofclockwork.net) groups=588200513(domain users@ad.angelsofclockwork.net) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 GDM also works. Jan 26 19:59:40 localhost gdm-password][37831]: pam_sss(gdm-password:auth): authentication success; logname= uid=0 euid=0 tty=/dev/tty1 ruser= rhost= user=label@ad.angelsofclockwork.net Jan 26 19:59:40 localhost gdm-password][37831]: pam_unix(gdm-password:session): session opened for user label@ad.angelsofclockwork.net(uid=588201103) by (uid=0) Please provide the following information: * AD Domain Forest Level * SSSD configuration - /etc/sssd/sssd.conf * SSSD debug logs - See https://sssd.io/troubleshooting/basics.html for debugging steps. Setting to needinfo. |
|
Thanks for taking a look at this with us, Louis. Domain and Forest functional level: Windows Server 2016. sssd.conf: --cut_here-- [sssd] domains = mscsnet.mu.edu config_file_version = 2 services = nss, pam [nss] enum_cache_timeout = 600 entry_cache_nowait_percentage = 60 [domain/mscsnet.mu.edu] ad_domain = mscsnet.mu.edu krb5_realm = MSCSNET.MU.EDU realmd_tags = manages-system joined-with-samba cache_credentials = True entry_cache_timeout = 28800 id_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = False use_fully_qualified_names = False access_provider = ad ldap_schema = ad auth_provider = ad chpass_provider = ad auto_private_groups = true ldap_access_order = pwd_expire_policy_renew dyndns_update_ptr = False --cut_here-- var/log/secure error (one of many examples): --cut_here-- Jan 26 13:53:17 npascal sshd[2066079]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=134.48.5.61 user=testuser Jan 26 13:53:17 npascal sshd[2066079]: pam_sss(sshd:account): Access denied for user testuser: 4 (System error) Jan 26 13:53:17 npascal sshd[2066079]: Failed password for testuser from 134.48.5.61 port 49737 ssh2 Jan 26 13:53:17 npascal sshd[2066079]: fatal: Access denied for user testuser by PAM account configuration [preauth] --cut_here-- I will grab some SSSD debugging logs for you this afternoon. |
|
Louis, No change here -- every time one of our Linux clients inadvertently upgrades to sssd-2.9.1-4.el9_3.5 and associated packages, all AD authentication breaks. No updates or bugfixes have been forthcoming from upstream that I can see. Our AD administrator says that our domain is actually Windows Server 2019, objectVersion = 88. We've started to see other reports of this problem popping up for others, but still no fixes. Our best guess is that something about our AD domain differs from whatever testing context you and others are using. But the AD domain has been the primary authentication source for both Windows and Linux clients here for at least four years. Please let me know if there is any more debugging info we can help provide to try to track this issue down. Thanks! |
|
I may have been hitting this same issue as well. Possibly. The symptoms look very similar. Updated from 2.9.1-4.el9_3 to 2.9.1-4.el9_3.5 and logins fail. GPO access control is in use. The issue was tracked, using the SSSD logs, down to SSSD not finding the computer account of the machine being logged into. It may be *partially* related to my configuration because `ldap_user_search_base` does *not* include computer accounts. Computer accounts are located in another OU not in this subtree. Changing `ldap_user_search_base` to an OU that includes both computer + user accounts make everything work again, just like in 2.9.1-4. It should be noted that specifying `ldap_host_search_base` to the subtree containing computer accounts did not help at all! @Dennis Brylow, can you please check if in your case computer accounts are also outside of the "user" search base configured for SSSD? |
|
Not trying to hijack this but further investigation looks like I was on the correct track. I hope the OP could share some information on whether GPO access control is used. My particular issue will be most likely resolved when https://github.com/SSSD/sssd/commit/29a77c6e79020d7e8cb474b4d3b394d390eba196 is picked to the RH SSSD release. While looking at this I noticed another possibly relevant to the OP fix that's not yet in RH SSSD: `RHEL-22288 - ssh pubkey stored in ldap/AD no longer works to authenticate via sssd` -- is OP logging in with SSH pubkey stored in LDAP? |
|
Rocky Linux 9.5 has been released with version 2.9.5-4.el9_5.1. Please confirm if this issue still exists on this version. Keeping "needinfo" status. |
|
Nope. Issue does *not* appear to exist in Rocky 9.5 with sssd-2.9.5-4.el9_5.1. Thanks for the attention! @Voja Molani sorry I didn't see your question back in March, but yes, our computer accounts are stored outside of the user search base. |
|
Issue appears to be fixed in 9.5. Closing. | |
Date Modified | Username | Field | Change |
---|---|---|---|
2024-01-27 01:45 | Dennis Brylow | New Issue | |
2024-01-27 03:03 | Louis Abel | Assigned To | => Louis Abel |
2024-01-27 03:03 | Louis Abel | Status | new => needinfo |
2024-01-27 03:03 | Louis Abel | Note Added: 0005710 | |
2024-01-30 15:39 | Dennis Brylow | Note Added: 0005743 | |
2024-03-01 19:39 | Dennis Brylow | Note Added: 0006271 | |
2024-03-19 09:27 | Voja Molani | Note Added: 0006469 | |
2024-03-20 04:02 | Voja Molani | Note Added: 0006502 | |
2024-11-27 01:44 | Louis Abel | Note Added: 0008917 | |
2024-12-01 22:42 | Dennis Brylow | Note Added: 0008944 | |
2024-12-02 08:41 | Louis Abel | Status | needinfo => closed |
2024-12-02 08:41 | Louis Abel | Resolution | open => fixed |
2024-12-02 08:41 | Louis Abel | Fixed in Version | => 9.5 |
2024-12-02 08:41 | Louis Abel | Note Added: 0008977 |