Diary of a SysAdmin

1. Is the date and time right on your computer?
2. Is the date and time right on server?
3. Does Kerberos SSH work from another PC?
4. Does ssh -K username@server work in a non-elevated PowerShell?
5. Does id username (or username@ad.example.com) work on server?
6. Can you resolve server?
7. Can you resolve server.ad.example.com?
8. Does the computer account for server exist in AD?
9. Is the computer account for server enabled in AD?
10. Does /etc/krb5.keytab exist on server?
11. Does the keytab have any data inside it?
12. Lock and unlock your PC (renews your TGT) and try again.
13. Are SPNs correct? (Note 1)
14. What does the PuTTY Event Log show? (Note 2)
15. What does the sshd server log show? (Note 3)
16. Has your password recently been changed?
17. Is sssd running?
18. Is server joined to a realm? (realm list)
19. Does sssd enforce fully qualified names? (Note 4)
20. Does sssd have any errors in its log? (Note 5)
21. Does sshd allow GSSAPI authentication? (Note 6)
22. Does disabling PAM help? (Note 7)
23. Are your /etc/krb5.conf and /etc/sssd/sssd.conf files sensible? (Note 8)
24. Does Kerberos work anywhere?
    

Note 1: SPNs

In a Windows environment, you want something like this:

RestrictedKrbHost/server.ad.example.com
RestrictedKrbHost/SERVER
host/server.ad.example.com
host/SERVER

host is a catch-all for a bunch of SPNs, defined here:

Note 2: PuTTY Event Log

Right-click the top of the window and click Event Log

Note 3: sshd server logs

Add LogLevel DEBUG3 to /etc/ssh/sshd_config, restart sshd and tail -f the logs

sudo echo "LogLevel DEBUG3" > /etc/ssh/sshd_config
sudo systemctl restart sshd
sudo tail -f /var/log/messages

Try log in again and watch the logs on the server

Note 4: sssd fully qualified names

Does Kerberos SSH work with username@ad.example.com but not username?

If so, set use_fully_qualified_names to False in /etc/sssd/sssd.conf

[sssd]
domains = ad.example.com
... omitted for brevity

[domain/ad.example.com]
... omitted for brevity
use_fully_qualified_names = False
... omitted for brevity

Note 5: sssd logs

See what journalctl -xeu sssd shows.

Note 6: sshd GSSAPI configuration

Make sure this is present in /etc/ssh/sshd_config:

GSSAPIAuthentication yes

Note 7: disabling PAM for sshd

Add this line to /etc/ssh/sshd_config:

UsePAM no

Note 8: sensible krb5.conf and sssd.conf files

/etc/krb5.conf:

[logging]
    default = FILE:/var/log/krb5libs.log
    default = SYSLOG

[libdefaults]
    default_realm = AD.EXAMPLE.COM
    allow_weak_crypto = false
    dns_lookup_kdc = false
    dns_lookup_realm = false
    ticket_lifetime = 96h
    renew_lifetime = 7d
    forwardable = true
    rdns = false

[realms]
    AD.EXAMPLE.COM = {
        kdc = ad.example.com
        admin_server = ad.example.com
    }

[domain_realm]
    ad.example.com = AD.EXAMPLE.COM

/etc/sssd/sssd.conf:

[sssd]
domains = ad.example.com
config_file_version = 2
services = nss, pam, ssh, sudo
disable_mkhomedir_flags = true

[domain/ad.example.com]
id_provider = ad
access_provider = ad

ad_domain = ad.example.com
ad_server = ad.example.com
krb5_realm = AD.EXAMPLE.COM
realmd_tags = joined-with-samba
cache_credentials = True
krb5_store_password_if_offline = True
default_shell = /bin/bash
use_fully_qualified_names = False
krb5_lifetime = 7d
krb5_renew_interval = 500
krb5_renewable_lifetime = 7d
ldap_id_mapping = True