Home > Uncategorized > Hacking sshd for a pass_file

Hacking sshd for a pass_file

September 28, 2009

First of all, I want to apologize for not getting around to writing part 2 of my previous post yet. I have more free time now and have started research for that post, but haven’t had a chance to write everything down yet. Hopefully I’ll get to it soon.

Update: some people have been confused at to my intention or my recommended use of the code I present here.  Let me make a few things clear:

  1. Don’t make these modifications on any production machine
  2. Don’t make these modifications on any machine receiving a lot of traffic
  3. This isn’t the best way to capture logins.

I called it a “hack” for a reason.  It’s something I threw together in a few minutes in order to gather the necessary data to conduct the analysis I did in my next post.  As dozzie pointed out, this can be done better by writing a PAM module.  My purpose here was not to write something robust, rather to write something quick in order to find a password file being used against me.  I apologize for any confusion.


This post concerns another topic I’ve been interested in for a while: honeypots.  Some friends and I run an SSH server and have that server registered with DynDNS so that we will always have an easy route to our box without remembering an IP address.  A botnet master may find it to be a profitable endeavor to scan predictable (ours is predictable) DynDNS entries for popular services offered over known ports and attempt to guess credentials for those services.

Keeping in mind that:

  • only a fraction of the subdomains pinged will actually be registered with DynDNS
  • only a fraction of the registered subdomains will be offering authentication services
  • only a fraction of the authentication services will allow predictable usernames*, and
  • only a fraction of those valid usernames will have predictable passwords

* root logins aren’t allowed by default on openssh and many other SSH implementations.

The (hopefully) small number of boxes that can be owned by brute forcing with this method apparently outweighs the cost to our adversary(ies).  As I later discovered, our IP address wasn’t enumerated via our DynDNS entry, but was brute forced.  Yeah.  They are trying IP addresses sequentially.

We noticed the SSH logs for our box were getting suspiciously long and it was pretty obvious why:

09/24/2009 12:49:19 PM: [FAIL] An error occured during key exchange auth done
09/24/2009 12:49:19 PM: [NOTE] Connection from 118.46.137.101 disconnected
09/24/2009 12:49:20 PM: [FAIL] An error occured during key exchange auth done
09/24/2009 12:49:20 PM: [NOTE] Connection from 118.46.137.101 disconnected
(repeat about 100 times)...

Our gateway Snort agreed that something was up:

[ ** ] [ 1:2001219:18 ] ET SCAN Potential SSH Scan [ ** ]
[ Classification: Attempted Information Leak ] [ Priority: 2 ]
09/17-10:49:59.339210 118.46.137.101:50905 -> ***.***.***.***:22
(repeat about 100 times)...

The attacking IP addresses would change periodically.

Perhaps I could discover if this is a single attacker or if this is multiple attackers:

  • If there is a single group behind these attacks, it would make sense that they would synchronize this work amongst the attacking IPs, allowing the attack to evade simple IDS rules and avoid duplication of effort.
  • If there are multiple parties behind these attacks, it would make sense that the same username/password combinations would be tried by different hosts, pointing to a lack of synchronization.

Of course this is a lot of assuming and is hardly scientific, but promises to be a fun experiment regardless.

My first thought was: I’ll build a honeynet!  After reading more about honeynets, however, I came to realize that a honeypot would require a lot of network work and a tough cost/benefit analysis. The problem is that a smart attacker will first check his/her newly compromised environment: is he/she root? is he/she in an obvious VM or jail? what others hosts are on this subnet?

If the attacker isn’t satisfied that what they’ve compromised is a unwitting user’s box (and not a honeypot), they may never execute telling commands or push interesting payloads. On the other hand, if you give the attacker too much access, the attacker may use your box to attack others, host child pornography or conduct other malicious/illegal actions.  To everyone else it will look like your box (and by extension, you) are doing these illegal things. In such a scenario, you would be presumed guilty unless you can prove you’re running a honeypot and aren’t actually the person breaking the law.

Having a severe lack of lawyers at my side (I do know a few poly sci majors), I opted to go a different route, at least for now:

I’ll modify sshd itself, causing it to log the time and origin of all attempts to authenticate, along with the complete usernames & passwords attempted.

This is not a new idea, in fact, it’s kinda what honeyd is for, but I thought it would be fun to do the ssh modification myself and follow the password trail to see where it leads.  (Where these harvested passwords lead will be the topic of my next post.)

For obvious reasons, openssh and others never log incorrect passwords (a mistype of your password would get winblowz logged when you meant winblows…such logging would make it trivial to escalate privilege).

Setting up the Server:

I chose to use VirtualBox on a Windows XP machine to virtualize Ubuntu 9.04 Desktop, on which I will be serving SSH with openssh.  VirtualBox is like VMware Workstation except it’s free (as in speech).  The process of creating and configuring a VM is outside the scope of this post.  Don’t do this on a production machine or any machine that has multiple users, as privilege escalation may become trivial.

The rest of these instructions will be valid for Ubuntu 9.04 Desktop’s default directory structure, installed software and openssh-5.1p1.  They can easily be adapted to other environments & versions of openssh.  The instructions listed here result in multiple installations of openssh-server.  I did’t really care about overlap in this throwaway VM environment, so long as I could get my modified sshd running with Ubuntu’s daemon manager.  A purist might do this another way.

1) Install the required dependencies for building openssh:

sudo apt-get install zlib1g-dev libssl-dev

2) Install openssh-server itself (we’ll modify the default installation):

sudo apt-get install openssh-server

3) Check which version of openssh you’re running:

ssh -v

4) Get the source code of the version of openssh you’re currently running (by the using the same version we may avoid odd version dependency issues).

5) (Optional) Download the corresponding .asc file & verify your copy of openssh.

gpg --recv-key 86FF9C48
gpg --verify openssh-5.1p1.tar.gz.asc openssh-5.1p1.tar.gz

(The above is the signing key for Damien Miller, maintainer of portable openssh. Try to guess which is him.)

Of course, if you’re really concerned about the integrity of your openssh download, you’ll want to verify gpg fingerprints as well.

Hacking sshd:

1) Extract the source & verify you can successfully build it:

tar -xvf openssh-5.1p1.tar.gz
cd openssh-5.1p1
./configure
make
ls -al | grep sshd

If you see an sshd binary, you compiled it.

2) Stop the sshd daemon:

sudo /etc/init.d/ssh stop

3) Install the openssh build you just created. (This is to put the config files, etc. in locations that our modified sshd will expect, while breaking very little of the Ubuntu package installation.  Since we’re not going to modify the config files, we don’t need to worry about syncing changes between them.)

sudo make install

4) Back up your current sshd binary (just in case):

sudo cp /usr/sbin/sshd /usr/sbin/sshd_original

5) Make the following modification to openssh-5.1p1/auth-passwd.c:

--- auth-passwd_original.c    2007-10-25 21:25:12.000000000 -0700
+++ auth-passwd.c    2009-09-28 21:35:04.000000000 -0700
@@ -53,6 +53,7 @@
 #include "hostfile.h"
 #include "auth.h"
 #include "auth-options.h"
+#include "canohost.h"

 extern Buffer loginmsg;
 extern ServerOptions options;
@@ -82,6 +83,23 @@
 {
 struct passwd * pw = authctxt->pw;
 int result, ok = authctxt->valid;
+
+    if (*password != '')
+    {
+        struct tm *timePtr;
+        time_t localTime;
+        char timeString[100];
+
+        localTime = time(NULL);
+        timePtr = localtime(&localTime);
+        strftime(timeString, 100, "%D %r", timePtr);
+
+        FILE *logFile;
+        logFile = fopen("/var/log/sshd_attempts","a+");
+        fprintf (logFile,"From: %s at: %s | user: %s, pass: %s\n", \
get_remote_ipaddr(), timeString, authctxt->user, password);
+        fclose (logFile);
+    }
+
 #if defined(USE_SHADOW) && defined(HAS_SHADOW_EXPIRE)
 static int expire_checked = 0;
 #endif

Or you can grab my modified auth-passwd.c file and throw it in your openssh source directory.

6) Rebuild sshd:

make

7) Stop SSH, replace sshd, Start SSH:

sudo /etc/init.d/ssh stop
sudo cp sshd /usr/sbin/sshd
sudo /etc/init.d/ssh start

You’re done.  You should now have a modified sshd binary running your openssh server and logging all connection attempts to /var/log/sshd_attempts.  I’ve been running my modified sshd for a few days now and have collected quite a few of these attempts. Click below to view the connection attempts against my server.

View my Log

Part 2: From pass_file to Script Kiddies

Advertisements
  1. September 29, 2009 at 12:57 pm

    Nice, reminds me of popular patching done a while back that “fixed” ssh to log ips/usernames/passwords

    • krupt
      September 29, 2009 at 10:30 pm

      reverse of a ssh backdoor nothing new here champ.

      • September 29, 2009 at 10:53 pm

        No argument there, although the specifics of doing this might be of interest to some.

        Regardless, the real point of this post is to serve as an intro to my next post, where I use the harvested passwords to identify the tools used against me and attempt to track down those running the attacks. I hope to have that posted tomorrow.

  2. Henri Salo
    September 30, 2009 at 12:18 pm

    “The problem is that a smart attacker will first check his/her newly compromised environment: is he/she root? is he/she in an obvious VM or jail? what others hosts are on this subnet?”

    My personal statistics say that 97% of attacks are automated. Percent is obviously high, because automated attacks are more frequent, but still a person running honeypot should not normally be “afraid” of this problem. One can still catch these attacks using other methods and besides it is good to know how attacker is getting information of the system.

    If you are interested on honeypots you really should read “Virtual Honeypots: From Botnet Tracking to Intrusion Detection” written by Niels Provos and check out his webpage .

    • September 30, 2009 at 12:31 pm

      Thanks for the tip; at least with these attacks, however, I found evidence of privilege escalation attempts on other hosts, mostly via Linux kernel exploits.

  3. blieb
    September 30, 2009 at 4:28 pm

    I received this error:
    auth-passwd.c:87:26: error: empty character constant
    make: *** [auth-passwd.o] Error 1
    Making it double-quotes did the trick:
    if (*password != “”)

  4. dozzie
    October 1, 2009 at 3:03 am

    That’s a stupid approach, especially for sysadmin. You don’t have any experience in administration, do you?

    First of all, you should have built a Debian package instead of `./configure && make mess’ procedure. How would you like to upgrade OpenSSH? And would you remember that you have patched sshd (and remember what patches were used) when building it for the second time? And if you want to provide scponly accounts, how would you like to install that package if it requires openssh-server, which is not installed?

    And the other thing, patching OpenSSH for grabbing passwords is stupid itself. You could just write a PAM module which gathers and logs all the passwords (or just passwords that failed to authenticate users). And the module could be used for collecting passwords for FTP, SMTP and POP3 services, not only SSH.

    • Mario Vilas
      October 1, 2009 at 4:57 pm

      Writing a PAM module seems like an awful lot of work for a quick test. I would have done the same, patching openssh is the smartest thing to do for a temporary hack.

      I do agree that if you’re setting up an actual honeypot (instead of just tracking down this particular attacker) the PAM module seems like a better move.

    • October 1, 2009 at 4:59 pm

      @dozzie: You either totally missed the point or I failed at explaining myself adequately. First, I didn’t care about upgrading OpenSSH – this was a one-off modification being used in a VM for the sole purpose of collecting some passwords being used against me in order to conduct the analysis I did in my next post.

      I was only concerned with credentials being used against me. No sysadmin of a proper installation would implement the changes I presented, nor should they. Sysadmins of proper systems have far more concerns that I had.

      Implementing the changes I above introduces DoS potential as well as privilege escalation issues, among others I’m sure. I’ve captured all the data I need; I’ve stopped the capture and have trashed the VM.

      As for modifying OpenSSH vs writing a PAM module, OpenSSH came to mind first. For my purposes, a proper implementation wasn’t required. I’ve added a note at the top of this post to outline what you’ve mentioned in the hopes that I don’t mislead others.

  5. df
    October 1, 2009 at 8:37 pm

    Are you still generating password logs?

    • October 2, 2009 at 1:12 pm

      Yeah – I’m not using the computer for anything else right now and don’t really care if it crashes.

      I’m up to ~3000 attempts. Want a copy? I suppose I could email it to the address you commented with… I’m not going to replace the file I currently have hosted, since I don’t want to host yet another password file for everyone to use.

  6. Algol
    October 2, 2009 at 10:42 am

    Paul, nice work, thanks for sharing the information you have collected.
    Thanks again and keep the good work.

  7. somedude
    February 18, 2010 at 9:48 pm

    Like you said, this is nothing new overall since it’s done in honeyd. But it’s new, specifically to me. I suppose I can verify this myself, but I think it’s worth asking:

    Can you hack a sshd such that it’s transparent to the accounts that have previously added the sshd’s public key to their ssh client’s known_hosts?

    • February 19, 2010 at 1:32 am

      Sure – no hacking required for that. What’s stored in a users known_hosts file is a list of the hashes/fingerprints of public SSH keys public SSH keys that user had previously accepted, along with what server they belong to.

      If you were to modify sshd and wanted to avoid generating a warning for those who had used your previous sshd installation, simple move the public/private key pair from the old installation to the new one. Having said that, I have no personal experience doing so.

      This article: http://bit.ly/co1sMS says that the keys are typically stored in /etc/ssh/ on Linux boxes. Assuming you are root, moving these around won’t be an issue.

      Chances are the public/private key generation happened without your knowledge/input on the old sshd installation. A new installation, by default, would generate a new key pair. Just replace your new key pair with the old one and your users won’t know the difference.

      Correction: the entire public key is kept in known_hosts; the hash/fingerprint is displayed to the user upon first connect to a new SSH server.

  1. September 29, 2009 at 4:40 pm
  2. October 1, 2009 at 1:43 am
Comments are closed.
%d bloggers like this: