Karl’s Work Mac Setup

(This post is in beta)

So, what all do I install, and why?

  • The OpenSSH shipped in El Capitan is partially broken (GSSAPI Key Exchange doesn’t work).  So, we build our own, including GSSAPI authentication and GSSAPI Key Exchange.
  • MacPorts’ OpenSSH depends on MIT Kerberos, so we install that.
  • MacPorts’ remctl also depends on MIT Kerberos.
  • OpenSSL is needed because macOS doesn’t give you OpenSSL headers.
  • I also do a lot of Perl development.
  • Finally, Proxychains is useful for tunneling remctl through SSH.

This assumes that you’re running macOS 10.9 or later, and that you have admin access on your system.

  1. If you don’t have one, get an Apple account.
  2. Install Xcode from the App store.
    Although it’s possible to download the installer package, it’s easier to just install it from the App Store.  The only reason I’d install the package directly is if I was using some sort of system-management platform to push out software.
  3. Install the Command Line Tools that match your version of Xcode and macOS.
    Unfortunately, there is no App Store entry for this.
  4. Open a Terminal window, and view/accept the Xcode license.
    xcodebuild -license
  5. Download and run the MacPorts installer, for 10.11 El Capitan, 10.10 Yosemite, or 10.9 Mavericks.
  6. In a Terminal window, update the package lists and build ported sofware:
    sudo port selfupdate
    sudo port install kerberos5 remctl openssl proxychains-ng openssh +kerberos5 +gsskex -xauth
  7. The MacPorts installer updates your PATH, but for some reason it doesn’t update your MANPATH.  Add the following line to your .bash_profile:
    export MANPATH="/opt/local/share/man:$MANPATH"
  8. If you use Eclipse, which ships its own Git implementation, you might want to ensure that it uses the OpenSSH you’ve installed, so add this line to your .bash_profile:
    export GIT_SSH=/opt/local/bin/ssh
  9. Copy and customize one of the SSH configs from my Stanford web space.  Even though the MacPorts version of OpenSSH is used, it’s not acting as the system ssh daemon, so you need to match that with the SSH config you download.
  10. Pull in BASH customizations:
    1. Copy all of my BASH files to your home directory, and then add a dot to the start of each filename.
    2. Make a symlink from .bash_stanford to either .bash_stanford_mit (if using MacPorts’ kerberos5 package) or .bash_stanford_heimdal (if using macOS’ built-in Kerberos).
    3. Finally, update your .bash_profile file to run the .bashrc:
      . ~/.bashrc
  11. To set up proxychains, create a file at ~/.proxychains/proxychains.conf, with the following content:
    tcp_read_time_out 15000
    tcp_connect_time_out 8000
    socks5 1080
  12. If you do a lot of Perl work, consider installing Perlbrew.  But, if you do, be sure to read my warning!  Also, consider if you want to enable Perlbrew in your .bash_profile before or after you set the PATH for MacPorts.  MacPorts can also install Perl, and Perl packages, so if you aren’t careful you’ll get weird clashing between your Perl and MacPorts’ Perl.
  13. If you do anything with PGP/GPG, including signing, encrypting, or authenticating, then install the GPG Suite.  In my case, I’m doing package signing (Debian packages) using a hardware key (on a Yubikey 4), so I need the functionality that the GPG Suite provides.

That’s it!  Most of the software (like Kerberos and OpenSSH) is documented elsewhere.  Here are my specific usage notes for other stuff.

To switch which Kerberos principal you’re using, use the aliases pag (to switch to regular principal), rootpag (root principal), adminpag (admin principal), or sunetpag (sunet principal).  The principal type appears in the command prompt, or the message (nc) to indicate that you don’t have a principal right now.

Unfortunately, I’ve noticed a tendancy for the OS to switch prinicpals in the background unexpectedly.  Send a noop command (that is, just press enter at shell prompt) to see if your active principal’s changed.

Using proxychains is as simple as prefixing your command with proxychains.  For example:

proxychains remctl my-server command subcommand arg1

That routes all network connections through the SOCKS5 proxy at localhost port 1080, which should be set up by the SSH connection to your bastion host.

Using an OpenPGP card/Yubikey with SSH authentication? Don’t forget about sshcontrol!

Here’s yet another thing that hit me at work today, and getting the answer involved annoying searching & testing, so here it is for you!

I use a Debian jessie workstation, and my SSH key lives on a Yubikey.  We don’t really use pubkey-based authentication in my area (we use Kerberos), so the first time I needed to do a pubkey-authenticated SSH connection was to clone one of my Github repositories.  When doing the clone, I got the mysterious message “Agent admitted failure to sign”.  GitHub talks about this error, but their article only talks about locally-stored keys (like id_rsa keys).  My private key lives on the Yubikey, so their article didn’t really work for me.

Doing some investigation, I could see that my SSH client was trying to authenticate, and it did see my Yubikey, but still the authentication was failing.  If you have this problem, there are two important things to check:

  1. Your GPG Agent must be running with SSH support.
  2. Your authentication key’s keygrip needs to be loaded in the .gnupg/sshcontrol file.

My problem was actually #2, but #1 is important to mention!

SSH Support in GPG Agent

The GPG Agent is able to act as an SSH Agent, speaking the ssh-agent protocol, as long as it has been told to do so.  In order for this to work, you need to specify –enable-ssh-support in the gpg-agent command line, or add “enable-ssh-support” in ~/.gnupg/gpg-agent.conf.

To confirm this, check your gpg-agent, and check for the appropriate environment variables:

blargh:~> env | grep SSH
blargh:~> ps xaw | grep gpg-agent
7631 ?        Ss     0:00 gpg-agent -c --daemon --enable-ssh-support
9425 pts/6    S+     0:00 grep gpg-agent

SSH_AGENT_PID contains the process ID of the ssh-agent, which is actually gpg-agent.  Also, SSH_AUTH_SOCK points to a special socket that gpg-agent creates, meant for connections speaking the ssh-agent protocol.

If SSH_AGENT_PID is pointing to a different process, you may have ssh-agent running somewhere else.  You should find out what is running ssh-agent, and stop it.  gpg-agent is able to manage locally-stored keys (like id_rsa), so I don’t think there’s a need to run ssh-agent, as long as you are running gpg-agent with SSH support.

If SSH_AGENT_PID isn’t specified at all, or you just added “enable-ssh-support” to ~/.gnupg/gpg-agent.conf, then you’ll need to restart gpg-agent.  In many cases, it’s easier to log out and log back in, so that everything picks up the new gpg-agent.

The sshcontrol File

This is what affected me.

The file at path ~/.gnupg/sshcontrol contains a list of “keygrips” for SSH that gpg-agent will use for ssh-agent authentication.  Even if gpg-agent is configured and can see your key, it will not work unless it’s listed in the sshcontrol file!  For local keys, running ssh-add will automatically add them to the sshcontrol file, but that doesn’t work for keys that live on an OpenPGP card.

I had gpg-agent running with SSH support, but gpg-agent does not automatically add keys that are already on an OpenPGP card, so it’s up to you.  What you need to do is…

  1. Extract the keygrip.
  2. Add the keygrip to the sshcontrol file.
  3. Restart gpg-agent.

Extracking the keygrip is pretty easy to do, using this string of commands:

blargh:~> echo 'scd learn --force' | gpg-connect-agent | grep KEYPAIRINFO
S KEYPAIRINFO 6E6D1675B29724F4D4A2250EDE5A0634B4E06884 OPENPGP.1

Each line of output contains the keygrip for one of the three keys on your OpenPGP card: The encryption key, the signing key, and the authentication key (which is the one we want).  The third line, for “OPENPGP.3”, contains the keygrip for the authentication key.

To update the sshcontrol file, simply add the keygrip to the file, on its own line.  So, in my case I would add the following line to ~/.gnupg/sshcontrol:


After you update the sshcontrol file, you need to restart gpg-agent.  Again, the safest way of doing that is to log out and log back in, so that everything picks up the new gpg-agent environment variables.

Identifying spammers in your shared web service (featuring Postfix, auditd, and iptables)

Over the last week, we’ve been having a problem with spam in our shared web service: Something was sending out lots of low-quality, easily-blockable spam, and the bouncebacks were filling up the Postfix queues in our outgoing email cluster.  The way we tracked down the spammer was interesting, so I’m writing it up here in case it’s of interest to anyone else!

Continue reading

Developing on El Capitan? Need OpenSSL? Install MacPorts!

I’m right now experiencing the joys of setting up a Perl development environment on Mac OS X 10.11 (El Capitan).  I’ve already talked about the weird linker warnings that appear when building Perl, and I’ve just come across another fun roadblock: A lack of OpenSSL header files.  This is not unintentional, and there is a solution.

Continue reading

Weird object file warnings building Perl on OS X 10.11 El Capitan

I just recently got a Mac laptop with Mac OS X 10.11 (El Capitan) installed, and one of the things I do in a new system is install a local Perl environment using perlbrew.  It allows me to install and upgrade modules without worrying that I am getting in the way of the system environment.  Problem is, when I built my first perlbrew environment on El Capitan, I saw some weird stuff happening in the build.

Continue reading

The 911 Test

On Twitter, Andrew Mayes posted a link to an October 3 story posted by The Verge titled This is what happens when 911 fails.  The article talks about experiences people had trying to call 911, via landline or mobile, as well as troubles that 911 dispatchers have sometimes experienced in trying to locate mobile callers.  One area not mentioned by the author was Voice over IP (VoIP).  Unfortunately, the story isn’t any better in the mystical VoIP wonderland.

If you have a business, especially if your business uses VoIP for outside phone service, have you done a 911 test?  If not, do one soon!

Continue reading

CAs, Name Constraints, and a Business Opportunity

Warning!  I am about to bring up a topic that others have talked about before: Name Constraints (as it applies to SSL, that is).  Name Constraints is a way for a Certificate Authority (a CA) to say “All of the certificates issued by me should only be used for these domains (x.net, or y.z.com, or all of .org, etc.).  Any other certificate should not be trusted.”

Name Constraints are not supported by everything yet, but they are supported in some places.  Not only would Name Constraints make things easier for sysadmins, it would give a new revenue stream for CAs, browsers would have an easier time accepting them, and it would “streamline the user experience” (whatever that means).

Continue reading

An Analysis of the CAs trusted by iOS 8.0

iOS 8.0 ships with a number of trusted certificates (also known as “root certificates” or “certificate authorities”), which iOS implicitly trusts.  The root certificates are used to trust intermediate certificates, and the intermediate certificates are used to trust web site certificates.  When you go to a web site using HTTPS, or an app makes a secure connection to something on the Internet (like your mail server), the web site (or mail server, or whatever) gives iOS its certificate, and any intermediate certificates needed to make a “chain of trust” back to one of the roots.  Using the fun mathematical property of transitivity, iOS will trust a web site’s certificate because it trusts a root certificate.

iOS 8.0 includes two hundred twenty-two trusted certificates.  In this post, I’m going to take a look at these 222 certificates.  First I’m going to look at them in the aggregate, giving CA counts by key size and by hashing algorithm.  Afterwards, I’m going to look at who owns these trusted roots.

Continue reading

My First iOS Security Bug

About an hour ago, I opened bug #18403015 on Radar, Apple’s bug reporting web site.

This is not the first time I’ve reported a bug to Apple:  I recently opened a different bug (#18347673) because iOS has problems handling identify certificates, and I’ve opened other bugs as well in the past, for other stuff.  What makes this bug different is that I’m reporting it as a security bug.

Continue reading