Can a Cloud Foundry app ssh to itself with key authentication?

Question

I have a CF app that needs access to an sftp server for integration testing and I'd like to take advantage of the local container being configured to enable it to act as one.

I understand it's possible as explained in the docs for an app to ssh to itself through the proxy, but obtaining a password to use with the cf:<application-guid> username is a complication that I'd like to avoid if possible.

If I ssh into the app container, I can indeed ssh through the proxy, but if I try to ssh to localhost:2222, I get a "public key denied" error message, suggesting that it would support key authentication.

Is there a private key available in the app container that the app can use to connect to ssh/sftp to itself?

Answer 1

Yes, there is a private key available in the app container environment: the diego-sshd SSH server process running inside the app container has its own private key stored in its environment as the SSHD_HOSTKEY environment variable.

Once you've used cf ssh to get a shell session inside the app container, here's a quick way to extract that PEM-encoded private key value to a file and then to use it to authenticate to the diego-sshd server:

$ strings /proc/$(pidof diego-sshd)/environ | awk '/-----BEGIN/,/-----END/' | sed 's/SSHD_HOSTKEY=//g' > sshdkey
$ chmod 0600 sshdkey
$ ssh -i sshdkey -p 2222 localhost

You need the chmod command to restrict permissions on the private key file to only the vcap user in the app container, as otherwise the SSH client will complain that permissions are too open.

It's hard to tell that you've done anything once you start that SSH session, as the shell prompt will look identical to the existing CF SSH session, but you can check by tracing your shell's PID through the process tree:

$ pstree -pT $(pidof diego-sshd)
diego-sshd(8)───bash(259)───pstree(323)

$ echo $$
259

$ ssh -i sshdkey -p 2222 localhost

$ pstree -pT $(pidof diego-sshd)
diego-sshd(8)─┬─bash(259)───ssh(341)
              └─bash(342)───pstree(353)

$ echo $$
342

In this case,

• 259 is the PID of the bash process started by the initial CF SSH session,
• 341 is the PID of the ssh process starting the nested SSH session,
• 342 is the PID of the bash process started by that client's session.

Some background on what's going on with the CF internals:

• When Cloud Foundry creates a new app with SSH access enabled, it generates an app-specific private host key and stores it in the routing information for the app in CF's Diego system. The Diego SSH proxy component uses this private key to authenticate to the app instances' diego-sshd SSH servers when brokering an app developer's CF SSH session into one of the app instances.
• Cloud Foundry also configures the app's diego-sshd sidecar server with that private host key in the -hostKey flag so that it matches the key information that the SSH proxy will use.
• That key is of course sensitive, so to prevent its accidental disclosure, on startup the diego-sshd server stashes the key value in that SSHD_HOSTKEY env var and then re-execs itself to clear the key value off the command line. But the key value is still in its environment, so we can extract its value from the proc filesystem inside the container.