Remote SSH commands require double escaping before hitting the DefaultShell

[penguin359]

"OpenSSH for Windows" version
1.0.0.0

Server OperatingSystem
Windows Server 2016 Standard

Client OperatingSystem
Fedora 27

What is failing
Attempting to run remote commands using SSH with my DefaultShell set to powershell.exe requires escaping the command to pass through both my local shell and the Command Prompt on the remote Windows server before PowerShell handles the command. As a simple example, this is the command as run directly inside Powershell:

echo "`"hello`""

Which produces this output:
"hello"

Attempting to run the above command from a Bash script on Linux, I expect to have to use single quotes around the command to avoid Bash interpresting the double-quotes and back-ticks, but not much else:

ssh windows 'echo "`"hello`""'

It's hard to read, but that ends in back-tick, double-quote, double-quote, single-quote. However, I find that this only produces the following output:
hello`

Which seems to be due to the Command Prompt first interpreting the double-quotes before handing it off to PowerShell. Instead, I must quote the command for the Command Prompt and then for Bash in order for it to pass all the way to PowerShell unencumbered. The following command produces the correct output:

ssh windows '"echo \"`\"hello`\"\""'

This is far from intuitive and gets worse as I try to do more complex commands including pipes, where-object and foreach-object using the ?, %, and {} metacharacters.

Expected output
"hello"

Actual output
hello`

[manojampalam]

Behavior wrt to double escaping is the same even with cmd.exe. See the following from Windows client to Windows server.

hello` output seems to the behavior specific to powershell.exe

[penguin359]

My concern isn't about the client-side escaping, it's that I need two levels of server-side escaping. My example still applies when running ssh.exe from a Windows box. When running a remote PowerShell command with ssh, I first have to write it thinking about proper PowerShell syntax, naturally, but then I have to escape it to pass through the remote cmd.exe process that is started by the sshd.exe server on the Windows server which then runs powershell.exe as it's child. Then, as a third layer, I have to escape it to pass through the local cmd.exe shell. It's the middle layer, the remote server cmd.exe that I would like to eliminate. This is in stark contrast to how sshd runs on any UNIX box, whether it's Linux, macOS, or cygwin on Windows layer. If I use csh as my personal shell on a UNIX box, it has a distinct syntax incompatible from standard Bourne shell or it's derivatives like bash. I only have to worry about writing a valid csh command, and then escaping it for my local shell whether that's bash, csh, cmd.exe, or powershell.exe. The command, as passed through the ssh protocol is handed directly to the final shell that interprets it, which I would like to be powershell.exe and not a cmd.exe wrapping powershell.exe.

[penguin359]

Looking at the process hierarchy from an interactive PowerShell session on Windows, I see this:

  Id Level IndentedName             ParentId
  -- ----- ------------             --------
1676     2     sshd                      668
1032     3       sshd                   1676
 984     4         conhost              1032
3852     4         sshd                 1032
1920     5           ssh-shellhost      3852
1336     6             cmd              1920
  84     7               conhost        1336
 960     7               powershell     1336

It does indeed look like PowerShell is being run as a child of cmd.exe. This is an interactive session to a Linux server where the user's shell is csh, and not sh or bash as is typical:

  PID TTY      STAT   TIME COMMAND
18695 ?        Ss     0:00 sshd: user [priv]
18701 ?        S      0:00  \_ sshd: user@pts/86
18714 pts/86   Ss     0:00      \_ -csh
18817 pts/86   R+     0:00          \_ ps axwf

Where as the desired UNIX shell is invoked directly on a traditional UNIX\Linux box. Even if the cmd.exe layer on the server side can't be removed, it would be nice for it to escape commands going through it so it is transparent to the remote user running a command in the selected target shell, PowerShell or otherwise.

[penguin359]

Here's a better step-by-step example to demonstrate the different behavior of sshd.exe on Windows and sshd on a Linux box. The local and remote shell for Linux is bash, but for Windows, I'll use cmd.exe as the local shell for ssh.exe and powershell.exe for the server-side shell called by sshd.exe on Windows. The goal will be to echo out the string "hello" with quotes included in the output. In Bash, the double quotes have a special meaning and need to be escaped. When running the command locally, there are two ways to do it:

echo '"hello"'
echo \"hello\"

To run this remotely from a Linux box, I will need to escape the command so that the local Bash shell passes it through ssh to the remote Linux shell. The simplest mechanism is to use single-quotes with the second command above. Inside single-quotes no characters other than a single-quote, which ends the quoting, have any special meaning. That makes it pretty simple to implement:

ssh linux 'echo \"hello\"'

With just two layers of quoting, I can print "hello". Now, to do the same from a cmd.exe shell running ssh.exe, it's not much different, but the quoting rules are different. With cmd.exe, only two characters have any special meaning in quotes, the double-quote and backslash. To keep the example simple, I'll use the first command from above that only has double-quotes. To quote a double-quote in cmd.exe, it just needs to be doubled up, but the string much already be inside double quotes so I add double quotes around the whole command after doubling each double-quote in the original command:

ssh.exe linux "echo '""hello""'"

It looks a little confusing, but it's just two layers, still, of quoting rules to handle. Now, to do this same action in a local PowerShell, there are again two ways of writing it:

echo '"hello"'
echo `"hello`"

Both commands will print "hello" to the screen when run in a local PowerShell. If I run this from a cmd.exe shell on Windows, I would expect to be able run either by just adding the quoting needed to pass through the local cmd.exe shell. Taking the second command from above, I double up the quotes and then surround the entire command in quotes:

ssh.exe windows "echo `""`"""

But that produces hello` as output, not "hello". It turns out, I need to apply the cmd.exe quoting rules twice, once for the local cmd.exe and once for the remote cmd.exe:

ssh.exe windows """echo `""""`"""""""

So now I have 4 quotes for double-quotes that must appear for PowerShell, and 2 quotes where I'm wrapping the command for the remote cmd.exe giving 7 quotes at the end once I wrap it again for the local cmd.exe. Running the command from a local Linux box to a remote Windows server using PowerShell, I see the same behavior, but this time, I quote the PowerShell command to pass through cmd.exe, and then to pass through Bash by using single-quotes around it:

ssh windows '"echo `""hello`"""'

For every remote PowerShell command I run on Windows, I have to also quote to handle passing through cmd.exe regardless whether it's initiated from a Linux or Window box.

This whole thing above might seem like a silly example because it really is, so I'll give a more realistic example based on what we are actually doing. Here's a pretty short PowerShell command that we would like to run from a Linux box. This is the command as run on the target PowerShell:

Get-NetAdapter | ?{$_.MACAddress -eq "12-34-56-12-34-56"}

And it gives us the information we need as output. Trying this from ssh.exe inside cmd.exe, I know I need to double up the quotes and then surround it in double-quotes. Luckily, there's no backslashes to worry about. So that produces this command:

ssh.exe windows "Get-NetAdapter | ?{$_.MACAddress -eq ""12-34-56-12-34-56""}"

However, I find that this produced a syntax error since I failed to account for the remote cmd.exe interpreting this command. Taking that into account and escaping again produces this command:

ssh.exe windows "Get-NetAdapter | ?{$_.MACAddress -eq """"""12-34-56-12-34-56""""""}"

Which is starting to get hard to read. However, the whole reason I'm using ssh is to be able to run PowerShell commands like this from a Linux box. If I do the same thing from from ssh in Bash, I will run into that same syntax error if I don't first quote the PowerShell command for cmd.exe and then escape it for bash:

ssh windows 'Get-NetAdapter | ?{$_.MACAddress -eq """52-54-00-AB-A7-1F"""}'

I would like to be able to take a working PowerShell command and run it through ssh only worrying about my local shell or scripting language whatever that may be. Currently, I need to always add in this cmd.exe translation layer and don't always get it right.

[manojampalam]

Thank you for the detailed analysis. I understand the issue. Its stemming from the fact that while launching default shell (other than cmd.exe) with PTY, its actually launched via an intermediate cmd.exe and this is consuming the extra escape.

To fix this, we need to internally excape the shell payload before passing it to cmd.exe. AFAIK, we only need to escaple any double quotes ("s), by doubling them. Is this understanding correct?

Here' how the command payload would flow through:

ssh ""command"" -->
"command" (over the wire to sshd) -->
base64encoded("command") to ssh-shellhost -->
"command"(decoded) in ssh-shellhost -->
""command"" (after escape'ing) to cmd.exe --> //this is the escape step that needs to be added
"command" to default shell

@bagajjal thoughts?

[penguin359]

After a little more research, I'm thinking that this might have a little more to do with a fundamental difference in process creation between Windows and UNIX-like operating systems. The command-line is actually passed to it's child process as a single string which the child parses with it's own rules. What I'm seeing as the remote cmd.exe layer of quoting might actually be the C/C++ Runtime library inside PowerShell.exe following these rules:
https://msdn.microsoft.com/en-us/library/17w5ykft.aspx
In that case, it might require shell-host.c to escape the string it receives over SSH to pass through as the argument to /c for the child cmd.exe/powershell.exe. I should also test with cmd.exe and bash.exe and see how the quoting works for those remote shells.

Here one test I tried from a local cmd.exe shell:

powershell.exe /c "Get-NetAdapter | ?{$_.MACAddress -eq \"12-34-56-12-34-56\"}"

Using the expected level of quoting to handle the local invocation of PowerShel.exe and produces the desired output. Turning that into SSH directly, though breaks:

ssh.exe windows "Get-NetAdapter | ?{$_.MACAddress -eq \"12-34-56-12-34-56\"}"

As before, I have to double up the backslashes to handle two layers of C/C++ Runtime parsing the same argument:

ssh.exe windows "Get-NetAdapter | ?{$_.MACAddress -eq \\\"12-34-56-12-34-56\\\"}"

The equivalent test on Linux, for comparison, works with the same quoting level:

bash -c "/sbin/ip -o link | grep \"12:34:56:12:34:56\""
ssh linux "/sbin/ip -o link | grep \"12:34:56:12:34:56\""

[penguin359]

OK, you beat me to my last comment. Thanks for looking into this!