Monday 26 September 2011

WinDBG remote debugging with SSH port forwarding

When I tried to apply my “tunnel” onto Java remote debugging, I found someone got a better approachSSH port forwarding.

It is very cool, so I’d like to try it on WinDBG.

If the machine with public IP has Linux OS installed, it is very easy to start a SSH daemon. However I got one machine which has windows 2003 installed, so I have to find a Windows version sshd (SSH deamon). I think that freeSSHd is the best one among the free editions.

As the port 22 (default SSH port) on my server is taken by another process, I use 9000 instead, and listen on “All interfaces”. Next, go to the tab “Tunnelling”, check on both “local port forwarding” option and “remote port forwarding”.

We are going to use WinDBG to do the remote debugging, certainly it is done on Windows system, so we need a windows version SSH client. I prefer a CLI version of SSH client, so I choose plink.exe (from the popular putty site).

Which parameters do we need in the SSH client? If you are new to SSH, please have a look of the hyperlinks in this document. However, I believe it is much easier to understand them with an example, so let’s give the instructions first.

On the client machine where the application you want to debug is running on, run debugger server (dbgsrv.exe), specify the port as 1012 (you can use any free one).

dbgsrv –t tcp:port=1012

Check the tab Processes in Task Manager to make sure the debug server is running. Also, open a notepad as the application we are going to debug.

Next, on the same client machine, run

plink –P 9000 -R 9001:localhost:1012 nanoart@sshserver -N

Or, you can run the following command on the relay machine instead.

plink –P 9000 -R 9001:[ip of client machine]:1012 nanoart@sshserver -N

This is very useful when you are debugging GINA or credential provider which prevent you running a user-mode application.

At this stage, we’d better to have some explanations on the plink command.

The parameter -R specifies a remote port forwarding. It asks SSH server to start a listener process on the remote machine (SSH server) with port 9001. Thus the remote machine can accept connections and forwarding them back over SSH back to the machine that started the SSH port forward session, and then from there onto the client machine. Because my SSH sever is not running on the default port 22, I have to specify it with the parameter –P. The last parameter –N means “don't start a shell/command”, this is optional. Of course, you need to provide a SSH login account (“nanoart” in this example).

On developer’s machine, run this command,

plink –P 9000 -L 1012:localhost:9001 nanoart@sshserver –N

The parameter -L means local port forwarding. It results in a process listening on a port (here 1012) on the developer’s machine . This process/port will accept connections and forward them via the remote authenticated SSH (sshd) process to another host:port combination (here it is localhost:9001).

Note: The localhost might appear at first sight a little confusing, but don't forget host:port pair is relative to SSH server in local port forwarding. If localhost is used, it is the SSH server itself. However it is relative to the initiating end of this SSH connection, i.e. the workstation in the case of remote port forwarding.

Now, the route is tunnelled between the developer’s machine and the client’s machine.

On the developer’s machine, we can run

windbg –premote tcp:port=1012, server=localhost

It is worth mentioning that you may need to switch the order of port forwarding (first local port forwarding, then remote port forwarding), otherwise it may not work if you are using the freeSSHd as the SSH server.

In WinDBG, we can see all the processes of the client’s machine at the remote site.

Also, we can attach to the sample process “notepad.exe”.

Bravo, the remote debugging with SSH port forwarding succeeded! Finally I can forget the dbengprx.exe issue.

Please read the full version which has some figures to help you understand it easily.

The foible of Apple PackageMaker

Using PackageMaker GUI, I created a package script which has a Postinstall action "Open File". The package (built with GUI) worked as expected, it did launch the installed application of mine after the installtion. However, I got a warning when I tried to build the same script with PackageMaker command line.

/Developer/usr/bin/packagemaker --doc makedma.pmdoc --out dmgsource/Installer.pkg
2011-09-26 11:59:59.533 packagemaker[212:903] Warning: unknown or deprecated action

In addition, the command line built package did not launch the installed application.

It sounds like no answer so far, as someone asked the same question on stackoverflow.

The command line PackageMaker is just one job in my shell script which builds xcode projects at the top and creates a dmg file at the end, so this foible caused some trouble to me.

I just wonder if it is another strategy from Apple - leave some glitches on purpose so that some people can make a good living on that? You know what I mean.

Wednesday 21 September 2011

Remote Debugging with Visual Studio 2008

I accidentally found that the port(4015 as the default) of MSVC(Visual Studio 2008) remote debugging monitor(msvsmon.exe) is only for the authentication, the debug data is transfered through other ports which are created by the authetication process.

This mechanism makes it impossible to do the remote debugging over firewall(s). Please see my experiment in details.

Sunday 18 September 2011

Tunnel implementation

In order to make WinDBG remote debugging work through firewall(s), I decided to write my own repeater, instead of using its dbengprx.exe, which doesn’t work with “Two Firewalls”(see my previous blog).

With Boost ASIO, implementing the prototype is not a big job. The instructions to use the tunnel are as following.

First, run dbgsrv.exe at the debuggee’s side.

Second, run tunnel.exe on the server with a public IP address. It has two listening ports, 9000 and 9001.

Third, at the debuggee’s side, run tunnel outward (public IP) 9001 (dbgsrv ip) 1012. Unlike “dbgsrv.exe”, many other debugger servers(like msvsmon.exe) do not have the feature “clicon”, so I made my tunnel be able to relay the traffic between the public server and the dbgsrv machine. As you can see, the connection direction is outward relative to the relay server itself.

You can run the above outward direction tunnel on the same machine where dbgsrv is running, however it is very useful to run it on another machine when you are debugging GINA or credential provider which makes impossible run a cmd application, because you haven’t logged in.

Finally, run windbg.exe at the debugger side, but the server parameter should be public IP:9000, instead of debuggee IP:1012

  • dbgsrv.exe -t tcp:port=1012
  • tunnel inward 9000 9001
  • tunnel outward (public IP) 9001 (dbgsrv ip) 1012
  • windbg -premote tcp:server=(public ip),port=9000

I did a test, it worked. I could attached the process which I wanted to debug.

Please read the PDF version for the details. You can also download the source code, I would be very happy if you can improve it and make it as a mature product.

Saturday 17 September 2011

My WinDBG remote debugging trial on "Two Firewall" scenario

You can do remote debugging with WinDBG, basically use the following instructions(with the displaying order).

  • At debuggee side, run dbgsrv.exe –t tcp:port=1012,password=longjump
  • At debugger side, run windbg –premote tcp:port=1012,server=debuggee’s ip,password=longjump

Pretty easy, isn’t it? Well, you probably know the question I am going to ask, what about the situation when the debuggee is behind a firewall?

As a matter of fact, the debugger himself is generally behind the firewall as well. As a developer, you might be able to persuade your network administrator to modify the firewall settings for your debug purpose, but it’s simply not practical to ask your client to change his firewall, so the forwarded connection to the Debuggee is unfeasible.

MS documentation provides an example titled “Two Firewalls” which sounds exactly what I wanted. Actually there is a bit difference, it is about kernel debugging while I only wanted to debug the user mode application. Anyway, it has a backward connection between the repeater and the debuggee, it is called “smart client” in WinDBG. I decided to have a go. Please read my full story here.

Monday 12 September 2011

Use Twitter to send out OTP in DualShield

Apart from the classical SMS and SMTP channels, DualShield can utilize Twitter to send OTP out. Obviously, you need to configure the Twitter Gateway. Deepnet provided a tool to generate the necessary keys, which is now obselete. You can use Twitter web service to get those keys, please read this pdf for the details.