Using PDB files to debug applications without source code

[Vagabond]

I've had some experience using PDB files in C#, but I wasn't sure if that concept would transfer to C++. Fortunately it does!

PDB (Program Database) files contain:

 * public, private, and static function addresses
 * Global variable names and addresses
 * Parameter and local variable names and offsets where to find them on the stack
 * etc

So basically, if you want to debug an application and use fairly sophisticated debug tools like breakpoints, variable watches, etc, you do not actually need the source code, you just need the PDB file that is generated when the file is compiled.

You can load the application into a new project in Visual Studio 2017 (or other IDEs for that matter), then start a new instance of the application with the debugger attached. Visual Studio will automatically search for and load the PDB file associated with the application/library. In this case, I just placed the PDB file next to the executable and Visual Studio automatically found it.

Since OP2Archive is a command line program, I needed to provide arguments before running. This worked just like when debugging normally in Visual Studio. So it was easy to give it the command CREATE test.vol abc.map.

See the image below of the application running in Visual Studio with the debugger attached.



Visual Studio even shows you the source code with line numbers and everything as you step through each function in the program.

Also, apparently since OP2Utility was statically linked, all of the information necessary to step through functions from OP2Utility called from OP2Archive were available for debugging without including a separate PDB file for OP2Utility.

The catch with PDB files is that each PDB file is unique to the compiled file and they are rather large. So, if you have a random PDB file for an application, it doesn't help unless it is the exact right one created on compilation.

For size reference, OP2Archive's release PDB is about 6,300 KB. This compressed down to about 1,400 KB with the standard ZIP algorithm. OP2Archive's debug PDB is about 14,100 KB, or over twice as large as the release PDB. OP2Archive is a small application and I could see how PDB files could quickly spiral into massive sizes for larger compiled files.

I know organizations like Microsoft will automatically create and archive PDB files for each release of their software. This way they can be debugged later as needed even without having to load and compile the source code. Probably important for widely used, operating system critical software.

Of course there is a security risk involved with storing the PDB file as someone could use it and basically just breeze through a thorough review of your source code.

I wonder how often when someone hires a company to create an application or library to fulfill some business need, they actually ask for copies of the PDB files for the final product just in case it needs debugged years later after development.

What a boon it would have been to get a hold of PDB files for Outpost 2's binaries.

I was thinking about placing the OP2Archive and OP2MapImager PDB files next to the executables when releasing them for fun. For such a non-critical application where the source code is public, it probably isn't a big deal, but it is a good habit to be in I think?

Thoughts? I'm done ramblig for now.

-Brett

[leeor_net]

One of the reasons I really love MS's development tools. They've improved considerably over the last decade or so.

[Hooman]

Interesting. I can see how that would be useful in a large organization like Microsoft. Especially with the automated crash reporting. If those crash reports were handed to an engineer, and their support system automatically pulled out relevant archived PDB files, that would make debugging much faster.

I'm not sure how we might use it though. We don't have automated crash reporting and support systems around our code. I think that greatly reduces the effectiveness of using PDB files.

You're right that being open source, anyone could checkout an appropriate version. Plus recompiling will regenerate the PDB files. Though that is true at Microsoft too for their internal engineers who would be using the PDB files. They should have access to the source code.

I think the main use of PDB files is saving time, and I think that's only really possible with the right support system in place. Otherwise, they're mostly just taking up space. Mind you, I've never really worked with them, so perhaps I'm missing something here. My gut reaction to them seems to come from a place of lack of understanding.

[Vagabond]

Hooman,

I think you really make money on older software that perhaps you would have to download an older compiler to recompile, etc. That could become really time consuming to locate and link in all the dependencies in the exact form they existed for the original compilation as you wouldn't want to use newer versions of the dependencies in case they hid the bug.

I was more thinking it would have been a boon for Outpost 2 if we had them from Sierra. Then you could literally step through the source code of the game and scenario DLLs in the IDE even though we didn't have a physical copy of the source code. You could probably even copy and paste source code files right out of the PDB files when reviewing them in Visual Studio (Maybe I should try this?).

Agreed, it isn't a big deal for what we are doing being open source and small projects.

On another project I work on in C#, I switched from XNA to MonoGame for the primary game framework. Since it is published in XNA though, it would have made sense for me to save the PDBs for debugging. Now if someone reports a bug, I would need to reinstall the older framework and pull the code from the REPO at the release state, etc to troubleshoot. It would have been a lot more trivial just to save the PDB files. Granted, bugs in a game don't really matter compared to something like a business application or operating system.

-Brett

[Hooman]

I think you really make money on older software that perhaps you would have to download an older compiler to recompile, etc. That could become really time consuming to locate and link in all the dependencies in the exact form they existed for the original compilation as you wouldn't want to use newer versions of the dependencies in case they hid the bug.

That's a very good point.

On another project I work on in C#, I switched from XNA to MonoGame for the primary game framework. Since it is published in XNA though, it would have made sense for me to save the PDBs for debugging. Now if someone reports a bug, I would need to reinstall the older framework and pull the code from the REPO at the release state, etc to troubleshoot. It would have been a lot more trivial just to save the PDB files. Granted, bugs in a game don't really matter compared to something like a business application or operating system.

Indeed. And you don't know you're going to do this until you do it.

This example is particularly interesting since it's a framework change, and not a compiler change. A compiler change might invalidate PDB files, perhaps using a different version or format, or not supporting the feature at all. With a framework change, there is no reason to expect a break in PDB files.


It would have been nice if Outpost 2 source code had been leaked somehow. Doesn't look like we'll be finding an accidentally released PDB file at this point though. (And yes, I just Googled)