Debugging problems with NetHelper on closing Outpost 2

[Vagabond]

Everyone, and probably Arklon most specifically,

I'm looking for some help with NetHelper. It is giving me a lot of problems when I close out of Outpost 2. Sort of three things are going on:

• Using the new version of op2ext that I have compiled, it takes an exceptionally long time to run through NetHelper's DestroyMod function.
• NetHelper cannot be compiled using Visual Studio 2015 or newer. (This one is solved, but should probably be fixed in the code base at some point)
• When testing a newly compiled NetHelper DLL using Outpost 2 1.3.6 as tagged in the Outpost2SVN, an unhandled error is thrown when shutting down Outpost 2. Whoever updated op2ext for the 2.3.6 release of Outpost 2, only place the updated version of op2ext.dll in the trunk and didn't clean up the tagged release on SVN. This section doesn't apply anymore.

Exceptionally long time to run

Using the newer version of op2ext, NetHelper's DestroyMod function is taking about 3 minutes to execute.

Within NetHelper's DestroyMod function, the following line takes about 21 seconds to run in either DEBUG or RELEASE mode:

Code: [Select]

WaitForSingleObject(hFwdThread, INFINITE);

Then the following line takes about 2 minutes, 42 seconds to run in either DEBUG or RELEASE mode:

Code: [Select]

for (int i = startPort; i <= endPort; ++i) {
    forwarder.Unforward(true, i);
}

I also verified it took about 3 minutes for Outpost 2 to close out without any debugger attached. When closing out of Outpost 2, from within the task manager, Windows moves Outpost 2 from the applications section to the Background application section, where it remains for 3 minutes.

So, I don't know if there is a problem in op2ext that is somehow affecting NetHelper's destruction time, or if there is a problem in NetHelper making it take so long to unload. To troubleshoot, I tried to compile a new version of NetHelper in order to get a PDB file to step debug through. See below.


Net Helper cannot be compiled with VS 2015 or newer

I wanted symbols (PDB file) in order to figure out why NetHelper was taking so long to destroy. I first attempted to compile NetHelper with VS 2017. However, I received errors caused by Microsoft making breaking changes to their toolset in the header stdio.h. This change was introduced with VS2015 and it looks like NetHelper was compiled with the Visual Studio 2010 (v100) toolset. The problem is these changes are inside the library file for miniupmp. So it cannot be used when compiling NetHelper with a toolset later than the one that ships with VS2013.

See quote below from this Microsoft article: https://msdn.microsoft.com/en-us/library/bb531344.aspx#BK_CRT

The printf and scanf family of functions are now defined inline. The definitions of all of the printf and scanf functions have been moved inline into <stdio.h>, <conio.h>, and other CRT headers. This is a breaking change that leads to a linker error (LNK2019, unresolved external symbol) for any programs that declared these functions locally without including the appropriate CRT headers. If possible, you should update the code to include the CRT headers (that is, add #include <stdio.h>) and the inline functions, but if you do not want to modify your code to include these header files, an alternative solution is to add an additional library to your linker input, legacy_stdio_definitions.lib.

To add this library to your linker input in the IDE, open the context menu for the project node, choose Properties, then in the Project Properties dialog box, choose Linker, and edit the Linker Input to add legacy_stdio_definitions.lib to the semi-colon-separated list.

If your project links with static libraries that were compiled with a release of Visual C++ earlier than 2015, the linker might report an unresolved external symbol. These errors might reference internal stdio definitions for _iob, _iob_func, or related imports for certain stdio functions in the form of _imp_*. Microsoft recommends that you recompile all static libraries with the latest version of the Visual C++ compiler and libraries when you upgrade a project. If the library is a third-party library for which source is not available, you should either request an updated binary from the third party or encapsulate your usage of that library into a separate DLL that you compile with the older version of the Visual C++ compiler and libraries.

So, this really isn't a problem right now because Microsoft allows downloading toolsets back to VS2010. However, for the long term health of the project, should probably be corrected. Just wanted to bring it up.


NetHelper throws an unhandled exception when closed using Outpost 2 1.3.6

Now that I had a version of NetHelper with symbols, I placed a releaseMinSize version of NetHelper that I compiled with VS2013 into Outpost 2 1.3.6, as seen at the repository here: https://svn.outpostuniverse.org:8443/!/#outpost2/view/head/GameDownload/Outpost2/tags/Outpost2-1.3.0.6-OPU. I did not change out op2ext.dll.

I was trying to step debug through the DestroyMod function. However, the function MemPatch::Disable throws an unhandled exception before the breakpoint in DestroyMod runs. I'm suspecting that DestroyMod isn't being called for some reason, but I'm not sure how to verify this. Below are the details of the exception.

... Doesn't matter anymore ...

When testing using the new version of op2ext that I have modified, DestroyMod is being called before the MemPatch::Disable function in NetHelper. I also remember there was a point in time when an outdated op2ext was released with Outpost 2 1.3.6 that was later fixed. So, perhaps the tag for Outpost 2 1.3.6 in the repo still has the older version? How can we check this, and if it isn't the problem, is there a way to test if this fatal error is occurring on shut down on the currently released version of Outpost 2 without symbols (since we don't have them)?



Okay, so I don't want to release the new version of op2ext without making sure it plays nice with NetHelper. But I've sort of hit these 2 roadblocks. I was hoping for some help moving forward since I don't really want to jump in and modify NetHelper code as it is Arklon's project, and I'm no good at networking anyways.

Update: It takes my computer a long time to unload/destroy the NetHelper module using either the version of op2ext.dll shipped with version 1.3.6 (the second version of op2ext.dll shipped with 1.3.6), or using what I'm calling op2ext.dll 2.0.0, for release on version 1.3.7 of Outpost 2. Hopefully not too confusing...

So, is it intended behavior for NetHelper to take this long to exit?

[Vagabond]

It looks like taking 3 minutes isn't unusual for NetHelper. So I'm going to continue with updating Outpost 2 1.3.7.

-Brett

[Arklon]

I'm going to test changing NetHelper to keep a global PortForwarder instance, instead of creating a new instance for opening ports and another instance for closing ports. That should help somewhat, though it could potentially break if your connection drops and reconnects while the game is running. UPnP is an incredibly crappy protocol though. If your router is taking its sweet time to process the messages to close ports, the game is just going to keep waiting and there's not much I can do about it. With the NAT-PMP path I can also change it to not wait for an ACK back from the router after telling it to close the ports, but most routers only support UPnP which I don't know if I can make a similar change for.

[Vagabond]

Hopefully it goes well. In the end, if a couple of minutes is taken to properly handle the cleanup, it doesn't bother me as long as it is documented in the readme or somewhere so it is expected behaviour.

-Brett