Updates for Outpost 2 1.3.8

[Vagabond]

Agreed the multiplayer issue is a big and clear problem.

For the next update we have completed:

* Updated NetHelper to version 1.5.3
   - Updated to newer versions of libnatpmp and miniupnp.
   - Cut down cleanup/destruction on application closing from 3 mins to 8 secs.
   - Removed dependencies on msvcr100.dll and msvcp100.dll by statically compiling.
   - Removed deprecated POSIX networking functions.

* Updated op2ext to version 2.0.1
   - Fix bug in Console Mod Loader that separated the executable's path into multiple arguments if the path included a space.

I'm working on 3 projects right now:

 - Adding a logger to op2ext that can be called by modules like NetHelper. Using the std library chrono header for pushing a time and date stamp into the log.
 - Creating a new coop multiplayer scenario. It is about 2/3rds coded but not yet tested. Working with a new C++ programmer on it, so it goes really slowly as I'm teaching as I go. Scary that someone is listening to my programming advice. :|
 - Trying to get OP2MapImager released with bug fixes. Getting the unit tests to work has turned into a slog. This effort is separate from the next Outpost 2 release.

I'm putting a record low amount of time into Outpost 2 right now, so besides that and working on 3 projects progress is really slow.

The other thing I'd like to work on is fixing the old bug in blackbox's Plymouth Cold War scenario. Annoying that it has been broken for something like 10 years? Otherwise we should probably pull it.

Actually, with the 2 bugfixes, we could release a new version any day. I suspect the new NetHelper build will solve a decent number of multiplayer issues, but don't have much data yet. If the new NetHelper compilation works, it might be worth pushing out immediately to fix the multiplayer issue instead of waiting for other development.

If you pick the Novella back up, I'll certainly read.

-Brett

[Arklon]

The NetHelper update alone is enough to warrant a minor update. The console mod loader bug in 1.3.7 was really annoying, so that bugfix alone would also warrant a minor update. I have some minor updates to NetHelper that I could merge in, but it was mostly just switching all the multithreading to use STL <threads>, and code cleanup which you've duplicated some of yourself anyway, so meh.

[Hooman]

I agree, both seem worthy of a patch update.

[Vagabond]

Ok, Give me a day or two to make final updates to the restored subversion repository and do some final tests. Does anyone want to volunteer to test/try to break the release before it goes live?

Hooman, do you want to upload to the website again?

Thanks,
Brett

[Hooman]

Let me knows when the new package is ready and tested.

[Vagabond]

Hooman,

Just finished uploading to the SVN repository and testing.

To test, I put Outpost 2 in the directory: C://Users/.../Outpost 2 Test. This way the directory had spaces in it. I then loaded cm1 using the console mod loader and saw that the colors had been updated. This proved the console mod loader was working with spaces in the directory. No annoying popups occured either.

Then I sent the package to someone else and attempted multiplayer. Without touching router configurations or Hamachi or anything else, we were both able to create and see each others' game using TCP/IP. For NetFix, he could see and join my game but I could not see his created game. This was consistent with earlier tests. Not sure why the discrepancy exists. I can send the routers involved if anyone wants the data. Considering I had never been able to play Outpost 2 without Hamachi before this, it seems to be in a much better place.

If you don't mind, it would be good to proof-read the change log and the publishing instructions I just wrote. Also, please test on Linux using Wine to make sure I didn't break any of the compatibility there I cannot test. Besides that unless someone else wants to test, I would like to see it published.

Thanks,
Brett

[Arklon]

Then I sent the package to someone else and attempted multiplayer. Without touching router configurations or Hamachi or anything else, we were both able to create and see each others' game using TCP/IP. For NetFix, he could see and join my game but I could not see his created game. This was consistent with earlier tests. Not sure why the discrepancy exists. I can send the routers involved if anyone wants the data. Considering I had never been able to play Outpost 2 without Hamachi before this, it seems to be in a much better place.

Issues like that are fairly common with OP2. In fact, I've seen a pattern of behavior like that among lots of older games in particular with the same people, and what they all seem to have in common is they use DirectPlay, which makes me really suspicious. I wonder if there's any good way to look into that.

[Hooman]

For NetFix, he could see and join my game but I could not see his created game.

Hmm, is it possible there is a terminology gap on the part of users reporting this problem? As described, it would be quite surprising. For that to happen there would be (multiple?) packets dropped between the clients and the game server when he was hosting. Either the server never saw his game, which means packet drops between him and the server (most likely), or you didn't see the game because of packet drops between you and the server for each subsequent periodic game list update (less likely if you stayed watching the game list for a while). As both connections from client to server are initially outgoing, we're dealing with the easy NAT case here. Even timeout of the mappings doesn't seem likely, since both informing the server of a new game, and searching for new games, is started by outgoing requests from the clients, which would reestablish any mappings.

On the other hand, if that is an accurate description of the problem, that may provide some clues as to what to look at.


I tested the updates on Linux. No strange popups. Game seemed to run normally.

[Vagabond]

Thanks for testing on Linux.

Maybe I can schedule a time when a third party can attempt to join a game with the 2 of us. This could rule out if it is his sending packets or my receiving packets from NetFix's server. Typically around 9PM Eastern time is when we test if that time generally works for someone else we can pick a date.

We can still both create and play games using the TCP/IP option with the newly compiled NetHelper, so maybe the fallback is people just use NetHelper only instead of NetFix?

I really like the idea of having a sort of paired down SIGS like server so you can see who is creating games. NetFix would fill this nicely if we can get it working well.

Still recommend we push forward with the release. If we keep slow-rolling the release, there might be another small update to the op2ext API to allow properly destroying allocated memory when calling GetCurrentModDir. While incorrect, I don't think it is a big deal and existing mods would have to be recompiled to take advantage of the change anyways. https://github.com/OutpostUniverse/op2ext/pull/9

-Brett

[Hooman]

I may be lurking on IRC by 9PM or 10PM Eastern time.

[Vagabond]

Just updated Outpost 2 in the repository to include the newest version of op2ext (2.1.0). Below is the changelog for the new version of 2.1.0. The highlights are the availability of a unified logger for modules, ability to load more than 31 volumes, and improvements to the module interface.

NetFix has not been recompiled to take advantage of the new logger. So with this release, it will still create its own log files separate of the unified log message. We need to get ForcedExport and NetFix itself compiling on VS2017 before NetFix can use the unified logger.

op2ext Version 2.1.0

 * Add message logger usable by op2ext and external modules.
 * Deprecate GetCurrentModDir due to improper memory management across DLL boundaries.
 * Add external API function GetConsoleModDir_s to replace deprecated GetCurrentModDir.
 * Mark argument volName from function AddVolToList argument as const. This allows passing const values into the function.
 * Lift 31 volume load restriction. Outpost 2 will be able to load effectively a limitless number of volumes.

---

Also, I played a 3 player multiplayer game. We couldn't get TCP/IP to work, so we had to switch to NetFix. After one person switched to a different computer, we were all able to join and play. The best troubleshooting steps I have right now is asking someone if the IP address in TCP/IP is showing their internal or external IP address. If it is showing their internal IP address, I have them verify their firewall settings and restart Outpost 2.

[Vagabond]

Recently I noticed when I attach the debugger to Outpost2 to debug a custom mission, NetFix does not properly load. Occurs both in a debug and release build.

However, when I attach the debugger to debug op2ext itself, NetFix loads properly.

I attach the debugger from the Visual Studio project. So if the project is op2ext, NetFix is fine, if the project is the custom mission, it isn't fine.

My next step to test is to try and bring in the PDB file for op2ext when debugging the scenario. Hopefully I can step through op2ext initializing NetFix after the custom scenario attaches the debugger. If the bug exists in NetFix itself, I won't be able to track it easily since we don't have NetFixClient compiling right now.

I don't think this should delay the next release, but wanted to bring it up.

-Brett

[Hooman]

I've encountered similar with OllyDbg. In the case of OllyDbg, when I loaded a LevelDLL, it first loaded a DllHost.exe, which was basically just a dummy executable used to create the process and address space to host the DLL being debugged. The problem was, the dummy EXE was at the preferred (and MSVC default) base address used by Outpost2.exe. When the levels made references to Outpost2.exe (as they do to call the exported methods used to interact with the game), it would cause Outpost2.exe to be loaded into the address space. However, the preferred base address was already taken, so it was relocated to another address.

The problem with NetFix, and other OP2Internal (formerly ForcedExports) projects, is they relied on having a known fixed base address for Outpost2.exe. Or rather, they relied on having a fixed relative offset between the Outpost2.exe module's base address and it's own module's base address. The two could be relocated independently. In NetFix, there is a safety check that causes it to abort loading with a failure code if the Outpost2.exe module is not loaded at its expected address.

I'm guessing something similar happened with the debugger you are using.

The workaround I used, was to have the debugger load Outpost2.exe first, and then load the level and debug it that way.

More long term, I'd like to change how OP2Internal works so modules can be independently relocated (which is how Windows works), rather than assume nothing will ever be relocated.

[Vagabond]

I'm resurrecting this thread...

I just finished uploading new versions of the following missions to the development copy of Outpost 2.

https://svn.outpostuniverse.org:8443/!/#outpost2/view/head/GameDownload/Outpost2/trunk

• Rising From the Ashes, Colony Game
• Evacuation Under Fire, Colony Game
• Rescue Escort, Colony Game & 2P Land Rush
• On the Run, Colony Game
• Yukon Trail, 4P RR

Would someone with Linux/Wine mind downloading the development copy and opening each of these missions in Wine? I'd like to make sure nothing was broken with at least basic Wine compatibility before publishing.

op2ext was also updated to version 2.2.0.

Specifics of what was changed is in changelog.txt if anyone is curious. Some of these missions contain legitimate bugfixes. The others are just incorporating code refactoring as they were relocated to GitHub.

@Hooman, I was planning on including a new build of NetFixClient with this download unless you thought it was better to pass. It has not been uploaded yet.

If anyone has content to include outside of this, now would be the time to speak up.

Thanks,
Brett

[Vagabond]

Well, I've stalled out on work for this release. Everything I planned on adding is included except for a newer compilation of NetFix which requires more testing than I'm willing to complete in the near future.

[Hooman]

I was just checking out the new updates.

All the above mentioned levels load up fine under Wine.

I noticed the NetFix module didn't seem to be loaded. Doesn't look like there were any changes to it, based on version control history. Might have something to do with my new computer setup. I'll have to investigate later.

[Vagabond]

Hooman,

Thank you for the wine testing! Assuming it is not your new computer setup, I would try inserting the last release of Outpost 2's version of op2ext on the new release. We changed from dynamic to static compilation and introduced the multi-project solution to op2ext. If op2ext isn't loading correctly, it would prevent the modules from showing up.

Also, you can check to see if the ip address drop down is showing up in TCP/IP join screen. If that isn't showing up, there is definetly a problem with op2ext.

Thanks,
Brett

[Hooman]

I just tried again, without changing anything, and it seems to be working. Not sure what happened. Maybe an update to wine? Or perhaps some transient issue on my computer caused by other stuff I was working on.

Both NetFix and the TCP/IP join screen drop down are working now.

[Vagabond]

Okay great. I'll replace the icon, scan, and zip. I'll pm you a link once I get it all done... (today or tomorrow)

-Brett

[Vagabond]

Ok, version 1.3.8 is now live and I'll post an announcement email shortly.

Feel free to post bugs or comments here or in the announcement thread.

Thanks for everyone who help by posting bug info on the missions and especially Hooman for the heavy lifting on the technical portions of the release.

Brett

----------------
Edit:

@Drakmar,

Would you be willing to update the Macintosh installer for the new version?