I wouldn't suggest sending everything. I'd prefer to keep the protocol as light as possible in terms of bandwidth requirements. If the information isn't strictly needed to support network operation, then I'd like to keep it out of the protocol. I'd also like to not waste disk space by actually recording available information.
My comment was more a thought that proper logs might help development, as would a slight increase in information, mostly concerning external port mappings, and possibly player network IDs. The later is not strictly needed to implement any useful features, but might be interesting to debug current apparent problems. I don't believe the player net ID is of a particularly sensitive nature. It essentially consists of the generation time (time since Windows was last booted), and the player's index in the player table. Neither is a permanent identifier for a player, and should change for each game played. Essentially I'm suggesting collecting a global view of the connection state to better understand the connection problems (and possibly the mapping of temporary player IDs to connections). What makes current development so difficult is only being able to see the connections from one point of view (which will different from other points of view for the problem cases that need debugging).
To auto-detect certain router issues, you'd need to be aware of certain external port changes. These results would need to be compared from two different external sources. The game server is one possible external source, but another one would be needed for auto-detection. I had previously thought two game servers working together might work for an enhanced mode that can warn of router problems. This would allow those properties to be determined on connect, even before a game is even hosted. However, I would prefer not having to run two game servers on two seperate boxes if possible, and I definately don't want to think about keeping game lists synched between two of them, or modifying one to be a custom non-full game server. Another alternative is somehow using the port information from connecting peers. The current problem is that a peer only knows how it sees other peers. There is no global view of the port mappings, nor is any given peer aware of it's external port mappings to other players. It's also worth noting that such a scheme would only work once players start connecting to each other, after a game has been hosted.
I think the solution I might like the best, is if each peer informed each other what it sees of that person. There would be no global view in this case, but each peer should learn enough about itself to determine if it's behind a problem router. It would require more packets to be exchanged than if the information was collected in a central place, although, the packets would be slightly smaller. It also negates any privacy concerns since a client would only be learning about itself using information already available to other clients. This doesn't create for an easy debugging/development environment though, and if something goes wrong it will require each player to submit whatever logs are generated on their computer to diagnose the problem.
Is there a timeout for people who did already complete the join process? I.e. if their port/ip changes after they joined a game, but before its started?
I believe so, but I don't quite remember anymore. I think a player who stops responding will get dropped. I vaguely remember this happening while debugging when one client was stopped at a breakpoint.
I would also like to avoid pestering the user with dialog boxes concerning IP changes (and mostly avoid having to write such code). I also don't want to add account login requirements. If anything, maybe share a randomly generated pairwise unique value between peers for this purpose, and do the switchover automatically if the secret is known. You'd want the number to be random to prevent abuse by anyone on the internet. You'd want the pairwise uniqueness property to prevent other players from that game from abusing otherwise shared information.
I'm still perplexed by the packets you say you've recorded. If you find anything else out about them I'd like to know. It looks like a possible bug to me. I don't think they are actually valid packets.
The JoinHelpRequest is not sent by either Outpost 2, nor the NetFix client. It's sent by the NetFix server. The client only sends JoinRequest packets. Essentially, the JoinHelpRequest is a third party join introduction. It was needed to support routers that filter out traffic from hosts it has not sent data to. This would normally be a problem for a game host. In this case, the join request is routed through the game server. This traffic should get through because the host sent data to the game server about the hosted game, and so a reverse mapping now exists in the router's table allowing the game server to send data back to the game host. The game server transforms the JoinRequest packet from the client that wants to connect into a JoinHelpRequest packet so the host knows it's not the game server trying to join, but rather is being informed of some third party that wants to join. To ensure this third party can connect, the host will then send an empty packet to the client trying to connect, thereby creating a reverse mapping in the host's router table. The empty packet will simply be discarded by the connecting client as invalid/not-of-interest. However, subsequent JoinRequest packets (such as those sent after a short timeout) should now make it to the host through the newly formed reverse mapping. Hence, even somewhat problematic routers should be able to host games, with a possible slight delay in initial connections.
Now I did make one simplification in the protocol. Partly out of laziness and wanting to keep the code simple, but also to avoid unnecessary delay. When a client tries to join a game, it sends a packet to both the game server (if an address is set), and to the game host, in that order. Strictly speaking, this is a bit wasteful, and tells the game server more than it needs to know in most cases. The game server only needs to relay the packet for hosts whose routers filter packets from unknown addresses. However, this is no way to know this ahead of time. I suppose you can assume if the first JoinRequest goes unanswered that the join packet was dropped, and this is likely because the host's router filtered the packet, and so try routing through the game server on subsequent attempts, but this introduces more delay when this feature is needed and still might inform the game server more often than needed due to random dropped packets. So, in short, the game server can potentially track which games you've joined, if it so desired. Not strictly needed in all cases, but I did hear about problems less frequently after this was implemented. I don't believe there is a foolproof way of only sending this help request when strictly needed, so instead I've chosen to minimize delay when it is needed.
