Author Topic: Game Recording System  (Read 4419 times)

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« on: June 27, 2006, 05:57:10 AM »
Well, it was a 'top secret project' for a while but I figure I may as well release the details on what I've written.

I've written a system to record OP2 games into a file and allow you to play them back at a later time, including multiplayer games.

So like the description says, pretty soon you will be able to throw away Camtasia studio, Fraps, or whatever video recorder you use to record OP2 games. And you can throw away the lag that comes with using such software.

Also, if you want to watch a replay of someone's game, it offers a couple of advantages over the traditional way of using a screen recorder program.

First of all, it doesn't lag the game at all like a screen recorder would. It doesn't need much processing power, it just works by intercepting commands from the game and recording them into a file before passing them on.

The files it produces are also very small compared to the XviD videos we have posted on the site before. While an XviD recording of a game might be 80 MB or even more, the recorder logs probably run a max of 30 KB / minute for a 2 player game. (It depends on how many commands are being issued to the game at a time, and the number of players (since more players mean more commands get logged)).

This works out to about 450 KB for a 15 minute game with 2 players, not even half a megabyte. The data inside the recording file should also compress very well, so it's rather trivial to send a recording of a game to someone else. The only other thing you need besides the recording file is a copy of the mission used. (Map, DLL, techtree if there's a custom techtree).

Also, the XviD videos have their resolution decreased just to reduce their filesize. This means it's a lot harder to see what's actually going on in the game. Also, by watching the video you are forced to follow the view of the player who is recording the game. With this playback mechanism, you can scroll around the map freely and see what other players are doing, at full resolution since the playback system runs inside of OP2.

If there is any downside to this system, it's probably the fact that you have to play back the replays in OP2. However, I think this is a small price to pay considering the benefits you get (and you can still use camtasia / whatever if you want to record a 'normal' video).

As for actually having a working recorder / playback system, Mcshay and I have been testing it over the last few days, it's had more than a few bugs (it tends to freeze after a minute or two into the game, and I still haven't tracked down why). It only does this in multiplayer.. and only did it when I tested with him. (I tested on my LAN with two computers and it doesn't happen. Presumably it has to do with dropping packets).

Also, it doesn't save data for all the players in the game (which is minor, I can just write the few extra lines of code needed to record all the player's data).

I haven't tested how it works between game loads / saves in single player. However, my priority is multiplayer recording, because you can't even save a multiplayer game, and you can't really take anything away to keep after the game ends.

Other things I plan in the future are UI integration. For example, on the summary screen after a multiplayer game ends, I plan on putting a Save Replay button there rather than forcing people to use some other strange methods. Also, I plan on a new menu button to allow loading recordings directly to play them back. (Currently the test code is pretty primitive. You have to run a special loader to start OP2 with the recording software enabled, which records the game to a fixed filename).

But anyway, if you have any suggestions, or would like to help me debug the system, let me know! Ideally, I would like to get the finished system into the next game release.

Offline Stormy

  • Hero Member
  • *****
  • Posts: 679
    • http://www.op3game.net
Game Recording System
« Reply #1 on: June 27, 2006, 10:56:35 AM »
Will you be able to move the view freely as things are automatically built and all that based on the recording?
`..`..`..`..`..`..`..
3D artist in Blender, MS3D, and Terragen.
Trying to get good with Scene composition and lighting.

Offline Mcshay

  • Administrator
  • Sr. Member
  • *****
  • Posts: 405
Game Recording System
« Reply #2 on: June 27, 2006, 11:14:49 AM »
Yes, basicaly you are an observer of a game that already happend.

Offline Sl0vi

  • Sr. Member
  • ****
  • Posts: 468
Game Recording System
« Reply #3 on: June 27, 2006, 12:32:57 PM »
This sounds nice, great work!
!!!YAY!!!

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« Reply #4 on: June 27, 2006, 12:36:33 PM »
basically its like watching a bunch of AI's fight. You can click on player 1 and see what they are doing, but can't issue any commands.

(I might make it so you can pick what point of view you want to see the game from -- what player)

Offline dm-horus

  • Hero Member
  • *****
  • Posts: 1047
    • Division 32 Community
Game Recording System
« Reply #5 on: June 27, 2006, 08:12:41 PM »
very nice work!

i plan to use this when it is done
Always outnumbered. Never outgunned.

Offline Leviathan

  • Hero Member
  • *****
  • Posts: 4114
Game Recording System
« Reply #6 on: June 28, 2006, 11:21:12 AM »
totaly great work! cant wait

so happy we have you op2hacker ! :D

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« Reply #7 on: July 10, 2006, 07:53:35 AM »
Project update:
The recorder now works fine, no more freezing, crashing, or using 100% CPU.

It now records the actions of all players in the game. It also now records the player names and colors (unlike before).

However there are still a few bugs / problems that I want to work out before it gets released:
- If two players send a command during the exact same game cycle, the packet will get dropped during playback (it will still get recorded however).
I need to look and see if multiple players can even send a packet at the same time like that, to see if this will be a problem.
- If you exit the game by clicking the close box / end task, the recording won't end properly. Currently the recorder installs a code hook where the red text gets displayed in the game, if you exit with the close box messages could get dropped.
- The recorder will act strange, perhaps not even work at all if you load a game, instead of starting it from scratch. (It's not designed to handle loading / saving of singleplayer games yet)
- Since it is still test code, you have to record by using a loader, which records to a fixed file test.rec, which you can then load by going to the debug menu > Run Script and loading the playback mission.

There are also some other caveats in the game itself that would prove too time consuming to work around: (read, this is the way the game works and that's that)
- You can only see the game from one player's POV at a time (that is, by clicking on their structures / units and see exactly what research topics they're doing / structures building / etc). It might be possible to switch this during game play (perhaps with a dropdown / something else)
- The recorder can't follow the player's view around (this could probably done but it might take a lot of code hooks to reliably store the player view at each cycle. It could balloon the size of the recording files as well)
- The chat log before the multiplayer game is actually started can't be easily recorded.
- If sheets.vol, the mission DLL, map, or techtree get modified, the playback might act strange (since the game environment will be changed). The playback system does check for this (by checksumming all these files). I might add an option to bundle these files into the recording itself so you could have a recording that will play back regardless of the state of the files in the game.

Anyway, if you're willing to test, let me know on IRC! I want to try this in a full length 6player game, to see how well it holds up.
 

Offline CK9

  • Administrator
  • Hero Member
  • *****
  • Posts: 6257
    • http://www.outpost2.net/~ck9
Game Recording System
« Reply #8 on: July 10, 2006, 08:14:52 AM »
This is awsome, great for tournaments.  Instead of having to wait for a referee to be on at the same time as your opponent, just send in the recording of the game for proof of the victory.
CK9 in outpost
Iamck in runescape (yes, I still play...sometimes...)
srentiln in minecraft (I like legos, and I like computer games...it was only a matter of time...) and youtube...
xdarkinsidex on deviantart

yup, I have too many screen names

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« Reply #9 on: July 10, 2006, 08:44:19 AM »
Only thing is, if the losing player had time, they could doctor the recording file and make it appear as if they had won. (The file format is pretty easy to understand, provided you understand how the game commands work). To be sure that a player doesn't cheat you could ask for the files from all the players in the game. If the command log doesn't match up closely then you know that someone cheated.

I was thinking of making it part of WD or something, where you could put stats on how many actions the player took / units built / etc.
« Last Edit: July 10, 2006, 08:45:51 AM by op2hacker »

Offline Leviathan

  • Hero Member
  • *****
  • Posts: 4114
Game Recording System
« Reply #10 on: July 10, 2006, 09:01:29 AM »
Great work. Thanks for the update :D

Wanted this for years.

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« Reply #11 on: July 10, 2006, 11:26:29 AM »
Well, today myself, Moogle, wizisi2k, and Mcshay played a 4 player game on La Corrida to completion with the recorder. It was N v S, so it was Moogle/myself vs wiz/Mcshay. Moogle and I lost (snuck a nova into my base *oops*, Moogle got hammered with emp / esg and Mcshay took a nova to the CC).

right now, the playback system isn't working. (The message queue gets flooded with chat messages, and then crashes at mark 25). I've had a look at the recording file itself though, which is in perfect condition, so the problem is in the playback system itself.

Hopefully this should be an easy problem to fix.

Offline CK9

  • Administrator
  • Hero Member
  • *****
  • Posts: 6257
    • http://www.outpost2.net/~ck9
Game Recording System
« Reply #12 on: July 10, 2006, 01:52:10 PM »
Quote
Well, today myself, Moogle, wizisi2k, and Mcshay played a 4 player game on La Corrida to completion with the recorder. It was N v S, so it was Moogle/myself vs wiz/Mcshay. Moogle and I lost (snuck a nova into my base *oops*, Moogle got hammered with emp / esg and Mcshay took a nova to the CC).
 
Hey!  They stole my tactic!  lol

Good luck with the error fix
CK9 in outpost
Iamck in runescape (yes, I still play...sometimes...)
srentiln in minecraft (I like legos, and I like computer games...it was only a matter of time...) and youtube...
xdarkinsidex on deviantart

yup, I have too many screen names

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 3857
Game Recording System
« Reply #13 on: July 11, 2006, 06:13:33 PM »
Wow, nice work.

Are you recording the messages as they are written to the buffer, or as they are extracted to be executed? You can probably save on space a little if you only bother with the ones extracted since messages can be overwritten if they're too close together. But it kinda sounds like you're doing this already.

As for the view, you could just store the x,y position of the viewport for each player as you store their command packet. It shouldn't baloon the file size too much. Or just record the view every game cycle. It's only 4 times more frequent than command packets are executed. The filesize overhead should be fairly minor. Or were you looking to record all keyboard commands and display the menus the player went through to do stuff? That'd probably be a lot harder to do.

So uhh, yeah, post some technical details for me.  :D  

Offline Mcshay

  • Administrator
  • Sr. Member
  • *****
  • Posts: 405
Game Recording System
« Reply #14 on: July 11, 2006, 06:23:04 PM »
Today we also tested the recorder with five players.

It was north vs south on root cannal, Hacker, Iso, and Bomber vs Beta and I. Everything went normaly at the start, no one rushed anyone with basic combat units. Shortly into the game, Hacker made a comment referencing our last test game where I nova'd his cc (something along the lines of Mcshay is going to nova me again). So I built some starflares and ran them north, avoiding Hacker's units and attacked Iso's base (but he lived). Eventualy (mark 500ish) no one had started a major attack, so I sent 3 novas around the back in the dark. I managed to take out Hacker and Iso's ccs  :P. The game ended shortly when Beta and I charged Bomber and mobed his cc. However he quit before we could get the final blow in  :(.

Anyway, the playback works nicely now. However something is still broken. Around mark 180 the player who the recording was focused on (Bomber) completed the Common Ore Smelter research  :blink: and connected a tokamak to his cc by tube where his advanced lab was. Shortly after the tubes were done, the game crashed. I believe this was caused by the player attempting to research at a tokamak.
« Last Edit: July 11, 2006, 06:25:18 PM by Mcshay »

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« Reply #15 on: July 11, 2006, 06:55:29 PM »
Yet another update.

I changed the playback code so that all the command packets get loaded into separate linked lists, one list per player. This takes care of packets getting dropped when two players send a command at the same time.. however, the crashes haven't gone away.

This time, instead of the game crashing at mark 25, it crashes at mark 55 on the recording file for the game we played the other day.

Today we played a 5 player game, of myself, Mcshay, Betaray, Isolocis, and Sirbomber.. this recording did a little bit better but not much.

It seems some sort of strange things, subtle corruption or whatever is happening. At mark 181 the game output a message "Common Ore Smelter research completed", complete nonsense. Also, it shows player 1 (Sirbomber) building a tokamak instead of an advanced lab, however the convec is an extra square away. (It is in the position to be building an advanced lab, but instead a tokamak is getting built).
The game crashes once the construction completes.

I'm not totally sure of what's happening, but I think some sort of subtle corruption is taking place. I plan to read through the command log manually and see if a command to build a tokamak is really getting sent. Hopefully I can get to the bottom of this.

Edit: ditto what Mcshay said mainly :P, I started writing the post before he posted his. ;)  
« Last Edit: July 11, 2006, 06:56:14 PM by op2hacker »

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« Reply #16 on: July 11, 2006, 07:06:50 PM »
Quote
Are you recording the messages as they are written to the buffer, or as they are extracted to be executed? You can probably save on space a little if you only bother with the ones extracted since messages can be overwritten if they're too close together. But it kinda sounds like you're doing this already.
I install a hook right before TApp::PlaybackCommand is called (I overwrite the mov ecx, offset gTApp etc. with a call instruction), this is where the packets are extracted to be executed. It's inside a loop where the game iterates thru all the players.

I push the EDI and EBX registers to the function that handles recording itself.. EBX happens to contain the player ID of the player whose command packet is getting read, and EDI is a pointer to the next command packet.

I then write 0x74 bytes to a private buffer, and then overwrite the timeStamp and unknown fields with the game tick and player number respectively. I then store the contents of the private buffer to the recording file.

I do some other patches as well. I patch right before the DLL's InitProc is called, and at this point I save all the game vitals (random number seed, player parameters like names, colors, difficulty, and the game params like day and night, morale, etc). I also install the game loop hook at this time.

I patch the method that displays the red text to close the recording file, though I should change this to a place that will get called regardless of how the user closes the game. (If they use the close box to exit, it won't get called)

As for playback, the hacks are even nastier. The playback system is a normal mission DLL which loads the real mission DLL on top of it. I do some unheard-of stuff like changing the map and techtree at runtime (from the InitProc of the playback DLL). The only other thing I had to do was patch a GetProcAddress call in the game so that I could do the GetProcAddress myself on the mission DLL instead of the playback DLL so the game could find trigger exports and other things.

I still patch the same place for command packets to be written into the game engine buffers.

If you have any ideas on the oddities with the crash and the tokamak being displayed, let me know. the game crashes after a virtual function call to the unit function that processes a game cycle (part of the unit vtbl).

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 3857
Game Recording System
« Reply #17 on: July 11, 2006, 08:54:03 PM »
Oh, interesting.

Did you ever take a look at that project I had that hooked the game loop right where the network packets were exchanged? Basically the last place possible where you could create a command packet and have it sent to other players to be executed. I placed the hook there so there'd be no trouble with overwriting player issued commands if both a player and an AI were controlling the same colony. Heh, AI assisted play.  :D I figured it might make developing an AI easier if I could fill in the gaps meanwhile while parts of the AI were still being developed. I never really got to the AI stuff though. I think the code location was at a deeper nesting that TApp:PlaybackCommand.

Anyways, if you hooked that area, probably after the network exchange code, it'd be a good place to handle this sort of stuff without too much fuss. Force write each packet to stop the player from accidentally issuing commands during playback, and you could also save packets there knowing that what's in the buffers to be executed at that point is sure to be executed. Well, executed every 4 cycles, but I guess this loop runs a little more often. Just do a tick check before saving or writing to make sure it matches up with the cycle that CPs are executed on.

As for the corruption, the CommandPackets are of variable size. You might want to use the size field in the header to save them properly. I'm pretty sure I've seen code create unusually large packets. Larger than the stack space typically reserved in functions that create and issue CPs. I'm not sure if it's an issue in network code though. Although, the network buffers do allow for CPs to be up to 512 bytes in length. Who knows, with any luck maybe the game designers remembered to properly set the size field for every CP they create in the game.  :P  (Actually, don't worry too much. The network code only copies the number of bytes specified in the size field).

Oh, and you shouldn't record and playback every CP the game executes. Remember that some will be auto generated and executed independent of the game cycle. These are not controlled by player issued commands, but rather by DLL or game code. If you saved and played these back, they'd probably be executed twice since the game would autogenerate them again.

Btw, nice work with all the game vitals stuff and getting the correct DLL loaded for playback and all. That actually sounds like quite a hassle.
 

Offline Arklon

  • Administrator
  • Hero Member
  • *****
  • Posts: 1120
Game Recording System
« Reply #18 on: July 11, 2006, 11:02:53 PM »
I wonder... maybe Sirbomber had multiple structure kits in the factory's storage (Tokamak and Advanced Lab), and the recorder doesn't record which structure kit is pulled out correctly?

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« Reply #19 on: July 12, 2006, 06:00:39 AM »
Quote
Did you ever take a look at that project I had that hooked the game loop right where the network packets were exchanged? Basically the last place possible where you could create a command packet and have it sent to other players to be executed. I placed the hook there so there'd be no trouble with overwriting player issued commands if both a player and an AI were controlling the same colony. Heh, AI assisted play.  :D I figured it might make developing an AI easier if I could fill in the gaps meanwhile while parts of the AI were still being developed. I never really got to the AI stuff though. I think the code location was at a deeper nesting that TApp:PlaybackCommand.
That's where I had originally hooked at, right before Network.GetGamePlayPacket was called (in fact using your code you sent me, the fact that I had that code was one of the reasons I was interested in writing this). Basically the first 'real' thing that happens in Dans_RULE_UIFrame.OnIdle (well of course, they call QueryPerformanceCounter to time the game loop).

The only problem with that was the random freeze-ups. I put a trace on several functions (Network.GetGamePlayPacket, TethysGame.ProcessGameCycle, some other places in there as well). What was happening, as soon as network lag occurred, (resent packet or a wait) the game would basically freeze.. my code would continue to be called, Network.GetGamePlayPacket would continue to be called, but the rest of the stuff like TethysGame.ProcessGameCycle would get skipped. The same packet would keep getting recorded for thousands of game cycles in the recording file.

This was a nasty problem to track down since it doesn't happen in single player or even when playing multiplayer with two sessions on localhost (or on the LAN for that matter). The lag was a hunch, so I wrote a wsock32.dll 'wrapper' that would allow me to create lag on one session. (It would randomly drop data given to sendto() but still return a success value).

After this I figured I would stay away from the network code and insert my hook in TethysGame.ProcessGameCycle.

--

Hmm, however, now that I think of it, I was modifying the packets inside the game engine. I was setting the timestamp to TethysGame.tick which would obviously cause problems if a packet was resent - it could have a different timestamp on it.
I noticed this after I switched where I was hooking the game loop, since by then I had started to use the .unknown member of the CommandPacket struct to store the player ID. What I figured was happening then, was the network layer was getting confused since it stores the net ID there.

Right now, I don't use Player.GetNextCommandPacketAddress at all, I just push the buffer that the game is about to give to TApp.PlaybackCommand as well as the player ID.

The other thing is, if I hooked before the network code as I had before, and stopped modifying the packets (which would probably prevent the game from freezing / blocking for more network data) couldn't I still drop data in the first tick or so? For example, if a player happened to send commands right at the beginning of the game.

Quote
Oh, and you shouldn't record and playback every CP the game executes. Remember that some will be auto generated and executed independent of the game cycle. These are not controlled by player issued commands, but rather by DLL or game code. If you saved and played these back, they'd probably be executed twice since the game would autogenerate them again.

I skip recording if it's a ctNop or ctMachineSettings packet. However, this was mostly to keep the file size down (recording either of these in addition to the rest of the commands had a tendency to double or triple the file size).
I figured ctNop = useless, since the game ignores it anyway, ctMachineSettings = useless as well since that's how the machine information is transmitted (for the net stats page, I think) but you can't access the net stats in the single player playback.

The playback system plays back any packet in the file.

Quote
As for the corruption, the CommandPackets are of variable size. You might want to use the size field in the header to save them properly. I'm pretty sure I've seen code create unusually large packets. Larger than the stack space typically reserved in functions that create and issue CPs. I'm not sure if it's an issue in network code though. Although, the network buffers do allow for CPs to be up to 512 bytes in length. Who knows, with any luck maybe the game designers remembered to properly set the size field for every CP they create in the game. :P (Actually, don't worry too much. The network code only copies the number of bytes specified in the size field).

I noticed you had a note saying that command packets have a fixed maximum of 0x74 bytes. Is that necessarily true? It's obviously a lot less than 512 bytes.

Quote
Btw, nice work with all the game vitals stuff and getting the correct DLL loaded for playback and all. That actually sounds like quite a hassle.

Thanks :P Actually, it wasn't that bad. What I did was load the DLL and store the addresses of the major entry points (InitProc, AIProc, GetSaveRegions, etc..) In InitProc I load the recording file and validate some headers (there's a chunk of text that I do a string compare on so I know that it's a recorded game file). Next I pull out the mission specifics and checksum all the files in the game set (DLL, map, techtree, the spreadsheets in sheets.vol) For the checksum in both recording and playback I just use ResManager.ChecksumStream. If the checksum fails on any of the files I put up a warning and give the user an option to continue.

Next I load the game settings like day and night, etc and put them into the game engine. (I just overwrite the locations where functions such as TethysGame.UsesMorale, etc read from) I also read the random number seed into the random number generator class. (8 bytes here although the seed is only four bytes. The SetSeed function they use does some math on the number and gets 8 bytes. I figured this would be easier than trying to catch the 4 byte seed they use since the seed is set far before InitProc is called {even before the player objects are initialized}).
I then load the player information. Most of these settings are set the same way, I just write into the player classes (or use the exported function to set it if possible so I can avoid messy hacks as much as possible). As for the player name, that's easy enough too. It turns out that there are functions to get and set the player name, which are both part of the player class.

At this point I check that all the entry points are valid for the DLL, and then load the map and techtree. Reloading the map / techtree seems odd but it's actually very straightforward to do. I just simply call Map.LoadMap again with the map name I want to load, same with Research.LoadTechtree.

Next I patch GetProcAddress like I mentioned before so the game will get the correct address of trigger exports, unit block exports etc.

Finally I call the mission InitProc. In the rest of the functions (AIProc etc) I just call the mission AIProc and such.

---

Quote
I wonder... maybe Sirbomber had multiple structure kits in the factory's storage (Tokamak and Advanced Lab), and the recorder doesn't record which structure kit is pulled out correctly?

It should pull out the correct kit provided they were in the correct bays. The command that handles putting a kit into the convec (ctMoTransferCargo) takes the unit ID of the factory which holds the kit, and the bay ID. (The vehicle ID isn't needed since when this command is issued the vehicle will be on the dock).

The only thing I can think of, if something finished building sooner or later than it was supposed to. The playback system uses a DescBlock with MultiLastOneStanding for the mission type so the times shouldn't be different. (Things take longer in single player than in multiplayer).

I plan to look through the commands by hand today, and see if there are any anomalies.

(and now, this very long post comes to an end :P)

Offline Sirbomber

  • Hero Member
  • *****
  • Posts: 3170
Game Recording System
« Reply #20 on: July 12, 2006, 06:04:23 PM »
Quote
Around mark 180 the player who the recording was focused on (Bomber) completed the Common Ore Smelter research  :blink: and connected a tokamak to his cc by tube where his advanced lab was. Shortly after the tubes were done, the game crashed. I believe this was caused by the player attempting to research at a tokamak.
I never built tubes anywhere near my Tokamaks...

And I never realized people liked talking about me so much!  :wub:

 :unsure:  
"As usual, colonist opinion is split between those who think the plague is a good idea, and those who are dying from it." - Outpost Evening Star

Outpost 2 Coding 101 Tutorials

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 3857
Game Recording System
« Reply #21 on: July 12, 2006, 08:59:16 PM »
Mmm, now there is a satisfying post.


Definately a bad idea to overwrite the tick if packets can get dropped. Remember that network test where I changed the tick and the game froze? That was because the tick on the packet is compared to the current tick, and packets can be dropped or overwritten if it's wrong. Or the packets won't be overwritten if it's wrong, I can't quite remember the details. But messing with the tick will definately cause some problems in a lagged game.

The netID probably isn't as big of an issue, but I still wouldn't mess with it. I don't think I ever fully understood how that field was used. Seems the lower order bits may have been a player number, but I've forgotten everything about it now. Why would you change these anyways?

I probably thought the command packet sizes were fixed early on due to local stack sizes on the functions that created and issued CPs, storing them as local variables. It seemed pretty consistent suggesting that CPs were declared as a fixed sized struct or enum. Although there is really nothing in the CP processing code to limit the CP size. You could pass it a CP of any size really. There is no validity checking. Just index size limitations on how big a useful one can get. There also isn't much checking when creating the CPs in the first place. I'm pretty sure I've seen at least one CP go over the size I thought they originally were.

To be safe, I avoided creating any that were too big, but assumed they could be bigger.

 

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« Reply #22 on: July 13, 2006, 12:40:51 PM »
Quote
Definately a bad idea to overwrite the tick if packets can get dropped. Remember that network test where I changed the tick and the game froze? That was because the tick on the packet is compared to the current tick, and packets can be dropped or overwritten if it's wrong. Or the packets won't be overwritten if it's wrong, I can't quite remember the details. But messing with the tick will definately cause some problems in a lagged game.

The netID probably isn't as big of an issue, but I still wouldn't mess with it. I don't think I ever fully understood how that field was used. Seems the lower order bits may have been a player number, but I've forgotten everything about it now. Why would you change these anyways?
Well, after I realized the problems, I stopped modifying the timeStamp and netID fields for the packet in the game engine. The buffers inside the game engine don't get modified during recording. (I just read a command packet to a private buffer then deal with it there).

I use them to record the current tick and player ID, the playback part replaces these with default values.

(If you take a look at the code of Player.GetNextCommandPacketAddress, they fill in 0x80000000 for the timestamp and 0 for the netID).

Also I'm taking into consideration that the timestamp and netID aren't really required for single player games -- they get ignored.

As for the command packet size that's probably where I'm going wrong. I can't really imagine a packet over 0x74 bytes but I'm sure it's possible (seeing as you said the network buffers can hold 512 byte command packets).

Offline CK9

  • Administrator
  • Hero Member
  • *****
  • Posts: 6257
    • http://www.outpost2.net/~ck9
Game Recording System
« Reply #23 on: July 16, 2006, 07:45:40 PM »
probably too late to be mentioning this now, but I just realized that a few of my games have recording devices that work in the same way you decribe thisone to be, and I was thinking it might be usefull t see how they are coded to help find the bugs in yours.
CK9 in outpost
Iamck in runescape (yes, I still play...sometimes...)
srentiln in minecraft (I like legos, and I like computer games...it was only a matter of time...) and youtube...
xdarkinsidex on deviantart

yup, I have too many screen names

Offline BlackBox

  • Administrator
  • Hero Member
  • *****
  • Posts: 3103
Game Recording System
« Reply #24 on: July 17, 2006, 05:47:14 AM »
Well, I'm not sure looking at those would be very helpful. Other games store and transmit commands differently when compared to OP2.

Also, there isn't a lot of comparison since those games have been designed to be able to be recorded. In the case of OP2, we're doing stuff that's not really 'intended' to be done. Both the recorder and playback have to do messy hooks inside the game loop, there isn't really any 'clean' way to just plug in to the game loop and start recording messages.