Author Topic: Remote Pair Programming  (Read 40392 times)

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #50 on: November 11, 2017, 06:20:11 AM »
The op2ext project has been pushed to GitHub:
https://github.com/OutpostUniverse/op2ext.git


Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #51 on: November 12, 2017, 12:55:21 AM »
Thanks Hooman for pushing to GitHub.

I'm using a different CPU than last time, so hopefully everything works well.

I'll plan to find you on the Outpost Universe mirc room. Backup will be Discord.

-Brett

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #52 on: November 13, 2017, 12:43:58 AM »
We completed another successful pair programming today. Check out the thread http://forum.outpost2.net/index.php/topic,6045.msg85715/topicseen.html#new for details of the progress.

Lag was noticeable today. It was bad enough that it made it difficult for Hooman to program at times. (He was the one remoting in to my CPU).

In retrospect, maybe we should have tried lowering my screen resolution to reduce the pushed bandwidth. My newer CPU has a resolution of 1920x1080, which may not have helped?

I'm not sure internet bandwidth is the problem since I'm able to stream high resolution videos without any lag.

-Brett

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #53 on: November 13, 2017, 01:07:21 AM »
I'd like to point out the main lag issue was simply with the mouse cursor, and that was only after adjusting some setting. Previously, the mouse cursor would be invisible over certain windows. It was a choice of either a laggy mouse cursor that was always visible, or a reasonably responsive mouse cursor that would sometimes be invisible.

I also noticed a slight delay in the audio. I could see Vagabond type something, and hear him still typing it after he'd finished typing.

It was still a very workable session though.

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #54 on: December 15, 2017, 10:19:17 PM »
Just checking when our next session is. And prodding other people that might want to give pair programming a try.

In particular, any plans for December 18th, 00:00 GMT (midnight/Monday morning GMT, Sunday night North America)?

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #55 on: December 16, 2017, 03:32:36 PM »
Hooman,

Thanks for the prompt. I hadn't posted yet because I was still working through a venue. I'm going to try this session from a library conference room. My wife and kids are kicking me out of the house for the afternoon.

I would like to move the session 1 hour earlier than usual, so starting at 17 December, 11PM GMT. I have a library room booked for a 2 hour period. After that, we could work another hour in the library itself which would probably mean chatting back and forth an not using Team Viewer.

Also, their internet connection may not support Team Viewer bandwidth as I've never used it for this. Just setting expectations...

First project: Trying to fix minimap colors when using non Outpost2 specific BMPs with the Map Editor. (I'm bringing almost zero knowledge of bitmap file structure to the session, FYI).

If that goes quick enough, we can locate and document where the version number and copyright year are set for display on the Outpost 2 ABOUT page.

The other option for a second project is start designing a module for Outpost 2 to serve as a test-platform for op2ext. I'm thinking we can just output debug messages within the IDE displaying results to ensure all the hooks and C ABI export functions are working. This could continue into actually using GoogleTest on the project. Since I've never actually written a module, I'd rather focus just on that instead of bringing in GoogleTest though...

-Brett

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #56 on: December 16, 2017, 11:34:14 PM »
Ok, I've set my alarm to wake up an hour earlier.

I'm hoping the map editor fix will be quick and easy, though we'd need to verify the source of the problem first, which could lead to surprises.

For the version number, it might help to have OllyDbg pre-installed, and a copy of the .udd file from the repository available. No guarantees we'll need it, but that's usually the tool I use when investigating the exe.

Will need to keep in mind there was an exe mod that invalidates the .udd file. I've been using SVN revision r915 for the Outpost 2 game download, as that is before the exe was modified/resized. I tagged that revision in my git-svn clone to make it easier to work with.

Trying to do the above, and then write a module for the first time, and unit tests does sound a bit ambitious for one sitting. We'll likely have to prioritize there, in which case trying to write a module and do manual testing is likely the place to start.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #57 on: December 17, 2017, 12:25:41 AM »
If we can cleanly fix the mapper in a single session, I'll be happy. The rest is just sort of a laundry list of where I'd like to go eventually.

I have Ollydbg on my CPU and the full SVN repo. We will have to pull the files manually from r915.

-Brett

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #58 on: December 17, 2017, 11:59:15 PM »
During the last session Hooman and I managed to get OP2Editor (the backend for the OP2Mapper) compiling using Visual Studio 2017. We made some general updates to the OP2Editor repository to clean up what is posted.

When loading the OP2Mapper using the newly compiled OP2Editor.dll, the program loads fine and opens vol files. However, when loading a map file into the mapper, OP2Mapper.exe crashes. We were not able to pinpoint the cause of the crash. We never got to actually trying to fix the bug.

Visual Studio 2017 was also appearing to stall out during the compilation of OP2Editor.dll. Visual Studio would continue to run fine, but it would never finish actually compiling the DLL. After some fiddling, it started compiling again although we never figured out what the problem was in the first place.

I did learn some basics about Microsoft COM/MIDL compiler and the Git squash command.

In the short term for me this effort has stalled out. I don't have the skills to continue troubleshooting the new bug without compiling the OP2Mapper 2.2.1 source code. This code is written in Visual Basic, a language I have no experience with.

Blackbox mega thread of goodies: http://forum.outpost2.net/index.php/topic,4909.msg71732.html#msg71732

Hooman mentioned getting the mapper's front end code into the repository, which seems like the next logical step.

-Brett

--------------------------
EDIT

I deleted OP2Editor from the SVN repo to reduce number of copies of the codebase floating around.
« Last Edit: December 18, 2017, 12:31:57 AM by Vagabond »

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #59 on: December 22, 2017, 06:47:03 AM »
I gave this a bit of a go in a VM.

If you run Mapper2 with debugging (F5), it will crash when you open a file. If you run Mapper2 without debugging (Ctrl+F5), it opens and runs normally. I have not investigated the cause, but felt this was worth mentioning.

Running Mapper2 straight from the source package wasn't working for me. I got a bunch of errors from unregistered components. After registering them, I still got a cryptic "Run-time error 440". I eventually tracked down an installer for Mapper2, and after running that, the copy from the source package was able to run. I'm not sure what setup the Mapper2 installer does, but it'd be nice how to replicate it in a development environment, or details on how the installer was made.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #60 on: December 23, 2017, 03:34:05 AM »
Hooman,

I just tried running without the debugger and can confirm it works on Visual Studio as well. Should have thought about that during our pair programming session!  :(

When using the debugger, I tried playing with the debugger's working directory by setting it both to the location of OP2Mapper.exe and where I was pulling the map/wells from. Either way, it didn't seem to help.

Now that it is working, have you taken a crack at fixing the actual bug yet?

-Brett

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #61 on: December 29, 2017, 09:51:53 AM »
I did some fairly rudimentary testing, but didn't come up with anything. Doesn't seem to be related to the 5/5/5 or 5/6/5 RGB bit encoding idea I had earlier. Though that's not a big surprise since it's using 8-bit palette based bitmaps, and not 16-bit color, so that branch of code isn't run.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #62 on: December 29, 2017, 06:29:18 PM »
I'm planning on actually playing some Outpost 2 this coming weekend, so probably no pair programming.

I may be able to swing 7 January, 11PM GMT if there is someone else with availability.


My priority would currently be locating and documenting where the version number and copyright year are set for display on the Outpost 2 ABOUT page.

Not sure if Hooman wants to work more on the OP2Editor minimap bug in a pair session now that we have the code running (even if the debugger isn't attached).

-Brett

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #63 on: December 31, 2017, 06:05:47 AM »
Have fun. I plan to sleep in New Year's Day.

The version number is probably the best thing to work on. I assume you mean change the year for the modifications done by OPU. I'd be very hesitant to change the original copyright year info.

I may look into the OP2Editor minimap bug on my own.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #64 on: January 02, 2018, 04:49:37 PM »
I just want to change the version number, not the copyright year. Could have been more clear in my post.

-Brett

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #65 on: January 06, 2018, 01:47:55 AM »
I'm available 7 January, Midnight GMT if there is someone else with availability.

As discussed, I'll be looking to figure out how to set the version number on the ABOUT page in the main menu primary.

Secondary if time permits would likely be trying to record a bit of Team Viewer and see how it turns out.

Or, open to other projects if people have other things they would like to work on.

-Brett

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #66 on: January 06, 2018, 08:02:55 AM »
I've marked the time in my calendar.


I've also recently had a couple of pair programming sessions to recreate the main menu of Outpost 2 as an HTML page. Bit of a silly project, but good fun, and a good review of CSS. The project was visual only, no actual links or functionality, though the buttons do highlight like in the game if you click them (using the CSS a:active pseudo-class).

If people have any silly learning projects they want to try, feel free to suggest something. And remember these sessions are open to more than just 2 people at a time.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #67 on: January 06, 2018, 08:04:50 PM »
Sounds good, I'll jump on the quakenet mirc forum room as usual and use Discord as a backup if mirc isn't working.

-Brett

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #68 on: January 07, 2018, 10:37:52 PM »
Hooman,

I missed you during the session today. Fortunately, _A_ new exactly how to change the about screen version number via Resource Hacker, so that was quickly solved. Thanks to _A_ for the help.

_A_ and Daver helped a lot teaching me how to actually use NetFix and NetHelper, which was much appreciated. Thank you both. I'm still going to need to fiddle with it some more. Part of the problem is the lack of a readme for NetFix and the other half is my complete lack of understanding modern networking.

We also had a productive conversation about the difficulties involved in automating terrain transitions for Outpost 2 with a possible solution to ease the difficulty. Thanks to Leeor, _A_, and daver for their insights.

-Brett

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #69 on: January 08, 2018, 01:13:27 AM »
Terribly sorry about missing the pairing. My fault, I marked the wrong time in my calendar, and also didn't ensure adequate rest. I woke up an hour too early, didn't see you around, and having nothing else planned, managed to fall asleep again before the appointed time. I only just now noticed your last message said midnight GMT, as opposed to the 11pm GMT from the earlier message.

I should really pay more attention to sleep before those early morning sessions.


In regards to Resource Hacker, that is a very useful tool for extracting resources such as bitmaps, icons, and dialogs. It also makes for a more user friendly editor than a hex editor. Though be aware that some resource editors will repack the resources when saving, which can move stuff around. Generally this shouldn't matter, but it can result in file size changes, which can have unexpected effects. If you're going to use resource editor, it might be a good idea to check there were no file size changes, or possibly even verify that it only actually changed the part you think it changed.

NetFix could use some usability improvements. For instance, displaying both the internal and external IP. Using only 1 open socket internally, rather than one unbound socket and potentially one bound socket (ForcedPort). The log output could also be improved, which might help diagnose network difficulties. Maybe we could look into some improvements sometime in the future.

The map editor terrain transitions is something I badly want. It's a messy problem, as I'm sure you know. This should probably be a much bigger priority than we've given it.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #70 on: January 09, 2018, 12:23:24 AM »
Hmm, I should have emphasized the time change I made in retrospect.

I'm unavailable this coming Sunday, so perhaps in 2 weeks for my next participation.

Didn't mention in the last post, but it was nice that people seemed generally excited on the forums about continued work on Outpost 2 even though a lot of the work is mundane right now.

-Brett

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #71 on: January 10, 2018, 05:10:28 AM »
I know. It's so much more motivating to work on stuff when you hear more than just crickets.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1015
Re: Remote Pair Programming
« Reply #72 on: September 20, 2018, 08:55:49 PM »
Hooman and I participated in another pair programming session yesterday. It had been quite a while. We used the newer version of Team Viewer (v13), in which we were unable to get the voice dialog to work. Discord filled in nicely for audio while we used Team Viewer for the screen sharing. Also, when we were using the Linux desktop, I was unable to control the mouse/keyboard. When we used the Windows desktop, Hooman was able to remotley control things. Not sure if this was a bug or user error.

We focused on extending our knowledge of the op2_art.prt. op2_art.prt contains all of the imagery metadata for Outpost 2 on how animations work, how frames are overlayed on top of each other, etc. It seems like a pretty sophisticated system for when it was created, although I doubt it would be useful now that more computing power is available.

We had already created new code on OP2Utility that allows for reading and writing op2_art.prt. However, I couldn't get the end of the writing routine to work.

After teasing out a small bug in the ArtWriter class, we were able to produce an exact replica of op2_art.prt. Except our replica stopped early. Turns out op2_art.prt stores more data after the animation structure array. It appears to be at least 2 more structure arrays and some more small fields at the end. We had to stop the session before finding anything in detail out about the new data.

I thought it was productive overall, especially considering that we were making progress with the TeamViewer shortcomings and the unintended background noise issues.

Unfortunately, I didn't get to dive into OllyDbg myself as much as I wanted to, but perhaps another session sometime for that.

The ArtWriter bugfix hasn't been pushed to GitHub yet. I'll plan to clean that up tomorrow. At some point, I'll also post in a different thread updating the new features in OP2Utility.

-Brett

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Remote Pair Programming
« Reply #73 on: September 30, 2018, 04:09:44 PM »
In regards to the extra data at the end of the Prt file, it is not loaded or used by Outpost 2. We have found extra unused data. This is very similar to the tile group data at the end of map files. The game never tries to load it. There is no code to analyse how the data is loaded or used, because it isn't loaded or used.

I spent some time examining the code we were looking at in OllyDbg. Turns out the code at the end of the Prt loading method corresponds to the UnknownContainer struct array, defined in Animations.h in the OP2Utility project. That particular code corresponds to data that is already being loaded, we just don't yet know what to do with that part yet.

The Prt loading method in Outpost 2 closes the stream and ends after loading the animation array. It never touches the data after that.

The only way to determine what that data means is to analyse the data itself.



In terms of the structure of the UnknownContainer, I strongly believe it represents a Rect. More specifically, I believe it represents two Point objects.

The reason for this conclusion is the peculiar and somewhat nonsensical stream error checks around that code, where it reads the first 4 bytes, and then conditionally loads the next 4 bytes, provided there were no errors from reading the first 4 bytes. As the only expected error might be end of file, it could have just tried to read all 8 bytes together.

This error checking behavior is consistent for other parts of the code that process Point structs. In particular, the clipRect struct exhibits the exact same checks. My guess is, the game's code had an inline method which read the x and y fields of the Point object separately, and returned early if there was a failure. It may have even returned an error code to indicate success or failure. Though the problem with returning error codes is they are inconvenient to check, and make a mess of the control flow of the program. As such, they are frequently ignored. That may explain the pattern in the code, which seems to be repeated for each Point or Rect that is loaded.

Possible example code:
Code: [Select]
/* Returns true if read was successful */
bool StreamIO::Read(void* buffer, std::size_t size);

/* Returns true if read was successful */
inline bool StreamIO::Read(Point& p) {
  /* Note: The && is short-circuiting */
  /* It only evaluates the right hand side if the left hand side is true */
  return Read(p.x) && Read(p.y);
}

/* Returns true if read was successful */
/* ... or get lazy, not check things, and return void */
inline void StreamIO::Read(Rect& rect) {
  Read(rect.p1);  /* Ignore return value */
  Read(rect.p2);  /* Ignore return value */
}

The Rect class doesn't seem to use the same short-circuiting behaviour. If one point fails to load, it still tries to load the second point.
« Last Edit: September 30, 2018, 04:12:25 PM by Hooman »