Outpost Universe Forums

Projects & Development => Projects => Topic started by: TechCor on March 02, 2019, 11:29:15 PM

Title: OP2 Scenario Project for C#
Post by: TechCor on March 02, 2019, 11:29:15 PM
Hey all, and thanks Leeor for marking my account as confirmed.

I've been playing Outpost 2 for the last couple months and completed both campaigns and Eden Starship on hard. Still wanting more, I started looking around at the various GitHub projects and how to make new scenarios.

The number one thing I noticed is that making scenarios is incredibly tedious, compounded by the fact that it is done in C++ which I haven't used seriously in 10 years, and had completely forgotten the pains of char* strings, header files and linker errors.

So last week I decided to move it all to C#. Still have to support groups and HFL, but it works. I wanted to make sure the community had access to the source, so you can access the project here:

https://github.com/TechCor8/OP2DotNetMissionSDK (https://github.com/TechCor8/OP2DotNetMissionSDK)

It also reads JSON for the initial setup and creating triggers and disaster zones. The goal being to move scenario development into an external editor.
Title: Re: OP2 Scenario Project for C#
Post by: leeor_net on March 03, 2019, 02:55:22 AM
Glad you were able to sign on!

This is wonderful! I've always loved C++ but recently learned how much easier it can be to develop with C#. I still prefer C++ because I'm a masochist but this could help to make mission coding more accessible. There has allegedly been an effort toward making it accessible via python scripting (glares at Arklon and BlackBox) but that seems to be an echo on the wind.

We've been pushing to migrate everything to GitHub to be as public as possible with it. Would you consider doing the same? Or would you prefer that we integrate this into the OutpostUniverse organization on GitHub?
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 03, 2019, 12:14:12 PM
Wow, what? I'm both surprised and intrigued.

I've manged to create simple DLLs with other compilers before, that the game sees as a level, and can load them, but they've never been able to do anything useful, like create initial units. The name mangling used by the C++ functions that Outpost2.exe exports always got in the way. The data exports (from the DLL) were simple enough to get the game to recognize the DLLs, but the function exports (from Outpost2.exe) limited the DLL from doing anything useful.

Yet you say this works? What about saving and then loading a game? Outpost 2 requires all data to be accessible in a fixed sized buffer, and there's no notification of when it's going to save or load data to/from that buffer, so it needs to be the live data.


I would love to see this on GitHub.
Title: Re: OP2 Scenario Project for C#
Post by: Arklon on March 03, 2019, 12:58:10 PM
There has allegedly been an effort toward making it accessible via python scripting (glares at Arklon and BlackBox) but that seems to be an echo on the wind.
That's still in development, but it's also larger in scope than just for writing missions. As far as missions goes, it's basically done, except for the parts of the public API we never bothered to document like Pinwheel groups, and exposing more non-public API game internals (though quite a bit is already exposed).

I've manged to create simple DLLs with other compilers before, that the game sees as a level, and can load them, but they've never been able to do anything useful, like create initial units. The name mangling used by the C++ functions that Outpost2.exe exports always got in the way. The data exports (from the DLL) were simple enough to get the game to recognize the DLLs, but the function exports (from Outpost2.exe) limited the DLL from doing anything useful.
He worked around this problem by just making extern C wrappers of all the OP2 API functions. I remember someone telling me that C++.NET/CLR is fully ABI compatible with C++ though (they're both still built using the MS toolchain so that wouldn't be a shock), so there's probably a better way to do this. That said, the general idea of using C++.NET as an interop shim between C++ and C# is actually a pretty good practice, and also the only real good use case for C++.NET.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 03, 2019, 08:01:41 PM
Quote
We've been pushing to migrate everything to GitHub to be as public as possible with it. Would you consider doing the same? Or would you prefer that we integrate this into the OutpostUniverse organization on GitHub?

I'll put it on GitHub. I thought about that as I was posting, but was too lazy to do it right then.

Quote
What about saving and then loading a game? Outpost 2 requires all data to be accessible in a fixed sized buffer, and there's no notification of when it's going to save or load data to/from that buffer, so it needs to be the live data.

I haven't tested the saving yet, but the general idea is to reserve a static buffer larger than what is needed for most missions in the unmanaged DLL, and forward the buffer to C# through GetSaveRegions where it can be written with a memory stream. It is a bit inefficient, but so is everything else about this. Hopefully, the data is loaded by the time InitProc is called. If not, I could test in a loop for some init flag that is always saved to the buffer.

-- Thinking about what you said, I may have misunderstood what GetSaveRegions does. Is it just getting a pointer to the buffer at initialization? This whole saving process is really arcane. I would have had Save()/Load() procs... If that's the case, I'll probably have to send the buffer at init and use some kind of indexing wrapper class in C# for accessing and modifying the buffer with live data.

This makes me wonder then, is it enough to create the buffer dynamically at DLL attach based on the size reported by C#? For example, in C# there could be a SaveBuffer class with GetBufferSize which is based on the total size of the variables in the class. Use that to new char[] the buffer and pass in GetSaveRegions.


It really is just a lot of wrapper boilerplate. My eyes started to glaze over while doing triggers.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on March 03, 2019, 09:56:09 PM
Hi TechCor,

Welcome to the forums and glad to here you are enjoying Outpost 2 20 years after its creation.

It sounds like you have a great start on the programming. One note on making scenarios is the DLL has to be loaded into the proper base address. Otherwise, if you save/load a game, the mission objectives will become garbled. If you haven't seen it yet, there are 2 posts in the wiki that go over a lot of the C++ settings details (just glance over the SVN part I guess). https://wiki.outpost2.net/doku.php?id=op2_sdk:projectcreation

Within the SVN repository you can find a decent number of the custom missions that have been added to Outpost 2 over the years for coding examples. Up to you to sift through the good/bad though. :| https://svn.outpostuniverse.org:8443/!/#outpost2/view/head/LevelsAndMods/trunk/Levels

If you want to move past basic level design, you may want HFL and/or IUnit to give some of the more advanced commands. HFL relies more on memory hacking. You can also add a modal dialog to the beginning of your scenario using odasl. Couple of examples in missions I've made and Plymouth Cold War.

There are a lot of undocumented bugs in the mission development SDK. Both internal to Outpost 2 and in Outpost2DLL and HFL. It would be good to get things fixed or at least documented, but it is all volunteer work.

I've been trying to split my time between improving the mission SDK and template on GitHub and trying to help improve NetFix, but am happy to try and answer questions about the SDK.

If you run through all the colony games and fan added colony games, there are some multiplayer coop missions with AI. You can bring up two instances of Outpost 2 on your machine and attempt them solo for a tough challenge. I think Danger Zone is barely doable single player. If you want to play multiplayer, I'm willing to schedule some time out, but mostly play coop missions now. The deathmatches never did much for me (probably because I'm a mediocre player).

Anyways, happy playing and coding.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 03, 2019, 11:47:46 PM
Yeah, I'm liking this idea of using C# for mission development. I guess it makes sense there would be some kind of interop.

I was looking through the code a bit the other night. Looks like the C# classes are proxies for the C++ classes. The C++ classes are in turn proxies for internal Outpost2.exe classes. I'm wondering if we can do something more direct there. Perhaps there's a way that adds less boilerplate, or that might make the Save/Load thing easier. The game really could have made use of Save/Load methods, or at least callback notifications. It would have made the system much more flexible. I'd love to see this up on GitHub so we could play around with this C# idea some more.  :)

And Triggers, oh boy.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 03, 2019, 11:53:05 PM
OK. I read the base address thing 5 or 6 times, and I think I finally get it.

So I assume Outpost 2 is doing something like:

GetSaveRegions to get the buffer address and size.
Saves the struct ADDRESS to the save file and copies the buffer.
At some point OP2 loads the buffer from the save file back to that saved address.

If the DLL or save buffer were dynamic addresses, it would end up copying the data back to the wrong place.

If this is true, the "oversized static save buffer" method with a C# wrapper that indexes properties will be the way to go.

What threw me off, is I would think it would just query GetSaveRegions for the most up-to-date address before loading the data.


--
Playing two copies of OP2 at the same time might be tough. I am more of a "versus AI" type, but especially enjoy the morale aspects of the game. Otherwise, I'd play Starcraft. I never did like competitive multiplayer, but maybe that's because the Starcraft guys were just too good.

Didn't realize there were projects on the SVN repo. I thought everything was on GitHub. Will look into it when I have time.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 03, 2019, 11:58:32 PM
Yeah, I'm liking this idea of using C# for mission development. I guess it makes sense there would be some kind of interop.

I was looking through the code a bit the other night. Looks like the C# classes are proxies for the C++ classes. The C++ classes are in turn proxies for internal Outpost2.exe classes. I'm wondering if we can do something more direct there. Perhaps there's a way that adds less boilerplate, or that might make the Save/Load thing easier. The game really could have made use of Save/Load methods, or at least callback notifications. It would have made the system much more flexible. I'd love to see this up on GitHub so we could play around with this C# idea some more.  :)

And Triggers, oh boy.

I'm not sure if it can be made more direct, but certainly if someone knows a way, let me know, and I'll update it. For now, "working" is better than not.

The trigger problem (where it will call the native DLL) is definitely a tough one. I am really hoping that HasFired() works properly. In which case, I will save all the triggers to a list and poll HasFired() to execute the result.

Also, it is now on GitHub. Check the first post.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 04, 2019, 01:45:22 AM
Ok, I've cloned the GitHub repo. I'll have to take some time to play with it.

I agree that working is better than not-working/nice/"perfect".

The thought I had the other night, is the C# code seemed to be using the C++ classes in much the same way as native C# classes. That's what made me think maybe it could be more direct. I admit my C# knowledge is pretty weak though, so perhaps I'm missing something obvious. I know C# has some differences in terms of memory management, and these sorts of wrappers could be to paper over the memory management differences. Though in the case of Outpost 2's exported classes, they are mostly proxy objects with simple value like semantics, so that might negate some of the memory management differences. You could basically treat those classes like you'd expect of structs. Hard to say at this point, just a sneaking suspicion I had.

Internally, Outpost 2 uses function pointers for triggers. If we use the memory hacking APIs, we could gain direct access to those pointers, and setup triggers without going through the named DLL function export/lookup process. Though I assume C# must have a way of exporting plain functions from DLLs, so maybe that wouldn't be needed.

As for the saved games, Outpost2.exe calls GetSaveRegions just after loading the DLL. It calls it before InitProc when starting a new mission. When loading a mission it calls it before resuming the game cycle. It stores the result, and never asks again. Whenever a player saves the game, Outpost 2 copies the contents of that buffer to the saved game file. There is no notification to the mission DLL, so the mission has no idea when a game has been saved. Similarly, it doesn't know when it's been loaded. It simply loads the DLL, calls GetSaveRegions to find the buffer, and then copies data from the saved game file to that buffer, then resumes the game cycle.

You could potentially write a detect for the load, using an initialized global/static variable outside of the saved region, and modified at the end of InitProc. If it has the original initialized value, rather than the InitProc set value, the game has been loaded since InitProc ran. There would be no way to write a detect for saving the game though. At least nothing short of hooking into the code section of the game. Hence that static buffer must always contain current live data, or you risk it not being stored when a game is saved. Well, minus the small window of time within a game cycle, when the DLL has complete control to run AIProc, or a trigger callback. Once it returns to the game engine though, the game engine can do whatever in terms of saving/loading. You would need to update the static buffer before returning if things were modified outside the save region.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 04, 2019, 05:57:20 AM
Thanks for clearing all that up!

Saving and loading appears to be working.

I went ahead and made it a packed C# class for storing variables + a native memory wrapper. Checks if "isLoaded" in the update loop, and calls Load if it isn't. Then saves at the end. Glad to know the buffer doesn't need to be declared in the native plugin.

I'll see what I can do for triggers next.

---
On the topic of bypassing the native wrapper, besides C++ name mangling, C# does not appear to support __fastcall which is used on some of the functions, most notably TethysGame. Not worried about it right now though.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 10, 2019, 11:56:57 AM
UPDATE:

Triggers have been added. It works a little differently in C# to avoid interop passthrough.

The process is as follows:

Of course, you can also specify all triggers and their IDs through the JSON file. Currently, you can't specify actions in JSON, but you can still reference the ID and perform the actions via C#.

---
The C# DLL can now be referenced under different file names.

The NativePlugin has a #define USE_CUSTOM_DLL in LevelMain.cpp for toggling loading.

USE_CUSTOM_DLL true = [NativePluginName]_DotNet.dll
USE_CUSTOM_DLL false = DotNetMissionSDK.dll (JSON only library).

By default, this is set to false for JSON only missions.

---
UPCOMING:


Unfortunately, groups are kind of a mystery to me. The wiki page does not exist, and there isn't much documentation to be found elsewhere. Might take some experimentation.
Title: Re: OP2 Scenario Project for C#
Post by: Arklon on March 10, 2019, 03:31:21 PM
Okay, I looked at your code again, it actually wasn't using C++.NET like I thought before. I did some more research on the C++.NET interop shim technique, and yeah, it's fully ABI compatible with C++ (you need to do native function calls in unsafe { } blocks) including all calling conventions etc., and it compiles to a DLL that's a .NET assembly that you can drop into C# and it just works, so it would serve both of the roles that the native and .NET layers do currently.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 10, 2019, 07:58:28 PM
Hmm. There were 2 interop layers and 1 C# layer before. I just merged the 2 interop layers (NativeInterop and DotNetInterop) into 1 layer (DotNetInterop). C++ can have both managed and unmanaged code in the same DLL by setting file properties.

I'm thinking of trying this technique on the NativePlugin layer to call C# directly. I didn't try this initially in case Outpost 2 had trouble with it.

If I understand you correctly, you are saying DotNetInterop should wrap the Outpost2 calls with managed classes instead of extern C. DotNetMissionSDK (C#) could then reference DotNetInterop instead of using DllImport.


EDIT:
Tried to get NativePlugin to work with the CLR on just one file. Linker complains of flag incompatibility, so I don't think this is an option. I would have to make project wide flag changes which apparently some of them are required for Outpost 2.
This means there will be a circular reference problem in DotNetInterop if I change the externs to managed classes. Referencing the C# DLL may need to be replaced with reflection to accomplish this.

I'm out of time now. Will look at it later.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 11, 2019, 02:06:42 AM
Ok, this is getting interesting.

Have you updated GitHub? I don't see any recent changes.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 11, 2019, 11:09:33 AM
There were 6 commits yesterday.

You can check activity here:

https://github.com/TechCor8/OP2DotNetMissionSDK/commits/master

UPDATE:

I just switched the dependencies. It actually makes sense now. C++ no longer depends on C#. C# now references the C++ DotNetInterop and implements the C++ managed interface for MissionEntry.

Should be possible to wrap all the Outpost 2 functionality inside of DotNetInterop as managed C++ classes now.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 11, 2019, 01:00:46 PM
Ahh, I see what happened. It seems the public history was rewritten. Perhaps a rebase was run or something. When I updated, it pulled down new commits, but then refused to merge them to my local master branch, since the histories had diverged. Hence when I checked the log of my local master branch, I didn't see the updates.

It's such a tiny little error message, in a paragraph of text, and no highlighting.

Ok, yes, I see you've been busy.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 11, 2019, 02:25:15 PM
Yes, I rewrote the history because I accidentally pushed the first few commits with the wrong author info. Was hoping nobody would notice.
Title: Re: OP2 Scenario Project for C#
Post by: BlackBox on March 11, 2019, 02:54:08 PM
This looks really promising!

Thinking about this a bit further, it might be useful to join forces with the Python SDK that Arklon and I have been working on (this isn't dead, we've been quietly working on it in the background, hoping to have something more or less ready for general consumption soon-ish).

It'd probably be possible to eliminate the need for the unmanaged wrapper/proxy DLL that loads the assembly.
For the Python SDK (which is implemented as a mod DLL loaded through op2ext/the INI file) we already hook/replace quite a bit of UI code related to the list of missions in the various menus (in order to show Python missions in the list). We also hook a whole bunch of code relating to loading the mission, as well as trigger callbacks and loading/saving the game (same reason, since the mission itself is no longer its own DLL).

Since we're already hooking these places it probably wouldn't be a huge amount of work to detect whether a mission is a .NET assembly, and if so, load it via CLR hosting (i.e. call directly into the assembly without needing the C++ proxy DLL). I'd expect the assembly that handles interop between the Outpost2 API and .NET would probably still be needed.

Also, regarding loading and saving from earlier in the thread, the game actually internally handles this by passing a file-like object (StreamIO) to the internal class that represents a mission DLL. This in turn calls the GetSaveRegions exported function and then reads/writes the buffer contents from/to this file.
Since we similarly can't assume a fixed buffer for Python missions (since we don't control where the interpreter places things in memory, etc) our current plan is to expose the low-level Load/Save API to the python code and the mission is responsible for loading/saving whatever it needs to reconstruct its state from a saved game. We can probably simplify this further by having the mission provide an object such as a dict that we pickle/unpickle to/from the file. I would guess something similar would work for C# as well.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 12, 2019, 10:46:21 AM
It would be nice to get rid of the extra dll per mission, but since full compatibility is possible, I will continue to support that.

If the mod calls the DotNetMissionEntry interface, it would be possible to bypass the unmanaged DLL. Would just need to add something for level details. This way the C# DLL can be used with or without the mod.

If the mod wants to support JSON missions without the unmanaged DLL, it would need to check for the JSON files, and call into the special/uncustomized C# DLL's DotNetMissionEntry with the appropriate file path.

If you don't want to use the CLR in the mod, you can just call DotNetInterop's C functions which will do it for you. That DLL will be required regardless, and I don't think the slight overhead is worth worrying about. Also, the DotNetInterop will handle switching to JSON for you. You just have to set the bool in Attach for whether to load a C# mission or JSON.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 18, 2019, 05:02:13 PM
Decided to put some time into things I can see. One of the goals is to be able to rapidly set up generic scenarios (two hours or less time invested).

I've just added Autolayout. AutoLayout lets you place a colony without worrying about tiles, collisions, tubes, etc. It's very simple to use and quite customizable. I was having fun toying with the build order and distances to see what kind of bases it would come up with.

There is still some work to be done on it with regards to tube placement and structures being placed too far away (crow flies vs walking distance). I'll be adding A* pathfinding to the project and using that to solve those issues.

Here is an example of the usage (JSON AutoLayouts section):

Code: [Select]

"PlayerID":0,
"BaseCenterPt": { "X":27, "Y":38 },

"Units":
[
{ "TypeID":"CommandCenter", "MinDistance":1 },
{ "TypeID":"StructureFactory", "MinDistance":0 },
{ "TypeID":"ConVec", "CargoType":"CommandCenter", "MinDistance":1, "SpawnDistance":3 },
{ "TypeID":"ConVec", "CargoType":"None","MinDistance":1, "SpawnDistance":3 },
{ "TypeID":"CommonOreSmelter", "MinDistance":0 },
{ "TypeID":"Agridome", "MinDistance":0 },
{ "TypeID":"Agridome", "MinDistance":1 },
{ "TypeID":"GORF", "MinDistance":0 },
{ "TypeID":"Residence", "MinDistance":1 },
{ "TypeID":"StandardLab", "MinDistance":0 },
{ "TypeID":"Nursery", "MinDistance":0 },
{ "TypeID":"Residence", "MinDistance":1 },
{ "TypeID":"University", "MinDistance":0 },
{ "TypeID":"RobotCommand", "MinDistance":0 },
{ "TypeID":"Residence", "MinDistance":1 },
{ "TypeID":"MedicalCenter", "MinDistance":1 },
{ "TypeID":"DIRT", "MinDistance":0 },
{ "TypeID":"Agridome", "MinDistance":1 },
{ "TypeID":"MedicalCenter", "MinDistance":1 },
{ "TypeID":"RecreationFacility", "MinDistance":0 },
{ "TypeID":"RecreationFacility", "MinDistance":1 },
{ "TypeID":"DIRT", "MinDistance":2 },
{ "TypeID":"VehicleFactory", "MinDistance":1, "CreateWall":true },
{ "TypeID":"Tokamak", "MinDistance":3 },
{ "TypeID":"Tokamak", "MinDistance":2 },
{ "TypeID":"GuardPost", "CargoType":"Microwave","MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"EMP", "MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"Microwave","MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"ESG", "MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"RPG", "MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"EMP", "MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"RPG", "MinDistance":2, "CreateWall":true },
{ "TypeID":"Tokamak", "MinDistance":3 }
]

And the base it generates:

(https://scoriastudios.com/myfiles/op2/Autolayout001.png)

EDIT:

Added some BFS and A* pathing for structures and tubes. Base now stays together properly. Autolayout vehicles spawn last to prevent structures from being affected by their placement. Wall placement should probably be moved to after structure placement.

Image above has been updated to reflect changes.

I think I'll add an option for single or multi tube setups. Image shows multi. Single tube is closest only. Multi is closest + all structures within minDistance.
Title: Re: OP2 Scenario Project for C#
Post by: Crow! on March 19, 2019, 11:51:19 AM
So with this, we could make a map which, at start-up, randomly selects a subset of buildings, and assigns that same set of buildings to all players, without the map needing to explicitly write out how to connect things with different sizes together.

Can you add buildings to the map via some method other than this base creator first, and let autolayout avoid collisions with that building's placement, or does autolayout only know the positions of map obstructions and its own base layout?
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 19, 2019, 02:40:29 PM
It will avoid map obstructions and units it knows about.

The base generator has a constructor for taking existing units. You can also set the "ignoreLayout" flag for a UnitData which uses manual placement when the generator gets to it.

Each call to Generate keeps the list of created units, so the number of bases generated isn't a concern either.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 19, 2019, 02:47:30 PM
Very neat.  :)

Sounds fairly similar to the BaseBuilder section of the OP2Helper project. Define a base as data, and allow people to create duplicate copies of it at various locations. BaseBuilder didn't have any auto layout though. All coordinates were specified relative to some fixed/center point.



Curious, would this project build with the .NET Core API? Seeing as how it's C#, the build system is a little more cross platform than C++, so I thought I'd give it a try on Linux. Unfortunately the .NET Framework API is Windows only. The .NET Core API is cross platform.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 19, 2019, 07:19:33 PM
I just converted DotNetMissionSDK to .Net Standard 2.0. Hopefully that is portable. Please give it a try and report back.

I don't think the rest of the project can be made platform independent, however. Tips are always welcome.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on March 19, 2019, 10:10:25 PM
We have some minor changes brewing in Outpost2DLL, OP2Helper, and HFL. Would be nice if we could get them into OP2MissionSDK since it sounds like TechCor is actively using it.

Nice to see some more C# development going on. Make me feel like a bit of hack with how quickly this is moving compared to the C++ code I am slogging through.

I enjoyed seeing the auto base builder results.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 20, 2019, 02:00:59 AM
Thanks for the update!

Now it runs package restore, though fails at a later step (edited paths):
Code: [Select]
/bin/sh: 2: /tmp/tmpcac72cbbf94540508eaf575e65118431.exec.cmd: copy: not found
/.../OP2DotNetMissionSDK/DotNetMissionSDK/DotNetMissionSDK.csproj(8,5): error MSB3073: The command "copy "/.../OP2DotNetMissionSDK/DotNetMissionSDK/bin/Debug/netstandard2.0/DotNetMissionSDK.dll" "*Undefined*Outpost2\DotNetMissionSDK.dll"" exited with code 127.

Looks like you have a shell command as part of the Post Build step. Shell commands can easily lead to a few incompatibilities. Here, Linux uses "cp" rather than "copy". There also seems to be a strange "*Undefined*" in the path, which looks like use of an environment variable.

I checked DotNetMissionSDK.csproj and saw this:
Code: [Select]
  <Target Name="PostBuild" AfterTargets="PostBuildEvent">
    <Exec Command="copy &quot;$(TargetPath)&quot; &quot;$(SolutionDir)Outpost2\$
(TargetFilename)&quot;" />
  </Target>

It seems $(SolutionDir) was undefined. That makes sense, since I tried to run the build from the "DotNetMissionSDK/" folder to avoid errors with the other projects in the solution file.

Doesn't look like there is much in that project file in terms of build settings, or even references to source files. I'm guessing C# is a little smarter about finding the source files for a project.



The following might help for fixing the Post Build command:
Best way to make a project with post-build script work on MonoDevelop and Visual Studio? (https://stackoverflow.com/questions/13150733/best-way-to-make-a-project-with-post-build-script-work-on-monodevelop-and-visual)
The MSBuild Task answer looked promising.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 20, 2019, 02:48:02 AM
Nice! I didn't know it could be done that way!

I replaced line 8 in DotNetMissionSDK.csproj with:

Code: [Select]
<Copy SourceFiles="$(TargetPath)" DestinationFolder="$(SolutionDir)Outpost2" />
It seems to work. You will need to run from the solution however.

This one with the $(ProjectDir) also works and may be the better choice:

Code: [Select]
<Copy SourceFiles="$(TargetPath)" DestinationFolder="$(ProjectDir)../Outpost2" />
The C++ post builds did not work however. The project won't load when I do that. I'm sure those projects have a lot more problems than a convenience post-build step though.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 20, 2019, 03:53:04 AM
That was quick.


I tried out the first option, and when I built the specific project file, it generated a weird "*Undefined*" directory and copied to that.

I tried out the second option and when I built the specific project file, it worked.


For either option, when I tried to build the whole solution, I see what looks like some partial success, and then a whole bunch of errors from the other projects. The output file was not updated. I don't think the Post Build step ran. Perhaps the Post Build step didn't run due to errors in the other projects. I assume it only runs after a successful compilation.



Edit:
As for C++ projects, there are much deeper problems that prevent them from building on Linux. The whole project file structure ends up getting significantly changed. C++ uses a different compiler on Linux than for Windows, so the references to source files end up in different tags referring to a different compiler. The compiler options are also completely reworked. I've never seen a C++ project file that contained both Windows and Linux settings in the same file. All the examples were either one or the other.

C# seems to be much more standardized in terms of compiler and compiler options across platforms.



Edit 2:
From the solution folder, I tried running:
Code: [Select]
dotnet build DotNetMissionSDK

With the first option, it appears to compile successfully, though the output file is not updated. No strange folders observed.
With the second option, it appears to compile successfully, and the output file is updated as expected.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 21, 2019, 03:22:53 PM
The last thing to do for AutoLayout is get map wrap working, but I'm having a bit of trouble understanding the values I'm getting back for clip.

on6_01.map is a 128x128 map.

When I clip the map, I get an area of (32,0) to 160,127) inclusive as valid tiles.
This comes out to a size of 129x128. That can't be right.

Maybe it would help if I knew why maps can start at arbitrary points.

newworld.map gets an area of (0,0) to (511,255) which comes out to a 512x256 size. Which is fine.

Here's the code I'm using to calculate this:

Code: [Select]
// Calculate map size and determine if map wraps
// Clip to top left
LOCATION mapSize = new LOCATION();
mapSize.ClipToMap();

// Initialize to top left corner
area = new MAP_RECT(mapSize, mapSize);

for (int i=0; i < MaxMapSize+10; ++i)
{
int x = mapSize.x;
int y = mapSize.y;

++mapSize.x;
++mapSize.y;

mapSize.ClipToMap();

// Clamped map will pull mapSize.x back to x.
// The max value is the current and previous value.
if (mapSize.x == x)
{
area.maxX = x;
doesWrap = false;
}

// Unclamped map will wrap mapSize.x back to 0.
// The max value is the previous value.
if (mapSize.x == 0)
{
area.maxX = x;
doesWrap = true;
}

// Y is always clamped.
// When mapSize.y is pulled back to y, and we have our clamped or wrapped values, end search.
if ((mapSize.x == x || doesWrap) && mapSize.y == y)
{
area.maxY = y;
break;
}
}

EDIT:

Since the area on the wrap map is correct. I was able to get base generation working there. I tested generating the base near the edges of the clamped map to see if it also worked, and it does.

I'm going to assume that 129 width is expected unless someone says otherwise.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on March 21, 2019, 10:26:19 PM
Map starting position has two options. Nonwrap maps have offsets of (31,-1). Typically I use the constants X_ and Y_ defined in op2helper.h when making maps.

Check the usage note on cliprect in map.h from the project op2utility for a bit more detail. https://github.com/OutpostUniverse/OP2Utility/blob/master/src/Map/Map.h

Hooman could maybe give you a bit more technical details about it. Also in the sun repo there is a ollydbg section that contains detailed notes about different files, structures, and code related to Outpost 2. Maybe a good reference to help with understanding.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 22, 2019, 12:18:30 AM
That's good to know that those offsets don't change arbitrarily from map to map.

I'm thinking I could clip -1,-1 to determine if the map wraps and hard code the lower bounds. Then use the result for wrap upper bounds, or clip again at 5000,5000 for the upper bounds for clamped maps. (The for loop used above is a bit inefficient. ;) )

The question is why am I getting the values I am from clip, as I'm using that function everywhere to resolve clamp/wrap. It seems I can't trust the values coming from it.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 22, 2019, 06:07:18 AM
For non-around-the-world maps, there is an internal memory offset of one 32-column wide section of tiles for the map data. As the internal map coordinates are 0-based, but the displayed values are 1-based, they also need to be offset by 1. Hence you end up with display coordinates to internal coordinates being offset by (x + 31, y - 1).

I assume the offset is to add padding to the map data, so when units drive off the edge of the map, it doesn't crash the game. I don't think they did proper clipping and bounds checking, so that was probably a hack to prevent crashes. In terms of memory allocation, non-around-the-world maps actually allocate twice as much memory as necessary. That gives them room to have padding on both sides of the map, with possibly a lot of extra padding on the right side.

For around the world maps, the coordinates are always wrapped, so they'll never be out of bounds. Hence they have no need for that internal offset.

You can find some details in:
FileFormats: Read down to post about the Saved Game File Format (https://forum.outpost2.net/index.php/topic,1250.msg27376.html#msg27376). In particular, the ClipRegion struct (or ClipRect in some sources).

The around-the-world aspect is controlled by the map width. From the above, if lgMapWidth >= 9 (or equivalently, if mapWidth >= 512) then it's an around-the-world map.

Additionally, the bottom edge of the map is marked as impassible. I haven't verified if this is all maps, or only around-the-world maps, or just some maps. I assume that's probably also to deal with clipping problems. Hence the ClipRect may exclude the last row of tiles.



Edit: The 129 is probably not correct.
(160, 127) - (32, 0) = (128, 127)
Those values should be the size, which for 0-based indexing would mark a non-inclusive endpoint. Valid values are (0..127, 0..126) inclusive.