Outpost Universe Forums

Projects & Development => Projects => Topic started by: TechCor on March 02, 2019, 11:29:15 PM

Title: OP2 Scenario Project for C#
Post by: TechCor on March 02, 2019, 11:29:15 PM
Hey all, and thanks Leeor for marking my account as confirmed.

I've been playing Outpost 2 for the last couple months and completed both campaigns and Eden Starship on hard. Still wanting more, I started looking around at the various GitHub projects and how to make new scenarios.

The number one thing I noticed is that making scenarios is incredibly tedious, compounded by the fact that it is done in C++ which I haven't used seriously in 10 years, and had completely forgotten the pains of char* strings, header files and linker errors.

So last week I decided to move it all to C#. I wanted to make sure the community had access to the source, so you can access the project here:

https://github.com/TechCor8/OP2DotNetMissionSDK (https://github.com/TechCor8/OP2DotNetMissionSDK)

It also reads JSON for the initial setup and creating triggers and disaster zones. The goal being to move scenario development into an external editor.


UPDATE [2019-3-30]:

Groups and HFL added.

Features exclusive to the C# SDK:

Things that are still missing in C# that are in the C++ SDK:
Miscellaneous functions that are missing:
Title: Re: OP2 Scenario Project for C#
Post by: leeor_net on March 03, 2019, 02:55:22 AM
Glad you were able to sign on!

This is wonderful! I've always loved C++ but recently learned how much easier it can be to develop with C#. I still prefer C++ because I'm a masochist but this could help to make mission coding more accessible. There has allegedly been an effort toward making it accessible via python scripting (glares at Arklon and BlackBox) but that seems to be an echo on the wind.

We've been pushing to migrate everything to GitHub to be as public as possible with it. Would you consider doing the same? Or would you prefer that we integrate this into the OutpostUniverse organization on GitHub?
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 03, 2019, 12:14:12 PM
Wow, what? I'm both surprised and intrigued.

I've manged to create simple DLLs with other compilers before, that the game sees as a level, and can load them, but they've never been able to do anything useful, like create initial units. The name mangling used by the C++ functions that Outpost2.exe exports always got in the way. The data exports (from the DLL) were simple enough to get the game to recognize the DLLs, but the function exports (from Outpost2.exe) limited the DLL from doing anything useful.

Yet you say this works? What about saving and then loading a game? Outpost 2 requires all data to be accessible in a fixed sized buffer, and there's no notification of when it's going to save or load data to/from that buffer, so it needs to be the live data.


I would love to see this on GitHub.
Title: Re: OP2 Scenario Project for C#
Post by: Arklon on March 03, 2019, 12:58:10 PM
There has allegedly been an effort toward making it accessible via python scripting (glares at Arklon and BlackBox) but that seems to be an echo on the wind.
That's still in development, but it's also larger in scope than just for writing missions. As far as missions goes, it's basically done, except for the parts of the public API we never bothered to document like Pinwheel groups, and exposing more non-public API game internals (though quite a bit is already exposed).

I've manged to create simple DLLs with other compilers before, that the game sees as a level, and can load them, but they've never been able to do anything useful, like create initial units. The name mangling used by the C++ functions that Outpost2.exe exports always got in the way. The data exports (from the DLL) were simple enough to get the game to recognize the DLLs, but the function exports (from Outpost2.exe) limited the DLL from doing anything useful.
He worked around this problem by just making extern C wrappers of all the OP2 API functions. I remember someone telling me that C++.NET/CLR is fully ABI compatible with C++ though (they're both still built using the MS toolchain so that wouldn't be a shock), so there's probably a better way to do this. That said, the general idea of using C++.NET as an interop shim between C++ and C# is actually a pretty good practice, and also the only real good use case for C++.NET.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 03, 2019, 08:01:41 PM
Quote
We've been pushing to migrate everything to GitHub to be as public as possible with it. Would you consider doing the same? Or would you prefer that we integrate this into the OutpostUniverse organization on GitHub?

I'll put it on GitHub. I thought about that as I was posting, but was too lazy to do it right then.

Quote
What about saving and then loading a game? Outpost 2 requires all data to be accessible in a fixed sized buffer, and there's no notification of when it's going to save or load data to/from that buffer, so it needs to be the live data.

I haven't tested the saving yet, but the general idea is to reserve a static buffer larger than what is needed for most missions in the unmanaged DLL, and forward the buffer to C# through GetSaveRegions where it can be written with a memory stream. It is a bit inefficient, but so is everything else about this. Hopefully, the data is loaded by the time InitProc is called. If not, I could test in a loop for some init flag that is always saved to the buffer.

-- Thinking about what you said, I may have misunderstood what GetSaveRegions does. Is it just getting a pointer to the buffer at initialization? This whole saving process is really arcane. I would have had Save()/Load() procs... If that's the case, I'll probably have to send the buffer at init and use some kind of indexing wrapper class in C# for accessing and modifying the buffer with live data.

This makes me wonder then, is it enough to create the buffer dynamically at DLL attach based on the size reported by C#? For example, in C# there could be a SaveBuffer class with GetBufferSize which is based on the total size of the variables in the class. Use that to new char[] the buffer and pass in GetSaveRegions.


It really is just a lot of wrapper boilerplate. My eyes started to glaze over while doing triggers.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on March 03, 2019, 09:56:09 PM
Hi TechCor,

Welcome to the forums and glad to here you are enjoying Outpost 2 20 years after its creation.

It sounds like you have a great start on the programming. One note on making scenarios is the DLL has to be loaded into the proper base address. Otherwise, if you save/load a game, the mission objectives will become garbled. If you haven't seen it yet, there are 2 posts in the wiki that go over a lot of the C++ settings details (just glance over the SVN part I guess). https://wiki.outpost2.net/doku.php?id=op2_sdk:projectcreation

Within the SVN repository you can find a decent number of the custom missions that have been added to Outpost 2 over the years for coding examples. Up to you to sift through the good/bad though. :| https://svn.outpostuniverse.org:8443/!/#outpost2/view/head/LevelsAndMods/trunk/Levels

If you want to move past basic level design, you may want HFL and/or IUnit to give some of the more advanced commands. HFL relies more on memory hacking. You can also add a modal dialog to the beginning of your scenario using odasl. Couple of examples in missions I've made and Plymouth Cold War.

There are a lot of undocumented bugs in the mission development SDK. Both internal to Outpost 2 and in Outpost2DLL and HFL. It would be good to get things fixed or at least documented, but it is all volunteer work.

I've been trying to split my time between improving the mission SDK and template on GitHub and trying to help improve NetFix, but am happy to try and answer questions about the SDK.

If you run through all the colony games and fan added colony games, there are some multiplayer coop missions with AI. You can bring up two instances of Outpost 2 on your machine and attempt them solo for a tough challenge. I think Danger Zone is barely doable single player. If you want to play multiplayer, I'm willing to schedule some time out, but mostly play coop missions now. The deathmatches never did much for me (probably because I'm a mediocre player).

Anyways, happy playing and coding.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 03, 2019, 11:47:46 PM
Yeah, I'm liking this idea of using C# for mission development. I guess it makes sense there would be some kind of interop.

I was looking through the code a bit the other night. Looks like the C# classes are proxies for the C++ classes. The C++ classes are in turn proxies for internal Outpost2.exe classes. I'm wondering if we can do something more direct there. Perhaps there's a way that adds less boilerplate, or that might make the Save/Load thing easier. The game really could have made use of Save/Load methods, or at least callback notifications. It would have made the system much more flexible. I'd love to see this up on GitHub so we could play around with this C# idea some more.  :)

And Triggers, oh boy.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 03, 2019, 11:53:05 PM
OK. I read the base address thing 5 or 6 times, and I think I finally get it.

So I assume Outpost 2 is doing something like:

GetSaveRegions to get the buffer address and size.
Saves the struct ADDRESS to the save file and copies the buffer.
At some point OP2 loads the buffer from the save file back to that saved address.

If the DLL or save buffer were dynamic addresses, it would end up copying the data back to the wrong place.

If this is true, the "oversized static save buffer" method with a C# wrapper that indexes properties will be the way to go.

What threw me off, is I would think it would just query GetSaveRegions for the most up-to-date address before loading the data.


--
Playing two copies of OP2 at the same time might be tough. I am more of a "versus AI" type, but especially enjoy the morale aspects of the game. Otherwise, I'd play Starcraft. I never did like competitive multiplayer, but maybe that's because the Starcraft guys were just too good.

Didn't realize there were projects on the SVN repo. I thought everything was on GitHub. Will look into it when I have time.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 03, 2019, 11:58:32 PM
Yeah, I'm liking this idea of using C# for mission development. I guess it makes sense there would be some kind of interop.

I was looking through the code a bit the other night. Looks like the C# classes are proxies for the C++ classes. The C++ classes are in turn proxies for internal Outpost2.exe classes. I'm wondering if we can do something more direct there. Perhaps there's a way that adds less boilerplate, or that might make the Save/Load thing easier. The game really could have made use of Save/Load methods, or at least callback notifications. It would have made the system much more flexible. I'd love to see this up on GitHub so we could play around with this C# idea some more.  :)

And Triggers, oh boy.

I'm not sure if it can be made more direct, but certainly if someone knows a way, let me know, and I'll update it. For now, "working" is better than not.

The trigger problem (where it will call the native DLL) is definitely a tough one. I am really hoping that HasFired() works properly. In which case, I will save all the triggers to a list and poll HasFired() to execute the result.

Also, it is now on GitHub. Check the first post.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 04, 2019, 01:45:22 AM
Ok, I've cloned the GitHub repo. I'll have to take some time to play with it.

I agree that working is better than not-working/nice/"perfect".

The thought I had the other night, is the C# code seemed to be using the C++ classes in much the same way as native C# classes. That's what made me think maybe it could be more direct. I admit my C# knowledge is pretty weak though, so perhaps I'm missing something obvious. I know C# has some differences in terms of memory management, and these sorts of wrappers could be to paper over the memory management differences. Though in the case of Outpost 2's exported classes, they are mostly proxy objects with simple value like semantics, so that might negate some of the memory management differences. You could basically treat those classes like you'd expect of structs. Hard to say at this point, just a sneaking suspicion I had.

Internally, Outpost 2 uses function pointers for triggers. If we use the memory hacking APIs, we could gain direct access to those pointers, and setup triggers without going through the named DLL function export/lookup process. Though I assume C# must have a way of exporting plain functions from DLLs, so maybe that wouldn't be needed.

As for the saved games, Outpost2.exe calls GetSaveRegions just after loading the DLL. It calls it before InitProc when starting a new mission. When loading a mission it calls it before resuming the game cycle. It stores the result, and never asks again. Whenever a player saves the game, Outpost 2 copies the contents of that buffer to the saved game file. There is no notification to the mission DLL, so the mission has no idea when a game has been saved. Similarly, it doesn't know when it's been loaded. It simply loads the DLL, calls GetSaveRegions to find the buffer, and then copies data from the saved game file to that buffer, then resumes the game cycle.

You could potentially write a detect for the load, using an initialized global/static variable outside of the saved region, and modified at the end of InitProc. If it has the original initialized value, rather than the InitProc set value, the game has been loaded since InitProc ran. There would be no way to write a detect for saving the game though. At least nothing short of hooking into the code section of the game. Hence that static buffer must always contain current live data, or you risk it not being stored when a game is saved. Well, minus the small window of time within a game cycle, when the DLL has complete control to run AIProc, or a trigger callback. Once it returns to the game engine though, the game engine can do whatever in terms of saving/loading. You would need to update the static buffer before returning if things were modified outside the save region.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 04, 2019, 05:57:20 AM
Thanks for clearing all that up!

Saving and loading appears to be working.

I went ahead and made it a packed C# class for storing variables + a native memory wrapper. Checks if "isLoaded" in the update loop, and calls Load if it isn't. Then saves at the end. Glad to know the buffer doesn't need to be declared in the native plugin.

I'll see what I can do for triggers next.

---
On the topic of bypassing the native wrapper, besides C++ name mangling, C# does not appear to support __fastcall which is used on some of the functions, most notably TethysGame. Not worried about it right now though.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 10, 2019, 11:56:57 AM
UPDATE:

Triggers have been added. It works a little differently in C# to avoid interop passthrough.

The process is as follows:

Of course, you can also specify all triggers and their IDs through the JSON file. Currently, you can't specify actions in JSON, but you can still reference the ID and perform the actions via C#.

---
The C# DLL can now be referenced under different file names.

The NativePlugin has a #define USE_CUSTOM_DLL in LevelMain.cpp for toggling loading.

USE_CUSTOM_DLL true = [NativePluginName]_DotNet.dll
USE_CUSTOM_DLL false = DotNetMissionSDK.dll (JSON only library).

By default, this is set to false for JSON only missions.

---
UPCOMING:


Unfortunately, groups are kind of a mystery to me. The wiki page does not exist, and there isn't much documentation to be found elsewhere. Might take some experimentation.
Title: Re: OP2 Scenario Project for C#
Post by: Arklon on March 10, 2019, 03:31:21 PM
Okay, I looked at your code again, it actually wasn't using C++.NET like I thought before. I did some more research on the C++.NET interop shim technique, and yeah, it's fully ABI compatible with C++ (you need to do native function calls in unsafe { } blocks) including all calling conventions etc., and it compiles to a DLL that's a .NET assembly that you can drop into C# and it just works, so it would serve both of the roles that the native and .NET layers do currently.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 10, 2019, 07:58:28 PM
Hmm. There were 2 interop layers and 1 C# layer before. I just merged the 2 interop layers (NativeInterop and DotNetInterop) into 1 layer (DotNetInterop). C++ can have both managed and unmanaged code in the same DLL by setting file properties.

I'm thinking of trying this technique on the NativePlugin layer to call C# directly. I didn't try this initially in case Outpost 2 had trouble with it.

If I understand you correctly, you are saying DotNetInterop should wrap the Outpost2 calls with managed classes instead of extern C. DotNetMissionSDK (C#) could then reference DotNetInterop instead of using DllImport.


EDIT:
Tried to get NativePlugin to work with the CLR on just one file. Linker complains of flag incompatibility, so I don't think this is an option. I would have to make project wide flag changes which apparently some of them are required for Outpost 2.
This means there will be a circular reference problem in DotNetInterop if I change the externs to managed classes. Referencing the C# DLL may need to be replaced with reflection to accomplish this.

I'm out of time now. Will look at it later.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 11, 2019, 02:06:42 AM
Ok, this is getting interesting.

Have you updated GitHub? I don't see any recent changes.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 11, 2019, 11:09:33 AM
There were 6 commits yesterday.

You can check activity here:

https://github.com/TechCor8/OP2DotNetMissionSDK/commits/master

UPDATE:

I just switched the dependencies. It actually makes sense now. C++ no longer depends on C#. C# now references the C++ DotNetInterop and implements the C++ managed interface for MissionEntry.

Should be possible to wrap all the Outpost 2 functionality inside of DotNetInterop as managed C++ classes now.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 11, 2019, 01:00:46 PM
Ahh, I see what happened. It seems the public history was rewritten. Perhaps a rebase was run or something. When I updated, it pulled down new commits, but then refused to merge them to my local master branch, since the histories had diverged. Hence when I checked the log of my local master branch, I didn't see the updates.

It's such a tiny little error message, in a paragraph of text, and no highlighting.

Ok, yes, I see you've been busy.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 11, 2019, 02:25:15 PM
Yes, I rewrote the history because I accidentally pushed the first few commits with the wrong author info. Was hoping nobody would notice.
Title: Re: OP2 Scenario Project for C#
Post by: BlackBox on March 11, 2019, 02:54:08 PM
This looks really promising!

Thinking about this a bit further, it might be useful to join forces with the Python SDK that Arklon and I have been working on (this isn't dead, we've been quietly working on it in the background, hoping to have something more or less ready for general consumption soon-ish).

It'd probably be possible to eliminate the need for the unmanaged wrapper/proxy DLL that loads the assembly.
For the Python SDK (which is implemented as a mod DLL loaded through op2ext/the INI file) we already hook/replace quite a bit of UI code related to the list of missions in the various menus (in order to show Python missions in the list). We also hook a whole bunch of code relating to loading the mission, as well as trigger callbacks and loading/saving the game (same reason, since the mission itself is no longer its own DLL).

Since we're already hooking these places it probably wouldn't be a huge amount of work to detect whether a mission is a .NET assembly, and if so, load it via CLR hosting (i.e. call directly into the assembly without needing the C++ proxy DLL). I'd expect the assembly that handles interop between the Outpost2 API and .NET would probably still be needed.

Also, regarding loading and saving from earlier in the thread, the game actually internally handles this by passing a file-like object (StreamIO) to the internal class that represents a mission DLL. This in turn calls the GetSaveRegions exported function and then reads/writes the buffer contents from/to this file.
Since we similarly can't assume a fixed buffer for Python missions (since we don't control where the interpreter places things in memory, etc) our current plan is to expose the low-level Load/Save API to the python code and the mission is responsible for loading/saving whatever it needs to reconstruct its state from a saved game. We can probably simplify this further by having the mission provide an object such as a dict that we pickle/unpickle to/from the file. I would guess something similar would work for C# as well.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 12, 2019, 10:46:21 AM
It would be nice to get rid of the extra dll per mission, but since full compatibility is possible, I will continue to support that.

If the mod calls the DotNetMissionEntry interface, it would be possible to bypass the unmanaged DLL. Would just need to add something for level details. This way the C# DLL can be used with or without the mod.

If the mod wants to support JSON missions without the unmanaged DLL, it would need to check for the JSON files, and call into the special/uncustomized C# DLL's DotNetMissionEntry with the appropriate file path.

If you don't want to use the CLR in the mod, you can just call DotNetInterop's C functions which will do it for you. That DLL will be required regardless, and I don't think the slight overhead is worth worrying about. Also, the DotNetInterop will handle switching to JSON for you. You just have to set the bool in Attach for whether to load a C# mission or JSON.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 18, 2019, 05:02:13 PM
Decided to put some time into things I can see. One of the goals is to be able to rapidly set up generic scenarios (two hours or less time invested).

I've just added Autolayout. AutoLayout lets you place a colony without worrying about tiles, collisions, tubes, etc. It's very simple to use and quite customizable. I was having fun toying with the build order and distances to see what kind of bases it would come up with.

There is still some work to be done on it with regards to tube placement and structures being placed too far away (crow flies vs walking distance). I'll be adding A* pathfinding to the project and using that to solve those issues.

Here is an example of the usage (JSON AutoLayouts section):

Code: [Select]

"PlayerID":0,
"BaseCenterPt": { "X":27, "Y":38 },

"Units":
[
{ "TypeID":"CommandCenter", "MinDistance":1 },
{ "TypeID":"StructureFactory", "MinDistance":0 },
{ "TypeID":"ConVec", "CargoType":"CommandCenter", "MinDistance":1, "SpawnDistance":3 },
{ "TypeID":"ConVec", "CargoType":"None","MinDistance":1, "SpawnDistance":3 },
{ "TypeID":"CommonOreSmelter", "MinDistance":0 },
{ "TypeID":"Agridome", "MinDistance":0 },
{ "TypeID":"Agridome", "MinDistance":1 },
{ "TypeID":"GORF", "MinDistance":0 },
{ "TypeID":"Residence", "MinDistance":1 },
{ "TypeID":"StandardLab", "MinDistance":0 },
{ "TypeID":"Nursery", "MinDistance":0 },
{ "TypeID":"Residence", "MinDistance":1 },
{ "TypeID":"University", "MinDistance":0 },
{ "TypeID":"RobotCommand", "MinDistance":0 },
{ "TypeID":"Residence", "MinDistance":1 },
{ "TypeID":"MedicalCenter", "MinDistance":1 },
{ "TypeID":"DIRT", "MinDistance":0 },
{ "TypeID":"Agridome", "MinDistance":1 },
{ "TypeID":"MedicalCenter", "MinDistance":1 },
{ "TypeID":"RecreationFacility", "MinDistance":0 },
{ "TypeID":"RecreationFacility", "MinDistance":1 },
{ "TypeID":"DIRT", "MinDistance":2 },
{ "TypeID":"VehicleFactory", "MinDistance":1, "CreateWall":true },
{ "TypeID":"Tokamak", "MinDistance":3 },
{ "TypeID":"Tokamak", "MinDistance":2 },
{ "TypeID":"GuardPost", "CargoType":"Microwave","MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"EMP", "MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"Microwave","MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"ESG", "MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"RPG", "MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"EMP", "MinDistance":2, "CreateWall":true },
{ "TypeID":"GuardPost", "CargoType":"RPG", "MinDistance":2, "CreateWall":true },
{ "TypeID":"Tokamak", "MinDistance":3 }
]

And the base it generates:

(https://scoriastudios.com/myfiles/op2/Autolayout001.png)

EDIT:

Added some BFS and A* pathing for structures and tubes. Base now stays together properly. Autolayout vehicles spawn last to prevent structures from being affected by their placement. Wall placement should probably be moved to after structure placement.

Image above has been updated to reflect changes.

I think I'll add an option for single or multi tube setups. Image shows multi. Single tube is closest only. Multi is closest + all structures within minDistance.
Title: Re: OP2 Scenario Project for C#
Post by: Crow! on March 19, 2019, 11:51:19 AM
So with this, we could make a map which, at start-up, randomly selects a subset of buildings, and assigns that same set of buildings to all players, without the map needing to explicitly write out how to connect things with different sizes together.

Can you add buildings to the map via some method other than this base creator first, and let autolayout avoid collisions with that building's placement, or does autolayout only know the positions of map obstructions and its own base layout?
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 19, 2019, 02:40:29 PM
It will avoid map obstructions and units it knows about.

The base generator has a constructor for taking existing units. You can also set the "ignoreLayout" flag for a UnitData which uses manual placement when the generator gets to it.

Each call to Generate keeps the list of created units, so the number of bases generated isn't a concern either.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 19, 2019, 02:47:30 PM
Very neat.  :)

Sounds fairly similar to the BaseBuilder section of the OP2Helper project. Define a base as data, and allow people to create duplicate copies of it at various locations. BaseBuilder didn't have any auto layout though. All coordinates were specified relative to some fixed/center point.



Curious, would this project build with the .NET Core API? Seeing as how it's C#, the build system is a little more cross platform than C++, so I thought I'd give it a try on Linux. Unfortunately the .NET Framework API is Windows only. The .NET Core API is cross platform.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 19, 2019, 07:19:33 PM
I just converted DotNetMissionSDK to .Net Standard 2.0. Hopefully that is portable. Please give it a try and report back.

I don't think the rest of the project can be made platform independent, however. Tips are always welcome.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on March 19, 2019, 10:10:25 PM
We have some minor changes brewing in Outpost2DLL, OP2Helper, and HFL. Would be nice if we could get them into OP2MissionSDK since it sounds like TechCor is actively using it.

Nice to see some more C# development going on. Make me feel like a bit of hack with how quickly this is moving compared to the C++ code I am slogging through.

I enjoyed seeing the auto base builder results.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 20, 2019, 02:00:59 AM
Thanks for the update!

Now it runs package restore, though fails at a later step (edited paths):
Code: [Select]
/bin/sh: 2: /tmp/tmpcac72cbbf94540508eaf575e65118431.exec.cmd: copy: not found
/.../OP2DotNetMissionSDK/DotNetMissionSDK/DotNetMissionSDK.csproj(8,5): error MSB3073: The command "copy "/.../OP2DotNetMissionSDK/DotNetMissionSDK/bin/Debug/netstandard2.0/DotNetMissionSDK.dll" "*Undefined*Outpost2\DotNetMissionSDK.dll"" exited with code 127.

Looks like you have a shell command as part of the Post Build step. Shell commands can easily lead to a few incompatibilities. Here, Linux uses "cp" rather than "copy". There also seems to be a strange "*Undefined*" in the path, which looks like use of an environment variable.

I checked DotNetMissionSDK.csproj and saw this:
Code: [Select]
  <Target Name="PostBuild" AfterTargets="PostBuildEvent">
    <Exec Command="copy &quot;$(TargetPath)&quot; &quot;$(SolutionDir)Outpost2\$
(TargetFilename)&quot;" />
  </Target>

It seems $(SolutionDir) was undefined. That makes sense, since I tried to run the build from the "DotNetMissionSDK/" folder to avoid errors with the other projects in the solution file.

Doesn't look like there is much in that project file in terms of build settings, or even references to source files. I'm guessing C# is a little smarter about finding the source files for a project.



The following might help for fixing the Post Build command:
Best way to make a project with post-build script work on MonoDevelop and Visual Studio? (https://stackoverflow.com/questions/13150733/best-way-to-make-a-project-with-post-build-script-work-on-monodevelop-and-visual)
The MSBuild Task answer looked promising.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 20, 2019, 02:48:02 AM
Nice! I didn't know it could be done that way!

I replaced line 8 in DotNetMissionSDK.csproj with:

Code: [Select]
<Copy SourceFiles="$(TargetPath)" DestinationFolder="$(SolutionDir)Outpost2" />
It seems to work. You will need to run from the solution however.

This one with the $(ProjectDir) also works and may be the better choice:

Code: [Select]
<Copy SourceFiles="$(TargetPath)" DestinationFolder="$(ProjectDir)../Outpost2" />
The C++ post builds did not work however. The project won't load when I do that. I'm sure those projects have a lot more problems than a convenience post-build step though.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 20, 2019, 03:53:04 AM
That was quick.


I tried out the first option, and when I built the specific project file, it generated a weird "*Undefined*" directory and copied to that.

I tried out the second option and when I built the specific project file, it worked.


For either option, when I tried to build the whole solution, I see what looks like some partial success, and then a whole bunch of errors from the other projects. The output file was not updated. I don't think the Post Build step ran. Perhaps the Post Build step didn't run due to errors in the other projects. I assume it only runs after a successful compilation.



Edit:
As for C++ projects, there are much deeper problems that prevent them from building on Linux. The whole project file structure ends up getting significantly changed. C++ uses a different compiler on Linux than for Windows, so the references to source files end up in different tags referring to a different compiler. The compiler options are also completely reworked. I've never seen a C++ project file that contained both Windows and Linux settings in the same file. All the examples were either one or the other.

C# seems to be much more standardized in terms of compiler and compiler options across platforms.



Edit 2:
From the solution folder, I tried running:
Code: [Select]
dotnet build DotNetMissionSDK

With the first option, it appears to compile successfully, though the output file is not updated. No strange folders observed.
With the second option, it appears to compile successfully, and the output file is updated as expected.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 21, 2019, 03:22:53 PM
The last thing to do for AutoLayout is get map wrap working, but I'm having a bit of trouble understanding the values I'm getting back for clip.

on6_01.map is a 128x128 map.

When I clip the map, I get an area of (32,0) to 160,127) inclusive as valid tiles.
This comes out to a size of 129x128. That can't be right.

Maybe it would help if I knew why maps can start at arbitrary points.

newworld.map gets an area of (0,0) to (511,255) which comes out to a 512x256 size. Which is fine.

Here's the code I'm using to calculate this:

Code: [Select]
// Calculate map size and determine if map wraps
// Clip to top left
LOCATION mapSize = new LOCATION();
mapSize.ClipToMap();

// Initialize to top left corner
area = new MAP_RECT(mapSize, mapSize);

for (int i=0; i < MaxMapSize+10; ++i)
{
int x = mapSize.x;
int y = mapSize.y;

++mapSize.x;
++mapSize.y;

mapSize.ClipToMap();

// Clamped map will pull mapSize.x back to x.
// The max value is the current and previous value.
if (mapSize.x == x)
{
area.maxX = x;
doesWrap = false;
}

// Unclamped map will wrap mapSize.x back to 0.
// The max value is the previous value.
if (mapSize.x == 0)
{
area.maxX = x;
doesWrap = true;
}

// Y is always clamped.
// When mapSize.y is pulled back to y, and we have our clamped or wrapped values, end search.
if ((mapSize.x == x || doesWrap) && mapSize.y == y)
{
area.maxY = y;
break;
}
}

EDIT:

Since the area on the wrap map is correct. I was able to get base generation working there. I tested generating the base near the edges of the clamped map to see if it also worked, and it does.

I'm going to assume that 129 width is expected unless someone says otherwise.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on March 21, 2019, 10:26:19 PM
Map starting position has two options. Nonwrap maps have offsets of (31,-1). Typically I use the constants X_ and Y_ defined in op2helper.h when making maps.

Check the usage note on cliprect in map.h from the project op2utility for a bit more detail. https://github.com/OutpostUniverse/OP2Utility/blob/master/src/Map/Map.h

Hooman could maybe give you a bit more technical details about it. Also in the sun repo there is a ollydbg section that contains detailed notes about different files, structures, and code related to Outpost 2. Maybe a good reference to help with understanding.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 22, 2019, 12:18:30 AM
That's good to know that those offsets don't change arbitrarily from map to map.

I'm thinking I could clip -1,-1 to determine if the map wraps and hard code the lower bounds. Then use the result for wrap upper bounds, or clip again at 5000,5000 for the upper bounds for clamped maps. (The for loop used above is a bit inefficient. ;) )

The question is why am I getting the values I am from clip, as I'm using that function everywhere to resolve clamp/wrap. It seems I can't trust the values coming from it.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 22, 2019, 06:07:18 AM
For non-around-the-world maps, there is an internal memory offset of one 32-column wide section of tiles for the map data. As the internal map coordinates are 0-based, but the displayed values are 1-based, they also need to be offset by 1. Hence you end up with display coordinates to internal coordinates being offset by (x + 31, y - 1).

I assume the offset is to add padding to the map data, so when units drive off the edge of the map, it doesn't crash the game. I don't think they did proper clipping and bounds checking, so that was probably a hack to prevent crashes. In terms of memory allocation, non-around-the-world maps actually allocate twice as much memory as necessary. That gives them room to have padding on both sides of the map, with possibly a lot of extra padding on the right side.

For around the world maps, the coordinates are always wrapped, so they'll never be out of bounds. Hence they have no need for that internal offset.

You can find some details in:
FileFormats: Read down to post about the Saved Game File Format (https://forum.outpost2.net/index.php/topic,1250.msg27376.html#msg27376). In particular, the ClipRegion struct (or ClipRect in some sources).

The around-the-world aspect is controlled by the map width. From the above, if lgMapWidth >= 9 (or equivalently, if mapWidth >= 512) then it's an around-the-world map.

Additionally, the bottom edge of the map is marked as impassible. I haven't verified if this is all maps, or only around-the-world maps, or just some maps. I assume that's probably also to deal with clipping problems. Hence the ClipRect may exclude the last row of tiles.



Edit: The 129 is probably not correct.
(160, 127) - (32, 0) = (128, 127)
Those values should be the size, which for 0-based indexing would mark a non-inclusive endpoint. Valid values are (0..127, 0..126) inclusive.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 22, 2019, 11:06:45 PM
You said "around the world" so many times I have ATC in my head.

So Clip() doesn't clamp to the playable area, but includes the max exclusive value for the x-axis unless the map wraps. I guess that's a bug? I changed everything to use the correct inclusive maximum. C# now uses its own clip instead of Outpost2DLL's.

When I saw the 32 value initially, I thought it was some cool map cropping feature that some games use to only show a portion of the map. Often completing objectives would "unlock" more playable area. What a let down.


I'm done with AutoLayout for now. The only thing that bothers me is the tube aesthetic isn't perfect, but it would be quite complex to cover all the cases to make it look nice. Might come back to it later.

Thanks for the help with tile offsets, guys. Here is your reward:  8)
(http://scoriastudios.com/myfiles/op2/AutoLayout002.jpg)
(AutoLayout at map edge)

Going to start making general-purpose AI.

I'm a little concerned about the limited API functionality.

It doesn't appear possible to get a structure's demand through Outpost2DLL or HFL, so I may just wing it with like pop / 25, but there could be other problems that can't be resolved. Time to see how well this goes.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 24, 2019, 04:25:24 AM
Lol @ATC  :P



Normally, when doing bounds checking, I generally expect some kind of Point + Size to determine a Rect, much like how a for loop will loop over arrays:
Code: [Select]
for (int i = 0; i < container.size(); ++i)

It's pretty easy and standard to write 0-based array code that is start point inclusive, and end point exclusive.

When looking at the Rect struct in Outpost 2, I assumed it would be the same. After reading your comment though, I decided to go check. In particular, there is the int Check(LOCATION& ptToCheck); method for hit testing if a point is in a Rect. It turns out to be a little more complicated and bizarre than I thought. There's a bit of code to determine if the Rect wraps around the edge of an around-the-world (ATC) map, but other than that complication it's basically checking: (y1 <= y < y2) and (x1 <= x <= x2). Note the end point conditions. It's exclusive for y, but inclusive for x. I'm thinking that's got to be some kind of bug.



I suppose you could still try modifying the clip rect, and try to unlock an area of the map. If you clip a section of the map, it's likely to still be visible, but the mouse cursor will likely change when you hover over an impassible area of the map. Maybe with a bit of day/night hacking, you could also make such a section of the map permanent night time until it is unlocked. Might not be quite what you were looking for, but perhaps close enough.



As for building requirements, I don't believe we currently have a way to get it, but I could probably find a way to get that info for you. I know where it is in memory, there's just no API written (yet) to access it. Same for the day/night position.



Oh and really cool image btw. Looks like it handles corners and restrictive complicated terrain quite well.  :)
Title: Re: OP2 Scenario Project for C#
Post by: Crow! on March 25, 2019, 11:26:04 AM
The biggest thing I'm thinking about for how I would actually use this function is that, in most scenarios, there will be one or two structures you want to demand are put in a particular place.  Smelters come to mind here, as well as enemy structures that are mission objectives in some way.  If you use the manual structure placement you mentioned before, will it know to build a tube from the final autolayout-ed base to wherever the manually placed structure is, or does it simply know not to cause unit overlaps with it?
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 26, 2019, 09:14:18 AM
The biggest thing I'm thinking about for how I would actually use this function is that, in most scenarios, there will be one or two structures you want to demand are put in a particular place.  Smelters come to mind here, as well as enemy structures that are mission objectives in some way.  If you use the manual structure placement you mentioned before, will it know to build a tube from the final autolayout-ed base to wherever the manually placed structure is, or does it simply know not to cause unit overlaps with it?
You make a good point.

I just added a "MaxTubes" attribute to UnitData that defaults to -1 (unlimited). This can be used to control the number of tubes to any structure passed to BaseGenerator. Tokamaks, Light Towers, Mines, etc will default to 0 if the value is -1, so you will be able to connect those by tubes as well, if desired. This attribute applies even with IgnoreLayout set to true. Same with CreateWall.

If all else fails, you can always use the PathFinder to generate a path between structures for creating tubes. It's a little more complicated, but can help create extra tubes if the BaseGenerator isn't putting them where you want them. All generated units are put in a list, so you can scan the list for the auto structure you want to connect.

Quote from: Hooman
It's pretty easy and standard to write 0-based array code that is start point inclusive, and end point exclusive.
[snip]
Saying this made me rethink using max inclusive. I looked at some other frameworks, and apparently you are right, it is standard to use max exclusive. It seems strange to me that a rect contains a point on its min edge, but not its max edge. However, I also think that a rect of zero size shouldn't be able to contain a point, even if it is infinitely small. Anyway, it's been standardized.




Just kind of want to outline how I'm going to set up the AI.

Managers
The AI is going to be set up as three managers. Managers assess the situation and prioritize goals using heuristics and goal weights. Goals are top-level tasks. Managers have no direct dependencies on each other. They assess the situation and perform their own goals to the best of their ability.

The managers are independent to prevent task blocking. For example, if the BaseManager needs a spaceport, but Space Program has not been researched, it will get stuck waiting for the research to complete rather than pursue other goals. CombatManager would block the other two managers if it's performing a raid, and so on.

BaseManager - Handles deployment, morale, and economic tasks. Also builds military.
ResearchManager - Chooses the next research topic for the colony. Base manager assigns the number of scientists to the lab.
CombatManager - Directs combat tactics with the military units available.

Tasks
Tasks represent what an AI must do to complete a goal. They are arranged as a tree. The current goal task is at the top. When a manager wants to achieve a goal, it calls PerformTask on it.

Tasks contain a list of dependent tasks underneath it that are required to complete the current task. Perform task first checks these tasks to see if they are complete (IsTaskComplete()). If all are true, the task performs itself. If any are false, those unfinished tasks get PerformTask called on them. This can continue all the way down to bottom-level tasks that have no dependencies.

Examples of top-level tasks include "LaunchSpaceportEvac", "CreateMilitaryUnits", "HarassEnemy", and "FixDisconnectedStructure".

If, for example, the goal task is "FixDisconnectedStructure", it would check IsTaskComplete on the "CreateEarthworker" task. "CreateEarthworker" would check "CreateVehicleFactory".

Some tasks can't be completed, and the AI needs to give up on the goal to pursue other things. PerformTask returns false if a task cannot be performed.

In the example above, "CreateVehicleFactory" may not be able to complete its task. If there is no structure factory (or convec with it) or Cybernetic Teleoperation is not researched, the task cannot be completed. The manager is informed via the return, and chooses a different goal.

Bot Types
Bot types or "personalities" will be based on goal weights. I intend to support the following predefined types:

Code: [Select]
PopulationGrowth,	// Bot focuses on growing population. Keeps enough defense to avoid being killed. Will build Recreation, DIRT and other optional structures.
LaunchStarship, // Bot focuses on launching starship. Keeps enough defense to avoid being killed.
EconomicGrowth, // Bot focuses on resource acquisition. Keeps enough defense to avoid being killed.
Passive, // Bot does not build new structures. Keeps enough defense to avoid being killed.
Defender, // Bot will build military units and defend itself and allies. Does not attack.
Balanced, // Bot will build military units and defend itself and allies. Attacks with best available strategy.
Aggressive, // Bot will build military units and won't defend itself or allies. Attacks with best available strategy.
Harassment, // Bot will build military units and harass trucks, power plants, and unescorted or poorly defended utility vehicles.
Wreckless, // Bot will build military units and send them to attack even against overwhelming odds.

A predefined type can be selected, and then the weights tweaked to get the desired effect. For example, you can choose "Aggressive" and tweak how much it focuses on population and economic growth to either confine it to its starting base, or have it attempt to build all over the map. You can also change these weights at any time, so a dormant AI can be triggered to suddenly turn Aggressive or Wreckless.



The greatest obstacle is going to be suboptimal access to player state. I'm thinking of creating a PlayerInfo class to start pooling information in an efficient manner at the beginning of each update.

For example, I am working on the LaunchStarshipTask, and I need to know if the evac module has been deployed. I believe I can iterate over the player units to find it, but I need to also check if the phoenix module has been deployed in another task, the orbital module in another, and... well.. you can see where this is going. Hundreds of loops to pull single items from the player state.

There are other things that are harder to get due to limited API - Beacon states (was it surveyed?), structure demand, and possibly many other things.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 26, 2019, 10:16:51 AM
Very neat. I'd started on a task and dependency management scheme years ago, but the efforts stalled out after I spent all my time tracing dependencies down rabbit holes. You can really take that stuff too far.

For instance, if any any stage, you don't have the immediate resources to do something, trace down to the way to get that resource:

Build structure Type at location (x, y) Depends on Moving ConVec with kit Type to (x, y)
Moving ConVec with Kit at (x, y) depends on having ConVec with kit type, (and (x, y) being move accessible)
Having ConVec with kit Type depends on Loading ConVec with kit Type at Structure Factory
Loading ConVec with kit Type at Structure Factory depends on having a ConVec docked at Structure Factory, and Structure Factory having kit Type in storage, and Structure Factory being enabled.
Having a ConVec docked at a Structure Factory depends on Moving a (free?) ConVec to dock location
Having kit Type at Structure Factory in storage depends on completing research to build kit Type, and having resources (common, rare) needed to build it, and on Structure Factory being enabled.
...
Moving a (free?) ConVec depends on having a (free?) ConVec
...
Having a ConVec depends on Having a Vehicle Factory, having completed research for building ConVecs, having the resources (common, rare) to build a ConVec, and the Vehicle Factory being enabled
...
Having a Vehicle Factory depends on building a Vehicle Factory ... (loop back to top)
...
Having resources (common, rare) depends on being able mine resources (common, rare)
Mining resources (common, rare) depends on ...


You get where that went. The resources were particular messy, as they effectively depended on themselves. If you didn't have a cargo truck, or smelter, or mine, or robo surveyor, you needed to build these things, and that depended on having resources (common, rare). Potentially things could deadlock with unsatisfiable requirements.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 26, 2019, 12:26:32 PM
Technically a player could receive an impossible task, too. No cargo trucks or metal and you're done.

Clearly if you have no convecs and no vehicle factory, you can't create a vehicle factory. Hence the need to declare the task impossible to complete. The question is at what point is it impossible.

Create Military Unit >
Need Vehicle Factory? Yes >
Need Structure Factory? Yes >
Need Convec with Structure Factory? Yes >
Fail out / Task impossible
...
Need Structure Factory? No >
Need free convec? Yes >
Fail out / Task impossible

But what if you have a vehicle factory? Well.. then you wouldn't need to go all the way down the tree to create a military unit.

Option 1:

Each goal can have its own set of tasks. Therefore, no circular dependencies. Downside? Massive code redundancy.

Option 2:

Each task tracks its parent. When you check if the task is possible, check if any prerequisites is an ancestor up the tree.

Create Military Unit >
Need Vehicle Factory? Yes >
Need Structure Factory? No >
Need free convec? Yes >
Is the VehicleFactoryTask prerequisite an ancestor of this task? Yes >
Fail out

Pseudo code:

Code: [Select]
CreateConvecTask : Task
{
    VehicleFactoryTask m_Prerequisite;
    Task m_Parent;

    if (!m_Prerequisite.IsTaskComplete())
    {
        if (IsAncestorTask(m_Prerequisite))
            return false; // Fail out
    }

    bool IsAncestorTask(prerequisite)
    {
        if (m_Parent == null)
            return false;

        if (prerequisite.GetType() == m_Parent.GetType())
            return true;

        return m_Parent.IsAncestorTask(prerequisite);
    }
}

I think that would terminate all circular tasks. Let me know if you think otherwise.

EDIT:

Here's Option 2 integrated into the current Task class.
Code: [Select]
public abstract class Task
{
private Task m_Parent;

protected Player m_Owner;
protected Task[] m_Prerequisites = new Task[0];


public Task(Player owner, Task parent)
{
m_Owner = owner;
m_Parent = parent;
}

public abstract bool IsTaskComplete();

public virtual bool PerformTask()
{
if (!PerformPrerequisites())
return false;

return true;
}

private bool PerformPrerequisites()
{
for (int i=0; i < m_Prerequisites.Length; ++i)
{
/* Skip completed tasks */
if (m_Prerequisites[i].IsTaskComplete())
continue;

/* If a prerequisite is an ancestor task, then we have a circular dependency. This task cannot be performed. */
if (IsAncestorTask(m_Prerequisites[i]))
return false;

/* Perform the task. Fail out if it can't be done. */
if (!m_Prerequisites[i].PerformTask())
return false;
}

return true;
}

private bool IsAncestorTask(Task task)
{
if (m_Parent == null)
return false;

if (task.GetType() == m_Parent.GetType())
return true;

return m_Parent.IsAncestorTask(task);
}
}

...And then presumably the manager picks a new top-level task if it fails.

I admit, the sheer amount of tasks that have to be considered is a PITA, but I don't think the architecture is flawed in itself.
Title: Re: OP2 Scenario Project for C#
Post by: lordpalandus on March 26, 2019, 05:43:14 PM
For my game I'm developing, I like to use something obvious to quickly cancel an activity, so that processing through various steps is quick, when failure is obvious. So for my game, instead of going through all of the code to setup variables for a technique, I first check if the player has enough energy to cast that technique in the first place. Energy is a single variable, so it is quick to check and if you lack energy, then it is quick failure state.

What I mean by this is:

In the case of construction of a military unit, the first thing that should be checked is cost. It is quick to check cost, as they are likely a globally accessible variable. If you lack the resources, there is no point checking all of the other steps.

Then, if you have the resources, the next step to check would be research. Why? Because I'd expect that all structures would be contained within a vector, and thus you'd have to search the entire vector to see if you have an active vehicle factory (as an inactive one would prevent them from building anything). If they lack the research to construct a unit, that then would be a quick check as well (even if research was stored in a vector, you'd likely be looking for a Boolean True for a researched topic)

Then check for an active vehicle factory. Then check to see if the vehicle factory isn't producing anything, and then build said unit.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 26, 2019, 10:16:15 PM
OK. Based on feedback, I've made a dependency graph for spaceship modules. The idea is to limit the decision tree so it doesn't sprawl and become unmanageable.

(https://scoriastudios.com/myfiles/op2/TaskDependencies.png)
NOTE: "IsEnabled" should be a CanPerform requirement for structures.

The top-level task order is not quite right. BaseManager should be performing a weighted situational assessment to determine priority in addition to whether or not the task can be performed.

So for example:

Choose DeployEvacModule, but if I don't have a smelter and trucks, prioritize that.
If common metal is below X amount for Y duration, prioritize CommonMetalExpansion instead.
If enemy is stronger than me, prioritize defense units over the spaceship.
If I have no structures at all, deploy a base.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 27, 2019, 01:02:11 PM
@lordpalandus: In terms of checking resources and failing early, that doesn't quite work with dependency management when you are trying to optimize time to goal completion. As a task may require many lengthy to perform dependencies, such as research, you shouldn't fail out on moving towards a goal because you don't currently have the metals to produce it. You still need to check what those lengthy dependencies are, such as doing research, or setting up mining operations, and if the prerequisites are satisfiable. It may also be that some prerequiste tasks can be done in parallel. For example, you can setup mining operations while doing research.



@TechCor: Will that algorithm work? I don't really know. I'm not sure I quite understand it yet.

As for the graph, I should point out land rush games. At the start, you don't have a Structure Factory, nor do you (usually) have empty ConVecs, but you do usually have structure kits loaded in ConVecs. As such, I think the graph needs an extra node for all structures, which is the single prerequisite of having a ConVec loaded with the proper kit. At that node, you would then fan out to having a Structure Factory, and having a ConVec. I suppose the ConVec need not be empty, since you would swap kits at the Structure Factory when loading a new kit, though generally speaking, if the ConVec wasn't empty, it was probably already given a different task.

I noticed you modelled that dependency for the Structure Factory, but not the Spaceport.

As for balancing priorities, that's a whole other ballpark. I don't really have a clear idea how to handle that.

I suppose if you get the dependency management right, you do get some minimal inbuilt priority system. Such as setting up mining to build the components for the spaceship. Such dependency tracking wouldn't be enough to shift resources between launching an evacuation module, or building an army for defense though. Or shifting research between starship components and weapons.



If you really want to complicate things, there are also maps that can get reinforcements from off the map. Perhaps a ConVec shows up with a needed kit at some delayed time. Though that's a complication that can probably be ignored starting out. There's no real way to know those sorts of things without advance knowledge. Would be nice to have custom pluggable inputs to let the algorithm know about such events though.



Looking more closely at the dependency loop detection, I'm thinking it may be more complicated. The cycle can close at any point, so you don't know where the head of the loop is. It might be 5 levels down before the start of the loop, and maybe another 10 levels until it closes. Those tasks may have seemingly very little to do with the target goal.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 27, 2019, 06:24:17 PM
I think you are looking at the tasks too granularly.

Let's look at how HaveSpaceport would work if we don't have a spaceport.

We start in HaveRocket and ask, HaveSpaceport.IsTaskComplete?

TaskComplete If:
Spaceport exists
Spaceport has empty bays (This should basically be always true)

Task is not complete, so we attempt to perform the task.

Prerequisites:
HaveStructureFactory
HaveConvec

If we don't meet prerequisites, perform those tasks first. If they fail, task fails, manager can decide what to do, if anything.

Now, we assume we meet all the criteria above and the task isn't complete, perform task:

If no CC, return FAIL (If a convec has a CC, the manager should be prioritizing SetupInitialBase task)
If spaceport inactive or idle, return FAIL (Fixing this issue is delegated to another task, BaseManager for tubes, EmploymentManager for worker/scientist distribution)

If spaceport is under construction, return
If spaceport in convec and convec is moving, return
If spaceport in convec and convec is NOT moving, build spaceport, return
If spaceport in structure factory and convec is empty and on dock, transfer cargo, return
If spaceport in structure factory and convec is empty and not on dock, move to dock, return
if spaceport in structure factory and convec is NOT empty and is moving, return (Wait for operation to complete)
if spaceport in structure factory and convec is NOT empty and is on dock, transfer cargo for spaceport
if spaceport in structure factory and convec is NOT empty and is not moving and not on dock, move to dock
If metals < cost) return FAIL (Wait for metal, or expand mining operations, decision to be made by manager)
If spaceport not in structure factory, build spaceport kit, return


In real life, tasks are delegated to the appropriate teams by a manager. Let's look at decision making:

Manager: "Deploy the satellite!"
Engineer: "Well, I need a spaceport, and I can build one for you, but I don't have the materials."
Manager: "Damn it, Bobby. Hurry up, blue collar guys!"
...
Manager: "Ok, you guys are taking too long gathering those metals, forget the satellite for now, ExpandCommonMetalSupply when you save up enough.
...
Manager: "Our tokamak just exploded! Change task to MeetPowerDemand! Priority one! If it fails, just wait for resources!

etc.

Managers decide what to do. The task tree is how to accomplish what the manager wants. If what the manager wants can't be done / is outside your field, tell him so he can update priorities as necessary.
Title: Re: OP2 Scenario Project for C#
Post by: lordpalandus on March 27, 2019, 11:38:34 PM
AI is computationally expensive. If you can provide a fail condition early in the AI tree, it will save you computational resources for other things. You could provide an event after failing to check back on the task in one minute (or 1800 frames -> 30FPS * 60SEC), and see if have resources then. Or fail out of the construction task, and create a new task to gather the resources necessary and then retry the original task when you have required resources.

AI for strategy games can get quite expensive fast, so creating ways of optimizing it and exiting out of a lengthy AI tree, when failure is ensured, will really help with that. If it cannot perform a task, there is no point in checking the other dependencies of the task.

Or if you want the AI to perform several mutually exclusive tasks at once (ie mine resources and research), you could provide several failure conditions to check and if any failed, it aborts the task and looks to turn the failed conditions into successful ones and then retry the task the moment all conditions read as successful. Think of trying to solve all failure conditions as a quest in an rpg; it is successful when you've "gathered" all the necessary components.

Ex.

If you want to build say a Vehicle Factory, it would have these mutually exclusive conditionals:
-> Have available adults
-> Have power
-> Have convec with kit
-> Have technology

If any of these fail, it aborts/postpones the task to construct the factory. Say, it fails on Adults, Power, Convec with Kit. So it generates three new tasks: Grow Worker Population, Construct Power Plant, and Build Kit. Once population reaches necessary amount, Adult task is complete. And so one... until all conditions are satisfied.

Then it produces a task to place the kit in a specific area. While it is being built, it checks to make sure that there is a tube connection, and if not, build one. If the building is built, and a power plant was destroyed, it will idle the structure. Etc...

Ughh... this got complicated fast. I'm going back to developing my roguelike... so much simpler.
Title: Re: OP2 Scenario Project for C#
Post by: leeor_net on March 28, 2019, 06:44:27 PM
AI doesn't have to be expensive -- in terms of games it's not a true AI, it's a fake AI. Have it run tasks in a separate thread maybe once every three to five frames and it need not be expensive. Many commercially successful games do this including World of Warcraft, StarCraft and Age of Empires (you can actually see the lag in AoE2 -- select a unit and direct it to move somewhere, you'll see it hesitate for a few frames before it actually starts to move). Quake III Arena does the same thing with its bots.
Title: Re: OP2 Scenario Project for C#
Post by: lordpalandus on March 29, 2019, 12:45:41 AM
Most things that get called AI are not true AI. No videogame in existence has a true AI... that would be way too computationally expensive to have. So, we fake it with something that appearances to have intelligence with decision trees, probabilities, build orders, etc... to create an illusion of an intelligent AI. Some games does it better than others; ie First Encounter Assault Recon (FEAR) had exceptionally smart "AIs" and as far as adaptability goes for multiplayer bots, Unreal Tournament still is the best with them.

The closest thing the world has seen of a true AI would be that "social experiment" back in 2017? with TayAI, which I think was a learning program based on a neural network fed information from the interactions the AI had with people and adapted their personality and knowledge-base based on the information provided to TayAI.

The world isn't ready for a true AI. If we cannot get along with eachother, avoiding wars, hatred and racism, then we stand no chance of coexisting with an AI. More likely the AI would take over, with either a "Matrix scenario" or a "Skynet scenario".
Title: Re: OP2 Scenario Project for C#
Post by: leeor_net on March 29, 2019, 03:43:14 AM
Thread is starting to derail. Philosophical discussion about what a true AI is and whether or not humanity is ready for it can be started in a new topic.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on March 29, 2019, 07:46:20 PM
TechCor, there is a new PR in OP2MissionSDK that pulls in new library changes. Wanted to bring it to your attention before merging since I think you are actively using the library. Unless you are using deprecated portions, I do not anticipate any issues (famous last words). Also opened a new forum thread if you want to post any concerns, additions, etc.

Thanks,
Brett
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 30, 2019, 01:31:41 AM
TechCor, there is a new PR in OP2MissionSDK that pulls in new library changes. Wanted to bring it to your attention before merging since I think you are actively using the library. Unless you are using deprecated portions, I do not anticipate any issues (famous last words). Also opened a new forum thread if you want to post any concerns, additions, etc.

Thanks,
Brett
Sounds good. I'm watching the other thread and will get the changes when they are ready.


HFL is now in the C# project (with the exception of panes).

Starting AI now, we'll see it how it goes.  ;)

Updated OP to reflect the current status of the project - what's missing, what's new.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 31, 2019, 04:58:46 AM
Does anyone know how to transfer cargo to the spaceport rocket?

I can't seem to figure this one out. I tried DoTransferCargo, but I'm pretty sure that's for the dock.

Scanned types in Unit/Building enumerators and couldn't find an SULV.

--
Can't seem to get the status of a structure either. Kind of necessary to redirect to another task if disabled.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on March 31, 2019, 07:28:42 AM
Haven't tested yet, but have you tried from Outpost2DLL:

Code: [Select]
void SetFactoryCargo(int bay, map_id unitType, map_id cargoOrWeaponType);	// [StructureFactory, Spaceport]  [Note: If items is an SULV, RLV, or EMP Missile, it is placed on the launch pad instead of in the bay]

Might be able to use unitType SULV and cargoType of spaceship part. I can update the comment based on your results.

---

Just a speculation, but someone may have to go into UnitInfo or UnitEx on HFL to get building status and figure out which unknown it is? You might want to check IUnit-HFL to see if it has a building status field (I have never used it). Since the AI cheats, I guess I've never worried about it.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on March 31, 2019, 08:57:34 AM
It doesn't work. It will place an empty SULV on the pad.

I'll try dumping out the OP2Unit struct to a log file, disable a structure, and dump it again to see what changed.

EDIT:

This wasn't as easy as I was hoping it would be:

Active:
unknown5 = -2144927203
flags = 8449037
unknown7 = -2144927203
timerStickyfoam = 0

Disabled:
unknown5 = 2556445
flags = 8440844
unknown7 = 2556445
timerStickyfoam = 7

---
The log was printing continuously (no way to detect keypress AFAIK).
timerStickyfoam changed a few times in there.

timerStickyfoam started at 0.
Then it went to 519.
Then it went to 263.
Then the other values changed (I assume from becoming disabled) and it went to 7.

I'm thinking it might represent the structure animation state, but I am just speculating.



Decided to assume it has something to do with the flags and ran multiple tests for each disable type.

00000000 10000000 11101100 00001101 - Enabled
00000000 10000000 11001100 00001100 - Power
00000000 10000000 00101100 00001100 - Tubes
00000000 10000000 00001100 00001100 - Idle
00000000 10000000 01101100 00001100 - Scientists
00000000 10000000 00101100 00001100 - Workers and scientists
00000000 10000100 00101100 00001100 - Infected

It looks to me:

byte 2:
bit 6 = Is infected

byte 3:
bit 1 = Has scientists?
bit 2 = Has workers
bit 3 = Has power

byte 4:
bit 8 = Lights (Already defined in file)

There doesn't appear to be a "Connected to CC" flag which is very confusing. I checked for other differences between "disabled CC" and "disabled workers" and found none. They are considered identical in OP2Unit.



Tested spaceport with empty SULV and one with Phoenix module loaded. The cargo is very clearly placed into "unknown14", as the value gets set from 0 to 99 (Phoenix module map_id).
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on March 31, 2019, 08:33:47 PM
Good question about loading rockets. I'm not actually sure. I'm not sure if an AI ever does that. I only remember the AI launching EMP Missiles, where there is a special function for in TethysGame. In terms of direct Unit control, the only exported methods for manipulating specific units seems to be:
Code: [Select]
	// Specific Building
map_id GetObjectOnPad() const; // [Spaceport]
void DoLaunch(int destPixelX, int destPixelY, int bForceEnable); // [Spaceport]
void PutInGarage(int bayIndex, int tileX, int tileY); // [Garage]
int HasOccupiedBay() const; // [Garage, StructureFactory, Spaceport]
void SetFactoryCargo(int bay, map_id unitType, map_id cargoOrWeaponType); // [StructureFactory, Spaceport]  [Note: If items is an SULV, RLV, or EMP Missile, it is placed on the launch pad instead of in the bay]
void DoDevelop(map_id itemToProduce); // [Factory]  [Note: Sets weapon/cargo to mapNone, can't build Lynx/Panther/Tiger/GuardPostKits]
void ClearSpecialTarget(); // [Lab]

A few are Spaceport related, but none seem to load cargo into a rocket. My guess is, the original game didn't ever use this functionality for the AI, and so there are no exported methods to do it.

It would still be possible to construct a CommandPacket which matches the Player's command to do it, and issue that. The OP2Internal (https://github.com/OutpostUniverse/OP2Internal) project has a fair bit of info on CommandPackets. Check src/Game/CommandPacket.h. Unfortunately Outpost2DLL has no direct way to issue a CommandPacket. That functionality was split off into Outpost2App.h. It's in class TApp:
Code: [Select]
int PlaybackCommand(struct CommandPacket *, int);

We should look at merging Outpost2App.h into the same project as Outpost2DLL. Maybe as separate subfolders of the same project or something.



In terms of Unit fields, we have some text files in the SVN repository (currently offline?) in the OllyDbg/InternalData/ folder that describes a lot of detail about the Unit structure. It's messy though. It's basically a union of a bunch of different structs. Each Unit type can potentially have it's own layout. It does actually follow an inheritance hierarchy, though we've never actually worked out the full hierarchy. Fields that are common to many unit types come first.

The flags field is reasonably well understood and documented in there.

We should port those text files over to GitHub.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 01, 2019, 06:49:16 AM
Hell Yeah! I reinvented the wheel! The OllyDbg confirms all those flags and the launchPadCargo. I assume HFL used an earlier version or something. The "connected to CC" flag most def isn't there, either.

CommandPacket.h doesn't appear to mention anything about transferring launchpad cargo. The only thing that might be it is TransferCargo which has an unknown variable. I guess it isn't necessary. I set unknown14 to the launch cargo and it loads the cargo. I mimic the command by doing a swap with the bay.


So that leaves two issues:

1. How to tell that a structure is disconnected.

2. HFL doesn't give me enough power. Do I desync my version by adding functionality?
Title: Re: OP2 Scenario Project for C#
Post by: Crow! on April 01, 2019, 09:43:30 AM
What is the "tubes" disable type, if not a reference to CC connection?
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on April 01, 2019, 11:19:25 AM
You can always submit a pull request for HFL updates.

Additionally, OP2Internal can be used with level building. Though it does currently complicate linking, and make the module not relocatable. The relocatable thing isn't normally a problem in practice though, particularly not for level DLLs. I'm planning to address those issues soon.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on April 01, 2019, 05:08:34 PM
TechCor,

If you have the skill to create a function that allows loading rocket cargo via memory hacks, I would encourage you to attempt to add it to HFL via a pull request. We can review the code and hopefully incorporate it into the SDK.

Better not to diverge unless we cannot come to agreement on implementation and this seems like the right thing to add to HFL.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: Arklon on April 01, 2019, 08:40:38 PM
You can always submit a pull request for HFL updates.

Additionally, OP2Internal can be used with level building. Though it does currently complicate linking, and make the module not relocatable. The relocatable thing isn't normally a problem in practice though, particularly not for level DLLs. I'm planning to address those issues soon.

Level DLLs are actually the most susceptible to the not being relocatable thing, since they are loaded last.

Launch pad rocket is an int32 at +0x6E from the base of the internal unit structure, launch cargo (starship module loaded in rocket) is an int32 at +0x72.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 03, 2019, 09:29:35 PM
Launch pad rocket is an int32 at +0x6E from the base of the internal unit structure, launch cargo (starship module loaded in rocket) is an int32 at +0x72.
How did you know it was an int32? I was having problems building a SULV after launching hacked cargo. Turns out changing the value from short to int fixed the problem.

The OllyDbg says it's 2 bytes.


Anyway, I've run into a couple issues.

1. For some reason "DoDock" sends the convec off the map. However, "DoMove" works fine. This is mostly an aesthetic problem that I'm not too concerned about right now, as the convec will still transfer its cargo and move on.

2. Robominer won't deploy. I'm calling DoBuild(map_id.CommonOreMine, beacon.GetTileX(), beacon.GetTileY()). It seems to spin in place on the beacon. Maybe it's the wrong command?
Title: Re: OP2 Scenario Project for C#
Post by: Sirbomber on April 04, 2019, 05:53:13 PM
If you're using .DoBuild you need to issue the command 1 tile down and to the right of the beacon, as if the Robo-Miner (or GeoCon) were a ConVec.  It's a bit of a cheat, since if you watch it happen the miner appears to teleport onto the beacon when construction starts, but it's the easiest way to do it.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on April 04, 2019, 08:18:39 PM
Yeah, the game doesn't provide a proper method to directly build a mine. You either have to hack it with the (+1, +1) tile offset, using a build group (AI only, I believe), or issue a command packet directly.



For the Spaceport, I see this in the InternalData notes from SVN:
Code: [Select]
Unit:Building:Spaceport
-----------------------
0x6E 4 launchPadUnitType  [SULV/RLV/EMPMissile]
0x72 2 launchCargo
0x74 2 boolRLVLanding
Title: Re: OP2 Scenario Project for C#
Post by: Sirbomber on April 04, 2019, 08:35:38 PM
I forked HFL and am working on a DoDeployMiner method, but I'm not sure how to do the bitwise stuff to get the X and Y coordinate into the low/high byte, respectively.  Thoughts?
Code: [Select]
Type: (0x06) "ctMoBuild" Build a building (ConVec, RoboMiner, GeoCon)
------------
Size in bytes:     0x13 (19 bytes) minimum to deploy RoboMiner/GeoCon/ConVec building
Size in bytes:     0xD + numUnits*2 + numWayPoints*4
        (13 + numUnits*2 + numWayPoints*4)
Offset     Size     Description
------     ----     -----------
0x0     1     numUnits
0x1     2*X     unitID - array of size numUnits
0x3     2     numWayPoints (BYTE)
0x5     4*Y     wayPointLocation - (pixelX:15, pixelY:14)
0x9     2     buildArea.TopLeft.x (bulldozed tile)
0xB     2     buildArea.TopLeft.y (bulldozed tile)
0xD     2     buildArea.BottomRight.x (bulldozed tile)
0xF     2     buildArea.BottomRight.y (bulldozed tile)
0x11     2     (Stored to Unit.+0x6A - value 0xFFFF)
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on April 05, 2019, 03:27:25 AM
Do you mean the waypoint location?
Code: [Select]
(pixelX & 0x07FFF) | (pixelY & 0x03FFF) << 15;

Ehh, actually, that is pretty awful and unnatural looking. Assuming I got that right.
Title: Re: OP2 Scenario Project for C#
Post by: Sirbomber on April 05, 2019, 09:06:55 AM
Thanks, that gave me the push I needed to figure it out.  I also fixed DoDock while I was at it.  Take a look here (https://github.com/Sirbomber/HFL).
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 05, 2019, 02:36:03 PM
Nice work. I'll take a look at it when I'm back at a testable point.


Currently implementing a trio of tasks:

CreateCommonMiningBase
CreateCommonMine
SaturateCommonMine

A mine isn't saturated if it has less than X smelters assigned to it. If there are unsaturated mines, build a smelter.
A command center has a "control area". A CC owns beacons inside the control area. If there are beacons without mines, build a mine.
If every beacon has a mine and every mine is saturated, build a new CC near a beacon to increase access.


In order to place buildings, I need to know the "connection grid" to determine where tubes need to be placed, etc. I assume how OP2 does this is unknown. If that's the case, I'm going to create my own grid that can be used as a lookup. A bit of an expensive operation - BF search from the CC checking for tubes and buildings on tiles.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on April 06, 2019, 06:55:00 AM
Ahh sweet. Would be good to get new functions into the development libraries.


Just had a thought about the waypoint calculation. If you used bitfields in a struct, the compiler should be able to pack the values for you. It could also properly handle signed/unsigned extensions. (I'm not sure what is needed here in regards to signed/unsigned).
Code: [Select]
struct Waypoint {
  int x:15;
  int y:14;
  int reserved:3;
};

Waypoint waypoint{pixelX, pixelY};
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 06, 2019, 11:34:52 PM
Things are going pretty well.

The SVN is down, and I can't check the OllyDbg notes. I guess I should have downloaded it. There are a couple new things that I need:

1. Power output and demand. I assume this is in OP2Player.

2. UnitInfo about capacity for residences, medical centers, recreation facilities and DIRT.

I need this to determine the number of each structure that needs to be built.


Hopefully that information is there.
Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on April 07, 2019, 07:17:08 AM
I zipped the OllyDbg folder from my local copy of the SVN repository. It may be missing a couple of changes if Hooman has updated since I pulled. He tends to update them slowly over time when researching.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 09, 2019, 07:05:28 PM
Thanks for the OllyDbg zip. Found what I needed.

---
I would like to check if a player has a particular unit's required technology researched to determine if a task can be completed. For example, you need to research "Space Program" before you can make a Spaceport.

In the UnitInfo class in HFL, there is a function called GetResearchTopic(). The problem is I don't know what this is returning or if it is valid. It returns 69 for the Spaceport, but "Space Program" has tech ID of 5405.

I've tried looking at OllyDbg, but it doesn't appear to match what HFL is doing at all. I'm thinking it's a discrepancy between unit types (vehicles and structures). HFL seems to assume all units are the same size struct.

Can someone give me some clarity on this?


HFL UnitInfo:

https://github.com/TechCor8/OP2DotNetMissionSDK/blob/master/NativeMissionSDK/NativeSDK/HFL/Source/UnitInfo.cpp

I'm looking at DataStructure UnitTypeInfo.txt in OllyDbg.
Title: Re: OP2 Scenario Project for C#
Post by: Arklon on April 09, 2019, 09:28:38 PM
In the UnitInfo class in HFL, there is a function called GetResearchTopic(). The problem is I don't know what this is returning or if it is valid. It returns 69 for the Spaceport, but "Space Program" has tech ID of 5405.
That's because the tech index in that context is by array index, which boils down to the order in which techs are defined in the tech .txt, rather than tech ID.

Quote
I've tried looking at OllyDbg, but it doesn't appear to match what HFL is doing at all. I'm thinking it's a discrepancy between unit types (vehicles and structures). HFL seems to assume all units are the same size struct.
The unit entries in the internal unit array are all the same size, based on sizeof mapGeneralUnit (max map_id) I believe.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on April 10, 2019, 10:42:53 AM
We should get those txt files moved to GitHub.

A long while back, I was paring down the txt files as that information got added into what is now the OP2Internal (https://github.com/OutpostUniverse/OP2Internal) project. Effectively functional documentation, rather than simple text. It may be a good alternate source of information for some things. Though the Unit and UnitInfo stuff was not really cleaned up enough to port over effectively.

In terms of storage, the game seems to use arrays of unions of all sub-types, so you effectively get an array of fixed sized data, even though the fields within each chunk have variable meaning based on unit type. Each Unit has 120 bytes of storage, even though not all unit types use the full 120 bytes. More like the largest Unit type used up to 120 bytes. UnitType info has something similar going on.

As Arklon stated, the research numbers are index numbers into the research array data. OP2Internal likely has some good detail there.

Title: Re: OP2 Scenario Project for C#
Post by: Vagabond on April 12, 2019, 07:49:57 PM
Sirbomber,

Did you want to move your new HFL code into a branch and issue a PR with HFL for review and merging (functions DoDeployMiner and DoDock). I can manually add them, but it would take the credit from you.

Don't want to derail topic, the fix was just posted in here.

-Brett
Title: Re: OP2 Scenario Project for C#
Post by: Sirbomber on April 12, 2019, 09:30:41 PM
I'll PM you about this, to not derail the thread.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 12, 2019, 09:41:23 PM
Just want to let you guys know that there are a bunch of HFL additions on the C# project including:

Research/TechInfo files (Research)
Player structure capacities and status info (PlayerEx)
DoBuildWall (UnitEx)
LaunchPadCargo (UnitEx)
Mining beacon data (UnitEx)

I won't be getting around to creating a proper fork for quite some time, so it could be useful to grab those whenever. I suppose it's possible only I will find those things useful.

--
While I am here, anybody have a good way of detecting when a unit gets stuck?

Sometimes I'll have a situation where a unit will want to move to an unreachable location. Maybe they were ordered to go somewhere, and now a building is in the way. Instead of giving up, it spins in place. I need to detect that this is happening and order it to stop so it can be reassigned.

---
TethysGame::GetRand appears to be returning 0 after the first tick. Is this expected behavior? I may switch to C# rand to get around this. I'm trying to randomize guard post types.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on April 15, 2019, 02:06:20 AM
If you're not using the build in Rand(), there's a pretty good chance multiplayer games will desync.

I believe Rand should return from 0 (inclusive) up to the max value (exclusive). Are you sure it's not behaving correctly? You're not messing with the seed, are you? The build in random number generator should auto seed at game start, based on the system time. If you try to re-seed it, you basically reset the random number sequence. Continually reseeding means it will continually return the same sequence of numbers.

Not sure about stuck units.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 15, 2019, 02:48:18 AM
Hmm. I went back and checked, and Tethys::GetRand works fine. Must have been really bad luck the handful of times I used it. Like flipping heads 15 times in a row... I have no idea.
Title: Re: OP2 Scenario Project for C#
Post by: lordpalandus on April 15, 2019, 10:58:16 AM
I use random number generation a lot in the video game I'm making, so some possible reasons it is returning 0:

1. You improperly set up the array / vector, so it is using an empty array/vector. Random number of 0 between the size of the array/vector would be 0, so it will always return 0.

2. It is choosing a random number as a float, and converted into an int. If the random number was 0.9, in floats that is nearly 1, but when converted to int it becomes 0. Some round up, some round down.

3. Some random number generators, like the one I use, utilizes different streams. If your random number generator requires a specific stream, that could be the issue.

Otherwise:

We will need to see what code you are trying to execute to provide more detailed assistance. Just because there is no syntax error, doesn't mean there isn't some unintentional logic error. Without the code in question, it is really hard to say.
Title: Re: OP2 Scenario Project for C#
Post by: leeor_net on April 17, 2019, 09:38:41 PM
Gonna go out on a limb and assume the core code is just using a really shit PRNG. Haven't looked too far into it but I'd suggest using literally any other PRNG than whatever OP2's engine provides (unless for some reason mission DLL's must use that function as Hooman has suggested).
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 19, 2019, 01:11:28 AM
Again, I don't know what happened. It's possible I coincidentally fixed some code the same time I swapped it out for C# Random. Tethys rand appears to be working, so I will leave it as is. It's easy enough to change the wrapper function to C# random later if needed.

---

I have need of accessing a lab's data for labor reassignment. Unfortunately, it appears that the lab variable addresses overlap the "bayItem" array, which are represented by different sizes. It's not simply a variable naming issue.

I assume then, that I need to put a union in here to make this work. Rather than hack a union in at that spot, what address marks the end of the base class and the beginning of each sub type?

Or am I approaching this entirely wrong?

--It just occurred to me while posting in another thread that instead of a union, I could create a separate struct and offset the address similar to how BeaconData is done. I will give this a shot.
Title: Re: OP2 Scenario Project for C#
Post by: Hooman on April 19, 2019, 01:37:14 AM
Correct about the union. Each unit type may have it's own specific unit related fields. That will make the non-common fields appear to overlap in memory, reusing the same offsets.

We don't have clear picture of the entire unit hierarchy, or how big each unit record is. We know a handful of them, and have some general sense of parts of the unit hierarchy, just not the full info. This is one of our long standing ToDo items.

In SVN: OllyDbg/InternalData/DataStructure Unit.txt, you can find many pieces of known memory layouts. Where base classes and hierarchy relationship are known, they are marked. Where the constructor sets a different set of flags (offset 0x44), they are marked. The meaning of flags may be dependent on unit type, and isn't fully detailed or separated out by unit type, though many details are known. Similarly which offsets and fields belong to which unit types or base types isn't fully worked out. The hierarchy has multiple levels, and is a bit lopsided, so there are many sizes and offsets involved in the unit structure breakdown. Many details are known about what values can be found at what offsets. It's the breakdown of those fields and offsets into unit types and base types that is lacking.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 24, 2019, 01:29:28 AM
I ended up going with a LabData struct with a direct pointer to the first variable's address.


Base, Labor and Research management are all at a nice first draft. Currently going through now and fixing bugs and adding extras such as DIRT, geocon, and magma well handling. Then I'll move onto task priorities, customization and finally combat management.

Looking forward to doing real world tests in the next month or so.

---
I really want to get this unit stuck bug fixed, so I no longer need to "help" the AI occasionally.

I'm thinking the best way is to find out where the unit is trying to move. If I knew that, I could check if it is occupied. Is there a way to get the adjacent tile that the unit is trying to move into? This would be different from its final destination. Then I could check if there is something in the way, and how to handle it.

The other "stuck" situation seems to be due to a bug in "UnloadCargo". 1 in 50 times, UnloadCargo will fail to trigger, and can't even be triggered manually in the agridome or smelter pane despite *not* being grayed out. I can't seem to figure out the pattern, but I've yet to see it happen while only a single cargo truck is trying to dock at the same port, making me think it happens when another truck is trying to push the one on the dock off. The truck on the dock reports "moDone", but building.UnloadCargo() is ignored. This bug is only an issue for Land Rush type games where you need to unload food and metals. Ore doesn't require UnloadCargo(), and therefore the bug never happens there.
Title: Re: OP2 Scenario Project for C#
Post by: Crow! on April 24, 2019, 06:45:47 AM
I've had the inability to unload cargo in the normal game from time to time.  The best way to fix it I've found is to order the truck to move onto the dock again, even if it's already on it.
Title: Re: OP2 Scenario Project for C#
Post by: TechCor on April 24, 2019, 03:44:05 PM
Yes, it seems issuing any command to the truck will resolve it. The issue is detecting that the problem is happening in the first place.

I'm thinking maybe adding a timer linked to the unit index to determine if they have been on the dock idling too long. Maybe a tenth of a second with "moDone" is all that is needed to determine that it's "gone bad". Not as elegant of a solution as I was hoping for, but any fix is better than no fix.

This solution might work for the movement problem as well, if pixelX/pixelY is a reliable way of telling if a unit is moving.

EDIT:

Checking for "spinning wheels" and waiting 200 ticks before issuing a stop command seems to work. Wish this kind of handling was built into the engine, but whatever.