Author Topic: OP2 Mission Editor (Read 42359 times)

TechCor · « **on:** November 13, 2019, 07:05:26 AM »

Hey there!

Welcome to another episode of TechCor's OP2 Projects!

Today, we are going to focus on the Outpost 2 Mission Editor.

This top post will be edited as development progresses - just like the C# Mission SDK.

Releases
https://github.com/TechCor8/OP2MissionEditor/releases

Source
Mission Editor: https://github.com/TechCor8/OP2MissionEditor
OP2Utility.NET: https://github.com/TechCor8/OP2UtilityDotNet

Currently designing the feature set for this project. Additional details in the following post.

TechCor · « **Reply #1 on:** November 13, 2019, 07:07:34 AM »

Tools:
This project is going to be built with Unity since I have a lot of experience working with it. It will make long-term maintenance a pain (annual upgrades), but it shouldn't take more than a few hours each year to update. It will be very important not to let the project rot to the point of being unsalvagable.

The editor's missions will work exclusively with the C# Mission SDK through the use of its JSON data files. There will be an extra feature to export the MAP portion through OP2 Utility, but for the most part, data will be saved in JSON. For devs that prefer to work in C++, don't forget you can link to the C# SDK and continue working in C++.

Feature Specification:

Code: [Select]

--------
Menu bar
--------
File:
New
Open...
Import Map...
Save
Save As...
Export Map...
Exit

Edit:
Map Properties
Mission Properties
Player Properties

--------------
Map Properties
--------------
Width
Height

------------------
Mission Properties
------------------
Description
MapName
TechTreeName
MissionType (DropDown)
MaxTechLevel (DropDown)
UnitOnlyMission (Checkbox)

DaylightEverywhere (Checkbox)
DaylightMoves (Require DaylightEverywhere=false, Checkbox)
InitialLightLevel (int inputfield)

Music Playlist
Scroll list of tracks
Add track (dropdown)
Remove track
Move up/down
Repeat start index

-----------------
Player Properties
-----------------
Scroll list of players
Add Player
Remove Player

TechLevel
MoraleLevel
FreeMorale
ColonyType (dropdown: Eden, Plymouth)
IsHuman (checkbox)
BotType (dropdown)
Color (dropdown)
Allies: Togglable scroll list of players
Center View X
Center View Y

Kids
Workers
Scientists
CommonOre
RareOre
Food
SolarSatellites

Scroll list of completed research
Add topic (dropdown: Data pulled from tech file)
Remove topic


----------
Paint Pane
----------
Dropdown (Terrain, CellTypes, Resources, Walls and Tubes, Disasters, Units, Structures, Locations)

Terrain:
Tiles (Extract BMP)

CellTypes:
CellType (dropdown)

Resources:
Prebaked Icon
Common Mining Beacon
Rare Mining Beacon
- Yield (dropdown)
- Variant (dropdown)
- *Spawn Area (width/height)

Fumarole
- *Spawn Area

Magma Vent
- *Spawn Area

Circle Marker
DNA Marker
- *Spawn Area

Wreckage:
- Type (dropdown: spaceship parts)
- IsVisible (checkbox)
- *Spawn Area

Walls and Tubes:
Prebaked Icon
Tube
Wall
Lava Wall
Microbe Wall

Disasters:
DisasterType (dropdown)
*SrcArea
*DestArea
Size
Duration
StartTime
EndTime
MinDelay
MaxDelay
(If disaster has both src and dest areas, must place both for disaster area to appear)

--NOTE--
*Areas are painted by clicking and dragging to create the size

Units/Structures:
Colony (dropdown: Eden, Plymouth)
Prebaked Icon
Direction
CargoType (dropdown: Convec=StructureKit, Truck=TruckCargo) Must be convec or truck
CargoAmount (int inputfield, cargo truck only)

Locations: (Used for triggers)
ID (alphanumeric?)
*Spawn Area

--------
Triggers
--------
TBD

-------------------
AI Base Preplanning
-------------------
TBD

Please let me know if you have any feature requests or changes you would like to see, and also if you see any problems with the current list.

Hooman · « **Reply #2 on:** November 13, 2019, 10:11:06 AM »

Slightly related, I had a project I was working on a while back that converted the Outpost 2 map file to JSON format. Not sure it would be useful to you though. It mostly just made use of OP2Utility, which you can use directly.

Would be good to have extra map data stored in JSON format, rather than forcing people to write executable DLL code.

Vagabond · « **Reply #3 on:** November 15, 2019, 08:31:26 PM »

techcor,

This sounds ambitious. Good Luck! I would be willing to do some light testing for you if it gets far enough along.

We should talk about your use cases for the sprites contained in Outpost 2. Currently, OP2Utility will just extract individual frames. You will probably want to show composite frames of structures. I wanted to program this, but wasn't good enough at the bitmap format at the time. I could take another look at it though if you think it would be helpful.

At any rate, please report any bugs are wanted features in OP2Utility. I'll see what I can do to accommodate the changes.

----

Hooman, if your ideas are fairly well fleshed out, maybe it would be good to get it in the repo so techcor can use it?

-Brett

Hooman · « **Reply #4 on:** November 15, 2019, 10:26:23 PM »

I'll have to dig up the project and upload it to GitHub.

Actually, I was just thinking, a C# project that could run with Mono would work on Linux. That would be wonderful. The current OP2 editor doesn't run on Linux, and trying to update it so it does sounds like a rather difficult undertaking.

TechCor · « **Reply #5 on:** November 16, 2019, 11:48:37 AM »

Just committed OP2Utility.NET's source to GitHub.

Unity can build to Linux, so the editor will work there as long as an ".so" can be built from the OP2UtilityForC project in OP2Utility.NET. Someone will need to check that source out and see if they can make an .so. No rush though, OP2UtilityForC has not been tested yet.

----

The editor has the menu options set up. Going to add the code behind them later tonight now that I have OP2 Utility for C#.

Hooman · « **Reply #6 on:** November 17, 2019, 07:26:45 AM »

For those who are curious, the link to the project is:
https://github.com/TechCor8/OP2UtilityDotNet

Edit: Ahh, I see now. You edited your first post and added a link.

Interesting how it converts the C++ API interface to a C compatible one, and then wraps C# classes around the C interface.

I gave the project a try from Linux. I needed to first get some Nuget packages with dotnet restore. The Microsoft build tools don't really know how to build C++ projects from Linux, but I was able to specifically build the C# project with either msbuild OP2UtilityDotNet or dotnet build OP2UtilityDotNet.

TechCor · « **Reply #7 on:** November 17, 2019, 04:24:23 PM »

Yes, the C compatible interface is what needs to compile on Linux. I suppose it needs a CMake file or something. I'm not familiar with it.

Since C# compiles to IL code, the DLL from Windows will work on Linux. The JIT compiler on the user machine should take care of it.

Although, I will need a special DLL anyway to change DllImport to point to the Linux C library. That is something I can prepare easily.

---

Looks like I am able to load and save maps and missions (File Menu). Going to look at rendering tiles to the tile map.

TechCor · « **Reply #8 on:** November 17, 2019, 11:41:50 PM »

EDIT: I fixed it! Turns out the map was upside down and the tile graphic indexes needed to be inverted.

Update:

Submitted editor source. It's currently just a "Map Viewer" where you can import and scroll around maps, adjust grid overlay color and alpha, and export the map. Not going to bother with a release build yet.

Hooman · « **Reply #9 on:** November 18, 2019, 01:44:51 PM »

I tried checking out the project. There didn't seem to be a project file. I saw some DLL files, but no EXE. Not really sure how to run it.

Meanwhile, the JsonMap project that was mentioned earlier has been uploaded to GitHub. Probably not directly useful for your editor though.

TechCor · « **Reply #10 on:** November 24, 2019, 08:13:35 AM »

Quote from: Hooman on November 18, 2019, 01:44:51 PM

I tried checking out the project. There didn't seem to be a project file. I saw some DLL files, but no EXE. Not really sure how to run it.

The folder itself is the project file. You need Unity to open it.

Update:

Added minimap with navigation
Added Mission Properties
Added Player Properties
Added Windows build for interested parties to take a gander.

Next major task is the paint pane.

Now that I have a better understanding of how maps work, it's clear that the mission API can't fully control the graphic in the tilemap. I thought the entire game was using one big atlas.

I'm thinking the two alternatives now are exporting the map and opm files separately (which is how these are currently read), and/or bundling them into a special mission vol file with a different extension that gets read by the C# Mission SDK. Maybe both.

So options in the File menu could be:

Open (omv / Mission Volume)
Import Map
Import Mission (opm)

Save (omv / Mission Volume)
Export Map
Export Mission (opm)
Export Plugin (Native C DLL hook)

Of course this would require modifying OP2 to look for maps from a different vol extension. Probably not worth it.

leeor_net · « **Reply #11 on:** November 25, 2019, 03:01:30 AM »

Suggestion, it would probably be better to use the GitHub Releases feature instead of providing the release directly in the repository. Just FYI.

TechCor · « **Reply #12 on:** November 25, 2019, 03:24:21 AM »

Done.

Quick brief about how to use the current version:

When you open the editor, you will be asked to select your Outpost 2 game folder where it can find the system vol files. If you set this to the wrong directory, go to File > Preferences.
By default, you are staring at a "New" map. You can't really do anything yet with a new map, so go to File > Import Map...
Set filter to .vol unless you have loose .map files lying around. Select maps.vol to get a list of maps.
Voila! Map Viewer 3000! Use arrow keys to move, shift to move faster, or use the minimap under the View menu.
Export Map and Open/Save mission should work, although I haven't tested in a while.
Mission and Player properties configure the mission json file. The output already works with the C# Mission SDK (In fact, it uses it to make the json file.)

That's it for now!

leeor_net · « **Reply #13 on:** November 25, 2019, 09:11:03 AM »

It's definitely a great working prototype! I gave it a shot last night and was generally impressed.

I dislike the UI as it's not native but that's truly a minor gripe.

Only thing I noticed is that when importing a map, it kinda hangs for a bit. Or at least, it appears to. If I had to guess I'd say it's doing a lot under the hood to convert to an internal format that the editor can work with. Might be good to add a loading text or change the cursor to a wait cursor, something like that. It's a comment about usability and UI's that are 'responsive' to the user. Granted this is early work so I wouldn't go too far into worrying about issues like that before functionality is in place.

Anyway, good work! I'm excited to see where this goes.

TechCor · « **Reply #14 on:** November 25, 2019, 12:56:08 PM »

Yes, I would prefer native UI also, but that's the price to pay to use Unity.

I already profiled the import map. Most of the load is coming from color averaging the tiles for the minimap (about 75%). Maybe there is a faster algorithm. I've thought about skipping pixels, but that might affect image accuracy.

Actually, a loading bar is not a bad idea. I can have the percentage be the number of tiles loaded / total tiles. It would would look pretty smooth.

leeor_net · « **Reply #15 on:** November 25, 2019, 06:10:18 PM »

A faster way to do it would be to average the colors of the tilemap itself, build an index table then build the minimap using the index table. E.g., tile index = color index = pixel or however you're generating the map. It's very fast in the prototype code that I built a couple years ago.

Could even pre-cache the index table, e.g., on first run as part of the set up go through all of the tilesets, determine an average color value and store in an index. Can save it out however you think is appropriate and then just load that if the file is present.

Hooman · « **Reply #16 on:** November 25, 2019, 11:56:13 PM »

That's pretty much how the MapImager works. It loads the tilesets, then precomputes scaled tiles for the given scale factor. When it renders the maps, it uses the scaled tileset rather than the original one.

pente · « **Reply #17 on:** December 09, 2019, 03:52:12 AM »

Likewise my python map code precomputes the scaled tiles for the desired scale. I put the code here https://forum.outpost2.net/index.php/topic,6275.0.html, it is pretty short and fast thanks to numpy magic.

TechCor · « **Reply #18 on:** December 13, 2019, 08:41:04 AM »

I finally have some time to work on this again.

I now cache a minimap tileset texture along with the regular one when they are first loaded, so that has been optimized. Thanks for the idea, Leeor!

Also removed unnecessary tile refresh which was already happening during SetTiles, and added a progress bar for importing.

Still can't get resourceManager working:

Code: [Select]

extern EXPORT unsigned __int64 __stdcall ResourceManager_GetResourceSize(const char* archiveDirectory, const char* filename, bool accessArchives)
{
	ofstream outputFile;
	outputFile.open("quicklog.txt");

	outputFile << "archiveDirectory = " << archiveDirectory << "\n";
	outputFile << "filename = " << filename << "\n";
	if (accessArchives)
		outputFile << "accessArchives = true\n";
	else
		outputFile << "accessArchives = false\n";
		
	outputFile.close();

	ResourceManager resourceManager(archiveDirectory);
	std::unique_ptr<Stream::BidirectionalReader> stream = resourceManager.GetResourceStream(filename, accessArchives);
		
	if (stream == nullptr)
	{
		ofstream outputFile;
		outputFile.open("quicklog2.txt");
		outputFile << "hit 0";
		return 0;
	}

	return stream->Length();
}

Prints out:

Code: [Select]

archiveDirectory = E:/Business/Freelance/Projects/Outpost2/Outpost2exe
filename = well0008.bmp
accessArchives = true

...and stream is null.

Tried different archive directories - with/without trailing slash, forward vs back slashes... Can't get it to work.

Will probably have to debug the utility itself to figure this one out.

Hooman · « **Reply #19 on:** December 13, 2019, 02:25:05 PM »

I looked into this. From cursory examination, there may be a bug when the archiveDirectory does not match the current working directory. In that case, when scanning the resource folder for .vol and .clm files, all non-existent files (using the current working directory, not the resource folder) are filtered out of the search results. This causes the archives to not be seen and loaded. That would result in no packed files being found and nullptr being returned for the stream.

Edit: To debug, consider using GetArchiveFilenames to see if any archives are loaded.

Despite my earlier comment, I'm not sure a trailing slash is actually needed. For reference though, I would have used the path:

Code: [Select]

E:/Business/Freelance/Projects/Outpost2/Outpost2exe/

The trailing slash makes the name unambiguously a directory name.

TechCor · « **Reply #20 on:** December 13, 2019, 05:25:58 PM »

You are right; it ignores the passed in directory and uses the working directory.

Confirmed by putting art.vol into the current program directory which is NOT the directory being requested.

The working directory should be used when a relative path is specified. Absolute paths should override the working directory.

Can the OP2Utility project owner please fix this?

I will pull the fix as my copy of OP2Utility has not been changed.

EDIT:
Just want to add that I did check the archive count for the absolute path - it was zero. No archives at the specified directory.

Vagabond · « **Reply #21 on:** December 13, 2019, 07:55:30 PM »

Hooman and I co-curate OP2Utility, although pull requests from the general OPU community would be welcome. I see he opened an issue against the described behaviour. We should be able to fix in fairly short order. Maybe best if Hooman picks the solution. My steering on the ResourceManager and XFile seems to have been poor...

TechCor, why don't you pull OP2Utility as a git submodule into your project so you can pull updates from OP2Utility easier into your project? Also might make it easier to push pull requests if you ever felt inclined.

Sorry for the defective code.

-Brett

TechCor · « **Reply #22 on:** December 13, 2019, 08:15:58 PM »

I've tried looking into submodules before, and couldn't figure it out. Possibly due to a limitation to referring to another user's repositories. I'm not sure.

Quote from: Vagabond on December 13, 2019, 07:55:30 PM

Sorry for the defective code.

Let him who has not coded bugs cast the first stone.

lordpalandus · « **Reply #23 on:** December 15, 2019, 02:22:28 AM »

I've never coded insects into my code

.

"Tosses a stone worth of insects at Vagabond."

Hooman · « **Reply #24 on:** December 16, 2019, 06:51:08 PM »

Quote

I've tried looking into submodules before, and couldn't figure it out. Possibly due to a limitation to referring to another user's repositories. I'm not sure.

The mechanics of Git submodules can be a bit strange and unintuitive at times. The worst seems to be that recursive cloning is not the default. There are no restrictions on adding someone else's project as a submodule though. The submodule reference is entirely in the host repository, and doesn't require changes to the submodule's repository. Read-only access is sufficient.

News:

Author Topic: OP2 Mission Editor (Read 42359 times)