Needed Future Projects

[Hooman]

I spent some time thinking about what projects would be most useful. Things that either solve an immediate need, or represent where we want to be at some point in our future. Here's what I've come up with.


Future project ideas:

• OP2ImageExtractor: Extract sprite images from OP2_ART.BMP/OP2_ART.PRT. This handles extracting of single component images, single compound animation frames, and sprite sheets of animations. Such images can be used to spruce up the website and wiki. They can also be made available for more general usage, such as spinoffs and game remake projects.
• OP2UnitAnimationMetaExtractor: Extract meta info for unit animations. The above project could extract images and animations, but there is no data to tie all the animations for a single unit type together. This requires hacking into Outpost2.exe to extract animation info for the buildings and vehicles, and sorting the data into related sets. Animation sequences would be associated with each unit type, and also categorized based on the animation type. For vehicles moving in each of the 8 directions, docking animations, etc. For buildings, idle animations, damaged animations, etc. Part of the project will be figuring out what types of animations are available, and how they should be grouped and classified.
• Tile Group Meta Info: The OP2 tile sets lack certain critical info that would aid a map editor. We have tile groups, but no classification info for them. Some groups are meant to be pasted as a whole, such as volcanoes and multi tile meteor craters. Others are meant as a palette, consisting of a string of alternative tiles to select a single tile of a certain type from, such as 1 tile meteor craters, or rubble. The first set, it would likely be wrong to paste just a single tile from the group. The second set, it would likely be wrong to paste more than one tile from the group. Other groups are perhaps a bit more free form, such as flat plains, where you could paste the group as a whole, or arbitrarily mix and match any of the individual tiles. We should added two new fields to each group, if it supports full pasting, or single tile pasting (or both). We could extract the tile group info into a plain text file format, such as JSON, and add the extra attributes. No programming experience required for the group sorting.
• Tile Cell Meta Info: The OP2 tile sets lack meta info associating cell types with tiles. This complicates pasting of tiles from tile groups, since there is no default cell type associated with each tile. Having the user patch up cell types later is not friendly. Using alternate means, such as copying from a palette map which has both tiles and cell types set is a hack. This data is strongly associated with tiles, and so should be stored with tiles. Meta data could be stored in a plain text format, such as JSON. No programming experience required for assigning default cell types to tiles.
• Tile Transition Meta Info: The OP2 tile sets lack meta info for generating smooth terrain transitions. This includes how sand transition into mud or rock tiles. There is no set pattern to these transition tiles, making this a somewhat vague and frustrating problem. The transition tiles will need to be analysed to figure out the direction of each transition tile, and also for what restrictions may exist for placing the tiles to have them blend smoothly with neighbouring tiles. Data can be stored in a plain text format, such as JSON. No programming experience required for the tile sorting.
• Map Editor: Using the data from above, big improvements can be made to the map editor. A lot can be done to ease generating new large maps, using auto-transition tiles, and large area fills.
• Auto Map Generator: Automatically generate new maps. This could be a useful plugin feature for the map editor. This would allow a quick base layer to be generated for a new map, which could then be modified as needed by the level designer. It could also possibly be used as a library for missions, that generate a new map each time you play them. Parameters could be set that could allow guiding of the map generation process. Perhaps regions can be defined to have certain characteristics, such as base terrain (sand, mud, rock), or roughness (meteor craters, bumps, raised areas). For anyone interested, Perlin Noise, and Procedural Generation might be some good background study topics. This would be a more advanced project.
• Hacking OP2 Map Format: Take a deep dive into the internals of Outpost 2. Figure out what it would take to get the game to support a new custom map file format. This could pave the way for future work, such as loading maps with packaged unit data or scripting, possibly removing the need to compile DLLs in C++ to produce a new mission. Currently the mission generation process is ... tedious, resulting in little new content. Ease the burden with a new map format that requires a less complicated process and tooling. This also paves the way for the map editor to become a more full features mission editor.
• AI: Write a general purpose AI that can play OP2. This could be done as a library, such that custom levels can include it to implement an AI player. If someone is feeling particularly ambitious, a game mod could be written that allows dropping the AI in to pre-existing levels,
   perhaps allowing multiplayer missions to be played as single player. This would be a more advanced project.

• D Programming: A lot of our utilities are written in C++, which is an old language with a lot of baggage, hard to learn well, and not always very expressive for certain high level tasks. It might be worth exploring the use of D, which offers a lot of the same low level expressibility as C++, but also includes a lot of higher level expressibility. It's faster to write, faster to compile, and generally more cross platform. Compared to C++, D has a lot fewer legacy issues, and much less baggage. Project might be to rewrite a C++ utility that already exists, and compare the solution in the two languages.

• Blender modelling: Generate 3D models of vehicle and buildings from OP2. These could be used to generate in game graphics for a sequel, as well as new cut scenes.

If anyone else has some must-have project ideas to move things forward here, feel free to post them. I'd also like to hear what people think is most important.

[Arklon]

OP2ImageExtractor: Extract sprite images from OP2_ART.BMP/OP2_ART.PRT. This handles extracting of single component images, single compound animation frames, and sprite sheets of animations. Such images can be used to spruce up the website and wiki. They can also be made available for more general usage, such as spinoffs and game remake projects.

Pretty sure BB made something like this in Python a while back. I think it was even capable of repacking those files, which the older OP2ArtViewer tool didn't properly support.

Hacking OP2 Map Format: Take a deep dive into the internals of Outpost 2. Figure out what it would take to get the game to support a new custom map file format. This could pave the way for future work, such as loading maps with packaged unit data or scripting, possibly removing the need to compile DLLs in C++ to produce a new mission. Currently the mission generation process is ... tedious, resulting in little new content. Ease the burden with a new map format that requires a less complicated process and tooling. This also paves the way for the map editor to become a more full features mission editor.

BB's Python mission API would be a solution to this. Since it was so far along already, it'd make the most sense to finish that.

D Programming: A lot of our utilities are written in C++, which is an old language with a lot of baggage, hard to learn well, and not always very expressive for certain high level tasks. It might be worth exploring the use of D, which offers a lot of the same low level expressibility as C++, but also includes a lot of higher level expressibility. It's faster to write, faster to compile, and generally more cross platform. Compared to C++, D has a lot fewer legacy issues, and much less baggage. Project might be to rewrite a C++ utility that already exists, and compare the solution in the two languages.

I hate to say it, but D is practically just a gimmick. Even though it's relatively new, it's been around long enough that if a strong ecosystem were to develop, it would have started happening by now, yet it hasn't because it really doesn't have any big-league backers, and the prospects of that ever happening looks slim. The few projects I knew that people started in D (such as OpenMW IIRC), they ended up scrapping their code and going back to C++ just because the ecosystem/documentation/etc. is very lacking. Rust would be more promising since it has Mozilla backing it, but even that I have doubts about if it will go anywhere, and I'm not sure I'm a fan of the syntax.

[Hooman]

Pretty sure BB made something like this in Python a while back. I think it was even capable of repacking those files, which the older OP2ArtViewer tool didn't properly support.

I was not aware of that. Is the tool or its source available anywhere? What features does it support? I would be most interested with something that supports sprite sheet extraction, along with any associated metadata needed to use the sprite sheets.

BB's Python mission API would be a solution to this. Since it was so far along already, it'd make the most sense to finish that.

I am aware of this. In particular, it seems the one part that was left unfinished is the part I was proposing above. It seems BlackBox has the scripting component largely working, but does not yet have the game listing new file types in the game list dialog boxes. This work could go to support those efforts.

Another discussion I'd like to have concerning that project, is if Python scripting goes far enough. A lot of things, like initial unit placement, and starting colony conditions, can be done with simple data, which would be easier for a map editor to process than generating and interpreting code. This is also what the vast majority of DLL code handles. Having a standard data format for initial unit placement, and starting colony conditions, could eliminate the need for 80% of the DLL code/Python script. Though perhaps that's a discussion for another day. If the Python scripting is working, that's still much better than compiling C++ code, so it's still a big step forward. I'd say work on adding support for new map types, use BlackBox's Python scripting project to implement a new map type, and from there see if it makes sense to extend things with a standard data format for initial level conditions.

I hate to say it, but D is practically just a gimmick. Even though it's relatively new, it's been around long enough that if a strong ecosystem were to develop, it would have started happening by now, yet it hasn't because it really doesn't have any big-league backers, and the prospects of that ever happening looks slim. The few projects I knew that people started in D (such as OpenMW IIRC), they ended up scrapping their code and going back to C++ just because the ecosystem/documentation/etc. is very lacking. Rust would be more promising since it has Mozilla backing it, but even that I have doubts about if it will go anywhere, and I'm not sure I'm a fan of the syntax

A fair concern. That's one reason why I suggested a smaller utility project be written in D, as a way to try it out. D does seem small, and does seem to lack a bit of ecosystem support. In particular, the standard library is not quite as extensive as I'd like, and the Dub package management system and build tool still seems a little rough around the edges. The D ecosystem has in the past been somewhat slow growing. I suspect it took a big hit from the earlier D1/D2 split, and the Phobos/Tango standard library split. That seems to have been resolved long ago, and there seems to be consensus in the D community not to let that happen again. Currently there is growth, and it appears to be accelerating. As for the documentation, I'd say it's actually pretty good. It's also aided by the built in support for documentation with special source comments.

I've been curious about Rust. I heard something about static checks for pointer ownership, which I thought was pretty neat, and something I'd once wished for. Though reading just now, it seems some of the pointer stuff has been removed from the language.

[Arklon]

I was not aware of that. Is the tool or its source available anywhere? What features does it support? I would be most interested with something that supports sprite sheet extraction, along with any associated metadata needed to use the sprite sheets.

Yes actually: https://github.com/phossy/op2art It even has a readme which should answer your questions about it.

Another discussion I'd like to have concerning that project, is if Python scripting goes far enough. A lot of things, like initial unit placement, and starting colony conditions, can be done with simple data, which would be easier for a map editor to process than generating and interpreting code. This is also what the vast majority of DLL code handles. Having a standard data format for initial unit placement, and starting colony conditions, could eliminate the need for 80% of the DLL code/Python script. Though perhaps that's a discussion for another day. If the Python scripting is working, that's still much better than compiling C++ code, so it's still a big step forward. I'd say work on adding support for new map types, use BlackBox's Python scripting project to implement a new map type, and from there see if it makes sense to extend things with a standard data format for initial level conditions.

Python itself would be a reasonable static data storage format, though any of YAML/JSON/protobuf would work too.

Something to be kept in mind is differing base layouts/general unit placement depending on game type, resource/difficulty level, etc.

A fair concern. That's one reason why I suggested a smaller utility project be written in D, as a way to try it out. D does seem small, and does seem to lack a bit of ecosystem support. In particular, the standard library is not quite as extensive as I'd like, and the Dub package management system and build tool still seems a little rough around the edges. The D ecosystem has in the past been somewhat slow growing. I suspect it took a big hit from the earlier D1/D2 split, and the Phobos/Tango standard library split. That seems to have been resolved long ago, and there seems to be consensus in the D community not to let that happen again. Currently there is growth, and it appears to be accelerating. As for the documentation, I'd say it's actually pretty good. It's also aided by the built in support for documentation with special source comments.

By documentation being lacking I suppose I was referring to that as an extension of the ecosystem's barrenness. C++ has been around forever so there's no shortage of references, tutorials, etc. but there isn't nearly as much of that around for D.

[leeor_net]

I have an issue with using python as it requires a runtime to be installed unless a runtime is provided. Much like Java -- another language I really hate using. Just putting that out there.

I'd really like to see this mission thing that you guys keep talking about. Makes me wonder if my mapper project is necessary unless the scripting thing is purely about developing new missions (in which case, python is an appropriate language though I personally prefer Lua... I suppose at some point we could add several language bindings... but now I digress).

If the mapper project isn't pointless, I'll put forth a bigger push toward getting it more functional. I have it loading but not saving (saving should be simple enough) and it can display tile groups. Tile groups look particularly useful specifically with terrain transitions -- I just need to develop the mappings. And of course the biggest thing is the terrain paint brush. That will take a bit of time to develop... perhaps a few hours.

On that note, what about Vagabond's efforts? He addressed a few of these no? Or am I imagining things? (sorry, migraine, trying to make sense, not sure if I am).

I need to second my hesitation to work with D. I don't see many projects developed in it and I see very little support across the board. My language of choice is C++ but I see that C# is taking off especially with .Net being open-sourced. If anything I'd recommend pushing C#.

[Arklon]

I have an issue with using python as it requires a runtime to be installed unless a runtime is provided. Much like Java -- another language I really hate using. Just putting that out there.

I really don't see this as really being any different than needing to provide an interpreter for any other scripting language like Lua. Obviously we'd provide a Python interpreter with the game and not use an external one.

I'd really like to see this mission thing that you guys keep talking about. Makes me wonder if my mapper project is necessary unless the scripting thing is purely about developing new missions

The Python thing is just about missions themselves, not mapping. It's an alternative to mission DLLs.

I personally prefer Lua

I would personally prefer continuing to use C++, since I like adding all these gameplay hacks to my missions. I'd figure we wouldn't just drop support for the existing mission API.

I need to second my hesitation to work with D. I don't see many projects developed in it and I see very little support across the board. My language of choice is C++ but I see that C# is taking off especially with .Net being open-sourced. If anything I'd recommend pushing C#.

I like using C# for GUI apps, mostly because writing a GUI in C++ is an exercise in self-flagellation.

[leeor_net]

I have an issue with using python as it requires a runtime to be installed unless a runtime is provided. Much like Java -- another language I really hate using. Just putting that out there.

I really don't see this as really being any different than needing to provide an interpreter for any other scripting language like Lua. Obviously we'd provide a Python interpreter with the game and not use an external one.

That eliminates my issue with python then.

I'd really like to see this mission thing that you guys keep talking about. Makes me wonder if my mapper project is necessary unless the scripting thing is purely about developing new missions

The Python thing is just about missions themselves, not mapping. It's an alternative to mission DLLs.

Ah, I had a thought about doing something like this before but never got around to trying anything. This would be a big boon I think and would make mission development far more accessible to less advanced programmers. As much as I dislike python it's still useful and is a lot easier for new developers to learn.

I personally prefer Lua

I would personally prefer continuing to use C++, since I like adding all these gameplay hacks to my missions. I'd figure we wouldn't just drop support for the existing mission API.

Good good. I've been thinking about revamping it a bit. I think Vagabond solved a few mysteries that we've seen a few times (namely in saving/loading missions).

I need to second my hesitation to work with D. I don't see many projects developed in it and I see very little support across the board. My language of choice is C++ but I see that C# is taking off especially with .Net being open-sourced. If anything I'd recommend pushing C#.

I like using C# for GUI apps, mostly because writing a GUI in C++ is an exercise in self-flagellation.

Glad I'm not the only one that sees this. Even with a simplified GUI API that I developed myself in C++, it's still a painful exercise. I've tried a few times to do Win32 GUI programming in C++ and .Net GUI development in C++. Painful painful painful!

[Vagabond]

Good good. I've been thinking about revamping it a bit. I think Vagabond solved a few mysteries that we've seen a few times (namely in saving/loading missions).

Leeor, I think what you are referring to is the fact that Outpost 2 assumes specific base addresses of the mission DLLs when loading them into memory. I think others were aware of this fact before I was around, it just hadn't been documented. Check out the randomized base address section of this page: https://wiki.outpost2.net/doku.php?id=op2_sdk:projectcreation.

I will second that C# works great for developing games. I have a lot of experience doing this and you can work a lot faster then in C++. I haven't managed to get things cross platform though except I have code that runs on both XBOX 360 and Windows. I cannot speak to C# on Linux or Macintosh though. There will probably still be some challenges running C# on Linux or Macintosh even though Microsoft continues to port .Net to Linux and Macintosh.

-Brett

[Hooman]

+1 for "self-flagellation"

As for C# on Linux, there is Mono. Though there is something concerning about the "Get Mono" heading. 

[Angellus Mortis]

As for C# on Linux, there is Mono. Though there is something concerning about the "Get Mono" heading. 

You would want to use .NET Core.

Also, I think it would be better to implement some type of high level API to design and build mission files/AI with. Maybe a library that provides binding to Python or some other higher level and easier to use language.

[Arklon]

Also, I think it would be better to implement some type of high level API to design and build mission files/AI with. Maybe a library that provides binding to Python or some other higher level and easier to use language.

Python bindings for the API were mentioned earlier in this thread. They're a thing being made.

[leeor_net]

...snip... They're a thing being made.

When can we see the fruits of this labor? We have a habit of "We're doing the thing!" and only 10 years later does it show up.