Expanding OP2Archive to include .PRT/.BMP extraction & creation

[Vagabond]

OP2Utility contains enough functionality to extract the contents of a .PRT file and .BMP file although it needs to be cleaned up a bit. We haven't decided on a front-end application yet. I was considering adding it to OP2Archive and thought this was a better idea than having another application floating around.

Looking for input from the community if anyone has preferences or ideas. Especially if anyone is interested in modding sprites for use with Outpost 2.

We would want to decide on a format for extracting the metadata contained in the .prt file. Hooman has mentioned JSON in the past which seems reasonable. I might have considered XML since it is popular. Whatever format we choose will likely require adding a third party library to OP2Utility. I'd rather not write a parser from hand unless people are against using a third party library. We would need to find one with an agreeable license.

If we want to go with JSON, might want to consider this one: https://github.com/nlohmann/json. It has an agreeable license, is popular, and is in active development. Not sure how I feel about everything being in one header file though.

Phossy has already accomplished .PRT/.BMP extraction and packing here: https://github.com/phossy/op2art. Basically I am repeating functionality using C++ and consolidating it into OP2Utility. Maybe not the best use of time. I believe this program uses YAML for the data.

Console Commands

The console commands get a bit weird since you typically pair op2_art.prt and op2_art.bmp together. The following assumptions might ease the command line interface:

 * The prt and bmp file contain the same filename and must have the extension of prt and bmp
 * The two files must to be in the same directory

One could have to specify the PRT file for OP2Archive to understand that you want to unpack:

Code: [Select]

# Extract everything
OP2Archive EXTRACT op2_art.prt

# Extract specific image (no metadata parsed)
OP2Archive EXTRACT op2_art.prt --image 1234

# Extract a couple of images (no metadata parsed)
OP2Archive EXTRACT op2_art.prt --image 1234 2345 3456

I'm not sure about adding special extraction for a specific animation or animation frame. It would require pulling some specific metadata and a presentation format for the pulled bitmaps. Maybe once basic extraction is working we can explore what works or doesn't work here.

-Brett

[Hooman]

I was actually using https://github.com/nlohmann/json for another project I recently started and haven't published yet. That one's got my vote. 


I was expecting a new command line utility for this. It never occurred to me to consider those two combined files as an archive. It actually kind of makes sense. Though it may have the downside of other people not thinking to use an archive program to extra sprite images. I think my preference is for a new standalone tool.

I think the goal of doing sprite sheet extraction goes beyond what other tools currently provide. I'd say that alone makes it worth it. Though you may have a point about priorities, and where we should best spend our limited resources of time and energy.

As for options, that seems like a reasonable start. I'd say allow a resource folder to be specified, which might default to the current folder. Allow the PRT and BMP filenames to be individually specified (relative to the resource folder), which would default to the standard filenames.

From there you can extract Layers, Frames, and Animations.

Internally, Layers are closest to the actual bitmap data, but don't generally represent an entire graphic. You need to compose several layers into a Frame to get what people would view as a complete image of a unit. You could then have a vertical strip of Frames forming an Animation sequence.

Animations sequences in particular should have metadata associated with them, describing the unit, the action, the direction, and the number of frames. I believe JSON would be good for that data.

My preference here is for JSON. XML seems more suitable for human readable content that has bits of machine readable markup. Think HTML. For computer processing of largely machine readable data, I think JSON is more suitable.

[Vagabond]

I am hoping to limit the number of applications we have floating around. What if we rename OP2Archive to something else that would seem more inclusive of VOL, CLM, PRT, and the weird BMP file that Outpost 2 uses? Maybe OP2ResourceSerializer or OP2DataManager?

The other random side project I had targeted for this application is converting tilesets (wells) into and out of the Outpost 2 specific format. I think this code is already written in OP2Utlity and would just need to be expanded a bit and exposed as a console command.

Would we prefer to serialize the data as a function of the library (OP2Utility) or the console application? Completing the serialization in the application would keep us from adding a dependency to OP2Utility. However if other applications want access to the serialization process it would be easier if the serialization is defined in OP2Utility. I would probably lean a little bit towards putting it in OP2Utility, but not committed either way.

I'm happy proceeding with the op2art project existing, just wanted to bring up the other application so others were aware. For lots of modding, a more fine grained tool would be nice, but if someone just wants the base data, it would probably be easier to take it in YAML format and massage for their purposes.

-Brett

[Hooman]

I am hoping to limit the number of applications we have floating around. What if we rename OP2Archive to something else that would seem more inclusive of VOL, CLM, PRT, and the weird BMP file that Outpost 2 uses? Maybe OP2ResourceSerializer or OP2DataManager?

I'm curious why? I suppose my concern on reading that, is having one big monolithic program that tries to do everything, and possibly does it poorly. Either that, or functionality becomes unclear, and the utility becomes so generically named, it becomes impossible to reason what it might be capable of.

I think I would prefer keeping JSON/YAML/XML serialization out of OP2Utility. It's not really related to Outpost 2, at least in the sense that those formats would not be usable by Outpost2.exe. It seems more suitable to have such code in an external data conversion utility. I also like your point about not adding a dependency to OP2Utility.

One caveat, is there may need to occasionally be a bit of support code added to OP2Utility. Either new features, or code that might be more generally useful when working with the data in the native Outpost 2 format.

[leeor_net]

Will agree with Hooman that we should have individual programs that each do one job and do it well versus having one giant program that tries to do everything or more than one thing at a time.

Smaller programs that do an individual task are easier to develop, easier to debug, easier to maintain and easier to document.

If the concern is "Well the user might be overwhelmed with too many programs with too many options", don't worry about that. If a user is going to be picky about that they have no business using CLI programs to begin with.

Also, later on down the line a front-end GUI type application can be developed which handles the CLI stuff for users. This can be the tool that uses multiple smaller CLI applications to do its tasks. Think IDE's and compiler suites. One program (IDE) interfaces with many smaller programs to do the tasks the developer needs.

Same deal when I used to develop maps for Quake/2/3 -- you'd use the GUI map builder which would then invoke three different compiler tools (map to bsp, bsp vis generator and finally radiosity lighting tool) before calling up the game itself with the newly compiled map.

[Vagabond]

I think describing having one console application that extracts and packs Outpost 2 related resources as big and monolithic is stretching it. The instruction set for OP2Archive is currently a single page on the console and very verbose.

Thinking of my use cases, I'm not sure I want to extract op2_art.prt from the volume with OP2Archive, manipulate it with this new application, then reuse OP2Archive to push it back in the volume.

This new tool will be extracting and packing resources for Outpost 2, which is pretty much exactly what OP2Archive is already doing.

We already have 2 console applications and a library (not including the half dozen legacy applications for resource management). If we add a separate data conversion utility library and a sprite specific extraction application, that brings us to 5 projects. Including all the projects involved in the extension modules, op2ext, forced exports, Outpost2DLL, OP2Helper, HFL, HFL-IUnit, template scenarios, custom levels, some sort of future git project that unifies everything for easy level creation because there are so many projects, we are out the ears in projects. We cannot even keep all the project files up to date.

We are more like a jigsaw puzzle than a monolithic application.

Guess I'm ranting some.

Is there anyone actually using OP2Archive and/or has interest in modding the in game sprites that has an opinion from a user and not programmer perspective? Outside of a dependencies argument, it might be helpful to have their opinion.

-Brett

[Arklon]

I prefer YAML over JSON where the file is largely intended to be human-edited rather than generated output, but it's more heavyweight as it's pretty much an extension of JSON with more rules to make it more condensed. The main issue with that though is there's not really any good third party standalone C++ YAML libs - by far the best one I've seen is the one used internally in LLVM, but you'd have to do some work to separate it out (which is doable because the YAML portion doesn't have very deep dependencies into LLVM's internals, just Support and ADT).

[Hooman]

When I first moved to Git, I thought the number of projects felt a bit excessive. I suppose it feels normal now.

Would be good to ask for some end user feedback. I'm actually kind of curious if there are non-programmers using these tools.

I wouldn't count the libraries though, if you're taking an end user perspective.


For my projects, I intend to use JSON. I've already started on something, and found a library I like. If someone wants to put in work to support other formats, such as YAML, I'm fine with that.