Author Topic: OP2MapImager Development  (Read 38312 times)

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
OP2MapImager Development
« on: June 03, 2017, 02:30:39 AM »
We have been working on a command line tool for rendering images of Outpost 2 Maps larger than the small thumbnail produced by the current mapper.

Some discussion has been made in the Remote Pair Programming Thread. To keep from crushing the thread on Pair Programming, I'm trying to move information specifically this program's development here.

Project Goal: Create a command line program that can consume and Outpost 2 map and create a custom scaled image of the map in common file formats.

Development Milestones
  • Write library code to load an Outpost 2 Map into a C++ struct that is platform agnostic.
  • Write library code get the tileID of specific locations from an Outpost 2 map.
  • Open the Outpost 2 specific BMP tile set image files.
  • Pull tile image from a the associated tile set BMP using the tileID
  • Choose an image library
  • Use a simple image manipulation library (preferably cross platform) to resize each tile and place it into a larger image file.
  • Allow searching through and pulling required MAP and tile set files from VOL files.
  • Create console line functionality for selecting either a single map, or all maps in the directory. Include basic error handling and final image type selection.
  • Attempt to make the code cross operating system friendly, or perhaps compile it on multiple operating systems.

Code Location: https://svn.outpostuniverse.org:8443/!/#outpost2/view/head/GameResources/OP2MapImager

Implementation Details
 * Repository Type: Subversion
 * Language: C++
 * Windows IDE: Visual Studio 2017
 * Image manipulation library: FreeImage

While Hooman and myself have been working on it so far, if anyone else if interested in helping out, we would be happy accepting the help.
« Last Edit: July 19, 2017, 06:58:05 AM by Vagabond »

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #1 on: June 03, 2017, 02:51:45 AM »
Selecting a library for image manipulation

I've found 2 major candidates so far, CImg and FreeImage.

CImg
CImg (http://cimg.eu/) is an easy library to incorporate, only requiring the addition of a single header file. They claim easy cross platform compatibility. One downside is CImg can only work with the following image formats: RAW, ASC, HDR, INR, PPM/PGM, BMP, PAN, and DLM. I would prefer to be able to output in BMP, JPG, and PNG. CImg can output in JPG and PNG, but it requires the end user to install a cross platform command line tool called Image Magick. Actually ImageMagick supports Unix, Windows, and Macintosh, so I guess it wouldn't work on Linux? ImageMagick's Windows download size is about 24 megabytes.

If we worked with CImg, it would probably be possible to push an error code if the user tried to export a map image to disk using PNG or JPEG telling them they have to download ImageMagick or use BMP.

FreeImage
Another option would be FreeImage (http://freeimage.sourceforge.net/). It supports Linux, Macintosh, and Windows and can work with BMP, JPG, and PNG files without another dependency like ImageMagick. However, I think you have to precompile the FreeImage code on your specific operating system before use. It would probably require some more work to function than using CImg.

.Net
The third option would be .Net. I know that Microsoft has been working to push native compiled versions of .Net for cross platform work, although I'm not sure how mature the effort is and I have a feeling it might take a lot of work and dependencies to get working over a smaller dedicated image manipulation library.



Does anyone have any experience with an image manipulation library they could recommend (or recommend against), or experience with CImg or FreeImage? The library would need to be able to:

* Create an image in BMP, JPG, and PNG
* Resize images to a smaller size
* Add small images to a larger image file (IE lay each tile's image into the larger image)

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #2 on: June 05, 2017, 03:02:42 AM »
Today we settled on using FreeImage for image manipulation. It allows saving in PNG and JPG format without a third party application being installed on the end user's machine.

We wrote functions to pull cell properties on the map based on provided (X,Y) coordinates.

We managed to get FreeImage compiling and interacting with our code base. Basic code is written for saving an image file as a PNG.

Unfortunately, the tile set BMPs in Outpost 2 are not written as standard BMP files. FreeImage had difficulty loading it. The OP2 Mapper project includes code for opening the Outpost specific BMP files (written by Hooman). We didn't have time to extract the required parts yet though.

Current progress has been pushed to the repository.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #3 on: June 08, 2017, 12:47:16 AM »
Over the last 2 days I put some more time into the OP2Imager.

In particular, I put together a first draft of code to accept and parse arguments provided as command arguments by the console.

On a side note, I learned you can add command arguments for debug sessions within Visual Studio:
  • Right click on the console project -> properties
  • Under Configuration Properties, click Debugging
  • Set Command Arguments in the Command Arguments Field.

I also created a library project called OP2Utility. I'm moving all the reusable code to this library so it will be available in future projects. While we could spend a lot of time building relevant code in this library, my plan is to just add the required functions to get the OP2MapImager completed. Once the OP2MapImager is completed, I'd like to create a SVN tag of the utility library and link the OP2MapImager to the tag. This way anyone in the future can modify the working copy of OP2Utility in any way they wish without breaking functionality on the OP2MapImager.

I've located all the required functions to scale and copy/paste image pixels in the FreeImage library. Now they need to be implemented in code and tested.

I'm curious how large a map image we can produce before seeing major performance issues. There should probably be some thought put into limiting the size of a map image to something reasonable. Each Outpost 2 tile is 32x32 pixels (I believe). A 512x256 tile map rendered at 100% would be 16,384 x 8,192 pixels. Not sure that is smart to let the user create an image that big, especially if a batch create is available.

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4954
Re: OP2MapImager Development
« Reply #4 on: June 10, 2017, 10:03:43 AM »
For reference, the NASA Blue Marble site has images of the Earth at 21600x21600 resolution. The files are huge though, and take a while to download and view.

Maybe just let the user choose what they want. They'll know when it's too big.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #5 on: June 12, 2017, 12:46:19 PM »
Hooman and I spent time working on the OP2MapImager last weekend. Most of the time was spent refactoring/fixing the code I had written through the week and developing better implementation details on how the images would be scaled by FreeImage and accepting inputs from the command line.

In particular we:
  • Rooted out problems with how the Library project OP2Utility was referenced by OP2MapImager
    Moved code out of header files and into .CPP files for a couple of classes.
  • Created and then fixed a nasty linker error. I renamed a file from .h to .cpp. Apparently Visual Studio sets a property on the file to indicate if it is compiled as a .h or .cpp file and doesn’t just fix itself when you swap the extension.
  • Removed console application implementation details from the functions that call the MapImager class.
  • Reordered the console argument input.
  • Took a brief look at the structure of the tile set BMPs

We definitely did not accomplish the primary goal of getting the BMPs into memory. However, I think we have a much better handle on how to use FreeImage to scale and create the map renders and have a better setup within the console application.

Hooman also taught me some tricks on properly handling the disposal of objects assigned to pointers and how to create destructors.

I plan on writing another post soon with how we plan to implement getting arguments from the console application. Will be looking for feedback on implementation before finalizing and coding it.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #6 on: June 15, 2017, 03:00:32 AM »
I've been thinking more on how to accept input from the console for the OP2MapImager. This is the first legitimate console application I've helped designed, so please bear with my learning curve.

Application Assumptions
  • OP2MapImager executable must be placed in the same directory as the OP2 Tile Set Bitmaps for the associated map
  • User must unpack tile sets and maps from their associated .VOL files manually. (Plan on eventually automating this)
Currently I was considering something like this
Code: [Select]
OP2MapImager [OutputFileType = PNG [PNG|JPG|BMP]] [PercentScaled = 25[%]] <MapFilename.[map|OP2]>
Examples
    OP2MapImager PNG 25 Ashes.map
    OP2MapImager 50 C:\Users\UserName\Documents\Ashes.map
    OP2MapImager ..\Ashes.map
    OP2MapImager PNG Ashes.map bhsurv.map atwmon.map

After doing some thinking and reading, I would like to change the syntax. I think it would be much easier to parse the data if we use the -Short and --Long syntax for parameters. This would allow the end user to re-arrange the order of parameters at will, and simplifies the code to figure out which paramter they are placing in which order. It also seems more standardized.

Code: [Select]
OP2MapImager [-if = PNG [PNG|JPG|BMP]] [-tw = 8] filename...

USAGE:
-?  --Help
-if --ImageFormat       File format of the output image
-tw --TileWidth = 8    Integer value indicating the length of pixels in each rendered tile.
                        32 = full size. 1 = one pixel per tile.
-od --OutputDirectory   Set a different directory for outputting map renders. Create the directory if required.
-o  --Overwrite = true  Indicates if new renders will overwrite files with the same name. Will output an error if unable to filename is already taken.
-v  --Version           Software version information
My sources for Command Line Input (CLI) Syntax Documentation:

Anyways, I'm open to suggestions on changing any of the syntax.



Formatting Input for Map Scale???

Initially I was thinking of accepting a percentage value for scale. Hooman pointed out this would require handling fractional tile sizes. For example, if the user selected 17% scale, each tile would be 5.44 pixels in length. I would rather not deal with handling fractional pixel placement. So I thought we could just force round the percentage. In this case, you could either force the image to have 5 or 6 pixels per tile. This is not very precise. As a user I would expect to get an image that was exactly 17% in size if that is what they asked for.

So, what if we allow the user to set the pixel width of each tile instead of the scale. In this case you would input 5 and get a render that was 5 pixels in length per tile. There could be 32 discrete scale choices (although the bottom end at 1 or 2 or 3 pixels per tile may be rather useless). This seems cleaner to me than percentages though.

Thoughts?...

Offline leeor_net

  • Administrator
  • Hero Member
  • *****
  • Posts: 2350
  • OPHD Lead Developer
    • LairWorks Entertainment
Re: OP2MapImager Development
« Reply #7 on: June 15, 2017, 06:48:21 AM »
For reference, the NASA Blue Marble site has images of the Earth at 21600x21600 resolution. The files are huge though, and take a while to download and view.

Side note -- it took me about 8 seconds to download one of these giant images. This seems to have more to do with an end user's broadband speed more than the size of the file. ;)

As a thought, couldn't you just have the program output the full size of the map and if the user wanted to scale it down they could do so in an image editor?

Or, you could have the utility render it full and then simply scale it down for output? Any reasonably good image library should be capable of doing this in memory.
« Last Edit: June 15, 2017, 06:50:12 AM by leeor_net »

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #8 on: June 15, 2017, 02:07:56 PM »
leeor_net,

FreeImage does contain scaling functions.

It didn't occur to me to consider scaling the entire map after rendering it. This would allow us to just accept a simple percent to scale by.

Our current plan is to initially scale the tile set images down to the proper size and then place them on a blank bmp that is created at the final render size. I think this method will reduce the memory footprint of the application and the processing time compared to what you are suggesting. Since we do not yet have the program loading the OP2 Tile Set BMPs into memory, we haven't run any tests to see the actual processing time and memory footprint yet. If the differences were not significant, perhaps it would make since to implement what you are suggesting.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #9 on: June 19, 2017, 04:00:15 AM »
I made some progress on the OP2MapImager this weekend. I managed to find a copy of the Tile Set BMPs on this forum that had been resaved in the non-Outpost 2 specific format. I used these images to do some tests with FreeImage. In the process, I learned a lot about BMPs. Fortunately Hooman was already aware of the issues we would be facing with the render and threw out some specific terms during the last pair session that directed my efforts and research. Otherwise, not nearly as much progress would have been made.

It turns out that all the Outpost 2 tile set BMPs are 8bpp and contain palettes that index up to 256 colors for use. Each tile set contains a different set of 256 colors on their palette. FreeImage can only paste pixels from one image to another when using palettes if the two image palettes are identical. Because of this, FreeImage is unable to paste different tiles together even with the standard formatted Outpost 2 BMPs.

SIDE NOTE: The current Tile Set BMPS (well00XX.bmp) packed in maps.vol at SVN location Outpost2SVN\GameDownload\Outpost2\trunk and the primary game download (http://outpost2.net/outpost2.html) contain the old image format. If we could replace these with the non Outpost 2 specific BMP format, it would simplify things somewhat. Also, why don't we have some release log somewhere with what has been changed in Outpost 2 with each release???

Anyways, Gimp can strip palettes from a BMP and export them as 24bit BMPs. This increases their size by about 2.5 times, which is probably why Outpost 2 used the palettes in the first place?

After resaving the BMPs without palettes, FreeImage was able to scale and paste tiles from the different tile sets into the larger render! Unfortunately, it is choosing the wrong tile set and indices for about 1/4 of the tiles on the test map. The test map contains most of its tiles from well0001.bmp, which are the ones that are mostly right. I'm unsure if I'm pulling the data out of the MapData struct improperly or if the MapData struct is pulling data from the .map file improperly (or both)?

After that success, I retested FreeImage, and as long as you set the final output image as 24bits, FreeImage will paste chunks from a 8 bit palettized image without problem. I had tried testing it at 8 and 16 bpp earlier with the palettes, not really understanding what I was doing.

OP2MapImager currently scales each tile set image down, then pastes each chunk onto the final render already pre-scaled. Creating a render at scale factor of 32 (32 pixels per tile) takes about 11 seconds for a 64x256 tile map. Some diagnostic tools of Visual Studio were running in debug mode, which may have slowed it down a little bit. It looked like the bulk of time is spent actually saving the image to disc and not pasting all the tiles in. Final file size for a PNG was about 15MB.

Code has all been uploaded to the repository. It is a moderate disaster right now since I was learning and testing while working.

I wanted to upload the full size render, but the forum limits attachment sizes, so this render is 512x2048 pixels. I think that is about 25% size... Anyways, I'm happy with the results so far, but need to take a break on it for a while. Still looking for comments on how to format the console input (about 3 posts ago in this thread).

« Last Edit: June 19, 2017, 04:09:12 AM by Vagabond »

Offline leeor_net

  • Administrator
  • Hero Member
  • *****
  • Posts: 2350
  • OPHD Lead Developer
    • LairWorks Entertainment
Re: OP2MapImager Development
« Reply #10 on: June 19, 2017, 03:21:24 PM »
This is remarkable progress... Nice work!

Looks like some basic indexing issues to clean up an go from there.

As for switches --

My personal suggestion is to keep it as simple as possible. I would choose a lossless output format and just leave it at that (png). End user can convert it to whatever they want afterward.

I would leave scaling at something like:

Code: [Select]
-scale 1.0

Simple floating point value. 1.0 being full scale, 2.0 being double scale, 0.5 being half scale, etc.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #11 on: June 21, 2017, 07:07:07 AM »
@leeor_net,

Thanks! It sounds like you are comfortable with switches (-S / --Scale) so I will look at going with this format.


I tinkered with the code some today. Looks like I was pulling tile set indices incorrectly from MapData.h. I fixed this and managed to get a perfect render! Below is Twin Valleys in scale factor of 8 (8 pixels per tile).

Unfortunately, I get an out of range on vector access for any map that is not 64 x 256 tiles in size. Well I didn't test every size, but the several I did outside this size didn't work. I'm getting a negative tileInfoIndex value which throws the exception when accessing the TileInfo vector.

I'm guessing the function Hooman and I wrote to translate an (x,y) coordinate into an index within the 1D array may be messed up and only working with this size of map??? The code seems to throw the out of range exception at different (x,y) coordinates based on the map size. I don't have the brainpower to verify/troubleshoot bitwise operations right now (and even if I did, I may not have paid enough attention to what Hooman was teaching me to solve on my own...). Also, this doesn't seem to match up with the culprit, since it is the tileInfoIndex and not the cellIndex throwing the error.

Code: [Select]
size_t MapData::GetCellIndex(unsigned int x, unsigned int y)
{
    unsigned int lowerX = x & 0x1F; // ... 0001 1111
    unsigned int upperX = x >> 5;   // ... 1110 0000
    return (upperX * mapHeader.mapTileHeight + y) * 32 + lowerX;
}

Twin Valleys ( Scale Factor of 8 )


Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #12 on: June 21, 2017, 03:39:19 PM »
Quick update. The problem is associated with certain tile sets. In particular, if the map contains the yellow/orangish sand or glaciers, it throws the exception. Just happens that I got lucky and Rising from the Ashes and Hidden Valley both didn't contain these tile sets within the map itself. When looking at the vector, it looks like within the tileDataVector (list of all tiles displayed on the map) some of the tileDataIndices (the index where the tile's global information is contained with the tileInfoVector) are being set to negative. I'm assuming it is these tilesets, but I haven't dug in and confirmed the correlation. Not sure what is causing the negative values either.

Plymouth 01 ( 64 x 64 tiles, Scale Factor 8 )
« Last Edit: June 21, 2017, 03:42:47 PM by Vagabond »

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #13 on: June 22, 2017, 03:18:03 AM »
Another Update: I found the problem! a couple of variables in the MapData structure needed to be set to unsigned so they could fit all the legal values. It was wrapping the values past their max value to a negative number.

Both glaciers and orange sand are rendering properly now.

I went ahead and generated a PNG image atwmon.map (512 x 256 pixels) at scale factor 32 (100%). It took 41 seconds on my computer in Visual Studio debug mode. My CPU is a little older, so a newer machine and running a final release build will see a little better results. Final PNG file size was 56,109 kb. Rendering it at a more reasonable scale factor of 4 was close to instantaneous.

I tested out JPG export and it worked fine. PNG will still be the default output though.

Now that the program is working as intended, I am considering working some minor polishes like handling overwriting files and errors. The other option would be looking at VOL decompression or cleaning up the console input arguments.

atwmon.map rendered at scale factor of 1 (1 pixel per tile)

Offline leeor_net

  • Administrator
  • Hero Member
  • *****
  • Posts: 2350
  • OPHD Lead Developer
    • LairWorks Entertainment
Re: OP2MapImager Development
« Reply #14 on: June 23, 2017, 11:15:00 AM »
Very nice work! I've been meaning to do something like this but never found the time to do it. So yes, you're better than I and I love it. :D

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #15 on: June 23, 2017, 08:50:15 PM »
Thanks leeor.

I added bool values to allow the program to execute quietly (do not cout string to console) and overwrite (whether to overwrite a previous map render or not). The application now iterates through numbering a file to keep from overwriting. Basically the application appends _X, where X is an integer to keep from overwriting previously saved files if not desired.

All map files in a given directory may now be iterated through by inputting the directory path instead of a file path. I also cleaned up cout messages explaining what the application is performing.

I tested the greenworld tileset and the program had no problems imaging greeworld maps. See Bridge Defense below at a scale factor of 6. I also tried loading a save file (since they share very close file structure to map files). Unfortunately, all values in the map header seem to be set wrong in memory. I didn't troubleshoot beyond this.

Using FreeImage default settings, JPG files are saving at less than a quarter of the PNG sizes, which surprised me. I figured they would be fairly similar in size. I'm not sure without reviewing documentation as to how much compression is going on with the JPGs.

-Brett

Bridge Defence rendered at scale factor 6

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4954
Re: OP2MapImager Development
« Reply #16 on: June 25, 2017, 02:55:08 AM »
Very nice progress on this. I'm thoroughly impressed!

Nice that you caught the signed/unsigned bug.

To get saved games files to work, you'll need to skip over the saved game header. I believe it's fixed size. Read the text file in the OllyDbg folder for details on the file format. You should be able to just seek/reposition the read pointer some fixed number of bytes ahead to get past the header for saved game files, then process the data as usual. If you want to be clever about it, you can also verify the saved game file string marker before seeking ahead.

For PNG versus JPEG, the size difference is likely due to the lossy compression used by JPEG (it throws away data, and hence changes the image). I believe PNG also has lossy compression options, but the default is lossless. That's why things like text and fine lines look so much better with PNG. For natural images it's harder to tell the difference between the two. If you can generate an image with wall tiles, or anything with straight hard edges, you may notice a quality difference between the two formats.

When appending numbers to filenames to avoid overwriting, maybe append the scale factor, or final dimensions. That's the only real input the should change the output, other than actually modifying the source map or tilesets. Considering the nature of the transformation, you probably don't need to worry too much about protecting from overwrites.

Using named switches might make the command line parameter processing cleaner. Remember to consider what happens if a filename just happens to coincide with a switch name. ;)

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #17 on: June 25, 2017, 02:42:59 PM »
As alluded in the other post, OP2MapImager can now take renders of save files. It is a bit silly though since it doesn't include any of the building or units, just the background terrain. So you could just take a picture of the underlying map and get the same results. There could be a fun project involved in parsing out all the information in a saved file and rendering it so you could capture a huge battle or share a high def render of your utopian colony layout. I think this will remain outside the scope of the project for me though.

I finished implementing standardized switch statements. Still some testing to work through though.

Quote
When appending numbers to filenames to avoid overwriting, maybe append the scale factor, or final dimensions. That's the only real input the should change the output, other than actually modifying the source map or tilesets. Considering the nature of the transformation, you probably don't need to worry too much about protecting from overwrites.

Since there is both a grnwld01.map and grnwld01.bmp (tile set), without file overwrite protection, it is entirely possible to make some interesting mistakes. Fortunately, the stock tilesets don't suffer from it.

I'd like to finalize the big picture of the project. Currently, there are 2 projects, OP2Utility and OP2MapImager. OP2MapImager is the actual console application. OP2Utility contains XFile (eventual cross platform file system support) and MapData struct.



OP2MapImager Reusability Concerns

Is anyone possibly considering reusing parts of this code in other projects? If so, what would they/you prefer structure wise from my end.

Option A would be moving the render code over to OP2Utility so anyone could call it and render a map from another project. However, it will carry the dependency to FreeImage into the dependency project, when everything else in the library does not require FreeImage.

Option B would be a third library project that just contains the FreeImage dependency and render code. This way other projects could reference OP2Utility and not worry about FreeImage, or you could reference both library projects if needed. The downside is you now have to hook in yet another library project.

Option C would be to leave all the render code in the console application. I think you could just call the console application in quiet mode from another application to render a map.

Anyways, I'm willing to work any of the three ideas, just curious if there are opinions to weigh in on before I dive in coding.



Render of a save file, MesaMissions (a scenario I've been working off and on for a while)
« Last Edit: June 25, 2017, 02:45:07 PM by Vagabond »

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4954
Re: OP2MapImager Development
« Reply #18 on: June 26, 2017, 04:28:12 AM »
Quote
Since there is both a grnwld01.map and grnwld01.bmp (tile set), without file overwrite protection, it is entirely possible to make some interesting mistakes. Fortunately, the stock tilesets don't suffer from it.

That's one of the reason why I suggested appending an output parameter, such as the scale factor:
grnwld01.01.bmp
grnwld01.32.bmp

People would have to go out of their way to have a problem, which I think is good enough.


Project organization can be a pain. Maybe just move the main method out into a Main.cpp, and let the rest implicitly be library like code. It might not be separated out into a library, but someone could easily cannibalise the project or make such changes if they wanted to.

Offline leeor_net

  • Administrator
  • Hero Member
  • *****
  • Posts: 2350
  • OPHD Lead Developer
    • LairWorks Entertainment
Re: OP2MapImager Development
« Reply #19 on: June 26, 2017, 01:29:07 PM »
I think Hooman pretty well covered it -- PNG is lossless, JPEG isn't. Personally I hate JPEG. PNG can be squashed some using external tools like pngcrush which removes superfluous data.

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4954
Re: OP2MapImager Development
« Reply #20 on: June 27, 2017, 05:06:29 AM »
I was just doing some reading on this and found some interesting stuff. Seems there are lossy pre-processors that take PNG encoding into consideration. It modifies the image before being saved as PNG, so the compressor is able to handle it better and compress to smaller images. The actual PNG encoding process itself is still loseless, but the pre-rprocessing step done before saving is lossy. That would make it pretty similar to JPEG, though with a 2-step process.

On a related note, if you do many edit/save cycles with a lossy encoder, there is generational loss. For a video: JPEG generation loss.

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #21 on: June 28, 2017, 07:05:39 AM »
Hooman and leeor,

Thanks for the input. I'll just plan to leave the render specific code in the console application for now. I'm working through adding VOL decompression right now.

Hooman, interesting video of the JPEG generational loss. Good to think about when choosing a format for a master image or sound file.

-Brett

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #22 on: June 30, 2017, 01:01:33 AM »
Another update,

I have managed to hook Voldecompress code into the MapImager. I fed the code Ply01.map and after not finding the file loose in the directory, the application extracted it from the proper vol file and rendered it.

I added a switch that allows toggling ArchiveAccess. It defaults to true if not included.

I copy and pasted the Voldecompress code into the OP2Utility project under a folder called Archives. As vaguely alluded in the pair programming thread, I was looking at making some changes to Voldecompress.

* Wrapping the Voldecompress library code into  a namespace called Archives. This way it isn't present in the global namespace by default when including OP2Utility. Also, it would help add clarity on what the classes are used for when seen in code.

* Removing the C prefix from all the class names and filenames.

So, you would call Archives::Volfile(filename).

* Adding a function that allows pulling a copy of an archived file into memory without saving it to disk. Currently OP2Mapper saves the map file to disk then opens it. It might be nice if it didn't have to Save the file first.

Thoughts on these changes?

I pretty happy with the progress so far.

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4954
Re: OP2MapImager Development
« Reply #23 on: June 30, 2017, 04:44:09 AM »
Quote
* Adding a function that allows pulling a copy of an archived file into memory without saving it to disk. Currently OP2Mapper saves the map file to disk then opens it. It might be nice if it didn't have to Save the file first.
That actually surprises me a little. The backend was written to support direct loading from VOL files. Though the frontend might just lack the interface to allow that. Guess I didn't really play around with the map editor enough.  ???

Offline Vagabond

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1013
Re: OP2MapImager Development
« Reply #24 on: July 02, 2017, 12:58:32 AM »
@Hooman,

I created a separate thread to discuss modifying the VolDecompress code here: http://forum.outpost2.net/index.php/topic,5973.0.html

I just finished a draft of the OP2MapImager ReadMe file. See below. If anyone notices any grammer issues or other suggestions, please let me know. I've been putting OP2MapImager through its paces, tyring out different combinations of commands and cleaning the code some. There is still a fair amount of work to do cleaning the code (I'm not a great C++ developer either, so clean to me will probably not be clean to someone more experienced).




OP2MapImager Ver 1.0 - Outpost 2 Map and Saved Game Imager
Developed by Hooman and Brett208 (Vagabond)

+++ USAGE NOTES +++
  * Capable of saving multiple map files and entire directories.
  * The OP2MapImager executable and FreeImage.dll must be in the same directory as the tileset BMPs.
  * Scale Factor (-s) determines the final render size and represents the final pixel length of a single tile
    * Max Value: 32, renders at full size, or 32 pixels per tile.
    * Min Value: 1, renders at 1 pixel per tile

+++ EXAMPLE COMMANDS +++
  * OP2MapImager mapFilename.[map|OP2]
  * OP2MapImager -s 16 -o -q Ashes.map eden01.map sgame0.op2
  * OP2MapImager --Scale 8 --ImageFormat BMP [Directory of choice]

+++ OPTIONAL ARGUMENTS +++
  -H / --Help: Displays Help File
  -Q / --Quiet: [Default false] Add switch to run application without issuing console messages.
  -O / --Overwrite: [Default false] Add switch to allow application to overwrite existing files.
  -D / --DestinationDirectory: [Default MapRenders]. Add switch and name of new destination path.
  -I / --ImageFormat: [Default PNG]. Allows PNG|JPG|BMP. Sets the image format of the final render.
  -S / --Scale: [Default 4] Sets Scale Factor of image.
  -A / --AccessArchives [Default true]. Add switch to disable searching VOL archives for map and well files.

For more information about Outpost 2 visit the Outpost Universe (http://outpost2.net/)
Image Manipulation accomplished through FreeImage (http://freeimage.sourceforge.net/)


+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +


+++ INSTALL DIRECTIONS +++

Ensure OP2MapImager.exe, FreeImage.dll, and well0000.bmp through well0012.bmp are all in the same directory. (I typically put them all in the root OP2 install directory for ease of use, but not required.)

Open a command prompt. Navigate to the directory with MapImager.exe and type 'OP2MapImager -h'.


+++ OUTPOST 2 TILESETS (WELL00XX.BMP) +++

The original tileset images shipped with Outpost 2 use a special format that prevents them from being opened by typical pixel image manipulation software (like Gimp or Paint). However, Outpost 2 is capable of reading normally formatted 8bpp indexed BMPs.

OP2MapImager contains a copy of the tileset images that have been reformatted from the OP2 specific format to general BMP format. These may be placed in the root directory of Outpost 2 if you wish. They must exist in the same directory as OP2MapImager.exe and FreeImage.dll.

If you attempt to render a map using the original Outpost 2 tileset images, OP2MapImager will throw an error.


+++ ARCHIVE (.VOL) FILE ACCESS +++

By default, OP2MapImager is able to search through .vol archives and pull maps or tilesets out of the .vol files. OP2MapImager currently leaves the .map file or tileset image file loose in the directory after pulling the file. These loose copies may be deleted manually by the user without affecting the ability of Outpost 2 to run.

OP2MapImager will first search the supplied directory, and if it cannot find the file of interest, then it will search alphabetically through all .vol files in the directory for the given file. So if the file exists both loosely in the directory and in a .vol file, the copy not in the .vol file will be used.


+++ RELEASE COMPILATION INSTRUCTIONS +++

Source code may be found at: https://svn.outpostuniverse.org:8443/!/#outpost2/view/head/GameResources/OP2MapImager.

 1. Set Solution Configuration to Release.
 2. Set Solution Platform as desired. (x86/x64).
 3. Compile Code.
 4. Determine version number for this release.
 5. Grab the following files, and place them in a zipped directory with name format 'OP2MapImager 1.0 x64':
    * OP2MapImager.exe (Release Directory)
   * FreeImage.dll (There are different DLLs for x32 and x64 compilations)
   * Well0000.BMP-Well0012.BMP (Ensure they are the reformated versions that a normal image editor may open.)
   * ReadMe.txt (this file)
   * FreeImage liscense-gplv3.txt
 6. Place the following files in a separate zipped directory with name format 'OP2MapImager 1.0 x64 Debug':
    * OP2MapRender.pdb
   * OP2Utility.pdb
 7. Place both zip files on the Outpost Universe Website.