File Formats

[Hooman]

Outpost 2 Tile Set file format
------------------------------

Note: This refers to the "well00??.bmp" files found in maps.vol.
Note: Tile sets must be in either 1 bpp, 8 bpp, or 16 bpp color modes. Others will cause the load to be aborted. (But 16 bpp seems to cause crashes, probably 1 bpp as well)
Note: The tile set files can be replaced by standard bitmap files.
Note: The bitmap data should contain a column of all tiles.


Offset    Size    Description
------    ----    -----------
0x0    4    "PBMP"
0x4    4    sectionSize (size of file - 16)
---------------------------
0x8    4    "head"
0xC    4    sectionSize (0x14 - 5 DWORD) Note: sectionSize bytes are read
      into an unchecked buffer of size 0x14 bytes (5 DWORDs).
      Any more data will overflow the buffer and corrupt the stack.
      (*Possible exploit*)
- - - - - - - - - - - - - -
0x10    4    numTagsLeft (2) - number of tags left (at this nesting level)
- - - - - - - - - - - - - -
0x14    4    pixelWidth (32 pixels wide)
0x18    4    pixelHeight (height of all tiles together)
0x1C    4    bitDepth
0x20    4    flags (always 8?) **TODO** figure this out
---------------------------
0x24    4    "PPAL"
0x28    4    sectionSize (1048 - size of subsections and sebsection data)
---------------------------
0x2C    4    "head"
0x30    4    sectionSize (4) Note: only 4 bytes of section will be read
- - - - - - - - - - - - - -
0x34    4    numTagsLeft (1) - number of tags left (at this nesting level)
---------------------------
0x38    4    "data"
0x3C    4    sectionSize (1024) - size of palette data
- - - - - - - - - - - - - -
0x40    1024    palette data - array of 4 byte palette entries (BGR order?)
---------------------------
0x440    4    "data"
0x444    4    sectionSize
- - - - - - - - - - - - - -
0x448    1024*X    tile data - pixel data for tiles (32x32 pixel tiles), 8 bits per
      pixel, indexing into the palette stored earlier in the file




Loading Notes:
--------------

The above format is not absolutely required. The file is processed by loading tags
and then processing the following block of data according to the tag type. There
are two levels of nesting in the file structure. Reading a "PPAL" enters the next
nesting level. Each time a section tag is read, a count of remainings tags to read
is decremented. When the count reaches 0, processing returns to the previous level.
If the count on the outer level reaches 0, the load routine returns indicating a
successful load. Any read errors cause an immediate abort (such as EOF).

Note:
The count is initialized to -2 in Outpost2.exe. A count value is -1 is reserved
for error checking and could cause an abort (seems like useless error checking
that could never execute). To ensure the first few tags are read, a "large enough"
initial value is needed, and so -2 is the farthest value (when decrementing) from
0 that is allowed.

The first level scans for the following tags:
    "PBMP"  - ignored
    "PPAL"  - enters second level, if ((functionParam2 & 1) != 0)
        **TODO** find out what happends for alternate parameter
        Note: Seems functionParam2 is hardcoded to 1 tile set loads
    "head"  - reads sectionSize bytes into array (only space for 0x14 bytes)
    "data"  - allocates bitmap space and reads in bitmap data
    "BM"*
(* - The last 2 bytes of the "BM" tag are not considered (wild cards). A check
 for this tag is only performed under certain conditions. Namely, the number
 of tags left to read at this nesting level is >= 0.)

During the second level (after reading a "PPAL" tag) scans for the following tags:
    "PPAL"   - ignored
    "RIFF"
    "head"  - reads numTagsLeft (only 4 bytes are read, ignores sectionSize)
    "data"  - allocates palette space and reads in palette data
    "pspl"
    "ptpl"



Illustration of numTagsLeft
---------------------------
"PBMP"
"head"   (2 sections left at this level)
"PPAL"
    "head" (1 section left at this level)
    "data"
"data"


Logical structure of file
-------------------------
There is a logical distinction between tags consisting of all upper case letters
and tags consisting of all lower case letters. Upper case letters detnote the start
of a major section of the file and all subsequent lower case tags belong to that
section (keeping nesting levels in mind). The first lower case tag in a major
section is the "head" tag. The fields following this tag are associated with the
major section. The numTagsLeft is expected to follow a "head" tag, but other fields
are dependent on the major section type. Data for a major section follows
subsection data.


Note: The logical structure of the file doesn't really matter to the loading code.
Note: If a section appears twice (and the numTagsLeft count is adjusted to
      compensate), then the previously read values are overwritten.
Note: Multiple data sections will cause memory leaks. There is no check when a
      data section is encountered as to whether or not memory has already been
      allocated. New memory is allocated, the data is read into the allocated
      memory, and the pointer to the returned memory is stored, overwriting any
      previously stored memory pointer.
Note: The first level scan doesn't not so a size check on the "head" sectionSize
      when reading the "head" fields. It is assumed there will not be more than
      0x14 bytes of data to be read. (**Routine is Exploitable**)

[Hooman]

Outpost2 CLM file format
------------------------

Offset    Size          Description
------    ----          -----------
0x0   28 (0x1C)      "OP2 Clump File Version 1.0", 0x1A, 0 (Case Insensitive compare?)
0x1C   4         0x00000000 (Alignment?)
0x20   18 (0x12)      WAVEFORMATEX structure
 0x20   2          WAVEFORMATEX.wFormatTag    = 1   (WAVE_FORMAT_PCM)
 0x22   2          WAVEFORMATEX.nChannels    = 1   (mono)
 0x24   4          WAVEFORMATEX.nSamplesPerSec    = 22050 (22.05 KHz)
 0x28   4          WAVEFORMATEX.nAvgBytesPerSec      (44.1 bytes/sec = nSamplesPerSec*nBlockAlign)
 0x2C   2          WAVEFORMATEX.nBlockAlign    = 2   (2 bytes/sample = nChannels*wBitsPerSample/8)
 0x2E   2          WAVEFORMATEX.wBitsPerSample      (16 bits per sample)
 0x30   2          WAVEFORMATEX.cbSize         (0 = Size of format data following WAVEFORMATEX structure)
0x32   2         0x00 (Alignment?)
0x34   4         trackSizeAlignment (65536) (abs of this value must be 0 mod 65536)
0x38   4         numPackedFiles
0x3C   16*numPackedFiles   IndexEntry[] - table of index entries describing all files stored in the archive
    8          FileName - Internal file name
    4          Offset - offset to file data
    4          Size - size of stored file data
0x3C+16*numPackedFiles      Raw file data





Loading Notes
-------------

Outpost2.exe reads the first 0x46 (76) bytes of the file and processes it.
Eventually, the total size of the file header is calculated and the already
read data is copied to the new buffer and then the rest is read in from the
file. (The IndexEntry[] array is included in this data.)

The header string is first verified ("OP2 Clump File Version 1.0", 0x1A, 0)
and reading is aborted if the string does not match. The string compare
function uses a global variable to determine if the compare is case
sensitive. (Or other format conversions?)

The WAVEFORMATEX structure is then checked for validity. The following
fields are checked for the following values:
 WAVEFORMATEX.wFormatTag    = 1
 WAVEFORMATEX.nChannels      = 1
 WAVEFORMATEX.nBlockAlign   = 2
 WAVEFORMATEX.nSamplesPerSec   = 22050

At this point, the total header size is calculated using the numPackedFiles
header entry and memory is allocated to hold the entire header. The data
that was already read in is copied to this buffer and the remaining data is
read in from the file.

A file index table is initilzed. It contains 0x1A (26) entries. All values
are initialized to -1. The table is filled in with the internal CLM file
index of each of the following files. If any of these files do not exist
in the CLM archive, the load is aborted.
 0) "EDEN11"
 1) "EDEN21"
 2) "EDEN22"
 3) "EDEN31"
 4) "EDEN32"
 5) "EDEN33"
 6) "EP41"
 7) "EP42"
  "EP43"
 9) "EP51"
 10) "EP52"
 11) "EP61"
 12) "EP62"
 13) "EP63"
 14) "PLYMTH11"
 15) "PLYMTH12"
 16) "PLYMTH21"
 17) "PLYMTH22"
 18) "PLYMTH31"
 19) "PLYMTH32"
 20) "PLYMTH33"
 21) "STATIC01"
 22) "STATIC02"
 23) "STATIC03"
 24) "STATIC04"
 25) "STATIC05"
These files are searched for in a linear order. (Not a binary search, so
alphabetical order is not required for this check.) Also, there is no
check to exlcude files not in this list from being packed in the CLM
archive.



Sound Initialization Notes
--------------------------
Outpost2.exe will attemp to initialize the buffer format of the primary sound
buffer in the following order.
 1)
   WAVEFORMATEX.wFormatTag      := 1       (WAVE_FORMAT_PCM)
   WAVEFORMATEX.nChannels      := 2      (Stereo)
   WAVEFORMATEX.nSamplesPerSec   := 22050   (22.05 KHz)
   WAVEFORMATEX.nAvgBytesPerSec   := 88200   (88200 bytes/sec)
   WAVEFORMATEX.nBlockAlign   := 4      (4 bytes/sample, including both channels)
   WAVEFORMATEX.wBitsPerSample   := 16      (16 bit sound samples)
   WAVEFORMATEX.cbSize      := 18      (18 bytes following WAVEFORMATEX structure,

 2)
   WAVEFORMATEX.wFormatTag      := 1       (WAVE_FORMAT_PCM)
   WAVEFORMATEX.nChannels      := 2      (Stereo)
   WAVEFORMATEX.nSamplesPerSec   := 22050   (22.05 KHz)
   WAVEFORMATEX.nAvgBytesPerSec   := 44100   (44100 bytes/sec)
   WAVEFORMATEX.nBlockAlign   := 2      (2 bytes/sample, including both channels)
   WAVEFORMATEX.wBitsPerSample   := 8      (8 bit sound samples)
   WAVEFORMATEX.cbSize      := 18      (18 bytes following WAVEFORMATEX structure,
                     but should be ignored for WAVE_FORMAT_PCM)
 3)
   WAVEFORMATEX.wFormatTag      := 1       (WAVE_FORMAT_PCM)
   WAVEFORMATEX.nChannels      := 1      (Mono)
   WAVEFORMATEX.nSamplesPerSec   := 22050   (22.05 KHz)
   WAVEFORMATEX.nAvgBytesPerSec   := 22050   (22050 bytes/sec)
   WAVEFORMATEX.nBlockAlign   := 1      (1 bytes/sample, including both channels)
   WAVEFORMATEX.wBitsPerSample   := 8      (8 bit sound samples)
   WAVEFORMATEX.cbSize      := 18      (18 bytes following WAVEFORMATEX structure,
                     but should be ignored for WAVE_FORMAT_PCM)
Summary:
 1) Frequency = 22050 Hz, Channels = 2 (Stero), BitsPerSample = 16
 2) Frequency = 22050 Hz, Channels = 2 (Stero), BitsPerSample = 8
 3) Frequency = 22050 Hz, Channels = 1 (Stero), BitsPerSample = 8

[Hooman]

Outpost 2 Saved Game File Format
--------------------------------

Note: Saved games share much of the same loading code as the .map file format.
They consist of a prefix to the .map format and (possibly) some data near the end.



Outpost 2 .op2 File Format (Saved Games)
----------------------------------------

Offset    Size    Description
------    ----    -----------

0x0   0x19   "OUTPOST 2.00 SAVED GAME", 0x1A, 0 - String must match exactly or else load error
---------------------------
      Note: The following is repeated as an array of 0x74 (116 decimal) elements
0x19+i*0x1E0    4    **TODO** Figure this out
0x1D+i*0x1E0    0x1DC   **TODO** Figure this out
---------------------------
      Note: From here, the format matches that of .map files





Outpost 2 .map File Format
--------------------------

Note: Logs discussed below are all base 2 logarithms.



Offset    Size    Description
------    ----    -----------
      Section: Header
0x0    4    tag (must be >= 0x1010 or map is not loaded)
0x4    4    boolNotSure (for .map, but not saved games? flag...?)
0x8    4    logMapTileWidth (lg(mapTileWidth)
0xC    4    mapTileHeight - this gets rounded up to a power of 2
0x10    4    numTileSets
---------------------------
      Section: Tile Data
0x14    X    tileData - array of DWORDs of size mapTileWidth*mapTileHeight
         The tileData consists of the following bit fields
         5   cellType - determines speed of movement, bulldozed, walled, tubed, rubble areas (see Note below) (returned by GameMap.GetCellType)
         11   tileIndex - this is an index into the tileInfo array described below (see Tile Set Section)
         11   unitID - unitID (index of unit into unit array) of the unit that occupies this tile
         1   lavaPresent
         1   lavaPossible
         1   expansion (lava or blight about to expand to here?)
         1   microbePresent
         1   unitImpassable - Set if a wall or building has been built on this tile
---------------------------
      Section: ClipRegion - unit can only move within this map area
X+0x14    4    clipRegion.x1 - internal coordinate of left edge (0x20 normally, or -1 for around the world maps)
X+0x18    4    clipRegion.y1 - topmost edge (0 = "1"-1)
X+0x1C    4    clipRegion.x2 - rightmost edge (mapTileWidth + 0x1F, internal coordinates of rightmost tileX) or (0x7FFFFFFF if logTileWidth >= 9, around the world maps)
X+0x20    4    clipRegion.y2 - bottommost edge (tileHeight-2 = (tileHeight-1) - 1)
---------------------------
      Section: Tile Set Sources
X+0x24    Y    tileSetSource table (refers to the "well00??.bmp" files)
         The tile set source table has up to 3 fields for each entry of the table
         4    stringLength - length of filename
         8    tileSetFilename
         4    numTiles - number of tiles in this tile set
      Note: The last two fields will not exist if the first field, stringLength == 0.
      The game will attemp to read numTileSets (see Header Section) copies of these fields.
      It will read 16 bytes for the fields were a string exists (stringLength != 0)
      and 4 bytes where the string doesn't exist (stringLength == 0).
---------------------------
         Section: Tile Set
   0xA       "TILE SET", 0x1A, NULL (10 characters, 0x1A, NULL terminated string) - Loading fails if string doesn't match
   4       numTiles - Specifies how many tiles from the loaded tile sets are referenced
   8*numTiles   tileInfo array - tileData array (see Tile Data Section) indexes into this array to find tile graphic
            The tileInfo array consists of an array of the following structure
            2   tileSetIndex   - index of tile set (see Tile Set Sources Section)
            2   tileIndex   - index of the tile in this tile set
            2   numAnimations   - number of following tiles that can be displayed to replace this one (See TileSet7 for more details TileNum 0) or CodeSection 0x0047030D
            2   animationDelay   - number of cycles elapsed before displaying next tile (See TileSet7 for more details TileNum 0)
   4       numTerrainTypes
   0x108*numTerrainTypes   Array of TerrainType objects - (list of tile indexes, provides mappings to new tile indexes when graphics change. eg. bulldozed, tube/wall built)
            2   firstTileIndex   - first tile index to which this list applies
            2   lastTileIndex   - last tile index to which this list applies
            2   bulldozedTileIndex   - tile index to set when tiles in this group are bulldozed
            2   rubbleTileIndex - index of common rubble tile (4 consecutive common rubble tiles followed by 4 rare rubble tiles)
            2*6   tubePartial[6]   - index of tube tiles (data is repeated **TODO** find out why)
            32*5   wall[5][16]   - arrays of wall tile groups (16 tiles in each array) (lava, microbe, normal, light damaged normal, heavy damaged normal)
            2   lavaStartIndex      - starting lava tile index
             6   LavaInfo
              2    animationLength   - [6]
              2    centerAnimationLength   - [6] [Rand(value-1)]  [numCenterTiles?]
              2    numCenterTiles      - [1] [centerAnimationLength?]
            32   tube[16]   - array of tube tile indexes (for tubes facing each direction)
            2   scorched   - scorched tile index (from vehicle explosion)
             12   scorchedRange[3]
              2    startTile
              2    endTile
             30   unknown[15]   - **TODO** find out - various tile indexes, or 0  [unused?]
---------------------------
   4   tag - must match tag at first DWORD of .map file else error (see Header Section)
---------------------------
      Note: extra data can be read here, depending on the parameters to the load routine.
      .map files shouldn't load any data here, probably only saved games (which do seem to load extra data here)
   4   numUnits
   4   **TODO** figure out what this is
   4   **TODO** figure out what this is
   4   **TODO** figure out what this is
   4   sizeofUnit - 0x78 (120 bytes) if sizeofUnit != 0x78 && numUnits != 0 then Error
- - - - - - - - - - - - - -
      Note: Reading continues here in a subroutine at 0x00446CF0
   4   numObjects1 (number of 512 byte objects)
   4   numObjects2 (number of 4 byte objects)
      Note: if (numObjects1 == 0) return (successfully)
   512*numObjects   objectArray **TODO** find out what this object is
      Note: if (numObjects2 == 0) return (successfuly)
   4*numObjects2   objectArray **TODO** find out what this object is
      Note: End of subroutine read
- - - - - - - - - - - - - -
   4   unitID - **TODO** find out significance
   4   unitID - **TODO** find out significance
   0x1DF88   unitRecord[1023] - 1023 unit records of 120 bytes each
      Note: End of "optional" data (not really optional for saved games files)
---------------------------
   4   tag - must match tag at first DWORD of .map file else error (see Header Section)
---------------------------
      Note: End of .map load.
      Extra data seems to follow the last read. Appears to be tagged data. (eg. "LAVA_LEFT", "midd_lava", "RIGHT_LAVA", etc.)
      This could potentially have been used by a map editor.
- - - - - - - - - - - - - -
      4      int numTileGroups ()
      4      
   A list of the following items is found
      4      int xSize
      4      int ySize
      4*xSize*ySize   int data[xSize][ySize]
      4      stringLen
      stringLen   string
- - - - - - - - - - - - - -
      End of .map file
      



Loading Notes:
--------------
The numTiles in the Tile Set section are read into an array, processed, and then released from
memory. Processing differs depending on whether or not numTiles == 0. The
element at index 7 (the lava tile set) is special cased.


Note: cellType (5 bits) has one of the 32 following meanings, as can be found at [0x4DEBA8]:
Value       Meaning
--------    -------
0 (0x0)    Fast Passible 1      - rock vegetation
1 (0x1)    Impassible 2      - meteor craters, cracks/crevasses
2 (0x2)    Slow Passible 1      - lava rock (dark)
3 (0x3)    Slow Passible 2      - rippled dirt/lava rock bumps
4 (0x4)    Medium Passible 1   - dirt
5 (0x5)    Medium Passible 2   - lava rock
6 (0x6)    Impassible 1      - dirt/rock/lava rock mound/ice cap/volcanoe
7 (0x7)    Fast Passable 2    - rock
8 (0x8)    North Cliffs      -
9 (0x9)    Cliffs - High side   -
10 (0xA)    Cliffs - Low Side   - cliffs, low side (and middle too)
11 (0xB)    Vents and Fumaroles   - fumaroles (only passable by GeoCons/RoboMiners?)
12 (0xC)    zPad 12
13 (0xD)    zPad 13
14 (0xE)    zPad 14
15 (0xF)    zPad 15
16 (0x10)    zPad 16
17 (0x11)    zPad 17
18 (0x12)    zPad 18
19 (0x13)    zPad 19
20 (0x14)    zPad 20
21 (0x15)    Dozed Area       - bulldozed, non tubed area (buildings are tubed underneath)
22 (0x16)    Rubble
23 (0x17)    Normal Wall
24 (0x18)    Microbe Wall
25 (0x19)    Lava Wall
26 (0x1A)    Tube0          - tubed (buildings are tubed underneath, including docks)
27 (0x1B)    Tube1
28 (0x1C)    Tube2
29 (0x1D)    Tube3
30 (0x1E)    Tube4
31 (0x1F)    Tube5

[BlackBox]

Btw, found a little more about that crap at the beginning of the saved game file.

1 byte past the end of the ASCII string at the top there is a 2 byte tag. about 20-30 bytes down (mostly a block of nulls) the tag repeats again. (Apparently the game checks this, I changed it before I realized it's purpose and the game didn't show up in the list). (I didn't look at the code, too lazy to, this was the result of mucking with a hex editor and randomly changing bytes. )

Right after that is picture data. It's definitely not an array of RGBQUADs or anything like that, from what I see (the colors weren't changing to what I expected, and by checking them in a simple test app I did in VB they didn't come out to expected colors on the map.
My guess is that they are indexes into palettes somewhere. (Probably the PRT ones, or maybe there are palettes contained in the save file itself - not likely though because I couldn't find evidence of RGBQUADs anywhere).

Or maybe it is raw color data, "mangled" by the game (as we all know, how the game has those oddities to it). That would explain how it can draw the colors for units/structures, and how it can save the tactical view mode of the minimap. (It saves whatever you see on the minimap, maybe even just a memcopy from the minimap buffer space to the stream - it doesn't save the globe mode view obviously though)

The arrangement is weird too, it seems to be interlaced (skips every other pixel, found that by hacking the values) but this isn't consistent (it jumps around to other pixels in odd places)

There are some flags at the top of the file, they seem to control colors as well. (I remember a long time ago, I edited something and the colors became inverted. I don't think I have that savefile anymore though).
 

[Hooman]

Yes..., I'll eventually want to complete that saved game/map file format. Anyways, here's my two cents on the data.

It does seem kinda likely that it's just a memcopy of the bitmap. In which case, it would probably use 16 bit color. So no RGBQUADs, and every 4 bytes will be 2 pixels, hence possible skipping. Still, it's all just a guess. After that possibility, I assum it would just recalculate/redraw it from the data. Probably not though.

What about breaking it into columns like the map tile data? If it's not a flat bitmap, it could be broken into 32 pixel wide columns or something.

The flag info could be useful. I'll see when I can find some time to look into this stuff. Finished my hardest exam today. Weut! Actually, I thought it was gonna be a gong show and was half tempted to try throwing up in the middle so I could be excused and have it deferred. It actually went alright by the end though. No idea how, but it actually went well. Gee, glad I didn't waste too much time studying for it.  

j/k