Author Topic: Proposed Level File Format Mod  (Read 5236 times)

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Proposed Level File Format Mod
« on: February 19, 2015, 08:01:51 AM »
Making new maps for Outpost 2 is a pain. One major reason is the map files also need an associated DLL. I propose a mod to support a new file format to allow all needed data in a single file.

The bulk of the code in DLL files is basically static data, but in code form. Instead, we should design an appropriate data structure to represent that data, and store it as actual data in the map file itself. This is true particularly of initial unit placement during level setup. There was an attempt with the OP2Helper API to convert the raw function calls into data structures, then pass the data structures to a pre-built function to create the initial units. If we extend this idea by moving the pre-built functions into a mod, and the data into the map file, we could eliminate the need for DLLs.

This could be done by appending the data to the map file format, in a new tagged section at the end of the file, and hooking the map loading code to recognize and process such appended data sections after running the already existing map loading code.

The next natural step would be to update the map maker to allow generating and storing this new data in an upgraded map file format. The map maker could then be a self contained utility with no need for a compiler to create playable levels, nor would there be any logic needed in an external C++ project.

I think initial focus should be on level setup, and a later version could include support for trigger callback functions. General scripting support is not a specific goal. That could perhaps be handled in some distant future version, or a DLL could be optionally specified. That use case is not the focus of this project though, so should be given low priority. Highest priority is static initial unit placement, and second might be trigger callback support to do simple things like create disasters.

Offline TH300

  • Hero Member
  • *****
  • Posts: 1404
    • http://op3game.net
Re: Proposed Level File Format Mod
« Reply #1 on: February 19, 2015, 03:54:58 PM »
Is a single file really needed? Or could it be enough if we just had a new file type which replaces dlls? Somewhere (if I remember correctly) you made the suggestion of adding support for level scripts, i.e. human readable files which replace dlls. These both ideas cannot be combined and if I have a choice, I'll vote for scripted levels. Also, the mapper can probably easily be extended to write such script files. Now, scripting support may sound complicated, but if we (for now) confine it to placement of resources and initial vehicles/buildings, should be possible. The difficult part would probably be to allow op2 to load a script file instead of a dll.

I also like the flexibility that we have with the two separate files. It allows us to have several missions on the same map without duplicating the map data for each mission.

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Proposed Level File Format Mod
« Reply #2 on: February 20, 2015, 05:19:23 AM »
I don't believe I ever suggested level scripts in a human readable format. Maybe BlackBox suggested it?

I like keeping things in one file for one simple reason. It makes it dead simple to send a map to someone else.

Having a single file also simplifies the design of a map editor. If the game were ever to support automatically sending map files to people in multiplayer games, it would make that easier too. I'd also like to point out you don't really want to allow automatically sending DLL files for obvious security reasons. A simple data structure, or a human readable script designed to only interact with the game engine would be much safer.

As for duplicating map data, disk space is pretty cheap these days, so I don't think it matters. Plus, you're assuming a map can't contain multiple data sets depending on level parameters. It shouldn't be too hard to place certain sets of units depending on difficulty level, or game type.

At any rate, if arbitrary data structures could be added to map files, it wouldn't be too hard to also support storing a script string. I wouldn't mind if someone wanted to work on such a project. It probably wouldn't be too hard to design something general enough for something like that to be added in. I just don't want to be designing a script engine myself, nor importing an interpreter for an already existing language, and hooking it up to all the in game objects. It sounds like a lot of work. I want something simpler that has a better chance of being finished.
« Last Edit: February 20, 2015, 05:21:27 AM by Hooman »

Offline TH300

  • Hero Member
  • *****
  • Posts: 1404
    • http://op3game.net
Re: Proposed Level File Format Mod
« Reply #3 on: February 20, 2015, 02:13:51 PM »
If you want single files so much, I'd prefer a format that packs all files which belong to a mission into a single zip file (some maps have custom tech trees or even tile sets). This would allow for easy editing of the separate components.

I'm strictly against extending the map file format, because such monolithic map files could only be edited with special tools (even the parts which are pure text). With a format that just zips everything together, no additional tools would be needed (but would still be possible). Its more flexible.

Offline Leviathan

  • Hero Member
  • *****
  • Posts: 4055
Re: Proposed Level File Format Mod
« Reply #4 on: February 23, 2015, 06:30:23 AM »
Agreed, making new maps for Outpost 2 is a pain/time consuming. The map is one project and the dll is another. There are already lots of maps that anyone can use and create a new dll for. Some of the multilayer .maps are fine, they just need a new .dll with balanced starting positions and mines so that the random element is removed from these maps. Often on Flood Plain one person gets a 3bar and the other does not, this give one player a big advantage.

The map maker could sure use some improvement.

I agree with TH300, I like the flexibility that we have with the two separate files. I don't feel there is any need for a single file format.

As for dll and security, is there any way we can tag some data into all dll's that can be read that says "this is a op2 map"? I guess its already there... as the map name data is already in there. And also it would be pointless as the dll could still have security issues elsewhere. I think the way to go about auto sending maps would be to hold maps in a central file store that maps only get added to if they have been checked first. I dont see how sending two files is that much more difficult than sending one file.

What I would like to see added to the .map format is description data; Map name, Author, map description, version etc. As currently all of this is in the .dll.

The issue of new maps being created is not that its too complex, its that there are few people around with the motivation to do so.

We should be working on things that will help bring new users like making the game easy to pick up and play, new content etc.

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Proposed Level File Format Mod
« Reply #5 on: February 24, 2015, 09:19:03 PM »
Quote
The issue of new maps being created is not that its too complex, its that there are few people around with the motivation to do so.
There are few people with the motivation because it's too complex. ;)

There is no way to reliable ensure a DLL is secure. The best you can do is have a central repository of pre-checked code. That is essentially the game download package. There isn't much demand for something better in terms of post-download pre-checked code, since very little gets released, but auto download of updated files would be useful for development and testing. It's just potentially dangerous to do with DLL files.

To be clear, I'm not proposing the removal of support for DLL files. I'm proposing an update that will make DLL files not required for 90% of levels. If you still want a DLL, you can still have it.

The map maker does need improvement. Particularly with auto selection of tiles to fill large areas, being able to set initial units, and being able to configure things like disasters or events after game start.

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Proposed Level File Format Mod
« Reply #6 on: March 03, 2015, 11:39:04 AM »
I'm going with the assumption there is no interest in this idea.

Are there any parts of this idea, or related ideas that interest people? An alternate map format based on zip files maybe?

Offline TH300

  • Hero Member
  • *****
  • Posts: 1404
    • http://op3game.net
Re: Proposed Level File Format Mod
« Reply #7 on: March 15, 2015, 03:14:58 AM »
I'm putting what I suggested on my ideas list. But I can't say when I'll find time/motivation to work on it.

Offline leeor_net

  • Administrator
  • Hero Member
  • *****
  • Posts: 2352
  • OPHD Lead Developer
    • LairWorks Entertainment
Re: Proposed Level File Format Mod
« Reply #8 on: April 06, 2015, 07:27:20 PM »
Not having read the entire thread, I'm on board with map format mods. I think the biggest reason we have issues with lots of new content is because of the fact that you need a compiler and code to make it work. For anybody who's familiar with other games like StarCraft, WarCraft, Age of Empires and more, the need for a compiler is scary for anybody who's not a programmer.

My first thought was to provide basic source code for the maps that don't require scripting and then have the mapper auto compile the DLL's and package things up neatly for testing/distribution.

It's possible even to build basic 'script' functions that non-programmers can piece together kind of like in StarCraft maps. OR, we can find a way to link Lua to the DLL's and allow scripting that way.

Just a few thoughts.

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Proposed Level File Format Mod
« Reply #9 on: April 07, 2015, 11:39:38 AM »
Your reply is encouraging.

I agree that needing to use a compiler is scary for new people. Having to program as well is far worse. Having an editor emit code that can be compiled relieves one of those burdens, but not needing a compiler at all would be much better. If we relied on template code, and hence a compiler, we would either need to ship a compiler with the map editor, which has licensing issues, and would make the download likely huge, or we'd need people to download a compiler separately, which is an extra step in an already complicated process to make new levels.

I like having something integrated like the StarCraft map editor. You edit, you save, you play. Nice and simple. If you want to do something more complicated, that's fine. I don't object to allowing scripting or DLL files. I just want a format where it's possible to play a level without a DLL or script.

I'm thinking maybe some of the OP2Helper code or data structures could be used as a base for starting something here. That project was very much centered on specifying initial game setup as data rather than a series of scripted function calls. Perhaps support can be added to the map loading code to load those structures from the map file instead of compiling the data into a DLL.

Offline leeor_net

  • Administrator
  • Hero Member
  • *****
  • Posts: 2352
  • OPHD Lead Developer
    • LairWorks Entertainment
Re: Proposed Level File Format Mod
« Reply #10 on: October 09, 2015, 05:59:35 PM »
Will have to look into this myself but I like the idea of having a pre-built DLL file that handles all of this with a script that gets loaded by the DLL.

I don't know if we want to consider say, Lua, or just develop our own markup and have it load that way. We can start with basics, specifically unit and structure placement as you suggested and then later we can try to extend the functionality to allow for more complex scripts that could even define AI behavior.

I'm down for helping with this once I start getting through the API code and have a better understanding of it all.

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Re: Proposed Level File Format Mod
« Reply #11 on: October 10, 2015, 03:28:28 AM »
Ahh yes, I've been thinking of this again. I was thinking maybe go as far as trying to write a generic hook, but leaving the data for now unspecified. The data wouldn't be a whole lot of use until people had a way of editing it and adding it to the levels.

Offline leeor_net

  • Administrator
  • Hero Member
  • *****
  • Posts: 2352
  • OPHD Lead Developer
    • LairWorks Entertainment
Re: Proposed Level File Format Mod
« Reply #12 on: October 10, 2015, 11:22:35 AM »
I'm game. You're a lot more familiar with the OP2 code than I am. Until I start working with it this week I doubt I'll be much use. On the other hand if we want to develop our own markup I can develop the lexical parser. Then again, if we want to use a full blown scripting language I'm still rooting for Lua.