Outpost Universe Forums

Projects & Development => Outpost 2 Programming & Development => Topic started by: RobbixTheBobbix on August 24, 2011, 06:44:01 PM

Title: Moving Pictures
Post by: RobbixTheBobbix on August 24, 2011, 06:44:01 PM
Project of mine I've been working on for a few months.

Youtube clip of demonstration (http://www.youtube.com/watch?v=UXcpQl71G9s)

Google Code project where you can download the work in progress (http://code.google.com/p/moving-pictures-op2/)

Requires Java 1.6, might run in 1.5, haven't tried it. It's packaged on the Google Code page as an Eclipse project, so you can import it and have your way with it. Runs on windows, should run on linux and mac.

The video was recorded on my circa 2005 laptop where I'm doing the programming and it can run smoothly and a couple times faster than the nominal speed, despite Java. When I tried it at work, it ran at about 90% the nominal speed and was sketchy despite it being a more powerful circa 2008 machine.

Results may vary.

The interface will be kind of awkward until I get all the command buttons working - they're a fairly new addition.

* Left click units to select them.
* Left click a space to give a move command.
* Right click to deselect or cancel a unit placement command.
* Middle click the red command names on the edge of the screen to give commands.

uh....

Questions?

-----------------------------------------------
PS:

I don't know if there's going to much excitement about this as I've read about the repeated, crippling disappointments that have come from op2 clone projects in the past. I intentionally didn't post about it until I had gotten somewhere.

I have a habit of working on proofs at concept at work over lunch or just whenever I don't feel like working. I have PoCs concerning

* Map scrolling (current maps are only the size of the screen)
* Terrain editing and automatic tile annotation (related to map generation where tiles that go together have to be automatically selected)
* Tube connectivity <- works like a charm
* Networking (which is actually far-off, but I'm trying to think ahead structurally)
* More advanced pathfinding and automatic map annotation
* There are some others, I'm not at work.

--------------------------------------------------
PPS - About contributing:

So far, I've been extracting animation sequences from the op2art program into individual files - one frame per file - and writing info.xml files that contain information about them.

This is very tedious.

It also runs slow as the program has to load and read a couple 1000 individual files each time it starts.

So I might be asking for help with do the extracting and writing the info.xml files or I might be asking for help in loading the original op2_art.bmp/op2.prt files. I designed Moving Pictures to be modular enough that changing the sprite load sequence requires only the editing of a few lines. (Well, and the addition of a 1000 more for the actual load code, but whatever...)

I have considered writing my own binary file format to store each unit's sprite set and info about as I like the idea of modular sprite loading, which MP uses, but performance concerns at load time may make me go the op2_art.bmp route.

Oh, and please don't inundate me with feature requests. I know all about the features of the game and I'm keeping in mind things like variations on multiplayer, extensions to trading, around the world maps, tilesets, new units, etc. Right now, suggestions on more fundamental elements of the engine would be preferred, if anyone wishes to offer any.

-------------------------------------------------------------------------------------
Addendum: Graphics

Right now, the game's graphics kind of awkwardly relies on Java's built-in Event Dispatch Thread which is used to paint normal UI components like buttons and text fields and handle input events. And since the mechanics, which includes unit and environment behaviors, since mechanics also runs in the EDT, the program is primarily single-threaded. That's why, if sprites are set to load on demand, when you create a new type of unit (especially the cargo truck), the program will hang while the files are loaded, which ALSO occurs in the EDT.

Just some insight into how it works.

I tried implemented an asynchronous approach where mech and graphics and loading all run in different threads but it didn't run as smoothly as it does now.

---------------------------------------------------------------------------
Addendum: Audio

I found it most convenient to use javax.sound.sampled.Clip objects for the sound bytes since they have a simple start() stop() interface and mix together. However, each Clip instance has it's own thread associated with it, so when all the sounds are loaded the program is running in a couple hundred threads.

On top of that, I decided to include loading sounds modularity on demand, just like sprites and unit types, so the first time the meteor hits the ground, it has to load the sound and a bunch of sprites for the death of the units it strikes. Unchecking "load ____ on demand" when launching the sandbox might help to prevent this, although it varies by machine and Java version.

I'll work on more well-thought-out version of the graphics and audio, but I don't think I'll be jumping for native libraries as that would start to defeat one of the only benefits of using Java for a game. (for a GAME of all things!)
Title: Moving Pictures
Post by: Savant_Ace on August 24, 2011, 08:21:54 PM
Looks quite promising, but you are correct, it won't catalyze much excitement based on all the new project blue balls that have been thrown around in the past.

Still, awesome! B)  
Title: Moving Pictures
Post by: Hooman on August 24, 2011, 09:25:25 PM
Based on the YouTube video, this is far more than I've seen posted of other attempts. I'm quite pleased to see this.
Title: Moving Pictures
Post by: BlackBox on August 24, 2011, 10:04:39 PM
I watched the youtube video and was pretty impressed. I haven't looked at the code yet but it seems potentially promising (if there are major guts of an RTS engine here).

It might not be terribly difficult to improve performance (I don't know if we would want to use OP2's original formats necessarily, as opposed to writing converters for them and using some format that is easier to work with using existing tools.. i.e. loading png images from zip files or something. I'm also assuming it uses java2d for graphics, that might not be as performant as other potential approaches like SDL or similar). I'm an enterprise software developer by day (including java, and we have to write for memory/performance constrained platforms such as smartphones) so I would definitely be willing to lend my assistance, I just don't know how much time I could devote.

If it seemed like this project has a good chance of making it to fruition (depends on who is contributing and how much people are able to realistically contribute due to time constraints) we could always try to get the community involved. It wouldn't be hard for us to add sections to the OPU site and so forth, if people were interested.
Title: Moving Pictures
Post by: evecolonycamander on August 24, 2011, 10:57:47 PM
that looks good. very much so. It reminds me of something i tried in school once with game maker. it didn't get far though. not like your project that seems to be a wow moment
 
Title: Moving Pictures
Post by: darwithe on August 25, 2011, 12:35:58 AM
I've got to admit, I'm very impressed.

Few things, first it doesn't look like would it really require much more than changing the XML, and XML reader slightly to make it take animation strips instead of completely individual frame files. That would let you reduce the loading from the current 8,000 3000ish files down to 1,000 100ish while retaining the vast majority of the useful modularity.

Next, the image extraction/infoization looks simple enough, thou granted the tedium, that I'm willing to help. Any area in particular you want me to focus, or should I just pick something not included in the release and start chugging away?

Lastly, while these probably don't qualify as 'Fundamental' they do need your design to keep from making assumptions to keep from being a pain to add, so I would like the ability to break down resources into finer chunks (IE: Iron, Copper, Tin, Etc from commons, Iridium et all from rares), create additional resource sources, and create additional damage types which effect specific extra health (IE: Viral weapon that does infection type damage, which doesn't hurt normal health but only a custom viral HP. For that matter EMP could be handled in a similar fashion.)

Edit: SVN folders inflated original file counts.
Title: Moving Pictures
Post by: Spikerocks101 on August 25, 2011, 11:17:05 AM
Hello Darwithe, I dont recall seeing you before. I don't think RobbixTheBobbix is tacking suggestions atm, since he/she is on a role with doing it them self.
Title: Moving Pictures
Post by: Highlander on August 25, 2011, 12:55:06 PM
Quote
Based on the YouTube video, this is far more than I've seen posted of other attempts. I'm quite pleased to see this.
You should see _A_'s Demo :)

I don't think she has posted it anywhere though - but it is playable :D
Title: Moving Pictures
Post by: darwithe on August 25, 2011, 06:04:30 PM
Quote
Hello Darwithe, I don't recall seeing you before. I don't think RobbixTheBobbix is taking suggestions atm, since he/she is on a role with doing it them self.
I only register on a forum very reluctantly, so I've never posted here before, but I've been an intermittent visitor since 2004 or so when I dug out my OP2 CD for the umpteenth time and suddenly wondered if there were any unofficial bug-fixes or whatnot.

So yeah, 'Hi!'

As for the suggestions: RobbixTheBobbix mentions in one of his edits that "Right now, suggestions on more fundamental elements of the engine would be preferred, if anyone wishes to offer any."
Title: Moving Pictures
Post by: RobbixTheBobbix on August 25, 2011, 06:18:49 PM
Quote
1) Few things, first it doesn't look like would it really require much more than changing the XML, and XML reader slightly to make it take animation strips instead of completely individual frame files.

2) Next, the image extraction/infoization looks simple enough, thou granted the tedium, that I'm willing to help. Any area in particular you want me to focus, or should I just pick something not included in the release and start chugging away?

3) I would like the ability to break down resources into finer chunks (IE: Iron, Copper, Tin, Etc from commons, Iridium et all from rares), create additional resource sources, and create additional damage types which effect specific extra health (IE: Viral weapon that does infection type damage, which doesn't hurt normal health but only a custom viral HP. For that matter EMP could be handled in a similar fashion.)
(*addendum added to OP*)

1) I was considering doing something like this as an automated process that would take a huge folder like "eCargoTruck" and lump all the info and bmps into a single "eCargoTruck.dat" file. This would effectively prevent the game from having to redo the same work of accessing all those files every time.

2) If you want to, you could look at units like "eCargoTruck" and build a mirror-imaged folder like "pCargoTruck" (plymouth cargo truck). If the units are the same except for coloring, then you won't have to go through the tedium of copying out the x,y offsets. Try something like that first, then go for "pVehicleFactory", "eRareSmelter", building the folders just like their Eden cousins. You can test it by putting the folder in with your downloaded copy of the game (/res/sprites) to try it out and zip/send me the results.

Oh yeah, you'll also have to make unit definition files for /res/units.

3) Those wouldn't be too hard to add, don't worry about it. But I'll keep it in mind.
Title: Moving Pictures
Post by: RobbixTheBobbix on August 25, 2011, 06:43:47 PM
Quote
You should see _A_'s Demo :)

I don't think she has posted it anywhere though - but it is playable :D
Any idea where it is?
Title: Moving Pictures
Post by: TH300 on August 25, 2011, 06:56:29 PM
This looks pretty nice.

Here are some suggestions (some of them have are not new):

Don't use that many files. Not only do they kill performance, they also take up more space on the filesystem than few big files. If you are interested I can contribute code to load tileset images from original op2 files.

Use opu git: http://forum.outpost2.net/index.php?showtopic=5159 (http://forum.outpost2.net/index.php?showtopic=5159) - this will make it easier for opu coders to contribute. I am generally interested in contributing if you need help. First I have to get familiar with the code though, which btw. looks really neat and clean.

Use a different solution for sound playback. Spawning a few meteors will almost freeze the whole application for me if sounds are on. If sounds are off it runs fluidly. You could use OpenAL, for example.

Quote
Quote
You should see _A_'s Demo :)

I don't think she has posted it anywhere though - but it is playable :D
Any idea where it is?
Its not public, yet. Hence I doubt you'll find it somewhere. Despite _A_'s remake seems to be pretty complete, work on your one won't be useless, since _A_'s is not open source and not platform independent.
Title: Moving Pictures
Post by: RobbixTheBobbix on August 25, 2011, 07:20:33 PM
Quote
First I have to get familiar with the code though, which btw. looks really neat and clean.

Use a different solution for sound playback. Spawning a few meteors will almost freeze the whole application for me if sounds are on. If sounds are off it runs fluidly. You could use OpenAL, for example.
Parts of the code are definitely not neat and clean. Some parts were hack together just to get a feature in there and weren't implemented in a very elegant way.

Which brings me to the audio...

I found it most convenient to use javax.sound.sampled.Clip objects for the sound bytes since they have a simple start() stop() interface and mix together. However, each Clip instance has it's own thread associated with it, so when all the sounds are loaded the program is running in a couple hundred threads.

On top of that, I decided to include loading sounds modularity on demand, just like sprites and unit types, so the first time the meteor hits the ground, it has to load the sound and a bunch of sprites for the death of the units it strikes. Unchecking "load ____ on demand" when launching the sandbox might help to prevent this, although it varies by machine and Java version.

I'll work on more well-thought-out version of the graphics and audio, but I don't think I'll be jumping for native libraries as that would start to defeat one of the only benefits of using Java for a game. (for a GAME of all things!)
Title: Moving Pictures
Post by: TH300 on August 25, 2011, 07:47:41 PM
When disabling sound loading on demand I get
Code: [Select]
java.io.IOException: javax.sound.sampled.LineUnavailableException: line with format PCM_UNSIGNED 22050.0 Hz, 8 bit, mono, 1 bytes/frame,  not supported.
        at com.robbix.mp5.ui.SoundBank.preload(SoundBank.java:48)
        at com.robbix.mp5.Game.load(Game.java:38)

You should rather find a solution that keeps all sounds in memory and doesn't use one thread for every sound.

I never programmed anything with audio, so I can help you little with that.

I suggested OpenAL, because its one of the (probably few) sound libraries with java bindings (namely JOAL). Its probably the most platform independent sound library. I don't know if it is suitable for your game. But its used by some serious games, so its probably worth being looked at.
Title: Moving Pictures
Post by: RobbixTheBobbix on August 25, 2011, 07:55:35 PM
Quote
When disabling sound loading on demand I get
Code: [Select]
java.io.IOException: javax.sound.sampled.LineUnavailableException: line with format PCM_UNSIGNED 22050.0 Hz, 8 bit, mono, 1 bytes/frame,  not supported.
        at com.robbix.mp5.ui.SoundBank.preload(SoundBank.java:48)
        at com.robbix.mp5.Game.load(Game.java:38)
Yeah, the JavaSound package acts differently on different platforms. Yours doesn't support the format the sound bites are in. It always worked on mine, so I never noticed. Sorry.
Title: Moving Pictures
Post by: BlackBox on August 25, 2011, 09:02:26 PM
Same thing happened on Mac OS X 10.6.8 with apple's JVM. (Though mine was somewhat less descriptive, the description was "no free voices" on the exception it threw).

I'd suggest using something like SDL, it's an extremely cross platform graphics/sound/input handing library. There are JNI bindings to it (SDL itself is written in C and is very portable).

As far as dealing with the slowness in reading files, you should definitely try to avoid the huge number of opens you do (one simple idea might be to use a ZipInputStream). If you are using something like SDL for example you can deal with loading bitmaps and drawing a lot more efficiently as well (since the bitmaps can be loaded from the native side into memory buffers, so there isn't all sorts of marshalling that continually needs to happen between the java and native side).

For metadata storage XML parsing is going to be extremely slow with the amount of data you're reading (XML is not an efficient format to parse in the first place). One possible idea might be some kind of database (SQlite comes to mind, it is also native C but is portable anywhere and there are also java bindings for it. If you wanted pure Java you could try something like HSQLDB as well). There are tools to work with databases created by these libraries, so you don't need to write tools to handle creation/maintenance of them.
Title: Moving Pictures
Post by: Hooman on August 25, 2011, 10:49:04 PM
Quote
Quote
Based on the YouTube video, this is far more than I've seen posted of other attempts. I'm quite pleased to see this.
You should see _A_'s Demo :)

I don't think she has posted it anywhere though - but it is playable :D
I qualified that with "posted" for a reason. :)


I believe there can be a less manual way of extracting the graphics. We do know a reasonable about about the file formats, and do have code written that's been able to parse and display it. I believe there is also a list of indexes for the unit animations that can potentially be used to extract all of those. The user interface animations and graphics would probably be the harder ones to extract and sort nicely. Those would likely require more manual work. The unit graphics though can probably be done in an automated way.

I will also agree with the others that have suggested lumping the graphics into fewer larger files. Personally, I think one file for all graphics will do. I find BlackBox's suggestion about the zip file curious, and strangely practical. (It seemed like the kind of solution I would expect in a workplace environment). It should solve the issues with the number of open calls and the filesystem size overhead, and shouldn't require extensive code changes or data format changes. I suspect working on that could quite drastically reduce your startup time.
Title: Moving Pictures
Post by: evecolonycamander on August 26, 2011, 01:08:59 AM
I think I have to say it. No other project(with in the time i have been here) has created such an uproar in this manor. kudos to you. what makes this even more amazing is that you did this as a one man team. few get as far as you in anything like this(despite historians glorifying the 'lone wolf' status.) if this continues, as i surely hope it will, i foresee us finally getting that magical op2 remake with included source.

sorry for the irrelevancy of this post, but i had to say it.
Title: Moving Pictures
Post by: TH300 on August 26, 2011, 07:07:07 AM
I like the idea of using zip archives as container for several small image files. When doing a little test with the Eden SF "fidget1" frames I get better compression than converting each file to png.

As for the choice xml or not, I can tell that its probably a bottleneck, but at a place where it won't hurt. You have to parse xml files only once at startup (either of the whole application or a single game) and that is still fast enough to be unnoticed. (Civilization IV uses xml and it has no performance issues)

I like the xml format, because it can be modified in a mere text editor.

One thing that I don't like is its verbosity, especially the lines
Code: [Select]
  <OffsetFrame offsetX="11" offsetY="41"/>
  <OffsetFrame offsetX="11" offsetY="41"/>
  ...
We should find a better solution for those.
Title: Moving Pictures
Post by: Hooman on August 27, 2011, 02:23:58 AM
Sadly, I think that's probably the more compact way to express that in XML. Be thankful it's not something like SLD style files used in map styling for various geographic information systems. I've always found them to be needlessly verbose, even for XML. They seem to not like tag attributes. I think their equivalent would have been more like:
Code: [Select]
<OffsetFrame>
  <Parameter>
    <OffsetX>11</OffsetX>
  </Parameter>
  <Parameter>
    <OffsetY>11</OffsetY>
  </Parameter>
</OffsetFrame>

Those files have always made me cringe.
 
Title: Moving Pictures
Post by: RobbixTheBobbix on August 27, 2011, 08:39:14 AM
Quote
I like the idea of using zip archives as container for several small image files. When doing a little test with the Eden SF "fidget1" frames I get better compression than converting each file to png.

I like the xml format, because it can be modified in a mere text editor.

One thing that I don't like is its verbosity, especially the lines
Code: [Select]
  <OffsetFrame offsetX="11" offsetY="41"/>
  <OffsetFrame offsetX="11" offsetY="41"/>
  ...
We should find a better solution for those.
Yeah, the frm-xxxx.bmp files compress fairly well, as there is alot of repetition in them. It's the sound files that make the download so big. And all the unused sound files are in there too.

I was thinking of re-writing them using JSON or YAML. In JSON, that XML sample would could look like this:
Code: [Select]
  ["11", "41"],
  ["11", "41"],
  ...
I don't want to use one big file as I like the idea of modularity. I have this overly ambitious idea that we could be able to pipe things like new maps, new sprite sets, new sounds, new tilesets, even new unit types over the network on the fly as a multiplayer game is loading.

I think I'll put it on the list to have the sprite loader read two formats:
* open set of folders and files with a xml config file
* a single binary file that contains the config info and all sprites for a unit/event. May also be zipped.

And an extra program that builds the .dat file. That way, it can be uncompressed for editing, and then compressed for deployment.
Title: Moving Pictures
Post by: Highlander on August 28, 2011, 01:52:23 PM
Quote
Quote
Quote
You should see _A_'s Demo :)

I don't think she has posted it anywhere though - but it is playable :D
Any idea where it is?
Its not public, yet. Hence I doubt you'll find it somewhere. Despite _A_'s remake seems to be pretty complete, work on your one won't be useless, since _A_'s is not open source and not platform independent.
Hmm, I'm not one for technical stuff, but according to _A_ her game is cross platform. I'm sure you can ask her on IRC for details if you want/need them - I'm content just to play the demos shes putting out :)




So far her game seems be coming along quite well, alltough as she says herself, it goes forward in her own time. Last time I spoke to her, she seemed to need some more testers even :D

Though I have to say I'm a bit puzzled her project has not recieved any sort of attention here. Most people seem aware it exists. Not to mention many of the guys on IRC seems to be using _A_ as a "Go to Guy" or perhaps I should say "Go to Gal" when it comes to coding questions.


Oh well, sorry for sort of hijacking a thread. I'm very glad people still choose to work on this ancient game. Hopefully your project will be fruitful Robbix!
Title: Moving Pictures
Post by: Spikerocks101 on August 28, 2011, 07:31:38 PM
I don't think this is the only/official thread to this project, since it seems alot bigger then 1 thread. As for _A_, let her introduce her project in her own time. I doubt she wants us to hype up her project while shes not even looking. And Robbix, what are you wanting done with this project? BB already stated that if you need stuff from the website, he can give it to you. Im pretty useless, so I just talk.
Title: Moving Pictures
Post by: TH300 on August 29, 2011, 04:15:28 AM
Quote
I was thinking of re-writing them using JSON or YAML. In JSON, that XML sample would could look like this:
Code: [Select]
  ["11", "41"],
  ["11", "41"],
  ...
Sometimes my ideas are a bit ugly... Judge for yourself:

Keep the xml format for the files

Introduce a new data type "matrix" that looks like this: "[11,41;11,41;...]". It represnts
Code: [Select]
11 41
11 41
Files could then contain lines like
Code: [Select]
<OffsetFrames matrix = "[11,41;11,41;...]" />

Handle this in your program by reading it as a string and parsing it with your own function.
 
Title: Moving Pictures
Post by: RobbixTheBobbix on September 04, 2011, 08:41:48 PM
Map scrolling added so now you can have maps bigger than the screen!!! This was an obvious lack of feature which actually only took about 20 lines of code to add, because of some planning-ahead I exercised earlier.

Youtube video of combat and mining on 48x48 map (http://www.youtube.com/watch?v=KatgRPbHS1w). Before I could only fit a 30x20 map on my 1024x768 laptop screen, so not much could fit.

The combat demo now has four groups of tanks numbering 150 each (mw, laser, railgun, rpg) converging on the center of the map where 81 acid clouds await. Running it reveals serious performance problems with all those vehicles on screen.

The mining demo isn't much different, just more smelters, mines and trucks. This one hardly slows down at all from the bigger map - a pleasant surprise.

So... considering the noted performance problems, and the fact that the combat demo ends up using about 150MB of memory, I would really appreciate a code review. The google code project is linked at the top of the page. It can be checked out through subversion or browsed on the site.

Memory-wise: there are literally hundreds of thousands of Position objects being created as the program runs. The Position class contains only two integer values: x and y. Because there's no way to allocate the pair on the stack in Java, they get allocated on the heap and sit until the garbage collector cleans them up - any ideas? The profiler also showed a similar number of HashMap entries - so that's something else to look for. Look at
Code: [Select]
com.robbix.mp5.basics.Position
and
Code: [Select]
com.robbix.mp5.ui.SpriteLibrary
.

CPU-wise: I ignored this other problem in an attempt to avoid premature optimization, but its time has come: Every mechanics cycle, the entire list of units on the map is copied and iterated over. Every draw cycle (immediately following a draw cycle) the entire list of units is copied (sorted) to a sorted set and iterated over. I suspect this takes some time for 100's of units. Look at
Code: [Select]
com.robbix.mp5.Engine
and
Code: [Select]
com.robbix.mp5.ui.DisplayPanel
.
Title: Moving Pictures
Post by: TH300 on September 05, 2011, 06:18:49 AM
You are obviously using Strings a lot. I suspect that String operations (expecially using the Stringbuilder) are costly. Although using words for everything makes the code more readable, you should use plain numbers to identify things, e.g. identify each unit type (scout, lynx, command center, ...) with an integer. Then you can simply store this integer in the unit class upon creation of a unit and everything specific to a unit type (including its sprite) can be referenced with this integer. This will also allow you to use arrays instead of HashMaps which will improve performance further.

You are using Swing which is based on AWT. AWT is said to be slow. You may improve performance by using SDL (or a Java binding of it, e.g. sdljava (http://sdljava.sourceforge.net/)).

I'm seeing that in LayeredMap.getUnitIterator(boolean) you are indeed copying the whole HashSet. I believe that you can rather easily avoid this. When drawing the map, I see two options:

1. you iterate several times over the map's tiles (probably only the visible ones) from north to south (mistake corrected) and draw stuff in the order: tiles, tubes, structures & walls, vehicles, gaia objects (disasters, weapon fire, ...), computer overlay (mining beacons, status lights & bars). This way you won't need a sorted list, because your objects are implicitly sorted.

2. you use a sorted list instead of a HashSet to store units. This way units are sorted in the moment they are created and since most of the time only one unit is created in a game cycle, the impact won't be noticeable. You'll need to find an other means to avid duplication, but that should be feasible.

As for the position objects, I didn't yet find where so many of them are created. Usually, when using the assignment operator, only references are copied.

Would be much easier to help you if your code was properly documented.
Title: Moving Pictures
Post by: RobbixTheBobbix on September 05, 2011, 11:24:39 AM
Herp-Derp... I just realized why there are so many HashMap$Entry objects... HashSet is backed by a HashMap!

@TH300 - iterating over the grid is actually the way is used to work before I re factored the TerrainLayer/UnitLayer/LayeredMap classes into a single class. (This was before the svn was set up) The Grid class even has an iterator that will only enumerate positions in a certain rectangular region of the map in preparation for map scrolling.

...Actually, looking back, the grid spots were being iterated over and added to a sorted set because of some z-order problem. In particular, there was some problem with structures having units being drawn on top of them as the unit moved diagonally around the structure's upper corner.

For the mechanics loop, there was a problem where a unit would be told to move a step, which would put it in a spot the next row down and it would be hit again. These two problems are what caused me to start using the HashSets.

As for removing strings... it might be acceptable to me if the ints are stored in constants and if everything is stored in config files as strings, and the code numbers are resolved at load time. I really don't like the idea of having code numbers all over the place. The game is intended to be easily extensible and understood, despite lacking any documentation.

And I'm not moving to Java/SDL until I see that the performance problems are the result of Java/AWT/Swing and not my code. As far as I know, when units and animations are off-screen, they are not being drawn. And when I shrink the window and scroll to an empty edge of the screen in the combat demo, the game still runs slow. It shouldn't be graphics that are slowing it down.
Title: Moving Pictures
Post by: TH300 on September 05, 2011, 12:30:19 PM
Quote
iterating over the grid is actually the way is used to work before I re factored the TerrainLayer/UnitLayer/LayeredMap classes into a single class. (This was before the svn was set up) The Grid class even has an iterator that will only enumerate positions in a certain rectangular region of the map in preparation for map scrolling.

...Actually, looking back, the grid spots were being iterated over and added to a sorted set because of some z-order problem. In particular, there was some problem with structures having units being drawn on top of them as the unit moved diagonally around the structure's upper corner.
There is probably some order that avoids this. Maybe if you draw things row by row from north to south, i.e.:
1. all buildings in row 1
2. all vehicles in row 1
3. all buildings in row 2
4. all vehicles in row 2
with other objects inserted at an appropriate place.

Quote
For the mechanics loop, there was a problem where a unit would be told to move a step, which would put it in a spot the next row down and it would be hit again. These two problems are what caused me to start using the HashSets.
Not sure if I understand this correctly. Do you say that units were drawn twice, because they moved during a draw-loop pass? If that is the case, you can easily avoid that by using some locking mechanism.

Quote
As for removing strings... it might be acceptable to me if the ints are stored in constants and if everything is stored in config files as strings, and the code numbers are resolved at load time.
What would speak against assigning numbers at runtime? This would allow one to extend resource files without keeping track of all the numbers that are already used.

You can probably avoid unique identifiers in most parts of the game by making the sprites of a unit part of its UnitType class. When you need a sprite, you can then do something like unit.getType().getSprite(direction, ...). You should use constant numbers for directions, because that is what op2 does.

Quote
I really don't like the idea of having code numbers all over the place. The game is intended to be easily extensible and understood, despite lacking any documentation.
I had some trouble understanding your mechanism of identifying images with certain strings.
Title: Moving Pictures
Post by: Hooman on September 05, 2011, 02:08:39 PM
I also noticed your heavy use of strings. I'm not sure how much it affects performance, but using an enumerated type is usually standard practice for the bits I noticed. Since enumerated types are backed by integers, most operations involving them are cheap inline code, while string operations resolve to somewhat more expensive function calls. It might not add up to anything particularly significant, but I'm sure you're wasting some cycles there.

Also, I don't recommend using integers in a stored text files, particularly if you're concerned about readability. I like your idea of translating them at load/save time though. That's probably the best way to go about it, and usually what I see elsewhere.


I have noticed in the past the units need to be sorted according to their Y-coordinate to display properly. This is related to the slightly inclined not-quite-top-down view that most RTS game of the Outpost 2 era used. Indeed, this is exactly what Output 2 does. It creates and sorts (by Y-coordinate) a list of visible units before it displays them. The X-coordinate is usually irrelevant.
Edit: But you probably do want a "stable sort" in relation to the X-coordinate if there is a chance units will overlap in the X-direction. Stable sort meaning the X-coordinates maintain their ordering if the Y-coordinates are the same. This is usually controlled by carefully selecting either "<" or "<=" for your sort comparison of the Y-coordinate. Basically, only move things if they really need to be moved. This should prevent flickering where units continually jump on top of each other where they overlap.


I was also puzzling over the large number of "synchronized" keywords used in Engine. Synchronization primitives can often create a lot of overhead and need to be avoided in performance critical code when possible. Again, I'm not sure if this creates any real performance issues here, but it may indicate other reasons for looking more closely at this code. I haven't really looked at this code in detail to know why those keywords are being used, but I suspect this area of the code can be written without relying on thread synchronization.

Outpost 2 does use multiple threads, but the threads it uses are quite independent of each other, and so don't require synchronization. You may have a design here that includes the overhead of threads, without truly benefiting from the increased performance they can offer. If two threads are always waiting on each other, they might as well be collapsed into a single one, because you're not going to get more than 1 CPU's worth of processing power out of the code anyway. I'm not sure what things are actually like in practice here but I get a sneaking suspicion there's something funny going on.
Title: Moving Pictures
Post by: RobbixTheBobbix on September 05, 2011, 02:31:29 PM
Fun experiment I just did: After adding code to display only units and terrain in the visible area of the display while I as adding scrolling, I realized that when the window in minimized, there is nothing in the visible area. I ran the combat demo and it was still sluggish, with intermittent sound fx. I minimized the window. The sound fx turned into a constant buzz. I restored the window. A few rail guns standing victorious.

So, yes, it is the graphics that are keeping it. I'm still reluctant to move to another graphics library as everything runs in AWT's event dispatch thread and I don't know how smoothly or how multi-threaded the move would be. Then again, it could be the event dispatch thread that's keeping everything up!

Did a bit of refactoring: Each unit now holds the sprite sequence relevant to its current status and refreshes it whenever those relevant statuses change.
Activity - general description of what the unit is doing, correlates loosely with Task: "move", "dock", "mine", "construct", "build"
Direction.
Cargo.

This way, not every sprite has to be looked up every time - as long as a unit is moving in the same direction, a SpriteLibrary lookup won't be needed.

@TH300, Hooman - I'll look into reducing string usage, as comparing strings is essentially comparing a series of ints instead of just one. The synchronization was being added when I thought there would be more threads - mechanics, display, resource loading - but right now, everything still runs in one thread.

Thanks for the suggestions, as I've never attempted game programming before. And for some of the things, I knew there was a problem, but was able to ignore until you reassured me that "yes, it is awful". Thanks for that.

There are actually alot of other refactors I'd like to do, so it might be a couple weeks before there are any new end-user features.
Title: Moving Pictures
Post by: TH300 on September 05, 2011, 03:05:23 PM
Quote
Did a bit of refactoring: Each unit now holds the sprite sequence relevant to its current status and refreshes it whenever those relevant statuses change.
Activity - general description of what the unit is doing, correlates loosely with Task: "move", "dock", "mine", "construct", "build"
Direction.
Cargo.

This way, not every sprite has to be looked up every time - as long as a unit is moving in the same direction, a SpriteLibrary lookup won't be needed.
Thats clearly an improvement. But you will still have to load a new sprite (from the sprite library) whenever a vehicle does a turn or when a building animates.

And when there are a lot of vehicles that might be a lot of duplicate data in memory.
Title: Moving Pictures
Post by: RobbixTheBobbix on October 13, 2011, 07:05:04 PM
I know it's been a month but I've been getting alot done. I was going to make another demo video, but I keep adding more stuff.

* earthworker build tube, bulldoze area
* map scrolling re-worked
* map zooming, including zooming all the way out to mini-map
* re-worked sprite library
* sprite library, unit types have browsers where modules can be manually loaded and unloaded
* sound lib/player as well, where sounds can be tested and audio configured (under construction along with SoundBank being re-worked)
* probably even more, I just don't realize how long it's been.

I'm dicking around with sound again, trying to avoid the 100-thread tardation I had before. I think once this round of sound stuff is done, I'll zip up another release and people can try it out.
Title: Moving Pictures
Post by: Spikerocks101 on October 13, 2011, 09:22:56 PM
Sweet job man! I really like that your sticking with it. Come to XMPP if you ever want to discuss any of it (not that I really could help thou).
Title: Moving Pictures
Post by: jcj94 on October 14, 2011, 10:30:22 AM
I heard 'java' and am now intrested :D  i was going to attempt something of the sort myself, but I am absolutely deplorable doing ANYTHING with rendering or graphics/ GUI code.  Once I finish my class, i might be able to help here and there, and YAY you use Eclipse =D.  I might take time out to finish the FIRST Bot here and there, but that isn't until January when we'll know what the game even is this year, so until then, I can help in small ways if neccisary.  
Title: Moving Pictures
Post by: RobbixTheBobbix on October 31, 2011, 05:58:39 PM
DAMMIT! It's almost been a whole 'nother month!

You know, if I didn't have a stupid day-job that I only need to get food, or... if I didn't have a stupid stomach that needs food, I'd make progress alot faster! I long to become one with the internets...

Seriously, though. At times I decide to do some more significant refactoring is preparation for new features or to make existing code more solid (i.e. remove bugs and possibilities for future bugs). Following that, there's a long period where I have to cleanup the mess I created to cleanup the mess. I did something like this with the draw, InputOverlay & geometry code the last week or so. AmbientAnimations like explosions and laserfire still don't work quite right.

On the other hand, I introduced some new practical features like the ability to command earthworks to build L-shaped and square-shaped tubes! And an asynchronous sprite loader so sprites load in the background while the simulation can go ahead and start! And fixed like a 100 problems introduced when I added scaling. You can zoom out and select/command units while being able to see 4x - 16x (or more) as much map!

Some cool stuff. I'll try to refrain from adding new features and get existing ones working just right instead, so I have a well-rounded demo to show. Sorry I haven't made another view or demo release in a while, but you don't have to take my word for it that I'm making progress, you can see updates to the repository here:
http://code.google.com/p/moving-pictures-op2/source/list (http://code.google.com/p/moving-pictures-op2/source/list)

Oh, and I tried a 256x128-sized map and it took up like 200M of ram, but whatever...
Title: Moving Pictures
Post by: drib retsam on October 31, 2011, 06:03:26 PM
I look forward to seeing the final project it sounds exciting. I look forward to seeing your demo :D  
Title: Moving Pictures
Post by: RobbixTheBobbix on October 31, 2011, 08:26:44 PM
Extra Credit: does anyone know where that large, user-drawn op2 icon is? it was 256x256 or something. I'd like to make use of it, if you'd permit me.
Title: Moving Pictures
Post by: Zhall on November 07, 2011, 04:27:53 PM
Quote
Sweet job man! I really like that your sticking with it. Come to XMPP if you ever want to discuss any of it (not that I really could help thou).
Ya he's not much help :D lol jk, he's alright i guess

Lookin good, im in the same boat with the project im working on, which is also a remake of an old game, not outpost related though...

Good luck with your optimization! (its a FEMALE DOG)
Title: Moving Pictures
Post by: RobbixTheBobbix on November 11, 2011, 06:11:48 PM
Sorry to say, there's a good chance there will be delays, my current dev machine (my circa 2005 laptop) is a sinking ship, having memory and hard drive problems. I have a decent desktop I could start using, so I'm not coming to a halt, just no longer galloping along.

Until the other day, the was a tool bar along the top of the window that held ALL of the command buttons and I thought: "It would be better if only the relevant command buttons were shown" and in doing so, introduced/uncovered some annoying bugs.

They may be related to the problems I've had with multiple DisplayPanels, which I discovering while trying to implement some new features that I thought would be really cool:

Instead of having just one view of the map (a Display) or one Display and a minimap, how about N-number of Displays that can be sized and zoomed and scrolled to wherever on the map? You could center one display over the entrance to an enemies' base and the other around your's - where your units are - and give them detailed go/attack commands without having to pan back and forth? Or if you're blessed with a multi-screen setup on your desk, you can exploit the extra screen space.

Half this project is about learning game programming, the other half is just coming up with ideas on how to improve games. So.... multiple displays?
Title: Moving Pictures
Post by: TH300 on November 12, 2011, 01:01:35 AM
Although this would be nice, players with several screens shouldn't have an advantage over players with just one screen. Otherwise winning would (in part) depend on the money that you can spend on hardware.
Title: Moving Pictures
Post by: Hooman on November 12, 2011, 01:23:57 AM
That thought had come to mind for me too. But then, that's probably already the case with many games due to lag or screen resoution issues and such. Plus, you might also think of it as penalizing people who have the money to spend just because they are being artificially limited to what other people are generally capable of. If you spent that much money on hardware, why not get some benefit from it? But yeah, definately some fairness issues involved however you look at it. I do think it's a pretty cool idea though.

Maybe some limitations could be put in place for multiplayer games? I see no reason not to allow this for single player. It might be easy to simply disallow multiple screens for multiplayer, although, that's probably not a popular option if everyone playing has multiple screens.
 
Title: Moving Pictures
Post by: TH300 on November 12, 2011, 12:42:40 PM
If people have enough money for two screens, but see no benefit in having two screens, they should simply spend the money on something else. That is the opposite of penalizing those players.

I have some rant in mind, but don't feel like debating right now.
Title: Moving Pictures
Post by: Zhall on November 16, 2011, 05:56:09 PM
Its not a big deal, Supreme commander did it, and it never saved anyone from getting owned by me. :D
Title: Moving Pictures
Post by: RobbixTheBobbix on November 18, 2011, 09:26:12 PM
Cool beans. I just uploaded r180 as a new demo (Download Page (http://code.google.com/p/moving-pictures-op2/downloads/list))

I'd like as many people to try it as possible, even for a couple minutes. Getting information from a variety of machines would be nice.

Download the zip and extract it somewhere. Go into MovingPictures/run and run launcher.bat (or launcher.sh if you're not in windows). Hopefully, a little dialog will pop up that allows you to choose which map or demo to load, which tileset to use, and if you want to have sound enabled when you start or want to preload all images/sounds up front or load them as needed.

*Addendum: I just tried this and you may have to edit the batch file to specify the full path to java.exe. Look for something like Program Files/Java/jre___/bin/java.exe*

Start with the small 16x16 plain map and go to the MenuBar>Unit>Add and add some buildings. Add a Command Center, Common Ore Smelter, Robo Miner, Cargo Trucks, etc. Go to MenuBar>Terrain>Add Ore>Common and place a mineral deposit. Build a mine with the miner or just place one from the Unit>Add menu. Tell the trucks to go the mine route. Just play around.

You can use the terrain menu to arbitrarily place tubes and walls (forget the geysers). Use the player menu to add players each in one of 360 colors (0=red, 120=green, 240=blue and other colors are in between those "angles"). Select a player to add units that belong to that player - there's a bug with that I forgot to fix. Oh, well.

Notice that when you place a building that needs a tube connection, the tubes are highlighted red or green to show if they're alive and the selection rectangle for placing the building turns yellow to warn you that if you place it there, it won't have a tube connection or that it will obscure a mineral deposit.

Place an earthworker and try building a tube at a diagonal. Press Shift to rotate between tube building options (only for earthworker, not when placing arbitrarily).

Go under MenuBar>Engine and select SpriteViewer, UnitViewer or SoundPlayer and give 'em a whirl.

Later, try the 16x16 textured map. Place a scout in one corner and tell him to go to the opposite. Notice he doesn't go in a straight line. Go MenuBar>Display>ShowCostMap and then tell him to move around some more.

There's more, but I've written enough for now. Let me know how it works out (or doesn't)!!!

*Addendum: Oh, and try to + and - keys and Ctrl+MiddleClick and Ctrl+MouseWheel and use the arrow keys to move around... I'm sure there's still more...*
Title: Moving Pictures
Post by: Venera on November 19, 2011, 01:40:19 PM
This is pretty awesome.  Good work!
Title: Moving Pictures
Post by: Zhall on November 23, 2011, 02:41:23 PM
Amazing progress!

Im sure you know this, but the geysers arent placeable, yet they become an obstacle in the pathing grid.
Title: Moving Pictures
Post by: evecolonycamander on November 24, 2011, 11:10:28 AM
Bit late on this i suppose but why not make multiplayer have a toggleable switch before starting a match that allows/disallows multiple screens. hovering over the switch would drop a box explaining what it is and what the switch dose.
Title: Moving Pictures
Post by: RobbixTheBobbix on December 20, 2011, 07:41:24 PM
Just checking in so people don't start going "not again!"

Apologies for the lack of updates and activity. I've been waiting for and working into my new machine. It's great, but there are two sides to the upgrade:

1. I can work more quickly.

2. I'm not going to notice performance issues. At all. Especially the delay in loading all the sprites modularly the way it does. I put two SSDs in a RAID 0 (Stripped) in this thing. Oh. my. god.



So I'm in the middle of refactoring current code in preparation for adding new features, the less interesting half of the cycle from an end-user perspective. I'm trying to get greater separation between units and environmental objects/events and their display. This division will allow for there to be a game host running on one computer which runs the mechanics of the simulation and clients across the network that only hear about what needs to be displayed and where. This way, if we were to introduce fog of war or some related concept, player's machines would only get info on objects visible to them, making it impossible for them to hack the game and see things they shouldn't. And it should minimize the amount of network activity necessary to keep the game going. The client just gets updates on what sprites are where and only when they move or change state.

I also have some prototypes for automatically laying out maps, in particular, if you have one tile family (gray dirt) that makes up a region and you start dragging around a paintbrush with another tile family (orange sand), transition tiles of appropriate size and orientation automatically get placed around the affected area. And even better than manually creating a file for the adjacency info, I'm working on an image analyzer that will determine which tiles are plausible neighbors automatically. Getting close on that one. I was also working on a process for cliffs, so you don't have to manually pick each cliff tile, you just have a cliff brush that you drag around. Didn't get too far with that one.

Zhall - I around to fixing the geysers in the process of doing some other refactoring, thanks for being so thorough, more thorough than I was anyway!
Title: Moving Pictures
Post by: Hooman on December 21, 2011, 12:56:34 AM
Is the game host going to be one of the players, or some external machine? If it's one of the players, then I'd hope that person is trustworthy, because they will have far more opportunity for cheating than most games allow. If it's not one of the players, then it seems there will need to be independent servers running for people to play against each other. This might limit the life of the game if the servers end up shutting down.

Also, if a client doesn't have complete state, have you considered how this will affect lag issues? Many games delay player commands, making the game run full speed, but with delayed responses to player input. The time taken to exchange player commands over the network is thus hidden by the command delay time, allowing the game to continue while still being correct. But if graphics are being transferred, how do you hide lag? I assume you'll be sending animation sequences of some kind, rather than frame by frame drawing instructions? Will all actions have some kind of lead animation before the action is felt to help hide lag?


Also, pretty sweet getting those SSDs. I've heard lots of nice things. I've also heard of a few problems too though. I've been considering getting some myself, but I'm a bit hesitant to buy in for another generation or two. I've heard there are occasional problems with stuttering from some manufacturers, at least for the older generation chips. Sometimes it's not noticeable until the drive has been in use for a while and it needs to start reclaiming previously used sectors. Plus, I've heard TRIM support is only present in newer OSes, such as Windows 7, and I'm not quite ready to upgrade that along with already expensive hardware.

I'd be curious to know what your experiences turn out to be like. Also, how fast does your computer boot now?
Title: Moving Pictures
Post by: RobbixTheBobbix on January 04, 2012, 09:08:18 PM
More delays - I'm now downloading every game I can think of. And I got Skyrim. I won't have anything resembling a human life for months.

As for Hooman's questions - there are too many of them.

I was envisioning the host could run on any machine, including being launched by one of the player's on their own machine, but for anti-cheating's sake, it would probably be run by a trusted 3rd party.

Yes, the communication will be limited to player commands and animation sequence descriptions. All of the images would be sent up front or as needed. Right now, if there is a unit that needs to be drawn and it hasn't asynchronously loaded yet, the DisplayPanel just draws a team-colored rectangle.

------------------

Yeah, the SSDs are running great. I start to get peeved if anything takes more than 1 second to load. Really, I have gotten spoiled.

However, I also live in constant fear that one will fail and bring down the pair. I've double the chances of failure on drives that are known to higher rates of spontaneous failure. And I've heard that my RAID controller might not pass on the TRIM command. I need to back this thing up.

Performance example: In order to get some large files (> 4G dvd images) from my old machine, I needed to copy them to a drive that was already formatted FAT32. FAT32 can't support files greater than 3-4 GB, so I used a stock FileSplitter to copy the file contents to a group of 2GB files and then stitch them back together on the other side.

So on the old machine (with a 160gb 7200rpm Seagate), it took about 40 mins to split the contents of an 8gb file this way, into 3 chunk files.

On the new machine, it took about 10 SECONDS to write them back together!

When I mentioned this speed improvement to a friend (he knew about the SSD RAID already), he asked "Do you think that's a Windows 7 thing?"

I said "No, I think it's a PAIR OF SOLID STATE DRIVES IN A STRIPPED RAID THING!!!!! AAAAAAAAAAAAAAAAAAA!!!!!!!!!"

Another performance example: It copied about 200 files totaling 700mb from one folder to another (not move, but copy) and it took about 1 second.

AAAAAAAAAAAAAAAAAA!!!!!!!!!!!!!!!

I have a usb stick with a Win7Pro installer and hardware drivers as a "rescue stick" and if I just get an eSATA drive to back up files, I'll feel much better.

(Modern Warfare 3 shows you these "inspirational" quotes when you die and the level loads - I don't have time to read them, it loads too quickly. This computer makes me feel like a mad scientist. THey laucghd when I said i was making a SSD RAID in a laptop! Who's laughing now?! ME! Because I've gone crazy!)
Title: Moving Pictures
Post by: Hooman on January 05, 2012, 04:51:03 AM
Lol. Things sound pretty sweet over there in performance land.

I was half wondering if some of those tasks were completed in the background, given the small times for the large amount of data, but I'm doing a few mental calculations, and both examples do work out to about double the typical speed of a fast SSD. Right about what you'd expect for RAID striping. Very nice.



Hmm, how would the client know to send a team colored rectangle? I would assume the graphics themselves would be on each client, so only simple things like the coordinates and team color would need to be sent. If you knew enough to draw a box, wouldn't you know enough to draw the full image? Or are you sending the actual image itself over the network?
 
Title: Moving Pictures
Post by: WooJoo on January 05, 2012, 04:58:15 PM
that sounds like quite the overhead your creating which is not needed. I would be simpler to set the area coordinates to each host and let them do the render work instead of sending image files all over the intertubes. or do i missinterpret this whole story?
Title: Moving Pictures
Post by: RobbixTheBobbix on January 05, 2012, 07:21:20 PM
WooJoo, I think you did. The server provides the clients with any sprites/tilesets it does not already have as my version of this game is intended to be more flexible, dynamic, whathaveyou. It's not going to send every image over every frame.

Hooman - the client would draw the colored box if it had not yet dynamically loaded the sprites; sprites can be pre-installed by the player before starting the game, installed automatically server->client before game starts, or even loaded on the fly after the game has already started, as necessary.

I'm thinking the clients will have LocalGameMaps which are updated by the client as it receives visible events from the server. There will be no LocalEngine that executes unit behaviors or environmental events.

I know this is confusing as this all seems unnecessary, but as this project is more an exercise for me in software design than an attempt to make a perfect replica, it sounded cool so I started adding it.
 
Title: Moving Pictures
Post by: TH300 on January 07, 2012, 08:25:35 AM
An idea that I once had for another similar project was to have each player's machine do the logic for that player's units (i.e. path finding etc.), but only send the events for the next few milliseconds over the network. For example, a complete path would be calculated on a players machine for a unit controlled by that player, but instead of sending the whole path at once, only the part that is needed immediately is sent and other parts are sent when needed. The advantage would be, that time intensive calculations would not be done all on one machine. It would also take away some of the network lag that would be there if every command had to go through the host.

I'm not sure how this could address the fog of war concern. Not sure if we'd even want fog of war. Apparently you want each player's machine to only receive what that player can really see on his/her personal map. For that, the sender may only send what that player can see and for that the sender must know what that player can see. If that is the case the sender knows already more about that player than is desired.

There are also things like research which you'd want to have monitored by a trusted machine, since otherwise it would be possible to claim that a research is completed when no scientist had been assigned to it.

So, the question is whether to have all machines know almost everything or have only one machine know almost everything.
Title: Moving Pictures
Post by: jcj94 on January 09, 2012, 08:16:52 AM
I'm back for a few, and figured I'd check up on this.

FOW seems like a good idea at times, but I've noticed that in a few RTS's, the AI doesn't like to follow the rulings on that.

I prefer OP2 -without- Fog of War, but still having units be invisible on a minimap when they have thier lights out.  Thats a stratagy I've grown used to as of late.

The AI should be done on the server machine, be that a players machine or on a 3rd party system.  

Player pathfinding should, as TH300 said, be done by the players machine.  It would also allow quicker changes to a swapping of pathfinding.  If a player needs to rapidly swap out a path and the server has already loaded a path, there might be increased lag.  Not a lot, but most definately some.

And what is the intended final max player size?  Personally I'd love an FFA with 8-10 people in it, even if 6 of them are AIs.

*waddles back off into the corner while school consumes my life*
Title: Moving Pictures
Post by: RobbixTheBobbix on January 15, 2012, 08:45:21 PM
I;m starting to consider re-making this as a C#/.Net app from Java. Using .Net at work as turned me, it seems. The current Java version is still pretty slow and uses incredible amounts of memory.

Of course, this would kill the cross-platform compatibility aspect of it. The current version runs fine under linux.

Is this a real problem for anyone? Who out there runs linux or osx? Is it that big of a deal if I can get to run perform much better?
Title: Moving Pictures
Post by: Hooman on January 16, 2012, 09:40:03 PM
Now to say something subversive, like...
Hey, why don't you code it in assembly? That could certainly help performance. Plus, since Windows/Linux/Mac computers all run on x86, the only non-cross-platform code will be calls to the OS API. ;)


Also, check into Mono. Apparently you can run some .Net stuff on Linux. Although, I hear the features are generally lagging a bit behind. I don't know any specifics though as I've never much looked into any .Net stuff. (Yeah, I'm a bit of a dinosaur now).
 
Title: Moving Pictures
Post by: jcj94 on January 17, 2012, 09:01:17 AM
Well I can play games, Amnesia and the like, on Linux no problem.  I haven't tried Terraria (which needs the .net and .xna frameworks) yet however, and will probably try sometime in the near future.  Wine is -so- useful for that.

As far as OS compatability.. I think everyone probably has a spare computer lying about with Windows XP on it.. or could easily get one.  I mean, its not like this lags to death on an older machine (4 copies of Op2 on my 512mH processor with 512mB of RAM, no lag whatsoever), assuming you keep true to the OP2 stats on performance.
Title: Moving Pictures
Post by: TH300 on January 17, 2012, 04:48:47 PM
I only have Linux.

Before you're considering different languages/frameworks, you should consider different Java libraries. I mentioned that in an older post. You could use SDLJava and/or JOAL or possibly LWJGL and the nio classes. All of those are not too much platform dependent.

And don't forget that speed and memory usage are also influenced by how your code works.

Last time I tried Mono it wasn't at all able to run the .Net program that I wanted to use. That was long ago, though. Btw. Mono and Wine are entirely different things. Wine works quite well as of today, but that tells us nothing about Mono.
Title: Moving Pictures
Post by: Fenrisul on January 20, 2012, 01:58:03 AM
Mono works just fine on Linux - its good enough for Android/iOS too.  It doesn't have piss all for graphics libraries that run well on Linux though :(
Title: Moving Pictures
Post by: RobbixTheBobbix on January 21, 2012, 11:32:46 AM
OK, don't worry too much about moving to .Net. C# has some neat-o features, but making this into a WinForms app didn't help at all. WinForms seems to have worse performance than Java2D. Just drawing a 16x16 terrain was sketchy.

Though it did seem to use less memory. I didn't get very far with a WinForms version, but the current Java version uses ~200MB of memory just for an empty 256x128 map. Seems like alot. The original OP2 used 14M for a map twice that big.

And if you add a bunch of units to one corner of the map and select them all and tell them to move to the other side, there's a several second delay.

Hmmm... Java7 has support for running multi-core tasks. I know my pathfinding is simple and inefficient, but I'll probably try to take advantage of that in any event.

Anyways, about OpenGL: I'm trying out OGL for java right now, so far, all I have is a spinning box and a single Scout sprite. Seems like it would take a long time to rework what I have for JOGL. I wonder what kind of performance impact there would be in switching, though?

----------------------------------------------------------------

Quote
"Now to say something subversive, like...
Hey, why don't you code it in assembly?"
No. Just No. Get out.
...of your own forum.

This project is intended to be something that is easily modifiable, even for novice programmers. Assembly code would make that difficult. We wouldn't be any closer to having OP2 source code than we are now. Would a version written in ASM be any more readable than a dis-assembly of the retail exe?


-------------------------------------------------------------------

Thanks for everyone's continued interest, btw. I appreciate input, even if it chills me to the bone (i.e. Assembler).
Title: Moving Pictures
Post by: Fenrisul on January 22, 2012, 11:05:10 PM
Theres always SDL for java
Title: Moving Pictures
Post by: Hooman on January 25, 2012, 02:12:42 AM
Ok, how about JavaScript then? ;)


Also, I would expect code changes to the Java code could drastically reduce memory size. I know Java is kind of known for being a memory hog, and some of that is just the runtime before your code even gets a chance to start, but mostly, I think the language design encourages certain coding patterns that create a lot of temporary objects. If you learn how to avoid creating excessive temporary objects, you can probably decrease the memory usage substantially, while also likely improving performance.
 
Title: Moving Pictures
Post by: RobbixTheBobbix on January 25, 2012, 06:35:11 PM
Quote
Ok, how about JavaScript then? ;)
You...
You...

-----------------------------------------------------------

Hmmm... I found a very wasteful part of the DisplayPanel where it's generating a whole ton of Position* objects, and factored it out. Still doesn't explain the 200MB. From what people are saying about Java games, this is to be expected. And when it loads all the sprites, memory grows even more, as the sprites are stored as individual composited frames instead of tiny parts of an image with metadata, like original OP2.

Position is a simple but very helpful class that basically consists of:

class Position{
final int x, y;
}

with lots of helper methods and uses elsewhere. Makes for cleaner code, but creating and passing all these around is quite a load. I was profiling a run of the game once and the profiler said there were 100's of thousands of Position objects floating around. Those, and int[]'s The int[]'s are probably existing as members of BufferedImage's though. Hm....

If only I could have value types in JVM. You know, I actually considered starting to pass around simple objects like Positions as two 32-bit values Or'd into a 64-bit long? Using some static methods. Not very OOP.

I'm entering the phase of the project where I feel the need to start from scratch as the project has developed fundamental problems, like memory usage and speed, and even the changes I can think of are so widespread, it will require alot of refactoring work.

That's why I've been looking toward other languages and platforms. However, simply moving to Groovy or Scala wouldn't help any - they still run in the JVM. And moving to .net/Mono seems to have performance and memory problems of their own. And I don't want to abandon non-windows.

------------------------------------------------------

As for sdlJava, I've read that it doesn't offer much of an improvement over Java2D. Except that Java2D doesn't hardware accelerate some operations, like alpha compositing.

I'm entering the phase of the project where I'm talking alot more than I'm coding. Worrysome.
Title: Moving Pictures
Post by: Hooman on January 26, 2012, 02:22:06 AM
Interesting that you've already profiled that and found where a lot of the memory is going. Still, hundreds of thousands of 8 byte objects is still only maybe a couple of megabytes. Mind if, if you have any non-final methods on that class, it will probably add an extra 4 bytes for a vtable pointer. I bring this up because you said there are a lot of helper methods, and if even one of them is non-final, that vtable pointer will be needed.

Also, I doubt there would be much to gain by replacing those objects with strange packed integers. I would be a little surprised to see any real speed or memory gains (and might even expect some speed loss, depending on how things work out). The cost to clarity and maintenance doesn't seem justified there.


At the moment I'm kind of thinking maybe you should just bite the bullet and start refactoring the existing code. I know refactoring can often seem more daunting than just giving up and doing a rewrite, but refactoring is often the best thing to do. There is a lot to learn from refactoring attempts. Time spent refactoring usually means you better understand the problems you're now facing, and will have a better idea how to solve or avoid these problems in the future. I know it can be a bit of a hassle to actually have to really think about the mess of code you've just realized you've written, but there's also a lot to learn from exactly those types of problems. It's quite helpful to step back from your code once in a while, and forget about how it solves a problem, and instead consider what problem is being solved, and also why you choose to solve this particular problem. I find that last statement is not easy to fully appreciate until you've actually spent some time trying to refactor large projects.


It also good practice to learn how to modify code in small steps. Particularly if you want to work as a programmer. The people paying the bills (such as your salary) are generally not very receptive to the concept of basically throwing away everything they've paid you to develop, and then to continue paying you to develop entirely new stuff that won't be functional for quite some time. If you refactor, it might be that you end up replacing basically everything, but people paying the bills are generally more receptive to the concept of replacing things small parts at a time, so that the whole system remains functional throughout the transition. Plus, it's kind of fun trying to figure out how to replace certain parts of a system without breaking anything. It takes a lot of discipline though. You have to avoid getting sidetracked. It's very easy to try changing too much at once. Just because you've noticed something small and easy to fix while in the middle of making another change, doesn't mean you should fix it now.

I find when I end up refactoring existing code, the majority of my commits only touch about 10 lines at most. I generally strive for the smallest possible change, that's still meaningful, self contained, and doesn't break anything or reduce functionality. Additions of new code are sometimes bigger, but they're usually done in a stepwise manner, so new tested code gets committed, and then another separate commit will modify existing code to place the new code into the code path. You might also remove old code at such a point, but I usually don't do that until a later commit. It's nice to have a commit around in the middle where you can easily switch back and forth between old and new sections of code for testing and verification purposes.



Oh, and btw, with new HTML5 features, such as the Canvas object, that JavaScript joke might have a bit more seriousness to it these days than many of us would like to admit. There are already games being written in pure JavaScript that don't need browser add-ons or embedded applets to run. I still think JavaScript is a bit of a sick and twisted language, and certainly has some maintenance drawbacks, but people who are determined enough certainly seem to be able to do quite a bit with the language.
Title: Moving Pictures
Post by: Marukasu on March 28, 2012, 09:00:06 AM
This definitely seems like a promising remake.

I notice it's been a few months since your last posting on the forum...
I hope you plan on continuing the project, or passing it on :)