Moving Pictures

[TH300]

You are obviously using Strings a lot. I suspect that String operations (expecially using the Stringbuilder) are costly. Although using words for everything makes the code more readable, you should use plain numbers to identify things, e.g. identify each unit type (scout, lynx, command center, ...) with an integer. Then you can simply store this integer in the unit class upon creation of a unit and everything specific to a unit type (including its sprite) can be referenced with this integer. This will also allow you to use arrays instead of HashMaps which will improve performance further.

You are using Swing which is based on AWT. AWT is said to be slow. You may improve performance by using SDL (or a Java binding of it, e.g. sdljava).

I'm seeing that in LayeredMap.getUnitIterator(boolean) you are indeed copying the whole HashSet. I believe that you can rather easily avoid this. When drawing the map, I see two options:

1. you iterate several times over the map's tiles (probably only the visible ones) from north to south (mistake corrected) and draw stuff in the order: tiles, tubes, structures & walls, vehicles, gaia objects (disasters, weapon fire, ...), computer overlay (mining beacons, status lights & bars). This way you won't need a sorted list, because your objects are implicitly sorted.

2. you use a sorted list instead of a HashSet to store units. This way units are sorted in the moment they are created and since most of the time only one unit is created in a game cycle, the impact won't be noticeable. You'll need to find an other means to avid duplication, but that should be feasible.

As for the position objects, I didn't yet find where so many of them are created. Usually, when using the assignment operator, only references are copied.

Would be much easier to help you if your code was properly documented.

[RobbixTheBobbix]

Herp-Derp... I just realized why there are so many HashMap$Entry objects... HashSet is backed by a HashMap!

@TH300 - iterating over the grid is actually the way is used to work before I re factored the TerrainLayer/UnitLayer/LayeredMap classes into a single class. (This was before the svn was set up) The Grid class even has an iterator that will only enumerate positions in a certain rectangular region of the map in preparation for map scrolling.

...Actually, looking back, the grid spots were being iterated over and added to a sorted set because of some z-order problem. In particular, there was some problem with structures having units being drawn on top of them as the unit moved diagonally around the structure's upper corner.

For the mechanics loop, there was a problem where a unit would be told to move a step, which would put it in a spot the next row down and it would be hit again. These two problems are what caused me to start using the HashSets.

As for removing strings... it might be acceptable to me if the ints are stored in constants and if everything is stored in config files as strings, and the code numbers are resolved at load time. I really don't like the idea of having code numbers all over the place. The game is intended to be easily extensible and understood, despite lacking any documentation.

And I'm not moving to Java/SDL until I see that the performance problems are the result of Java/AWT/Swing and not my code. As far as I know, when units and animations are off-screen, they are not being drawn. And when I shrink the window and scroll to an empty edge of the screen in the combat demo, the game still runs slow. It shouldn't be graphics that are slowing it down.

[TH300]

iterating over the grid is actually the way is used to work before I re factored the TerrainLayer/UnitLayer/LayeredMap classes into a single class. (This was before the svn was set up) The Grid class even has an iterator that will only enumerate positions in a certain rectangular region of the map in preparation for map scrolling.

...Actually, looking back, the grid spots were being iterated over and added to a sorted set because of some z-order problem. In particular, there was some problem with structures having units being drawn on top of them as the unit moved diagonally around the structure's upper corner.

There is probably some order that avoids this. Maybe if you draw things row by row from north to south, i.e.:
1. all buildings in row 1
2. all vehicles in row 1
3. all buildings in row 2
4. all vehicles in row 2
with other objects inserted at an appropriate place.

For the mechanics loop, there was a problem where a unit would be told to move a step, which would put it in a spot the next row down and it would be hit again. These two problems are what caused me to start using the HashSets.

Not sure if I understand this correctly. Do you say that units were drawn twice, because they moved during a draw-loop pass? If that is the case, you can easily avoid that by using some locking mechanism.

As for removing strings... it might be acceptable to me if the ints are stored in constants and if everything is stored in config files as strings, and the code numbers are resolved at load time.

What would speak against assigning numbers at runtime? This would allow one to extend resource files without keeping track of all the numbers that are already used.

You can probably avoid unique identifiers in most parts of the game by making the sprites of a unit part of its UnitType class. When you need a sprite, you can then do something like unit.getType().getSprite(direction, ...). You should use constant numbers for directions, because that is what op2 does.

I really don't like the idea of having code numbers all over the place. The game is intended to be easily extensible and understood, despite lacking any documentation.

I had some trouble understanding your mechanism of identifying images with certain strings.

[Hooman]

I also noticed your heavy use of strings. I'm not sure how much it affects performance, but using an enumerated type is usually standard practice for the bits I noticed. Since enumerated types are backed by integers, most operations involving them are cheap inline code, while string operations resolve to somewhat more expensive function calls. It might not add up to anything particularly significant, but I'm sure you're wasting some cycles there.

Also, I don't recommend using integers in a stored text files, particularly if you're concerned about readability. I like your idea of translating them at load/save time though. That's probably the best way to go about it, and usually what I see elsewhere.


I have noticed in the past the units need to be sorted according to their Y-coordinate to display properly. This is related to the slightly inclined not-quite-top-down view that most RTS game of the Outpost 2 era used. Indeed, this is exactly what Output 2 does. It creates and sorts (by Y-coordinate) a list of visible units before it displays them. The X-coordinate is usually irrelevant.
Edit: But you probably do want a "stable sort" in relation to the X-coordinate if there is a chance units will overlap in the X-direction. Stable sort meaning the X-coordinates maintain their ordering if the Y-coordinates are the same. This is usually controlled by carefully selecting either "<" or "<=" for your sort comparison of the Y-coordinate. Basically, only move things if they really need to be moved. This should prevent flickering where units continually jump on top of each other where they overlap.


I was also puzzling over the large number of "synchronized" keywords used in Engine. Synchronization primitives can often create a lot of overhead and need to be avoided in performance critical code when possible. Again, I'm not sure if this creates any real performance issues here, but it may indicate other reasons for looking more closely at this code. I haven't really looked at this code in detail to know why those keywords are being used, but I suspect this area of the code can be written without relying on thread synchronization.

Outpost 2 does use multiple threads, but the threads it uses are quite independent of each other, and so don't require synchronization. You may have a design here that includes the overhead of threads, without truly benefiting from the increased performance they can offer. If two threads are always waiting on each other, they might as well be collapsed into a single one, because you're not going to get more than 1 CPU's worth of processing power out of the code anyway. I'm not sure what things are actually like in practice here but I get a sneaking suspicion there's something funny going on.

[RobbixTheBobbix]

Fun experiment I just did: After adding code to display only units and terrain in the visible area of the display while I as adding scrolling, I realized that when the window in minimized, there is nothing in the visible area. I ran the combat demo and it was still sluggish, with intermittent sound fx. I minimized the window. The sound fx turned into a constant buzz. I restored the window. A few rail guns standing victorious.

So, yes, it is the graphics that are keeping it. I'm still reluctant to move to another graphics library as everything runs in AWT's event dispatch thread and I don't know how smoothly or how multi-threaded the move would be. Then again, it could be the event dispatch thread that's keeping everything up!

Did a bit of refactoring: Each unit now holds the sprite sequence relevant to its current status and refreshes it whenever those relevant statuses change.
Activity - general description of what the unit is doing, correlates loosely with Task: "move", "dock", "mine", "construct", "build"
Direction.
Cargo.

This way, not every sprite has to be looked up every time - as long as a unit is moving in the same direction, a SpriteLibrary lookup won't be needed.

@TH300, Hooman - I'll look into reducing string usage, as comparing strings is essentially comparing a series of ints instead of just one. The synchronization was being added when I thought there would be more threads - mechanics, display, resource loading - but right now, everything still runs in one thread.

Thanks for the suggestions, as I've never attempted game programming before. And for some of the things, I knew there was a problem, but was able to ignore until you reassured me that "yes, it is awful". Thanks for that.

There are actually alot of other refactors I'd like to do, so it might be a couple weeks before there are any new end-user features.

[TH300]

Did a bit of refactoring: Each unit now holds the sprite sequence relevant to its current status and refreshes it whenever those relevant statuses change.
Activity - general description of what the unit is doing, correlates loosely with Task: "move", "dock", "mine", "construct", "build"
Direction.
Cargo.

This way, not every sprite has to be looked up every time - as long as a unit is moving in the same direction, a SpriteLibrary lookup won't be needed.

Thats clearly an improvement. But you will still have to load a new sprite (from the sprite library) whenever a vehicle does a turn or when a building animates.

And when there are a lot of vehicles that might be a lot of duplicate data in memory.

[RobbixTheBobbix]

I know it's been a month but I've been getting alot done. I was going to make another demo video, but I keep adding more stuff.

* earthworker build tube, bulldoze area
* map scrolling re-worked
* map zooming, including zooming all the way out to mini-map
* re-worked sprite library
* sprite library, unit types have browsers where modules can be manually loaded and unloaded
* sound lib/player as well, where sounds can be tested and audio configured (under construction along with SoundBank being re-worked)
* probably even more, I just don't realize how long it's been.

I'm dicking around with sound again, trying to avoid the 100-thread tardation I had before. I think once this round of sound stuff is done, I'll zip up another release and people can try it out.

[Spikerocks101]

Sweet job man! I really like that your sticking with it. Come to XMPP if you ever want to discuss any of it (not that I really could help thou).

[jcj94]

I heard 'java' and am now intrested  i was going to attempt something of the sort myself, but I am absolutely deplorable doing ANYTHING with rendering or graphics/ GUI code.  Once I finish my class, i might be able to help here and there, and YAY you use Eclipse =D.  I might take time out to finish the FIRST Bot here and there, but that isn't until January when we'll know what the game even is this year, so until then, I can help in small ways if neccisary.  

[RobbixTheBobbix]

DAMMIT! It's almost been a whole 'nother month!

You know, if I didn't have a stupid day-job that I only need to get food, or... if I didn't have a stupid stomach that needs food, I'd make progress alot faster! I long to become one with the internets...

Seriously, though. At times I decide to do some more significant refactoring is preparation for new features or to make existing code more solid (i.e. remove bugs and possibilities for future bugs). Following that, there's a long period where I have to cleanup the mess I created to cleanup the mess. I did something like this with the draw, InputOverlay & geometry code the last week or so. AmbientAnimations like explosions and laserfire still don't work quite right.

On the other hand, I introduced some new practical features like the ability to command earthworks to build L-shaped and square-shaped tubes! And an asynchronous sprite loader so sprites load in the background while the simulation can go ahead and start! And fixed like a 100 problems introduced when I added scaling. You can zoom out and select/command units while being able to see 4x - 16x (or more) as much map!

Some cool stuff. I'll try to refrain from adding new features and get existing ones working just right instead, so I have a well-rounded demo to show. Sorry I haven't made another view or demo release in a while, but you don't have to take my word for it that I'm making progress, you can see updates to the repository here:
http://code.google.com/p/moving-pictures-op2/source/list

Oh, and I tried a 256x128-sized map and it took up like 200M of ram, but whatever...

[drib retsam]

I look forward to seeing the final project it sounds exciting. I look forward to seeing your demo  

[RobbixTheBobbix]

Extra Credit: does anyone know where that large, user-drawn op2 icon is? it was 256x256 or something. I'd like to make use of it, if you'd permit me.

[Zhall]

Sweet job man! I really like that your sticking with it. Come to XMPP if you ever want to discuss any of it (not that I really could help thou).

Ya he's not much help lol jk, he's alright i guess

Lookin good, im in the same boat with the project im working on, which is also a remake of an old game, not outpost related though...

Good luck with your optimization! (its a FEMALE DOG)

[RobbixTheBobbix]

Sorry to say, there's a good chance there will be delays, my current dev machine (my circa 2005 laptop) is a sinking ship, having memory and hard drive problems. I have a decent desktop I could start using, so I'm not coming to a halt, just no longer galloping along.

Until the other day, the was a tool bar along the top of the window that held ALL of the command buttons and I thought: "It would be better if only the relevant command buttons were shown" and in doing so, introduced/uncovered some annoying bugs.

They may be related to the problems I've had with multiple DisplayPanels, which I discovering while trying to implement some new features that I thought would be really cool:

Instead of having just one view of the map (a Display) or one Display and a minimap, how about N-number of Displays that can be sized and zoomed and scrolled to wherever on the map? You could center one display over the entrance to an enemies' base and the other around your's - where your units are - and give them detailed go/attack commands without having to pan back and forth? Or if you're blessed with a multi-screen setup on your desk, you can exploit the extra screen space.

Half this project is about learning game programming, the other half is just coming up with ideas on how to improve games. So.... multiple displays?

[TH300]

Although this would be nice, players with several screens shouldn't have an advantage over players with just one screen. Otherwise winning would (in part) depend on the money that you can spend on hardware.

[Hooman]

That thought had come to mind for me too. But then, that's probably already the case with many games due to lag or screen resoution issues and such. Plus, you might also think of it as penalizing people who have the money to spend just because they are being artificially limited to what other people are generally capable of. If you spent that much money on hardware, why not get some benefit from it? But yeah, definately some fairness issues involved however you look at it. I do think it's a pretty cool idea though.

Maybe some limitations could be put in place for multiplayer games? I see no reason not to allow this for single player. It might be easy to simply disallow multiple screens for multiplayer, although, that's probably not a popular option if everyone playing has multiple screens.
 

[TH300]

If people have enough money for two screens, but see no benefit in having two screens, they should simply spend the money on something else. That is the opposite of penalizing those players.

I have some rant in mind, but don't feel like debating right now.

[Zhall]

Its not a big deal, Supreme commander did it, and it never saved anyone from getting owned by me.

[RobbixTheBobbix]

Cool beans. I just uploaded r180 as a new demo (Download Page)

I'd like as many people to try it as possible, even for a couple minutes. Getting information from a variety of machines would be nice.

Download the zip and extract it somewhere. Go into MovingPictures/run and run launcher.bat (or launcher.sh if you're not in windows). Hopefully, a little dialog will pop up that allows you to choose which map or demo to load, which tileset to use, and if you want to have sound enabled when you start or want to preload all images/sounds up front or load them as needed.

*Addendum: I just tried this and you may have to edit the batch file to specify the full path to java.exe. Look for something like Program Files/Java/jre___/bin/java.exe*

Start with the small 16x16 plain map and go to the MenuBar>Unit>Add and add some buildings. Add a Command Center, Common Ore Smelter, Robo Miner, Cargo Trucks, etc. Go to MenuBar>Terrain>Add Ore>Common and place a mineral deposit. Build a mine with the miner or just place one from the Unit>Add menu. Tell the trucks to go the mine route. Just play around.

You can use the terrain menu to arbitrarily place tubes and walls (forget the geysers). Use the player menu to add players each in one of 360 colors (0=red, 120=green, 240=blue and other colors are in between those "angles"). Select a player to add units that belong to that player - there's a bug with that I forgot to fix. Oh, well.

Notice that when you place a building that needs a tube connection, the tubes are highlighted red or green to show if they're alive and the selection rectangle for placing the building turns yellow to warn you that if you place it there, it won't have a tube connection or that it will obscure a mineral deposit.

Place an earthworker and try building a tube at a diagonal. Press Shift to rotate between tube building options (only for earthworker, not when placing arbitrarily).

Go under MenuBar>Engine and select SpriteViewer, UnitViewer or SoundPlayer and give 'em a whirl.

Later, try the 16x16 textured map. Place a scout in one corner and tell him to go to the opposite. Notice he doesn't go in a straight line. Go MenuBar>Display>ShowCostMap and then tell him to move around some more.

There's more, but I've written enough for now. Let me know how it works out (or doesn't)!!!

*Addendum: Oh, and try to + and - keys and Ctrl+MiddleClick and Ctrl+MouseWheel and use the arrow keys to move around... I'm sure there's still more...*

[Venera]

This is pretty awesome.  Good work!

[Zhall]

Amazing progress!

Im sure you know this, but the geysers arent placeable, yet they become an obstacle in the pathing grid.

[evecolonycamander]

Bit late on this i suppose but why not make multiplayer have a toggleable switch before starting a match that allows/disallows multiple screens. hovering over the switch would drop a box explaining what it is and what the switch dose.

[RobbixTheBobbix]

Just checking in so people don't start going "not again!"

Apologies for the lack of updates and activity. I've been waiting for and working into my new machine. It's great, but there are two sides to the upgrade:

1. I can work more quickly.

2. I'm not going to notice performance issues. At all. Especially the delay in loading all the sprites modularly the way it does. I put two SSDs in a RAID 0 (Stripped) in this thing. Oh. my. god.



So I'm in the middle of refactoring current code in preparation for adding new features, the less interesting half of the cycle from an end-user perspective. I'm trying to get greater separation between units and environmental objects/events and their display. This division will allow for there to be a game host running on one computer which runs the mechanics of the simulation and clients across the network that only hear about what needs to be displayed and where. This way, if we were to introduce fog of war or some related concept, player's machines would only get info on objects visible to them, making it impossible for them to hack the game and see things they shouldn't. And it should minimize the amount of network activity necessary to keep the game going. The client just gets updates on what sprites are where and only when they move or change state.

I also have some prototypes for automatically laying out maps, in particular, if you have one tile family (gray dirt) that makes up a region and you start dragging around a paintbrush with another tile family (orange sand), transition tiles of appropriate size and orientation automatically get placed around the affected area. And even better than manually creating a file for the adjacency info, I'm working on an image analyzer that will determine which tiles are plausible neighbors automatically. Getting close on that one. I was also working on a process for cliffs, so you don't have to manually pick each cliff tile, you just have a cliff brush that you drag around. Didn't get too far with that one.

Zhall - I around to fixing the geysers in the process of doing some other refactoring, thanks for being so thorough, more thorough than I was anyway!

[Hooman]

Is the game host going to be one of the players, or some external machine? If it's one of the players, then I'd hope that person is trustworthy, because they will have far more opportunity for cheating than most games allow. If it's not one of the players, then it seems there will need to be independent servers running for people to play against each other. This might limit the life of the game if the servers end up shutting down.

Also, if a client doesn't have complete state, have you considered how this will affect lag issues? Many games delay player commands, making the game run full speed, but with delayed responses to player input. The time taken to exchange player commands over the network is thus hidden by the command delay time, allowing the game to continue while still being correct. But if graphics are being transferred, how do you hide lag? I assume you'll be sending animation sequences of some kind, rather than frame by frame drawing instructions? Will all actions have some kind of lead animation before the action is felt to help hide lag?


Also, pretty sweet getting those SSDs. I've heard lots of nice things. I've also heard of a few problems too though. I've been considering getting some myself, but I'm a bit hesitant to buy in for another generation or two. I've heard there are occasional problems with stuttering from some manufacturers, at least for the older generation chips. Sometimes it's not noticeable until the drive has been in use for a while and it needs to start reclaiming previously used sectors. Plus, I've heard TRIM support is only present in newer OSes, such as Windows 7, and I'm not quite ready to upgrade that along with already expensive hardware.

I'd be curious to know what your experiences turn out to be like. Also, how fast does your computer boot now?

[RobbixTheBobbix]

More delays - I'm now downloading every game I can think of. And I got Skyrim. I won't have anything resembling a human life for months.

As for Hooman's questions - there are too many of them.

I was envisioning the host could run on any machine, including being launched by one of the player's on their own machine, but for anti-cheating's sake, it would probably be run by a trusted 3rd party.

Yes, the communication will be limited to player commands and animation sequence descriptions. All of the images would be sent up front or as needed. Right now, if there is a unit that needs to be drawn and it hasn't asynchronously loaded yet, the DisplayPanel just draws a team-colored rectangle.

------------------

Yeah, the SSDs are running great. I start to get peeved if anything takes more than 1 second to load. Really, I have gotten spoiled.

However, I also live in constant fear that one will fail and bring down the pair. I've double the chances of failure on drives that are known to higher rates of spontaneous failure. And I've heard that my RAID controller might not pass on the TRIM command. I need to back this thing up.

Performance example: In order to get some large files (> 4G dvd images) from my old machine, I needed to copy them to a drive that was already formatted FAT32. FAT32 can't support files greater than 3-4 GB, so I used a stock FileSplitter to copy the file contents to a group of 2GB files and then stitch them back together on the other side.

So on the old machine (with a 160gb 7200rpm Seagate), it took about 40 mins to split the contents of an 8gb file this way, into 3 chunk files.

On the new machine, it took about 10 SECONDS to write them back together!

When I mentioned this speed improvement to a friend (he knew about the SSD RAID already), he asked "Do you think that's a Windows 7 thing?"

I said "No, I think it's a PAIR OF SOLID STATE DRIVES IN A STRIPPED RAID THING!!!!! AAAAAAAAAAAAAAAAAAA!!!!!!!!!"

Another performance example: It copied about 200 files totaling 700mb from one folder to another (not move, but copy) and it took about 1 second.

AAAAAAAAAAAAAAAAAA!!!!!!!!!!!!!!!

I have a usb stick with a Win7Pro installer and hardware drivers as a "rescue stick" and if I just get an eSATA drive to back up files, I'll feel much better.

(Modern Warfare 3 shows you these "inspirational" quotes when you die and the level loads - I don't have time to read them, it loads too quickly. This computer makes me feel like a mad scientist. THey laucghd when I said i was making a SSD RAID in a laptop! Who's laughing now?! ME! Because I've gone crazy!)