Author Topic: Hex Editing  (Read 3150 times)

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Hex Editing
« on: June 11, 2008, 12:37:34 AM »
This post is about applying patches to Outpost2.exe with a hex editor. It does not cover how to write the patches, only how to apply them. Learning how to write them would require learning assembly, which is outside the scope of what is being done here.


First of all, there are two main tools discussed here that are used to create and apply these patches. They are:
OllyDbg
HxD


The OllyDbg comment file (.udd) in the zip attached to the post in this thread might also be useful to have. It will give about 2000 user defined symbols (function addresses, and global data addresses) to make navigating the exe much easier, and lots and lots of user entered comments.


This first part decribes how an instruction change relates to a patch of a specific byte or range of bytes. If you already have the new machine code byte values to write to the file, you can skip this part. If all you have is the assembly mnemonic change, then this part will tell you what the new machine code bytes will be.


I'll give an example of one of the changes needed for the unit limit increase patch. We'll start by finding the function GetNewUnitAddress. If you have the .udd file, you can use Ctrl+N (while the code window is active), and scroll down the list until you find that function name. It should default to sorted in alphabetical order. If you don't have the .udd file, then just navigate to the address in the screen shot. The function starts at address 439A10, and the highlighted line we will change is at address 439A3E. You can use Ctrl+G to jump to those address. Scrolling there is a bit difficult since scroll bars are a bit sensitive for files this large.



From the comments you can see that it first checks if 1023 (0x3FF) units have been allocated yet, and if so it returns with an error. If not, then it continues with the rest of the function to allocate the memory. We want to change this limit to 2047 (0x7FF) units.

Hit spacebar on the selected line, and change the constant from 3FF to 7FF. It should highlight the machine code and assembly mnemonic changes in red.



The red machine code bytes are the ones that need to be changed in the exe file. It is probably a good idea to record the old bytes to a txt file or something as well as the new bytes. When making changes by hand you'll probably want to double check you're changing the right bytes, so keeping track of the original values will help. If you forgot to record the old bytes, you can highlight that line and use Ctrl+BkSpc to undo the change. Optionally, you can also open the Patch Window to apply or remove patches. It can be accessed from the "/" toolbar button. Highlight a line in the patch window and pressing space will enable or disable the patch.



Note that OllyDbg only applies these patches to the in memory copy of the exe, and not to the copy on disk. Each time you start an exe within OllyDbg, you must reapply all the patches you want to test. If it crashes, you'll need to restart and apply them all again.


Now we turn to translating the virtual address given by OllyDbg into a file offset for use with a hex editor. For this part it helps to understand a tiny bit about the sections in the exe. You can view a list of loaded sections from the memory window (the "M" button on the toolbar).



There you can see that Outpost2.exe starts at address 0x400000. That first section is the file header that describes the rest of the exe. The code section starts at address 0x401000. This is the section the contains the change we want to make. (You can verify that by comparing the virtual address we edited with the base address of the section, and the size of the section). The important bit of information we need about this layout, is that the size of these sections is different in memory than on disk. Each section of the exe in the file is usually aligned on a 512 byte boundary. This is typically the size of one disk sector. In memory, each section is usually aligned on a 4096 byte boundary. This is typically the size of one page of memory. When loading an exe from disk, it will pad the end of each section until it's a multiple of the page size. (Note that 4096 = 0x1000, so the code section starts on the second page of memory for the exe).

To translate the virtual address of the change to the file offset, we need both the virtual address, and the file offset of the start of the segment we are editing. The previous screen shows the virtual address of the start of the segment (0x401000) for the code segment. The file offset can be read from the section table. You can also use a more ad hoc way to guess this by just comparing the first few bytes of the code section to the first few bytes at that file offset.

Here's the offset of the code section in the section table (in little endian byte order).



Each section header has 8 bytes for the name (in this case ".text", followed by three 0 bytes). Note that many section names start with ".", which is also used for unprintable characters, so it may not be obvious that it's part of the section name. The section name is then followed by a number of DWORD fields, which are 4 bytes each. The first few are VirtualSize, VirtualAddress, SizeOfRawData, and PointerToRawData. A few more fields follow here, but this last one is the one we want. It is highlighted in the previous image, and has value 0x400.


If we scroll down to the start of the code section, we can do our ad hoc compare of the first few bytes. In HxD we see:



Checking the start of the code section in OllyDbg we see:


The bytes match, so that's a pretty good indication that we didn't mess up anywhere so far.


So, we want to edit the instruction bytes, for the instruction at 0x439A3E, in the code section starting at virtual address 0x401000, and that code section has a file offset of 0x400. We use the following formula:
Code: [Select]
virtualAddress - sectionBaseAddress + sectionFileOffset = fileOffset

This gives a file offset of: 0x439A3E - 0x401000 + 0x400 = 0x38E3E.

Going to this address, we see the bytes for the original instruction:



We change the one byte we need by simply placing the cursor at the start of that byte and typing the new hex value. The change is highlighted in red.





Now, at this point, if you changed just that one byte, saved it back to the file, and then tried running the game, it'd probably crash. The unit limit increase patch requires quite a number of changes, and you'd need to complete all of them before it worked reliably. The complete list of changes can be found attached to this post. Most of the file offsets are filled in at the beginning, but I got a bit lazy near the end of the file, so you'd have to do the address conversion yourself. There should be enough information in the file that you only need HxD to make the changes, and the address conversion sample in this post. All the rest of the info about previous and new bytes that you'd get from OllyDbg should already be in the file.
 

Offline Leviathan

  • Hero Member
  • *****
  • Posts: 4055
Hex Editing
« Reply #1 on: June 11, 2008, 06:10:06 AM »
Thank you for this article Hooman. Great work!

Offline Sirbomber

  • Hero Member
  • *****
  • Posts: 3238
Hex Editing
« Reply #2 on: June 11, 2008, 11:09:04 AM »
You lost me at step 13 (Unit Drawing code change) in the text file. Perhaps you could explain your notation?
« Last Edit: June 11, 2008, 11:14:18 AM by Sirbomber »
"As usual, colonist opinion is split between those who think the plague is a good idea, and those who are dying from it." - Outpost Evening Star

Outpost 2 Coding 101 Tutorials

Offline Hooman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4955
Hex Editing
« Reply #3 on: June 11, 2008, 04:55:43 PM »
Oh, heh. That was were I really stopped caring. There were so many changes, I just stopped formatting things. All that stuff is basically the new code bytes. I didn't bother listing the old code bytes for that step. I also didn't title each group of changes with "Change:". You'll notice the addresses jump a bit at the blank lines. I also didn't clean off the "|" and "." that sometimes gets copied out of OllyDbg. Those were the little arrows and such that OllyDbg draws to show flow control, such as loops, and functions.

Just treat it all as a list of changes.