Welcome, Guest!

Here are some links you may find helpful

Digging inside a ROM file

BetaGuy64

Active member
Joined
May 30, 2019
Messages
40
Reaction score
38
Points
18
Location
The only country that matters
AG User Name
BetaGuy64
AG Join Date
July 19th, 2014
I've been in this community as a lurker for a few years and I've been really fascinated by the amount of work you guys have done digging into game files and finding unused stuff. Looking at TCRF makes me wish I could do that stuff and I'd really like to familiarize myself with the tools/languages/etc required to contribute to the community at that way. How does one attempt to do that? I understand ROM files aren't like conventional file systems with branching files, and whatnot. I just want to get a grip on what's required to go hunting for missing content inside a file. If you need specifics, I want to look inside an N64 rom
 

puch666

New member
Joined
Jun 21, 2019
Messages
2
Reaction score
1
Points
3
AG User Name
puch66
AG Join Date
Mar 21, 2010
A good idea is search for technical documentation about the machine.
Example: In NES roms the data is splitted in CHR (graphics) and PRG (the game code).

That's a good start.

I don't know too much about n64 but i think that there's a few plugins for emulators that can give you access to vram, wireframe data, textures and more.
Also a debugger could be usefull for watch the main memory.
Even an hexeditor could be usefull for string search.

You should search if there are any rom manipulation utiliies available. I mean, like a texture or sound extractor.
 
Joined
Aug 1, 2019
Messages
1
Reaction score
0
Points
1
PS1 is pretty easy to start poking around in. jPSXdec is what I use. It does a good enough job of interpreting some file types and showing them in the program.
 

Trimesh

Donator
Donator
Registered
Joined
May 30, 2019
Messages
238
Reaction score
207
Points
43
AG User Name
Trimesh
AG Join Date
Jul 4, 2008
I've been in this community as a lurker for a few years and I've been really fascinated by the amount of work you guys have done digging into game files and finding unused stuff. Looking at TCRF makes me wish I could do that stuff and I'd really like to familiarize myself with the tools/languages/etc required to contribute to the community at that way. How does one attempt to do that? I understand ROM files aren't like conventional file systems with branching files, and whatnot. I just want to get a grip on what's required to go hunting for missing content inside a file. If you need specifics, I want to look inside an N64 rom

The biggest problem with the N64 is that most of the data in the ROM is generally compressed in order to save ROM space and hence reduce the cost of the cart. Unfortunately, the compression method is not standardized and each developer used their own - but you can often figure out what format is being used either from the developer (I.E. Nintendo used a format known as Yaz0) or by looking for signature bytes in the ROM. This project handles a number of different formats (the "GE" is because it was originally written to unpack GoldenEye, but it's been heavily extended since then).


You will still need to figure out how to unpack the ROM, which also varies from one developer to another. For a concrete example, if you get the Ocarina of Time ROM image and search for "Yaz0" you will find multiple hits. each aligned on a 16 byte boundary - these are all compressed assets stored in Nintendo's archive format.
 

Cooljerk

Well-known member
Joined
Jun 8, 2019
Messages
45
Reaction score
51
Points
18
AG User Name
The Perfect K
With sonic the hedgehog related stuff, I actually built my own tools from scratch (clarification: I used the community decompression tools in a few cases). What that means is basically treating the rom like a giant block of memory then examining byte offsets, packing them into more conventional variables into custom data structures, then interpreting them through something like SDL or OpenGL.

example:

wqQuRhE.png

As to how one finds offsets for relevant information themselves, it's (generally) a process using emulators or some sort of hardware that can dump RAM (like a devkit). For example, what you can do is take a saved state of an emulated game in one instance, do some action which would de-increment the value you want to isolate, then take another saved state, and compare the two files to see what offsets have changed.

An easy to understand example, say you want to figure out where in memory the game stores your lives in a game. You could do that by creating a saved state, then dying, then creating another, and looking for values that decreased by one. You'll undoubtedly get many such offsets that had their values decreased by one, so you just have to pick around at the results until your figure it out. Once you figure that out, you can start looking around for things that call to that memory location. Say you want to change it so that, like, an extra life box gives you 3 lives instead of 1. You can find things that modify that memory location that stores the lives, and change it there. It's obviously a lot of trial and error doing this, mind you.

Once you get a handle on where the game stores and pulls data from, you can then go about checking those areas and seeing if there are unused things. Like lost badnik art in Sonic games get stored in groups, so finding art for a never before seen badnik, for example, means going through a block of ROM and basically testing it out piece by piece to see if there's anything unused. That's obviously an extremely simple example, and it gets way more difficult when you talk about things like compressed files (like nearly all Sonic art is) and things that don't necessarily live in ROM (like dynamically created objects), but that's the super basic gist.

This is for example how Wood Zone was found way before the Simon Wai Sonic 2 beta popped up. We were save state hacking with KGen or some emulator from back then on the final Sonic 2 built ROM. All art assets and level layout (collision, etc) data related to Wood Zone no longer existed in the game, as they'd stripped that data out for ROM sizing reasons. But by watching RAM states taken at the end of one level, and then again at the beginning of another level, lead to understanding where the game stores palettes, which wound up being a large block of memory. By offsetting into this block of memory by the size of a single zone's palette, you could break that block of memory up into zones, like an array. Emerald Hill Zone might have been block [1], Chemical Plant Zone might have been block [2], etc. But it became apparent that the block was larger than the number of zones actually present in the game. By contrasting the palette information with what you could actually see in game, a mapping of which blocks went to which zones was created, and there were the existence of palettes in the game not used. One belonged to Hidden Palace Zone, which was easy to recognize thanks to screenshots. By changing which palette was loaded into levels, you could actually test out other zone's palette. And that's how this was found:

hqdefault.jpg

This palette didn't match anything in the game or any known pre-release screenshots. We called it the "Lost" zone for a while until the Simon Wai Sonic 2 Beta was found and we realized it belonged to Wood Zone, a zone nobody had ever heard of before.
 
Last edited:
shape1
shape2
shape3
shape4
shape5
shape6
Top