What is JMA?

[Nach]

So use MD5 / SHA-1

If I had a dedicated rig for breaking these things it wouldn't take long either. It's been done, it just requires resources I normally don't have available at all times.

And in the case under discussion, creating a matching CRC32, MD5, and SHA1 would take less than a minute. Think about it.

[byuu]

If I had a dedicated rig for breaking these things it wouldn't take long either. It's been done, it just requires resources I normally don't have available at all times.

I've read that they haven't been able to generate fake matching MD5s yet except through brute force using really, really high end PCs and lots of time. WHY anyone would do this to trick an SNES header though, is a topic for another discussion.

And in the case under discussion, creating a matching CRC32, MD5, and SHA1 would take less than a minute. Think about it.

I'd rather not
From what I can gather, though... you could match the CRC32's, but then that would break your MD5 / SHA-1. You could make a matching MD5, but that would break the CRC.
It's like instead of having to find a number that is a multiple of 4 (CRC), 7 (MD5), or 13 (SHA-1), you now have to find a number that is a multiple of 4, 7, *and* 13. Of course, this is really easy to do (4 * 7 * 13), but it's harder than just finding a multiple of 4. Which is the point.

[Nach]

If I had a dedicated rig for breaking these things it wouldn't take long either. It's been done, it just requires resources I normally don't have available at all times.

I've read that they haven't been able to generate fake matching MD5s yet except through brute force using really, really high end PCs and lots of time. WHY anyone would do this to trick an SNES header though, is a topic for another discussion.

With 2GB of RAM, and a 2GHz processor you can do it in a few minutes using rainbow tables, look it up on google.

And in the case under discussion, creating a matching CRC32, MD5, and SHA1 would take less than a minute. Think about it.

I'd rather not
From what I can gather, though... you could match the CRC32's, but then that would break your MD5 / SHA-1. You could make a matching MD5, but that would break the CRC.
It's like instead of having to find a number that is a multiple of 4 (CRC), 7 (MD5), or 13 (SHA-1), you now have to find a number that is a multiple of 4, 7, *and* 13. Of course, this is really easy to do (4 * 7 * 13), but it's harder than just finding a multiple of 4. Which is the point.

No. You simply rebuild the hashes with the new data. The checks match it's not a problem.

If you don't get it, consider how you would go about self verification without it being faked, it's not possible.

[byuu]

No. You simply rebuild the hashes with the new data. The checks match it's not a problem.

No, no. I was meaning from your end. Sorry, I should've been more clear.
You would keep the CRC32/MD5/SHA-1 of Super Mario World inside your NSRT tool, and then if anyone were to try and 'fake' their hack as being the real SMW in your tool, they would have to make the ROM match what your program has for all 3 checksums. If we were to keep the checksums inside the SNES header, that would be a breeze to fake.

On a side note, it's kind of scary that MD5's can be faked so easily on only a 2GHz machine. Perhaps we need to start using more advanced algorithms with, say, 2mbit keys :) At least for large files, like ISOs of OpenBSD, etc.

[Nach]

No. You simply rebuild the hashes with the new data. The checks match it's not a problem.

No, no. I was meaning from your end. Sorry, I should've been more clear.
You would keep the CRC32/MD5/SHA-1 of Super Mario World inside your NSRT tool, and then if anyone were to try and 'fake' their hack as being the real SMW in your tool, they would have to make the ROM match what your program has for all 3 checksums. If we were to keep the checksums inside the SNES header, that would be a breeze to fake.

NSRT already stores hashes. But the point is storing it in the header for 100% verification purposes is not valid.

[evilant]

All this talk of falsifying hashes on headers is ridiculous. In order to put a header on a ROM, the tool would have to be able to calculate the proper hash for it. Anybody else who wanted to make their own header could obviously do that too...we aren't talking about digital signatures here! The hashes would only be a means of identifying the header and knowing it is not corrupt. They wouldn't be any kind of "tamper-proof" authentication.

unless the memmap info for a ROM comes from the cart that originated it, its suspect.

That is a wise statement. If we did all this header stuff, there's no guarantee that the headers would prove any more reliable than the information that is currently embedded in ROMs. Unless we physically inspect real carts and build a database of the address mappings for tool use. But (1) we might still accidentally introduce errors into the database, and (2) not all ROMs came from physical carts (i.e. translations/hacks/homebrew) so there is no 100% reliable way *whatsoever* to do the address mappings for all those other ROMs.

In the end, the primary value of this header would be, that it is separate from the ROM contents and can specify any necessary address mapping *without* depending on certain bytes in the ROM contents. For maximum robustness, emulators would still need the heuristic detection strategies they need today. So whether it is all worth the effort, I don't know.

[Noxious Ninja]

we aren't talking about digital signatures here!

Obviously, what we need _is_ for Nach to manually verify and sign each good SNES ROM image.

[m1t0s1s]

JMA features an incredible efficiency of compression for roms. For single OR solid archives, jma is better than zip, rar, and 7z.

Sounds cool! Now if we could only get some sort of universal header onto SNES roms so that future emulators did not have to heuristically detect the correct memory mapping settings.

First, there needs to be agreement on the header format. Then, tools like NSRT can start putting the headers on roms, and emulators can start reading and maybe even using the header info If its a 512-byte header it should be compatible with non-header-aware emulators, as long as they already know about copier headers.

How about emulating the complete system like Mame does? Would that solve a lot of the problems? In Mame, oftentimes they just compile in support for the rom's hash and it works perfectly.

Also Mess is a computer/console system emulator based on Mame. Perhaps you could look at its ideas?

Further, there is the Xe emulator styled after Mame, but for console emulation. It's at xe-emulator.com.

[Noxious Ninja]

How about emulating the complete system like Mame does? Would that solve a lot of the problems? In Mame, oftentimes they just compile in support for the rom's hash and it works perfectly.

In this case "the complete system" is the SNES itself + the cartridge. You'd have to emulate the circuitry of each individual cartridge, which is infeasible.

[grinvader]

A header is unnecessary. We can get all the info we want from the rom itself. We intend to port/clean memory mapping code in a not-so-far future.

[m1t0s1s]

How about emulating the complete system like Mame does? Would that solve a lot of the problems? In Mame, oftentimes they just compile in support for the rom's hash and it works perfectly.

In this case "the complete system" is the SNES itself + the cartridge. You'd have to emulate the circuitry of each individual cartridge, which is infeasible.

My non-silicon substrate brain is having trouble comprehending how it's different from arcade systems. Isn't a Sega system 16 game is actually a board that plugs into the main board?

[Noxious Ninja]

How about emulating the complete system like Mame does? Would that solve a lot of the problems? In Mame, oftentimes they just compile in support for the rom's hash and it works perfectly.

In this case "the complete system" is the SNES itself + the cartridge. You'd have to emulate the circuitry of each individual cartridge, which is infeasible.

My non-silicon substrate brain is having trouble comprehending how it's different from arcade systems. Isn't a Sega system 16 game is actually a board that plugs into the main board?

You're talking the difference between two or three dozen cartridges and 2,000 cartridges. There probably aren't 2,000 different board layouts, but I'm sure there are quite a few. Maybe someone who knows more about it can comment so I don't have to randomly pull figures out of my ass.

[Nach]

If you're asking about mapping and stuff, there probably exists ~3 dozen possibilities out there.

[Noxious Ninja]

I was only off by two orders of magnitude.

[Jipcy]

I know you don't care to do anything Win32 platform specific, but I think lack of ease of use and flexibility holds back your format. Zip files are nearly transparent these days with shell integration. I can extract and create zips with just a right mouse click and selecting a path.

Umm... it seems that if someone is comfortable using a command-line utility like NSRT, then compressing/uncompressing using JMA wouldn't be that hard. One could probably write batch files and/or windows shortcuts with commands to make zipping and unzipping easy.

Also, my thoughts on why JMA is better at compressing:
If you know the type of data you will be compressing, it seems like it would be MUCH easier to program a compression format. If you know the target data, you can optimize compression more. I imagine there are certain data strings common between all ROMs, so now you can replace those strings with a single number or what have you.

For those of you out there considering if JMA is useful, I was able to compress 126mb of roms into 55mb. This is better than 50%, and (naturally) lossless. That's pretty freaking amazing.

On a different note, can anyone inform me on the ROM documentation scene? I once used Cowering's tools to organize my roms, but I now use NSRT. I still use GoodN64, for lack of a better alternative. What are the different "principles" that Cowering uses vs. Nach?

[Nach]

I know you don't care to do anything Win32 platform specific, but I think lack of ease of use and flexibility holds back your format. Zip files are nearly transparent these days with shell integration. I can extract and create zips with just a right mouse click and selecting a path.

Umm... it seems that if someone is comfortable using a command-line utility like NSRT, then compressing/uncompressing using JMA wouldn't be that hard. One could probably write batch files and/or windows shortcuts with commands to make zipping and unzipping easy.

Already done, I made some right click integration stuff for Windows explorer.

http://board.zsnes.com/phpBB2/viewtopic ... =100#38181

What are the different "principles" that Cowering uses vs. Nach?

My principals are collecting info on ROMs and stuff.
Cowering's principals are come up with a unique filename for anything that resembles a ROM.

[Talbain]

So, is this essentially the same as how something like the MAME32 GUI behaves? Or is this what this intends to become? An archiving utility?

[adventure_of_link]

Or is this what this intends to become? An archiving utility?

You are correct sir. A winnar is you. Have a cookie.
PS: IIRC JMA is supposed to be the best compression scheme for SNES ROMs, and I believe it works on GBA ROMs.

[Aerdan]

Also, it is *the* best compression format for SPC sets.

[adventure_of_link]

Alright, thanks Aerdan.

[grinvader]

Some proof.

Code: Select all

-rw-r--r--  1 nach nach  229420 Mar  8 16:45 top_spc.7z
-rw-r--r--  1 grin users 228122 Mar  8 15:15 top_spc.jma
-rw-r--r--  1 grin users 262826 Mar  8 15:24 top_spc.rar
-rw-r--r--  1 nach nach  150617 Mar  8 16:49 so_spc.7z
-rw-r--r--  1 grin users 149685 Mar  8 15:22 so_spc.jma
-rw-r--r--  1 grin users 171655 Mar  8 15:23 so_spc.rar

JMA beta 6... better than 7z. Er, I meant 'better than sex'.
My finger slipped.


Honest.

[kieran_]

Some proof.

Code: Select all

-rw-r--r--  1 grin users 228122 Mar  8 15:15 top_spc.jma
-

JMA beta 6... better than 7z. Er, I meant 'better than sex'.
My finger slipped.


Honest.

How come that archive is bigger than the rest? I didn't know that you could choose the level of compression. Or... it's a more advanced version of Jma...

[grinvader]

How come that archive is bigger than the rest?

Err, what ?
It's clearly smaller than the Rar and 7z archives... Check the filenames, they are pretty self-explanatory.

I didn't know that you could choose the level of compression.

You must do something now. Did you try coffee ? It can help waking up.
Almost all (if not all) archivers provide you with several compression choices.

[kieran_]

How come that archive is bigger than the rest?

Err, what ?
It's clearly smaller than the Rar and 7z archives... Check the filenames, they are pretty self-explanatory.

I didn't know that you could choose the level of compression.

You must do something now. Did you try coffee ? It can help waking up.
Almost all (if not all) archivers provide you with several compression choices.

Holy Jesus, you're right! That was pretty stupid of me.

[Nach]

Propaganda explaining JMA:







If you want to make more, feel free to post 'em.