bsnes v0.033 released

[Nach]

Emulators unfortunately defy the whole no magic number principals

bsnes however has a few exceptions to this, like putting certain initial values or other things in a config file, which I applaud byuu for doing.

[byuu]

The console didn't need a DB to load a game.

True, it had the cartridge connected to it, rather than just one linear ROM. The DB isn't ideal, I agree. If all games could be tagged, that would be best.

A database is just a different way of putting what should be external into the emulator (well, it's still an external file ...), but at least it gives us something much better than our current blended mapper: we get concrete mappings. But it's still broken -- it won't compensate for hacked / modified games.

So yes, I prefer having the full cart mapping info included with games.

Because it's not needed. I can generate for you right now a valid possible PCB for every commercial dump we have.

At this time, I don't really believe that. But if it really were possible with 100% accuracy, then I suppose that would suffice. But you can't even prove such a claim if you aren't recording the PCBs. Verifying that 100 of 4000 match your generated IDs is nowhere near fool-proof.

But why make people have to look up a bunch of meaningless PCB IDs?

They shouldn't have to. NSRT should do it for them. ROM hackers very well should consider the mapping issues of their games. You'd be amazed, but 99% of SNES ROM hackers think they can just append data directly onto the end of the ROM and it'll always work. And it does -- in emulators.

Yes, I don't believe in spending a ton of extra work to everyone at every level involved just to get exactly the same results we currently are already getting.

I don't think it will have a visible impact on existing games, but I do believe it's a superior way of doing things. We can say "this is exactly how this game was mapped on hardware", or we can say "this is good enough to run this game."

I however wholeheartedly support coming up with an easy way for hackers to specify how to map things, and even making NSRT generate these headers on every ROM, so each one becomes a cartridge, and leaving mapping insanity patterns to NSRT.

I don't like ROM headers much, but I'm also not vehemently opposed to them. The biggest problems for me is that it would continue to complicate soft patching, and the new format would allow copier headers to survive with a new format. This is one thing I don't want backwards-compatibility on. I guess we could make it a non-512 byte header, but I'd prefer we just go to external data that anyone can edit by hand. Those who don't care can leave the files alone inside the ZIP/JMA archives.

bsnes however has a few exceptions to this, like putting certain initial values or other things in a config file, which I applaud byuu for doing.

Well, you really shouldn't. I get onto ZSNES about their Uniracers hack, and yet I have my own magic value hack for it. And a hack is a hack.

hello^^
I create a alt. Japanese localization file.

I almost missed this before o.O
Thank you so much for making this for me! :D
I know my version was really bad (I have very little experience with the language), so this is definitely much appreciated!

[creaothceann]

XML is a bit funky. especially when you throw XSLT into the mix. If it is as fancy as they say it is why aren't there OS level services for parsing it?

I say plain text, and lets not limit ourselves to 8.3 filenames(pcb might be taken by another program). At the very least the extension should be .txt.

I've always been partial to INI files.

- easy to read and edit
- easy to parse, and therefore easy to support
- can be extended to a hierarchical structure (e.g. [SNES], [SNES.PPU], [SNES.PPU.CGRAM] etc.)

It'd only need a standard for linebreaks and comments - e.g. "linefeed" character, and '#'.

[Nach]

Because it's not needed. I can generate for you right now a valid possible PCB for every commercial dump we have.

At this time, I don't really believe that. But if it really were possible with 100% accuracy, then I suppose that would suffice. But you can't even prove such a claim if you aren't recording the PCBs. Verifying that 100 of 4000 match your generated IDs is nowhere near fool-proof.

We can debate this back and forth to eternity, so lets not bother.

But why make people have to look up a bunch of meaningless PCB IDs?

They shouldn't have to. NSRT should do it for them. ROM hackers very well should consider the mapping issues of their games. You'd be amazed, but 99% of SNES ROM hackers think they can just append data directly onto the end of the ROM and it'll always work. And it does -- in emulators.

NSRT should do what for them? Figure out based on some pattern to generate a PCB? ROM hackers can't even figure out to correct a checksum, or not to overwrite the basic map info byte when they stick in their translated name, we should expect them to now follow every rule Nintendo laid out, every rule which I think they enforced, even though it's undocumented, and every rule which I extrapolated?

Many ROM hackers find out that when they append data, it DOES NOT always work in emulators. Happened with both Front Mission, and Dual Orb, and probably others as well. Most hackers don't have a clue about mirroring, or they did, but we didn't at the time.

So now when a ROM hacker wants to get a game working in their emulator, they shouldn't have to sift through some published list of PCBs we recognize, and have to figure the list out, or find we don't offer what they need. We should offer a very clear and concise way to allow them to do EXACTLY what they want, within the confines of reason. A simple text format to allow them to specify banks and some bank info and no strings of gibberish. Then if they ask for help, we can provide form them a list, text to copy and paste, and a FAQ of pointers or whatever.

Yes, I don't believe in spending a ton of extra work to everyone at every level involved just to get exactly the same results we currently are already getting.

I don't think it will have a visible impact on existing games, but I do believe it's a superior way of doing things. We can say "this is exactly how this game was mapped on hardware"

But you can't, because games came mapped multiple ways, and often did come in multiple mappings. All you can say is that this is a way that it may have been mapped, and that it works.

or we can say "this is good enough to run this game."

Which means this is just one of the ways to run the game, which reduces to have both methods meaning the exact same thing.

I however wholeheartedly support coming up with an easy way for hackers to specify how to map things, and even making NSRT generate these headers on every ROM, so each one becomes a cartridge, and leaving mapping insanity patterns to NSRT.

I don't like ROM headers much, but I'm also not vehemently opposed to them.

When I say header, I don't necessarily mean header per se, but a header or a companion file, or something along those lines.

The biggest problems for me is that it would continue to complicate soft patching, and the new format would allow copier headers to survive with a new format.

External files are fine with me too. Although I'm in the midst of some heavy rewrites in NSRT, which will need to be completed before I can properly keep track of which companion files belong with which dump.

bsnes however has a few exceptions to this, like putting certain initial values or other things in a config file, which I applaud byuu for doing.

Well, you really shouldn't. I get onto ZSNES about their Uniracers hack, and yet I have my own magic value hack for it. And a hack is a hack.

I'm not talking about hacks, but stuff like initial WRAM values, or whatever you stuck there. My ideal is that an emulator program should have NO magic values located in the source at all.

[byuu]

All you can say is that this is a way that it may have been mapped, and that it works.

Wrong, you can say it was, beyond a shadow of a doubt, definitely mapped this way. At least on one cart with said data. It does say it may have been mapped other ways, as well. But you have one guaranteed correct way now.

Heuristics are what's saying "a way that it may have been mapped."

I'm not talking about hacks, but stuff like initial WRAM values, or whatever you stuck there. My ideal is that an emulator program should have NO magic values located in the source at all.

Ah, thought you meant like the PPU scanline render position value. That actually would be nice to externalize all the magic values.

[tetsuo55]

So you download a thousand of 'em and throw em in your rom folder? What happens if the names don't match up? What if you want them zipped with the rom? Then, after spending weeks getting them in that format, what happens when errors are found and they change them? This is a disaster, it incurs practically all the costs and dependencies as headers.

Well thats the best part, its all based on checksums and all common compression formats are supported, so this works almost transparently for the end user.


-----
Response to all the XML(header) bashing:

XML doesn't have to be a problem at all, nothing is stoping anyone from parsing an XML as plain text with a little function that strips all the XML crap

Code: Select all

XML
PCB_ID : NNTD
XML

Ofcourse that would defeat all the pro's

Also the XML would be an add-on file, so nothing would physically change with the original ROM file, also if you do decide to parse the XML data the name would not even matter either as the idea is to identify them by hash

so the XML file would look like(disclaimer this is completely made up crap to get the idea across)

Code: Select all

XML
file: bla.ext
map 3 mb to bank 1
map 1 mb to bank 3
file: bla.sram
map to sram bank 1
XML

---------------

About the thing set about internal cpu's

The would ofcourse be coded into the emulator, but the emulator would not know to use the chip unless defined.

Code: Select all

XML
File: 1
map to bank 1
DSP1 on bank 2
XML

---------

And then on to the part that Nach disagrees with most, getting everything related to the game in 1 file (completely optional)

Code: Select all

XML
*Important technical info
XML
File: cover.jpg
file: Cheat.dat
File: translationpatch.dat
File: hack.dat

So basically, you can make it as simple as just defining a PCB code to as complex as to completely define the pcb laybout or completely define the retail box.

[Nach]

All you can say is that this is a way that it may have been mapped, and that it works.

Wrong, you can say it was, beyond a shadow of a doubt, definitely mapped this way. At least on one cart with said data. It does say it may have been mapped other ways, as well. But you have one guaranteed correct way now.

I'm arguing on your "exactly", since it's not a given. Although yes, it means we know for sure this worked (unless someone screwed up at the plant).

[henke37]

I want to join in on the xml bashing.
XML is a nice format to store complex heriacal data, like html documents.
People just don't get the conceptual differences between an element attribute and an additional child element.
I think that many people use XML because it's cool, not because it's the best tool. View it as what it really is and you will see that for basic configuration files, it's not a good idea.

But for cart defenitions? I'd use it.

[tetsuo55]

Based on some of the previous post i think there a is a big misunderstanding of XML format:

The Proposed XML format is not just about some lines of text:

The idea is to put all the "media dumps" into a single ZIP file, add into this zipfile a XML file that explains what all the contained files are.

so:

an example snes rom in XML format would be:

A zip file containing 2 files
-Rom.ext
-XML.xml

-------------------

PS

I have no clue if XML is a good format or not, I'm just glad that a group of people have decided to work toghether to fix the rom/header chaos for many systems (to be hones the snes doesn't REALLY need it at all)

---

Edit: i keep forgeting to add my favorite part

The single ZIP file could(should) contain all possible variants of the original rom

1 zip file, game.zip containing
-xml.xml
-game.beta.ext
-game.usa.ext
-game.jap.ext
-game.eu.ext


i do realise for this to work well it would need some extra work.
-Define region choice from commandline
-Popup a region selector
-Pre-Define prefered region in the emulator itself
-Treat the zip as folder, but only show the rom choices

i bet there are a lot of other ways to do it

[funkyass]

XML is primarily indented for stateless, context-free transfer of information. the additional overhead isn't really worth it for one file that contains information that would not be useful outside of emulation.

[FitzRoy]

I also desire extra features such as having the emulator know which inputs were used for a particular game, so if desired, the user could have special control setup automated. This is something already implemented in NSRT/ZSNES/Snes9x via NSRT headers.

This is not the job of the cartridge or the game any more than cheat codes. Find a list like the one we have posted in the compatibility thread and print it out if you can't remember. It's that easy.

Yes, I don't believe in spending a ton of extra work to everyone at every level involved just to get exactly the same results we currently are already getting.

Your "results" are creating a program dependency, are not applicable to other systems, and frustrate the heck out of dumpers like myself who need virgin roms for their DAT files. Imagine if every system did exactly as you are suggesting, each with its own separate program, proprietary compression and header format. You think rom hackers who work on more than one system would like that? I doubt it.

But why make people have to look up a bunch of meaningless PCB IDs?

What's one more document to them? Won't be that hard in a month when my site is up.

Benefits to my system (emulator-end external text based pcb format, not enabled by default):
1. does not require users to ask about, find, or learn to use separate program(s) in order to use pcb files
2. does not require users to trust that program's non-public methods of verifiability when it comes to pcb ids
3. does not require users to see and/or navigate around these files while performing rom management activities on their collections
4. just as external and true to hardware as SRAM or external headers, simply in a different folder
5. allows users to more easily select different mappings without having to delete or copy pcb files (since many games used more than one pcb type)
6. allows advanced users to create and select their own pcb files

Drawbacks:
1. not as fast for actual rom loading, requires users to reference a text file or print-out of documented game pcbs when using this option

The single ZIP file could(should) contain all possible variants of the original rom

Stop trying to daughter/parent your roms, keep them as unique entries. Problem solved.

[Nach]

I also desire extra features such as having the emulator know which inputs were used for a particular game, so if desired, the user could have special control setup automated. This is something already implemented in NSRT/ZSNES/Snes9x via NSRT headers.

This is not the job of the cartridge or the game any more than cheat codes.

That's why I used the words "extra features".

Find a list like the one we have posted in the compatibility thread and print it out if you can't remember. It's that easy.

Never. I would use "nsrt -control".
However that doesn't automate it, nor does it solve the problem on handling games that don't accept input changes in the middle, and only track what was plugged in when the power went on. In ZSNES, I found for those games you had to set the controls, then reset the game, which was very annoying. Sure we could offer in the load menu some radio options...

Yes, I don't believe in spending a ton of extra work to everyone at every level involved just to get exactly the same results we currently are already getting.

Your "results" are creating a program dependency, are not applicable to other systems, and frustrate the heck out of dumpers like myself who need virgin roms for their DAT files.

I have no idea what you're talking about. This doesn't even seem related to the PCB discussion anymore.

Imagine if every system did exactly as you are suggesting, each with its own separate program, proprietary compression and header format.

Where did I suggest any of that?

You think rom hackers who work on more than one system would like that? I doubt it.

Well, I have no issues of making the format be generic enough to work for multiple systems. However if it's too generic, it may also be a problem, as someone who only hacks for one system wonders why there's all these completely unrelated options involved, and the simple options he needs are overcomplexified.

But why make people have to look up a bunch of meaningless PCB IDs?

What's one more document to them? Won't be that hard in a month when my site is up.

It's not a matter of another document. It's why should I have to know that meaningless gibberish "S5JRL!" means some kind of ROM map? Why can't I just say directly the ROM map I want?
I.E.:

Code: Select all

$50-$6F = ROM, 0 to size/2
$70-$7F = SRAM
$80-$9F = ROM, size/2 to size

Or something similar which makes sense and is easy to comprehend?

Code: Select all

Benefits to my system (emulator-end external text based pcb format, not enabled by default):
1. does not require users to ask about, find, or learn to use separate program(s) in order to use pcb files
2. does not require users to trust that program's non-public methods of verifiability when it comes to pcb ids
3. does not require users to see and/or navigate around these files while performing rom management activities on their collections
4. just as external and true to hardware as SRAM or external headers, simply in a different folder
5. allows users to more easily select different mappings without having to delete or copy pcb files (since many games used more than one pcb type)
6. allows advanced users to create and select their own pcb files

All these things you pointed out is not why emulators should have a database, but why there should be collaborative efforts on making database files which multiple tools can work with. Well except the part that you dislike both headers and external files.

[byuu]

The single ZIP file could(should) contain all possible variants of the original rom

Noooo definitely not. That just encourages the piracy aspect, as nobody would reasonably be expected to own all regional variants of a game; and it greatly complicates emulator support.

You'd need menu selections and/or preferred language selections, etc. Then people would want IPS/UPS patch support to allow game.fanlanguage.ext.ups, etc.

Then you'd realize that different games use different PCBs, different box art, etc etc and end up duplicating everything anyway, and the only 'advantage" would be solid compression archiving (combined with much slower ROM loading as a result.)

Noooo, leave it at one file = one game, please :)

However that doesn't automate it, nor does it solve the problem on handling games that don't accept input changes in the middle, and only track what was plugged in when the power went on. In ZSNES, I found for those games you had to set the controls, then reset the game, which was very annoying. Sure we could offer in the load menu some radio options...

It's definitely not related to what real hardware does (you have the same issues with a real system), but it is pretty convenient. So long as it's just optional fields in a file, and emulators that support it have an opt-out, I don't see this as a negative thing.

And it's too bad they don't have a three-state radio option. It'd be nice to flag the input types as "not supported" for games that don't utilize certain input devices to save novices a bit of frustration. But again, not something real hardware does either. Just a fringe benefit, like filters.

Could put other useful stuff too, like genre and such. Would be good for a high-end ROM loading screen. Not something I'd probably do myself.

But really, this is borderline another feature entirely. Some people were recommending per-game filter selection. We could make that into some sort of preferences file. It could include saving your controller setup, game info, various other emu preferences, etc.
I don't see it being too bad having to set the input to Super Scope one time to play an SS game. It is annoying having to set it back and forth to regular controller, though. And is Metal Combat SS or Justifier? We know, but an end user may not right off the bat.

It's not a matter of another document. It's why should I have to know that meaningless gibberish "S5JRL!" means some kind of ROM map? Why can't I just say directly the ROM map I want?

Why do we say MOSFET instead of Metal-Oxide Semiconductor Field-Effect Transistor? If you don't know what it stands for, then you shouldn't be screwing with it.

SHVC-1J3M-20 is brief, to the point, and easily verifiable with just a hex screwdriver and the game. Figuring out bank mapping without guessing requires a copier and somewhat strong analytical skills.

If someone really wants to know what it means, they can look it up in an "acronym"-like dictionary of PCB IDs.

Or something similar which makes sense and is easy to comprehend?

What is this "SRAM" that you speak of? And what is $50? Is that how much the game costs? :P

Well except the part that you dislike both headers and external files.

He seems to dislike headers, not external files.

[FitzRoy]

Never. I would use "nsrt -control".
However that doesn't automate it, nor does it solve the problem on handling games that don't accept input changes in the middle, and only track what was plugged in when the power went on.

If the actual system didn't allow you to do this, tough cookies. You make it sound like a big deal, what's having to remember something's plugged in before you start it?

In ZSNES, I found for those games you had to set the controls, then reset the game, which was very annoying. Sure we could offer in the load menu some radio options...

Not as annoying as headers, I assure you.

I have no idea what you're talking about. This doesn't even seem related to the PCB discussion anymore.

I thought you were talking about NSRT headers.

Well, I have no issues of making the format be generic enough to work for multiple systems. However if it's too generic, it may also be a problem, as someone who only hacks for one system wonders why there's all these completely unrelated options involved, and the simple options he needs are overcomplexified.

Yeah, I wouldn't know and you should look into a universal format when you're thinking about the code structure for this (especially systems with multiple rom games like the NES and many arcade boards). Not that NSRT is bad for scanning and fixing SNES games, I use it all the time for that. It's just have nightmares about new header and compression formats.

It's not a matter of another document. It's why should I have to know that meaningless gibberish "S5JRL!" means some kind of ROM map? Why can't I just say directly the ROM map I want?

That's just the name of the file, which sort of has to be short. It's not the contents. You can rename the files in my scenario to whatever you want.

All these things you pointed out is not why emulators should have a database, but why there should be collaborative efforts on making database files which multiple tools can work with. Well except the part that you dislike both headers and external files.

I don't know if you're meaning this, but it's not the same as bsnes' old database, a database stores checksum entries of the commercial games themselves, scans a loaded rom's checksums and if it matches an entry, uses that entry's pcb attribute (if it's known) to utilize certain internal pcb code. Since not all pcb codes are even close to being known, and new dumps (revisions) continue to be found, it's far more update-heavy for emulator authors. And then it requires the user to modify it since it will not work out of the box with their hacks, betas, homebrews, etc. It also makes it impossible to define and use a custom pcb.

Go back and read my suggestion again, my files would be external. It's an emulator-end common deposit of all known pcb types, selected on load. The "external header" approach would be pairing files like this with each and every rom and implementing detection. The problem is, nobody distributes the roms this way and never will because it's extremist relative to default solutions. The files wouldn't even come with the emulator, you'd have to download them and manually place them in every container, or use a (currently) non-existing program to create them from its own internal database. NSRT could be modified to do it, but that's just one system, and how do we know your pcb ids are kosher? You're not as strict as I am, you don't really know the original sources of many of what you currently have.

EDIT: Okay, I have a great idea. Why not just do both? With pcb detection enabled (advanced option?), bsnes will scan the zip contents for a pcb file. If it finds one, it uses it. If it doesn't, it will give the selection prompt to choose one from the emulator's pcb folder. If that selection is canceled, it uses Nach's "creation" method.

[henke37]

This reminds me of a story. But I am not going to recite it.
The moral is, don't ask the user questions it can't answer.

If the game has a known pcb id, use it.
If not, check the field in the load settings advanced tab labeled "pcb id override". If it's usable, use it, otherwise, use the magic detection.

Accurate for everybody and easy to use for people who don't fricking care about this and just want to game.

[Nach]

It's not a matter of another document. It's why should I have to know that meaningless gibberish "S5JRL!" means some kind of ROM map? Why can't I just say directly the ROM map I want?

Why do we say MOSFET instead of Metal-Oxide Semiconductor Field-Effect Transistor? If you don't know what it stands for, then you shouldn't be screwing with it.

In the case of where they're not changing anything, whatever is there doesn't need to be changed, so that's all irrelevant.

SHVC-1J3M-20 is brief, to the point

It's to the point? Tell me right now without looking it up exactly what the map is.

Figuring out bank mapping without guessing requires a copier and somewhat strong analytical skills.

When you are hacking a game, and changing the mapping, you already know what you changed it to, no need to analyze anything else.

If someone really wants to know what it means, they can look it up in an "acronym"-like dictionary of PCB IDs.

You're enforcing people to know what it means when they want to change the existing mapping. And now, not only do they have to say what they want, they have to determine which provided list of PCB IDs is best fit, by sifting through a whole list of things they don't want.

Or something similar which makes sense and is easy to comprehend?

What is this "SRAM" that you speak of? And what is $50? Is that how much the game costs?

If I didn't know what that was, I wouldn't be changing the mapping in game, would I?


What is so wrong with being able to spell out exactly what I want in game with a simple documented format as opposed to needing to sift through a list of things I don't want?

Never. I would use "nsrt -control".
However that doesn't automate it, nor does it solve the problem on handling games that don't accept input changes in the middle, and only track what was plugged in when the power went on.

If the actual system didn't allow you to do this, tough cookies. You make it sound like a big deal, what's having to remember something's plugged in before you start it?

How many emulators ask you what's plugged in before you start it? How annoyed will you be if you have an extra box pop up asking you what to keep plugged in before starting the game?

You may be against something we call "features", and love bsnes's minimal no support of controllers, but other people out there actually play Super Scope and Mice games. I don't want to force anything on anybody, but there should be some sane mechanisms that allow auto controller setup for the layman.

In ZSNES, I found for those games you had to set the controls, then reset the game, which was very annoying. Sure we could offer in the load menu some radio options...

Not as annoying as headers, I assure you.

I have no idea what you're assuring me. I've never found headers to be annoying. Before you think it's because I don't use copiers or other tools, thing is, I do have cases when I need to use a ROM without a header on it. I have cases where I want to merge ROMs, hexedit them, work with certain programs that don't support headers on them, and a slew of other things. But I automate all my work with header removal and addition where needed so no issues here. I also got annoyed about needing to set a controller and reset, and you can't assure me one is worse than the other, it all depends on what you're doing. If you have to debug Lamborghini - American Challenge, where you're constantly recompiling and restarting your emulator every 2 minutes, it can be a total nightmare. Hence I automated that too. See as I programmer, I have the right to automate everything. Don't tell me I can't, or that it annoys you because you didn't.

Well, I have no issues of making the format be generic enough to work for multiple systems. However if it's too generic, it may also be a problem, as someone who only hacks for one system wonders why there's all these completely unrelated options involved, and the simple options he needs are overcomplexified.

Yeah, I wouldn't know and you should look into a universal format when you're thinking about the code structure for this (especially systems with multiple rom games like the NES and many arcade boards).

I'm not planning on creating another new format, only going to join in with my viewpoints, so we can get a new format I feel is sane. I'll let other people worry about how universal with everything it is. I'll keep to worrying how well it works with the 5 or so consoles I care about.

Regarding multiple ROM games, I really don't see where that comes up at all. One ROM or multiple ROMs from the point of emulation doesn't really mean anything. I can break one ROM into as many arrays as I like, or join separate ROMs into a single array, or any combination in between. As long as whichever format has no confusion on the merge or load order, it doesn't really make a difference at all.

Not that NSRT is bad for scanning and fixing SNES games, I use it all the time for that. It's just have nightmares about new header and compression formats.

Something which you're free to ignore, by not using any "features" NSRT provides by going "above and beyond". I have never made something NSRT does a requirement to run a ROM in an emulator, and I don't plan on doing anything like that unless other SNES emulator authors almost universally demand it.

It's not a matter of another document. It's why should I have to know that meaningless gibberish "S5JRL!" means some kind of ROM map? Why can't I just say directly the ROM map I want?

That's just the name of the file, which sort of has to be short. It's not the contents. You can rename the files in my scenario to whatever you want.

You're still thinking about your directory idea filled with PCBs. This is about an XML/Text External/Header based mapping setup where instead of saying what should be mapped, byuu is proposing to only offer a PCB ID, which can then be looked up in an internal emulator DB to map the game. Which continues the MAME idea of having a DB to play anything, requires an emulator to keep up with the latest in magic numbers, and from a real purist point of view, gives elevated status to some code number on a board, as opposed to the actual wiring. Lets remember that the SNES didn't read the ID on the board, it was dictated solely by the wires and circuits on the board, many of which could be changed independently of the ID, even though it's more likely the sun will go nova before we find a board sold where someone changed the ID name.

All these things you pointed out is not why emulators should have a database, but why there should be collaborative efforts on making database files which multiple tools can work with. Well except the part that you dislike both headers and external files.

I don't know if you're meaning this, but it's not the same as bsnes' old database, a database stores checksum entries of the commercial games themselves, scans a loaded rom's checksums and if it matches an entry, uses that entry's pcb attribute (if it's known) to utilize certain internal pcb code. Since not all pcb codes are even close to being known, and new dumps (revisions) continue to be found, it's far more update-heavy for emulator authors. And then it requires the user to modify it since it will not work out of the box with their hacks, betas, homebrews, etc. It also makes it impossible to define and use a custom pcb.

I'm not talking about emulation authors, I'm talking about tool authors sharing a unified DB, created from a unified community online DB where trusted dumpers post data, and trusted admins oversee the project.

Go back and read my suggestion again, my files would be external. It's an emulator-end common deposit of all known pcb types

I disagree that an emulator should recognize any PCBs at all. A real console accepted anything thrown at it, as long as the contacts connected.

NSRT could be modified to do it, but that's just one system, and how do we know your pcb ids are kosher?

You don't have to trust me, nor should anything be limited to one system. I have however been ranting and raving about having a community online DB for the past 6 years, yet no one wants to listen, and everyone who is remotely interested just offers some interim kludges somewhere in the middle

You're not as strict as I am, you don't really know the original sources of many of what you currently have.

I know the source of everything I have marked down, namely any dump info anyone ever sent me.

EDIT: Okay, I have a great idea. Why not just do both? With pcb detection enabled (advanced option?), bsnes will scan the zip contents for a pcb file. If it finds one, it uses it. If it doesn't, it will give the selection prompt to choose one from the emulator's pcb folder. If that selection is canceled, it uses Nach's "creation" method.

I dislike all that. Emulation should model the system, it should not need to do anything besides load up the memory dump of the ROM, and read in plain text info on how to map it. Once an emulator supports that, everything else should be thrown away, and leave default modes up to tools.

To ease transition, it might be an option to allow a tool to be a filter into an emulator, so a tool can automatically add/update headers/extern files/whatever associated with a ROM not on read only media transparently, which can slowly be phased out, and leave us in a situation where everything has finally truly become a cartridge, and emulators stop including lists of magic numbers just to load a ROM which would make any independent professor trying to read the source not have a heart attack from information overload, or having to learn encyclopedia's worth of data just to figure out or debug some source.

[FitzRoy]

How many emulators ask you what's plugged in before you start it? How annoyed will you be if you have an extra box pop up asking you what to keep plugged in before starting the game?

None, why should they? Look at the input selection menu if you're about to play a game with a special controller. What do you do, think Metal Combat first, super scope second? Once zsnes brings its gui up to speed, maybe it won't be so cumbersome.

You may be against something we call "features", and love bsnes's minimal no support of controllers but other people out there actually play Super Scope and Mice games.

Huh? When have I been against special controllers? You're trying to justify headers and tool dependencies with barely useful crap. It's no harder using an input selection menu than using a real system.

I have no idea what you're assuring me. I've never found headers to be annoying.

No one of your capacities or tendencies would. I'm sure the average person or dumpers who work on DATs are the furthest thing from your thought process. You probably don't even realize that you rail against people exactly like you when you bash ALSA. Formats of manufactured importance aren't so fun are they?

I have cases where I want to merge ROMs, hexedit them, work with certain programs that don't support headers on them, and a slew of other things. But I automate all my work with header removal and addition where needed so no issues here.

Great, let us know when you think things people who can't program are doing are important as well.

I also got annoyed about needing to set a controller and reset, and you can't assure me one is worse than the other, it all depends on what you're doing. If you have to debug Lamborghini - American Challenge, where you're constantly recompiling and restarting your emulator every 2 minutes, it can be a total nightmare. Hence I automated that too. See as I programmer, I have the right to automate everything. Don't tell me I can't, or that it annoys you because you didn't.

There's absolutely nothing stopping you from just keeping it inhouse for developer work. There's no sense in pushing endless header formats on users when you can just make an emulator option or modify the rom.

Regarding multiple ROM games, I really don't see where that comes up at all. One ROM or multiple ROMs from the point of emulation doesn't really mean anything. I can break one ROM into as many arrays as I like, or join separate ROMs into a single array, or any combination in between. As long as whichever format has no confusion on the merge or load order, it doesn't really make a difference at all.

It would make a difference if it afflicts the popular format in which the games are stored or distributed. When you combine separate roms into one file, you immediately throw my world into chaos. It's one of the reasons so much bad NES data was floating around, people were still dependent on your tool-dependency model of verification. Because let's say there's a 7-rom game and it's combined with a header attached. First of all, who decides the order in which they're combined? Then, to find the checksum of one of those parts, I have to manually clip it out of the file and scan it. Sure, your emulator can read your header and play it fine. But if people like me can't easily verify the parts of thousands of these, then you're going to get a lot of bad data floating around. And then people are going to experience bugs they otherwise wouldn't have, and you're going to get assaulted with questions you otherwise wouldn't have. The only "dumpers" willing to deal with such a format are programmers capable of creating elaborate tools like Cowering and yourself, and I refuse to argue endlessly with all of them. I'm not repeating work until the end of time because of your casual verification and naming standards or Cowering's further inability to issue prompt, error-free updates because he's so knee-deep trying to database thousands upon thousands of infinitely creatable garbage. It's a huge restriction.

I disagree that an emulator should recognize any PCBs at all. A real console accepted anything thrown at it, as long as the contacts connected.

And why couldn't you do exactly that with my method? You can make custom files and select them.

I know the source of everything I have marked down, namely any dump info anyone ever sent me.

As far as I know, people like the Dumper never kept detailed records of their own dumps versus "stuff someone told them," so when you get information from him, it may not be from him just because you got it from him. It's a giant game of telephone. This is why I don't jumble everything into one big database because that's exactly what happens, everyone should have a personal dumping record explicitly stating only what they dumped.

I dislike all that. Emulation should model the system, it should not need to do anything besides load up the memory dump of the ROM, and read in plain text info on how to map it. Once an emulator supports that, everything else should be thrown away, and leave default modes up to tools.

To ease transition, it might be an option to allow a tool to be a filter into an emulator, so a tool can automatically add/update headers/extern files/whatever associated with a ROM not on read only media transparently, which can slowly be phased out, and leave us in a situation where everything has finally truly become a cartridge...

I guess I don't understand how a pcb file is not modeling the system. Sounds more like you're trying to sell your tool.

[tetsuo55]

woah Nach, you made some very good points!
But so does FitzRoy

Nach is right imho when he says these changes can be done almost transperatly to the end users.

the container format will probably be something like ZIP, anyone can open a zip file, unzipping is even built into explorer.exe, so checking out each individual contained rom is not a problem for most people.

What ever the eventual "added" text file looks like internally(can be opened and edited with any text viewer) doesn't really matter, what is important is that that single file supports all those things everyone wants/needs

What we need is information about the actual mapping of the cart, so the FILE(in most snes cases a single rom and a text file in a zip file) becomes a cartidge instead of a binary image of the data stored on the rom chips.

What some people want is things like pre defining controllers, thats fine! and if you don't want that leave it out!

The mandatory parts of the format are(need to) be kept to an absolute minimal, but the optional parts could be huge, or as simple as pre defining controllers.

So Nach and me get to do our extra thing, and FitzRoy and Byuu get to keep their minimalist approach.

The minimalist approach should also be the one that gets auto-added and spread. Any additions can be added laterby the user, exactly like things are right now.

Translators could simple add their "patchdata" to the zip file and edit the text file to remap that data to the desired locations OR add a manual-selection system to have a supported emulator do some action
Or the text file could contain instructions for soft-patching the rom with a old-school or new style patch.


EDIT:

It might be a better idea to take this discussion to the mess board, as the people who are actually designing the concept can respond way better this feedback...
The format is not meant for mess/mame only its supposed to be universal and supposed to eventually eliminate the mess/mame databases to a certain extent (maybe even fully)

[byuu]

You're enforcing people to know what it means when they want to change the existing mapping.

You like tools so much -- make a tool to provide a list of valid PCBs for a given explicit ROM layout. And if none exist, tell them that.

Cross reference the PCBs with actual games released with it, and now an EE can easily locate a donor board to put his game on a physical cart by just replacing data ROMs.

What is so wrong with being able to spell out exactly what I want in game with a simple documented format as opposed to needing to sift through a list of things I don't want?

Absolutely nothing. I propose allowing both. Take a PCB if it exists, and it will for commercial games. Otherwise, make your own.

byuu is proposing to only offer a PCB ID, which can then be looked up in an internal emulator DB to map the game. Which continues the MAME idea of having a DB to play anything, requires an emulator to keep up with the latest in magic numbers, and from a real purist point of view

I never said that. I want the PCB to be the #1 priority, use it if it exists. If not, fall back on manually mapping. If that doesn't exist either, then fall back on parsing the ROM header as we do now.

Lets remember that the SNES didn't read the ID on the board, it was dictated solely by the wires and circuits on the board, many of which could be changed independently of the ID, even though it's more likely the sun will go nova before we find a board sold where someone changed the ID name.

Seriously, can you really not even try and understand that there's a huge difference between reading an ID from an external file that specifies mapping, and from reading bits out of a ROM header?

Anyone with a hex editor can easily change header values, say by overwriting the mapper byte with a long new cart title. But virtually nobody can physically create perfect replicas of SNES PCBs, put new games on them, and then sell them commercially and fool anyone into believing it's a legitimate game, whilst secretly it has a fake PCB ID on the board.

Of course, the SNES used the actual circuits connected to the cart. We can't store those in files, so absolutely no method will ever work for that.

PCB is the most compact, direct method of specifying a physical hardware mapping. All commercial games that ever would be have been released. We can get all of their IDs. There will be ~50 or so IDs, and they will allow mapping every commercially released game perfectly. You will never get perfect mapping with ROM header parsing, and your method of explicitly defining maps will be 10-20x more verbose while truly accomplishing nothing more than my method.

If a printed ID doesn't match, then yes, we'd have to change it. Yes, just like changing your magic ROM header parsing code. I see your analogy, but it's flawed. The difference is between software and hardware here. Changing an incorrect PCB ID would require changing the mapping code for one game, not changing it for every game ever released.

If it weren't for ROM header parsing, we wouldn't have had to hack the header of Starfox 2 to play in emulators.

If you still can't see the difference between a PCB ID and reading bits in the ROM data itself ... then I give up. We will each do our own thing. I still wish you would at least be willing to write down the PCB IDs for my sake, but since you're not -- I'll find a way to get as many as possible anyway without you.

I disagree that an emulator should recognize any PCBs at all. A real console accepted anything thrown at it, as long as the contacts connected.

How does anything you propose do the same? The real hardware didn't take a "readable" mapping like:
00-3f:8000-ffff, ROM, 0
either.

A PCB ID is just a compact, easily verifiable to the layman, method of storing the exact same information. This isn't debatable, it's fact.

You may be against something we call "features", and love bsnes's minimal no support of controllers but other people out there actually play Super Scope and Mice games.

What does that have to do with this discussion? Make a "let's bash byuu for being one person and not being able to do everything himself" thread if you like.

[grinvader]

it's more likely the sun will go nova

PROTIP: The sun won't 'go nova'. Not fat enough.

[Nach]

FitzRoy, I hear your arguments, you're right about many of them, will discuss more later when I have time.

byuu: I think we're not on the same page here. I'll try to discuss this online with you so we can better see eye to eye when I get some time, still trying to optimize a lot of SPC7110 code.

grinvader: Ha!, although I guess you missed how I was doing a lot of para-quoting.

[byuu]

Okay, I've spoken to Nach more on this. I was pretty much willing to give up on the idea of compromise, but thankfully he came up with a great one.

Really, so long as the PCB IDs are saved somewhere, and that information is publicly available, I'm happy.

Cart dumpers already need to provide them so that we know how to map their dumps (unless they want to analyze the entire 24-bit address range to manually determine all possible mappings themselves, and do this 4000x instead of 50x), so simply writing them down then would work.

No, we won't ever have all variations of all cart PCB IDs, but so long as we have one, we are able to map the game that is guaranteed to be hardware accurate for at least one copy of the game, which is far better than our current system.

So long as NSRT can generate .pcb files from these stored IDs quickly and easily for all ROMs, then the same goal is essentially accomplished. So if the IDs are saved, then I'll omit direct support for them in bsnes, relying instead only on explicit mappings on a per-game basis.

The pcb file can be a standalone file, or included in the ROM compression archive.

The only thing I ask is to please not generate the PCB IDs based on the ROM header information. That defeats the entire point. You would no longer be able to say with certainty that a given game with an ID was mapped that way. We can fall back on ROM header detection to generate .pcb files for unknown PCB IDs, of course. Just don't make an ID up for the DB is all I ask.

And if we can get this ROM parsing hooked in between tools and emulators somehow to ease the transition ... I would very much enjoy removing all of my ROM header detection code. That most certainly doesn't belong in emulators, and I think we all agree on that.

---

So, with that hopefully resolved, I suppose we can work on the pcb file format now.

[tetsuo55]

Below is the current revision of the XML proposal, Bsnes and zsnes could just ignore everything except the explicitly defined "pcb id=" statement.

Source: http://mess.toseciso.org/people:mizapf: ... _proposal2

Code: Select all

<?xml version="1.0" encoding="utf-8"?>
<romset>
   <data>
       <rom id="rom1" file="romfile1.bin"/>
   <data>
   <deploy id="std">
      <pcb id="pcb0" type="stdpcb">
         <socket number="1" uses="rom1"/>
      </pcb>
   </deploy>
   <software>
      <item deploy="std"/>
   </software>
</romset>

[byuu]

And without the ridiculous XML red tape:

Code: Select all

# SPC7110 (cheating example, SRAM needs to go thru SPC7110 chip to support SRAM disable command)
00:6000-7fff, Linear, RAM
00-0f:8000-ffff, Shadow, ROM
30:6000-7fff, Linear, RAM
50:0000-ffff, Direct, SPC7110
80-8f:8000-ffff, Shadow, ROM
c0-cf:0000-ffff, Linear, ROM
d0-ff:0000-ffff, Direct, SPC7110

Code: Select all

# BSC-1A7M-20
00-1f:8000-ffff, Linear, ROM
20-3f:8000-ffff, Linear, ROM, 100000
70-7f:0000-7fff, Linear, RAM
80-9f:8000-ffff, Linear, ROM, 200000
a0-bf:8000-ffff, Linear, ROM, 100000
c0-ef:0000-ffff, Direct, BSX-Flashcart
f0-ff:0000-7fff, Linear, RAM

Direct means whatever IC gets the read/write command gets the literal address requested. Linear means the address given starts with the value after the IC name (zero if not specified), and it increments each page by the range provided. Shadow means the same as linear, except that you don't ignore the unused portions of each bank, instead you use all 64k. Shadow is for things like how $00:8000 = $c0:8000, and $01:8000 = $c1:8000, not $c1:0000.

With these three, I've been able to support everything in bsnes thus far, and I can support SFX and SA-1 mappings easily.

[neviksti]

Really, so long as the PCB IDs are saved somewhere, and that information is publicly available, I'm happy.

Cart dumpers already need to provide them so that we know how to map their dumps (unless they want to analyze the entire 24-bit address range to manually determine all possible mappings themselves, and do this 4000x instead of 50x), so simply writing them down then would work.

No, we won't ever have all variations of all cart PCB IDs, but so long as we have one, we are able to map the game that is guaranteed to be hardware accurate for at least one copy of the game, which is far better than our current system.

So long as NSRT can generate .pcb files from these stored IDs quickly and easily for all ROMs, then the same goal is essentially accomplished. So if the IDs are saved, then I'll omit direct support for them in bsnes, relying instead only on explicit mappings on a per-game basis.

The pcb file can be a standalone file, or included in the ROM compression archive.

The only thing I ask is to please not generate the PCB IDs based on the ROM header information. That defeats the entire point. You would no longer be able to say with certainty that a given game with an ID was mapped that way. We can fall back on ROM header detection to generate .pcb files for unknown PCB IDs, of course. Just don't make an ID up for the DB is all I ask.

And if we can get this ROM parsing hooked in between tools and emulators somehow to ease the transition ... I would very much enjoy removing all of my ROM header detection code. That most certainly doesn't belong in emulators, and I think we all agree on that.

Yes. Yes. Yes!
I agree with everything.

I would like to very much emphasize this however:

We can fall back on ROM header detection to generate .pcb files for unknown PCB IDs, of course. Just don't make an ID up for the DB is all I ask.

I hope all the game -> PCB and PCB -> "memory map" database stuff is easy to check online as well. I assume so, right? Rom tools are nice, but I hope the database stuff is human readable somewhere (I assume this is the plan, just making sure).