Try my Java SNES emulator? :)

[creaothceann]

Which leads us to the problem with source code comments:

Yeah, it could've used a comment or two - but most of the functions require "inside knowledge" like that.

I can never think of a comment that won't be either so terse as to be pointless, or so verbose that the comments will be 5-10x larger than the code, making the functions as a whole very difficult to read.

The reader is basically expected to have the same detailed knowledge about the SNES as the writer. One could write a document or two about the whole rendering system (and the other parts).

"Four times for each OAM priority"???! Does this mean that [my interpretation of] Anomie's explanation is wrong and there really are four sprite layers and not one like it says? ARGH!

It depends on how you render it. My approach was to treat each priority as an extra layer, but that's slow.
See my explanation above - look only at the sprite with the lowest index at the current position, and then use its priority for the layering.

I've confirmed that a per pixel top-down style render is out of the question. It wasn't even a competition. Even not properly working and not supporting modes apart from 0, or background tile priorities, or 16-pixel tiles, or more than 2 layers, or sprites, or any other logic at all, it was just appallingly slow. Given the extraordinary amount of per pixel work that was required I'm not remotely surprised. But now I have no ideas left.

What byuu said: "it worked a hell of a lot better to not focus on speed at all on the initial implementation". Besides, there are not many SNES Java emulators. It would be interesting to know what speed a complete emulator could achieve.

What you'll probably see is that you can write many functions that do almost the same - i.e. removing branches. Maybe that could speed things up, even if it means more to write.

[byuu]

"Four times for each OAM priority"???! Does this mean that [my interpretation of] Anomie's explanation is wrong and there really are four sprite layers and not one like it says? Confused Sad ARGH!

Essentially, yes. OAM priorities work almost like BG priorities, in that you can layer them between other BGs. The only difference is that when you have multiple sprites on the same pixel, the first sprite to plot the pixel wins, not the highest priority. At least, IIRC. I haven't written rendering code in 3-4 years now.

Also, something that wasn't noted when I wrote my renderer: only the second half of sprites (192-255) can have color math applied. The first half (128-191) are exempt and always solid. I believe those indexes refer to the palette indexes.

Happily I haven't encountered this problem much yet. I've not tested many games but all the ones I have tested have been completely forgiving of my borky timings.

Games in general are way more forgiving when you run 20x too fast than when you run 2% too slow. Especially the SuperFX. Don't mean to scare you off, if you don't want absolute 100% perfection, just keep things a little too fast and you'll be okay :)

It wouldn't be! I have more than a hundred unfinished projects collected over years.

Okay, sorry. I just didn't want to push you if you didn't want to be.

I put so much work into my operating system. That had its crippling battle too -- trying to understand how to set up the virtual page tables when they're stored in memory that's being translated by the virtual page tables.

Wow, that's really cool. I worked on my own OS for a while as well. I got the bootloader, switched to protected mode with GDT+IDT setup, and could execute programs compiled with GCC and special flags not to use any of libc. I had a couple libc functions up as well, along with a keyboard driver using the PS/2 port.

For me, the deal breaker was getting lost in trying to set up VM86 mode to execute BIOS interrupts from protected mode. I needed that to set VESA2 and start on some actual graphical applications. After several weeks I just gave up.

That takes some real dedication, because an OS is something absolutely no human can complete on their own. Even with decades of work. I really think the SkyOS guy is cheating his ass off somehow :P
And even with all of that, knowing you'll never penetrate even 0.1% of the market ... ouch.

Thanks once again for a long and detailed reply byuu.

Any time, you can catch me on IRC as well if you have questions that have stumped you.

EDIT:

I only rebuild them if OAM was modified (it tests if writes to $2104 actually changed anything) since the last time, which means it often rebuilds the sprites less than once per frame.

I added this, it increased speed on a blue screen from 145fps to 152fps, in Zelda 3 from 96.5fps to 100.5fps, and in Star Ocean from 78.5fps to 80.5fps. The reason for less gains is due to increased work elsewhere, of course.

Thanks again ^_^

[spiller]

*Gasp!* Someone has tagged me as a developer. That's really cool! Except of course, now I have to somehow live up to it.

See my explanation above - look only at the sprite with the lowest index at the current position, and then use its priority for the layering.

This sounds to me like they are one layer then. If you were to print them out on clear plastic, there'd be (up to) eight sheets for the backgrounds, but 1 for the sprites, right?, whose position in the stack is determined by the first sprite stuff.

Games in general are way more forgiving when you run 20x too fast than when you run 2% too slow.

That is interesting! I had no idea. I'll add an option to overclock the CPU if it becomes necessary.

Wow, that's really cool. I worked on my own OS for a while as well. I got the bootloader, switched to protected mode with GDT+IDT setup, and could execute programs compiled with GCC and special flags not to use any of libc. I had a couple libc functions up as well, along with a keyboard driver using the PS/2 port.

For me, the deal breaker was getting lost in trying to set up VM86 mode to execute BIOS interrupts from protected mode. I needed that to set VESA2 and start on some actual graphical applications. After several weeks I just gave up.

I wonder how many developers have some variant of that story. There must be thousands of unfinished skeleton OSs around. I suppose it's a learning experience though.
Excellent news about the speed increases.


Well I came to say there's a new version of my Java emu now, based on feedback from this thread: http://tinyurl.com/jsnes-0-02-alpha (1 file, 401 kB)

- Fixed a few non-emulation related bugs including the keyboard focus bug.
- Added support for loading and saving SRAM. By default it saves them in the directory of the ROM, but it can be overridden to some common folder for SNES saves.
- Also, added support for loading Snes9x savestates. This is extremely useful for debugging because it means I can get to exactly the same point in the Snes9x debugger and my debugger. It can't save them though because it doesn't track/store a lot of the info that Snes9x seems to need.

It's 5 a.m. so I hope I didn't forget anything when compiling this.

EDIT: don't run 0.01-alpha because when it saves its ini file it will lose some of the new SRAM-related settings in 0.02-alpha (they'll go back to the defaults).

Once again, every bit of feedback is welcome. (But I already know it's full of graphics glitches and the debugger is buggy.)

[creaothceann]

This sounds to me like they are one layer then. If you were to print them out on clear plastic, there'd be (up to) eight sheets for the backgrounds, but 1 for the sprites, right?, whose position in the stack is determined by the first sprite stuff.

Well, the PPU "acts" as if there's only one layer for a given position, because it only looks at the index to determine the sprite z-order. For the game designer there are 4 layers, and the game programmer has to make sure that the index thing doesn't interfere. (Sometimes they used it for other things though.)

It's a bit like the wave-particle-duality. Both are high-level concepts and don't apply 100% to what happens at the low level. Same with SNES graphics features - sometimes there are strange effects (Mode5/6/7 mosaic, the leftmost tile in offset-per-tile modes, hires color math) where the nice abstractions no longer fit.

Well I came to say there's a new version of my Java emu now

Nice.

It passes the Nintendo test (didn't test it with the previous version).

Btw. in the Chrono Trigger intro, there are three lazy bats at Magus' castle that stay at the bottom of the screen. Maybe a CPU bug?

[spiller]

It passes the Nintendo test (didn't test it with the previous version).

Btw. in the Chrono Trigger intro, there are three lazy bats at Magus' castle that stay at the bottom of the screen. Maybe a CPU bug?

It always passed. Sadly the bats turned out to be sprites usually hidden behind the trees (for some reason). Since my emu draws sprites on top, they're visible. (So it's the PPU's fault. Again.)

Why can't I get them layered right? I just don't know what it's doing wrong. :'(

By the way, does anyone have a Mac? I don't have one and I don't know what Java is like on a Mac. I'm curious to know if JSNES works at all, and runs smooth enough, and that its UI looks okay. I'm also told that it works in Linux though I've not seen it personally. Such is the magic of Java.

[schuylerv]

I just tried out the emulator and it's looking good. I tested it on vista 64 bit and linux(with the Sun Java 6 runtime and the OpenJDK Java 6 runtime) and both seemed to be working as they should have been.

Here is some screenshots of it running on linux, it all looks good to me but there could be a problem I'm missing.
http://img16.imageshack.us/img16/3720/screenshotccu.png
http://img208.imageshack.us/img208/8700 ... shot2q.png

Overall it's looking like this could turn into a great emulator and I wish you luck in the project.

[spiller]

Thanks very much for testing it. It's über-nifty to see something I created running on an OS where I never tried it. I also hadn't expected it to work with a different runtime (at least quickly). The Mario sky is missing of course, but that's a general bug.

I note from your screenshot that .sfc is an extension for SNES ROMs so I'll add it to the default file filter.

[creaothceann]

Why can't I get them layered right? I just don't know what it's doing wrong. :'(

Turn your rendering engine upside-down? Otherwise you'd end up writing a lot of line-rendering functions, their number doubling each time you add a hardware feature.

Sprites have the two priority relationships mentioned above, which is really incompatible with bottom-up rendering.

[davideo7]

I'm not sure if this helps at all with the 3D stuff with your Java SNES emulator, but I found this that enables 3D to be done easier with Java
http://www.lwjgl.org/index.php

Just wanted to help anyway possible.

[Deathlike2]

The SNES renders in 2D. Anything that resembles 3D, isn't true 3D in reality.

[grinvader]

Sprites have the two priority relationships mentioned above, which is really incompatible with bottom-up rendering.

We shall see, we shall see.

[davideo7]

Deathlike2: I figured that was the case but I just thought it could possibly help with games like Mario Kart that use Mode 7, if not no big deal.


Spiller: If there's anything you need help on, please let me know, I'd like to help out in any way possible

[spiller]

Sprites have the two priority relationships mentioned above, which is really incompatible with bottom-up rendering.

Meaning no offense to you at all, but I hope that's not true.

I'm not sure if this helps at all with the 3D stuff with your Java SNES emulator, but I found this that enables 3D to be done easier with Java
http://www.lwjgl.org/index.php

Probably not useful for mode 7; they're different concepts.

Although, one way or another, figuring out how to use proper OpenGL in Java, either with JOGL or LWJGL, would be advantageous.

I think I can be forgiven for being somewhat annoyed with Sun Java's poor built-in graphics performance. It's more than disappointing, it's outright offensive when it takes 115 milliseconds to bilinearly blit a frame to a maximized window when I was able to emulate the hardware and generate that frame in less than 5 milliseconds. I tried enabling the built-in OpenGL and DirectX pipelines via command line switches but the DirectX one doesn't seem to work and the OpenGL one makes everything, even the UI code that I didn't write, about 50 times slower, and it's buggy as ass. I can see why those pipelines are disabled by default.

Anyway, as for JOGL and LWJGL, I found out about them just the other day while perusing Wikipedia. They're external, native code libraries that offer access to true OpenGL from Java. I didn't investigate them properly or whether there's a nice way of using them from an applet, but some demos left me impressed. Full screen 3D performance was as good as you'd expect from any commercial game. (So why couldn't they have built that into the standard library?!)

If I hadn't already got bored with this project again I probably could have added basic support for them by now. But meh. You see, I've a big todo list file for JSNES. I've opened it and looked at it on occasion, but none of the entries on it look particularly appealing or easy, and so I've ignored it.

(I liked this project better during the period when I was doing basic CPU tests and discovering bugs in opcodes. The opcodes are my favorite bit of the code, and they're extremely concisely and clearly written, even if I do say so myself. Sadly, they don't seem to have any bugs left, meaning that the main problems left to tackle are the horrible confusing bits in the PPU.)

[creaothceann]

Sprites have the two priority relationships mentioned above, which is really incompatible with bottom-up rendering.

Meaning no offense to you at all, but I hope that's not true.

None taken. If it's any help to you, after reading the thread again I just had an idea.

Let's look at the rules again:

1. Sprites with a lower index are always in front of sprites with a higher index.
2. A sprite's priority determines between which background layers it is located.

So you can first render (still with your fast rendering technique) a line with all active backgrounds, and a second line with all sprites (if they're enabled on main/sub, see registers 212C and 212D). Then go through the second line and copy all appropriate pixels onto the backgrounds line.

Note that you'll need to remember the source/priority of each pixel in both lines, not just the color. The sprites must be drawn by index, not by priority.

For color math you'll have to do this two times (once for mainscreen, once for subscreen) using the appropriate registers, and combine the results.

[spiller]

1. Sprites with a lower index are always in front of sprites with a higher index.
2. A sprite's priority determines between which background layers it is located.

This is the clearest explanation that I've yet seen, and yet it confirms my suspicion that the rules don't make sense.

Consider this basic layering example in mode 0:

(back)
- Backdrop
- BG4 priority 0
- BG3 priority 0
- Sprites priority 0
- BG4 priority 1
- BG3 priority 1
- Sprites priority 1
- BG2 priority 0
- BG1 priority 0
- Sprites priority 2
- BG2 priority 1
- BG1 priority 1
- Sprites priority 3
(front)

Now imagine we have two sprites 0 and 1. Sprite 0 has priority 1, and sprite 1 has priority 2. FirstSprite is 0.

Now both the rules cannot possibly hold at the same time. Rule 2 wants sprite 0 to go behind sprite 1 because that's their priorities in relation to the background layers, and rule 1 wants the sprite with the lower index (sprite 0) in front. That's why I'm confused.

Anomie's PPU doc offers the solution to this apparent paradox:

Note that only the priority of the topmost sprite is
considered relative to the backgrounds.

This means that the sprites among themselves are sorted according to their indexes relative to FirstSprite, but the only sprite whose priority value is paid attention to is FirstSprite itself, meaning that there is one sprite layer, sandwiched somewhere in the background stack. Yet that's exactly what I tried from the very beginning, and I couldn't get it to work. Sprites kept blinking in and out, not even staying consistent for a full frame.

I'm pretty sure I checked that FirstSprite was being calculated correctly, using the OAM word address and not the OAM byte address, and using the current value rather than the reset value.

[byuu]

This means that the sprites among themselves are sorted according to their indexes relative to FirstSprite, but the only sprite whose priority value is paid attention to is FirstSprite itself, meaning that there is one sprite layer, sandwiched somewhere in the background stack.

You can both sort sprites by order and sort the final sprite against background priorities, the two aren't exclusive.

Take for instance mode 0. You have FirstSprite that has OAM priority 0. But then the sprite immediately after it has priority 1. Doesn't matter, since FirstSprite already plotted a sprite pixel. It wins, OAM priority for this pixel = 0. Now look at the backgrounds, if you had BG3 priority 0, the OAM would win and you'd see the sprite pixel. But if you had BG3 priority 1, that would win and you'd see BG3 instead.

Now move to the next pixel, this time FirstSprite doesn't have a pixel here. So you look at the second sprite and it does. Thus, this pixel becomes OAM priority 1 for now. And in this case, there was only BG3 priority 1 set as well, so the second OAM's pixel wins out and is shown.

The way I do it is to pass over the sprites and render the 34 possible tiles per scanline to a line buffer. Then at the end, I go through and Z-combine the resultant priorities against the background layers. A real SNES almost certainly interleaves each BG priority and OAM priority test together. Real hardware is much better about parallelization than software.

[spiller]

Ohhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh. I finally get it.

[creaothceann]

The way I do it is to pass over the sprites and render the 34 possible tiles per scanline to a line buffer. Then at the end, I go through and Z-combine the resultant priorities against the background layers.

Nice, that's what I meant above with the line buffers.

[grinvader]

Sprites have the two priority relationships mentioned above, which is really completely unrelated to compatibility with bottom-up rendering.

fixed

[creaothceann]

Bottom-up rendering is based on the layering model, which is broken by that peculiar sorting issue. Hence the incompatibility.
The rendering technique used by byuu is no longer strictly bottom-up.

[byuu]

Mine isn't bottom-up at all. Since 90% of the processing both for BGs and OAMs are the same for all priorities, it's a waste of processing time to go through each of them from the bottom to the top.

I break the entire rendering process down to one render_bg() per background, and one render_oam() for all sprites, and Z-compare as each pixel is written.

[grinvader]

The rendering technique used by byuu is no longer strictly bottom-up.

I'm not talking about byuu's in particular.

What you're saying amounts to "if you use the incorrect order, you cannot use that order to get the right result". DUH.

Bottom-up rendering is perfectly feasible when you have a clue how stuff is ordered, because drawing bottom to up CANNOT FAIL at getting the topmost pixel at the end.

I.E. if you can't do it, it means you are not sorting things the right way, e.g. failing.
Sucking at something doesn't make it impossible or incompatible or whatever.

[byuu]

I think he's meaning that you can't bottom-up render sprites, which is true. They don't work that way. But if you generate that sprite line in advance, then bottom-up will work just fine, albeit it won't be the most optimal way to do things.

[grinvader]

You can bottom-up render sprites like anything else, by drawing the least prioritary of them first.

Why does having a different order of priority than the rest mean you can't handle them in the right order ?

[byuu]

Because sprites do not care about priority when compared to other sprites. They only care about which sprite ends up plotting a pixel first.

If you start on the bottom layer and don't process the other layers at the same time, how would you know when all 32 sprite fetches / 34 tile fetches were exhausted?

I mean, I get what you're saying ... just about anything is technically possible. You could do all the RTO calculations and then explicitly go through each layer one-by-one ... but that's just stupid, since you already have all the information on all four layers. You're making something four times slower just to say you can do something, which is silly. One might even say you're cheating by building the RTO list in advance, though I do that myself even with my combined-priority renderer.