Logo

The DDR Research Master Thread - UPDATED 10-20-2021 - Added how max possible DP is calculated.

Register Log In Back To Forums

Post #321 · Posted at 2020-10-27 05:03:16pm 11.9 months ago

Offline travelsonic
travelsonic Avatar Member
168 Posts
Not Set
Reg. 2006-10-18
Log in to send this user a private message
Steam: terminalcancer

Last updated: 2020-10-27 05:03pm
Quote: Wan
This is defined in chunk type 01 (something that it's not visible in memory as it's in the SSQ file because the game loads the values somewhere else).

I've noticed that the entire SSQ file, in the PS2 games at least, is in fact loaded into memory in certain cases at least (for example, edit mode, and training mode) - and that in a game stage, the non-stepdata chunks pertaining to everything else in the song are loaded into memory along with the chunk for the correct stepchart..

Actually, I've noticed that this happens in the 573 AC titles too - overwritten when a new song is loaded.

Quote: Wan
In the stepchart chunk first you have the Metric Value of all the arrows and then the TYPE of each arrow (so a left+right jump is not two separate arrows but a "left+right arrow type")

Specifically, I noticed that the values for individual arrows are powers of 2 (1 for left, 2 for down, 4 for up, and 8 for right), and that pairs (or even groups of 3 arrows, or even having 4 arrows) are just the sum of these values. For example, 1+2+4+8 = 15 (0xF), which shows up as all 4 arrows at once. 1+4+8 gives you a value a value of 13 (0xD), which shows up as left + up + right)

@Wan BEAUTIFUL writeup on the SSQ format btw, thank you very much for that - I learned a lot.

Post #322 · Posted at 2020-10-28 11:57:09am 11.9 months ago

Offline Wan
Wan Avatar Member
387 Posts
Chile
Reg. 2008-01-13
Log in to send this user a private messageTwitter: WannyTiggah
Steam: WannyTiggah
"I want to change my username =("
Most of the info I posted is thanks to pkgingo's investigation. If you're in youknowwhich tracker lemme know.

Post #323 · Posted at 2020-10-28 02:00:17pm 11.9 months ago

Offline travelsonic
travelsonic Avatar Member
168 Posts
Not Set
Reg. 2006-10-18
Log in to send this user a private message
Steam: terminalcancer

Last updated: 2020-10-28 02:00pm
There is one thing that confuses me though - the assertion that the entries before the step values are all pure metric values, multiples of 4096

I am confused because if we go to an excerpt of the excerpt I posted before

00 01 04 00 // Measure 16 Beat 4
00 01 08 00 // Measure 16 Beat 8

These are not even multiples of 4096 - which means I am missing something clearly (not your fault of curse Big Grin )

Maybe it's also my fault for trying to analyze this shit just after waking up / before having my coffee.

Side tangent... just realized that I've been registered here for 14 years and 10 days exactly... 0___0 Gotta love having ADHD... Laughing Out Loud )

Post #324 · Posted at 2020-10-28 02:28:53pm 11.9 months ago

Offline Arcorann
Arcorann Avatar Member
45 Posts
Not Set
Reg. 2015-01-29
Log in to send this user a private message
Quote: travelsonic
There is one thing that confuses me though - the assertion that the entries before the step values are all pure metric values, multiples of 4096

I am confused because if we go to an excerpt of the excerpt I posted before

00 01 04 00 // Measure 16 Beat 4
00 01 08 00 // Measure 16 Beat 8

These are not even multiples of 4096 - which means I am missing something clearly (not your fault of curse Big Grin )

Maybe it's also my fault for trying to analyze this shit just after waking up / before having my coffee.

Side tangent... just realized that I've been registered here for 14 years and 10 days exactly... 0___0 Gotta love having ADHD... Laughing Out Loud )

I think there's a mistake there - these values are written as big endian, right? They should be the 2nd and 3rd beats of the 17th measure (beat 65 and 66, if the start is beat 0), as 1 measure = 4 beats = 4096 units, and 1 unit is therefore 1/1024 beat.

Quote: Wan
You need a small crash course in the SSQ format.

There's 2 concepts involved in the format, not THAT different to SM:
- The stepchart itself, defined in chunk type 03. The arrows use a metric value whose calculation is fixed (read below).
- The timing and tempo of the song. Put it simply, it's the speed at how the stepchart scrolls. This is defined in chunk type 01 (something that it's not visible in memory as it's in the SSQ file because the game loads the values somewhere else). The calculation of this is definitely variable... it doesn't have a "BPM" per se, but definition(s) of when in time happens a tempo section of the stepchart (expressed in tick values of its beginning and end).

- In the stepchart chunk first you have the Metric Value of all the arrows and then the TYPE of each arrow (so a left+right jump is not two separate arrows but a "left+right arrow type")
- The metric value is written as
(number of measure) x 4096
This means that the format actually allows to be VERY precise for where they want to place an arrow!
So, for example... for that arrow at measure 16 you have 00 01 00 00 = 65536 = 4096 x 16.

Extra info you may come across:
- For freezes there's separate definitions of when a freeze starts and when it ends.
- As long as the metric value + arrow type match, the pair can technically be anywhere. For example, if they wanted to be masochist, all the freeze starts of the song can be defined at the beginning of the chunk and the stepchart would still be the same.
- The "timing resolution" of the SSQ format has changed along the years. Between 4th Mix and MAX2 it was 1/75 sec per tick, in Extreme changed to 1/150 sec and now DDR A/A20 allows for variable timing per song (Endymion's is 1/1000 sec!)

EDIT:
I had the impression that the timing resolution affected the precision of the stepchart, but after writing the above I understand that the timing resolution affects how precise can be the "BPM changes"... the stepchart's precision has been up to 1/4096 all this time.

Interesting info, didn't know that the "timing resolution" was only 1/75 sec in the MAX2 era. Do you know what it was for the early mixes (1st to 3rd)?

Incidentally, it's well known that 12ths and 24ths were rounded to the nearest 64th for a long time (e.g. bag). Based on my previous understanding of SSQ files I'd assumed that the format had changed to allow proper 12ths, but this seems to suggest that there was no such change, is it possible to check?

Post #325 · Posted at 2020-10-28 03:03:21pm 11.9 months ago

Offline travelsonic
travelsonic Avatar Member
168 Posts
Not Set
Reg. 2006-10-18
Log in to send this user a private message
Steam: terminalcancer

Last updated: 2020-10-28 03:09pm
Just got IDA reinstalled on my computer (at home - am at work now) so I am finally able to get back to work on my disassembly of MAX2 JP. Hopefully the info Wan et all provided will allow me to document the routines that actually handle stepdata. Big Grin

Not to mention being able to get back to work on trying to do disassemblies of 573-based AC mixes (and perhaps others in future).

Post #326 · Posted at 2020-10-28 04:17:29pm 11.9 months ago

Offline Wan
Wan Avatar Member
387 Posts
Chile
Reg. 2008-01-13
Log in to send this user a private messageTwitter: WannyTiggah
Steam: WannyTiggah
"I want to change my username =("

Last updated: 2020-10-29 09:02pm
Quote: travelsonic
There is one thing that confuses me though - the assertion that the entries before the step values are all pure metric values, multiples of 4096

I am confused because if we go to an excerpt of the excerpt I posted before

00 01 04 00 // Measure 16 Beat 4
00 01 08 00 // Measure 16 Beat 8

These are not even multiples of 4096 - which means I am missing something clearly (not your fault of curse Big Grin )

Maybe it's also my fault for trying to analyze this shit just after waking up / before having my coffee.

Side tangent... just realized that I've been registered here for 14 years and 10 days exactly... 0___0 Gotta love having ADHD... Laughing Out Loud )
I never said that the measure value has to be an integer number Tongue

00 01 04 00 = 66560 = 4096 x 16.25.
The arrow is right at the 1st quarter of the measure. Converting the SSQ to SM and opening it in ArrowVortex, it's this arrow (the position being confirmed in the status bar):
https://i.imgur.com/L2b33L9.png
Fun fact: using Burning Heat's decrypted audio from Extreme AC, the song is offsync by 0.06 secs but with the audio from MAX2 it's only by 0.03

Quote: Arcorann

Interesting info, didn't know that the "timing resolution" was only 1/75 sec in the MAX2 era. Do you know what it was for the early mixes (1st to 3rd)?

Incidentally, it's well known that 12ths and 24ths were rounded to the nearest 64th for a long time (e.g. bag). Based on my previous understanding of SSQ files I'd assumed that the format had changed to allow proper 12ths, but this seems to suggest that there was no such change, is it possible to check?

1.- Sadly nope, as the step format used is different. IIRC it's 1/75th as well.

2.- It's not that the format didn't support 12th/24ths/48th, it's the game engine that didn't.
So it's not that bag's steps were "rounded", the chart was made like that. That's why they didn't have problems in purposely adding the old Extreme chart in DDR X2 after they fixed the quantization in DDR X.

Have in mind that DDR STILL uses SSQ as of A20+

Post #327 · Posted at 2020-11-09 04:50:35pm 11.5 months ago

Offline travelsonic
travelsonic Avatar Member
168 Posts
Not Set
Reg. 2006-10-18
Log in to send this user a private message
Steam: terminalcancer

Last updated: 2020-11-09 05:56pm
Quote: Wan

I never said that the measure value has to be an integer number Tongue

I might be doing something wrong OR overlooking something (been bouncing between this and juggling a dozen other projects + work, so I am not as focused as I can be/should be), but it doesn't seem like these values adhere to the IEEE 754 floating point representation standard...? that and from my observations at least, it seems like the literal stepdata processing doesn't even touch the PS2's Floating Point Unit (as opposed to the drawing of the arrows on screen - which does seem to touch the FPU (at least in my observations). If those observations are correct, I am a bit more confused about how we are looking at this through the lense of non-integer values, and feel like a total duncecap for admitting it... XD

Is it possible that they (being Konami - and within the purview of the SSQ format) went with a completely unconventional route to do the equivalent of decimal values? Or am I REALLY derailing myself thought process wise? Holy crap, I feel stupid.

Post #328 · Posted at 2020-11-09 11:15:35pm 11.5 months ago

Offline Wan
Wan Avatar Member
387 Posts
Chile
Reg. 2008-01-13
Log in to send this user a private messageTwitter: WannyTiggah
Steam: WannyTiggah
"I want to change my username =("
You're definitely derailing yourself and thinking it too much xD
It's much simpler: the decimals are relative to the position IN THE MEASURE, forget about the PS2 FPU.
A value of X.5 is the middle of measure X.

Another example
https://i.imgur.com/hh5UOqj.png
In this case, the arrows are (in order) in measures:
4.000
4.375
4.500
5.000

Post #329 · Posted at 2021-03-26 07:39:32am 6.9 months ago

Offline travelsonic
travelsonic Avatar Member
168 Posts
Not Set
Reg. 2006-10-18
Log in to send this user a private message
Steam: terminalcancer

Last updated: 2021-03-26 07:39am
So I started an additional disassembly project in Ghidra to see if I can glean insight from the decompilation functionality it has.

This is the function where the player's earned DP is calculated, and a percentage is gained:
https://media.discordapp.net/attachments/540290331623686144/793543415920852992/unknown.png

Now, I bet in actuality they used a for loop to go through this process of processing the earned DP, and that was just unrolled upon assembly (and this decompilation is the interpretation of said unrolled loop) - that is, the original code wouldn't look this ... horrid, for a lack of better terms. That said, it is still useful, IMO, for understanding the game engine, and its inner working. For example, the parameters look correct for this function (param_1 being a pointer to an array that holds the step counts for each judgment, param_2 holding the number of non-freeze steps in the chart, and param_3 holding the number of freeze steps in the chart).

It does leave me to wonder: Why didn't they just calculate the max earned DP when the stage began / store it in RAM? I mean, once the stepchart is loaded, the number of non-freeze steps, and the number of freeze steps doesn't change (and the multipliers are constants hardcoded in the game ROM anyways).

Actually, that's another question I have ... why if they are getting the max DP earned do they go about multiplying (non-freeze-steps * perfect_multiplier), but use a 2nd definition of the OK multiplier for calculating the max freeze step DP, rather than just using the existing one? This multiplier is in the game ROM twice.


ER OK, 3rd quandry that comes to mind: What are these other 3 things in the array that are multiplied together ( param_1[5]+ a value at 0x26D684; param_1[8] + a value at 0x26D694; param_1[9] + a value at 0x26D698 From what I see, in a normal stage, at least, those elements are all 0'd out (the values in array, and the values at 0x26D684, 0x26D694, & 0x26D698 that said values are multiplied by, I mean)

_______________________________________________________________________________________________________________________________

Began putting in pages for DDR data structures onto the GitHub I set up (Thanks to Root760's debugging data extraction efforts! ):
https://github.com/travelsonic/Dance-Dance-Revolution-Game-Engine/tree/master/DDRMAX%20ENGINE/Data%20Structures

Right now, it's mostly just structure definitions (aside from music_info, for which I began the process of flushing out a detailed explanation for each struct member data field), but I will - like with the music_info struct page, work to flush out information about each member data field of each data structure.

If anyone has any corrections for the structure definitions provided on each page, definitely let me know, as accuracy matters to me when it comes to this information, and any information I make public on REing DDR games.
_______________________________________________________________________________________________________________________________
So apparently a month and a day ago, a prototype of Dancing Stage Fusion was dumped by Hidden-Palace.org

And, according to TCRF, the FreeZe! graphic that went unused in prior versions was actually translated into French, German, Italian, and spanish:
https://tcrf.net/images/5/51/DSfusionPS2-freezeEN.png
https://tcrf.net/images/9/91/DSfusionPS2-freezeFR.png
https://tcrf.net/images/4/4c/DSfusionPS2-freezeGE.png
https://tcrf.net/images/0/02/DSfusionPS2-freezeIT.png
https://tcrf.net/images/d/d5/DSfusionPS2-freezeSP.png
Huh. Interesting.

I think I just re-enabled that FreeZe! graphic in MAX2!

Please tell me I actually did it.... holy crap!

https://pbs.twimg.com/media/ExY6gdOWgAAmuI0?format=png&name=small

Post #330 · Posted at 2021-04-02 02:05:44pm 6.7 months ago

Offline Arcorann
Arcorann Avatar Member
45 Posts
Not Set
Reg. 2015-01-29
Log in to send this user a private message
System 573 emulation was recently improved in MAME thanks to Windy Fairy (Github, Twitter). Lately it seems that they've been doing some hacking on DDR Extreme arcade.

In particular, they found some leftover code for Solo mode, and wrote out cheats to enable it in 5th, MAX, MAX2 and Extreme. 6-panel can also be enabled, but it seems that the arrows show up in the wrong place.

Also, they've hacked the game's autoplay to hit all Marvelouses, after some effort:

Post #331 · Posted at 2021-04-04 11:03:44pm 6.6 months ago

Offline travelsonic
travelsonic Avatar Member
168 Posts
Not Set
Reg. 2006-10-18
Log in to send this user a private message
Steam: terminalcancer

Last updated: 2021-04-04 11:04pm
Dude, that's fucking sick!

You know, call me crazy, but between finding the 5thMIX song select BPM font/hud graphic (which I added on the TCRF page recently), between these awesome dudes finding SOLO mode leftover in 5th/max/max2/extreme, and whatnot, I think it actually gives legitimacy to this screenshot from a prototype of DDRMAX AC:
http://www.arcade-museum.com/images/118/1181242101265.gif
Which many here, in another thread, dismiss as a mere mockup. I mean, I had my doubts about it being a mockup before - between leftovers, and old graphics being used as placeholders not being too odd in game dev, from what I saw in the development of other games like Sonic the Hedgehog ... same with successive games using and modifying an existing engine (CS DDR games using code from AC mixes, for instance - and, to go back to Sonic for a second, Sonic the Hedgehog 2 being built off of Sonic the Hedgehog 1, Sonic the Hedgehog 3 being built off of Sonic the Hedgehog 2, and so on), and the like.

Which makes me wonder what other leftovers can be found in previous titles.

IMO, this screenshot from the development of DDR EXTREME AC proves the legitimacy of the above MAX AC prototype screenshot in so far as that they'd use assets from previous mixes. Look at those garbled MAX2 graphics, and all those ABSOLUTE slots. 😂😂😂😂😂
https://web.archive.org/web/20030220001017im_/http://www.konami.co.jp/am/ddr/ddr_new/images/l_bag.gif

Post #332 · Posted at 2021-04-05 12:48:04am 6.6 months ago

Offline GhostSpirit
GhostSpirit Avatar Member
186 Posts
Not Set
Reg. 2018-08-17
Log in to send this user a private message
" ( 0﹏0)"
Seeing all of these findings made me wonder something, is it possible to force the game to run at 480p all the time? I know it lowers the resolution to 240p during gameplay but I wonder if there's a way to make it run at that resolution? Or even run it at 60fps through overclocking?

Post #333 · Posted at 2021-04-05 01:23:14pm 6.6 months ago

Offline travelsonic
travelsonic Avatar Member
168 Posts
Not Set
Reg. 2006-10-18
Log in to send this user a private message
Steam: terminalcancer

Last updated: 2021-04-05 01:23pm
Quote: Arcorann
System 573 emulation was recently improved in MAME thanks to Windy Fairy (Github, Twitter). Lately it seems that they've been doing some hacking on DDR Extreme arcade.

In particular, they found some leftover code for Solo mode, and wrote out cheats to enable it in 5th, MAX, MAX2 and Extreme. 6-panel can also be enabled, but it seems that the arrows show up in the wrong place.

Also, they've hacked the game's autoplay to hit all Marvelouses, after some effort:


Not only that, but the dude also found leftover source code from earlier builds of DanceDanceRevolution4thMIX (in 4thMIX itself, ofc)

Quote: GhostSpirit
Seeing all of these findings made me wonder something, is it possible to force the game to run at 480p all the time? I know it lowers the resolution to 240p during gameplay but I wonder if there's a way to make it run at that resolution? Or even run it at 60fps through overclocking?

Hm, that is an interesting enigma. Definitely worth looking into.

Post #334 · Posted at 2021-04-06 12:09:03am 6.6 months ago

Offline Wan
Wan Avatar Member
387 Posts
Chile
Reg. 2008-01-13
Log in to send this user a private messageTwitter: WannyTiggah
Steam: WannyTiggah
"I want to change my username =("
Quote: GhostSpirit
Seeing all of these findings made me wonder something, is it possible to force the game to run at 480p all the time? I know it lowers the resolution to 240p during gameplay but I wonder if there's a way to make it run at that resolution? Or even run it at 60fps through overclocking?
There's not much you can do. The gameplay graphics are made for 320x240.
They were redone for the PS2 games (where gameplay runs at 640x480)

Post #335 · Posted at 2021-09-27 01:15:38pm 3.4 weeks ago

Offline travelsonic
travelsonic Avatar Member
168 Posts
Not Set
Reg. 2006-10-18
Log in to send this user a private message
Steam: terminalcancer

Last updated: 2021-09-27 01:20pm
From looking at the leftover code fragments in 4thMIX, I realized that I'm gonna have to restructure my entire github repository AND my initial post on this thread - as it became clear that what I dubbed the "MAX engine" really is much older (though IDK if it goes back to 4thMIX, or even earlier - though the latter is absolutely possible). Variable names, of data types whose definitions could be ascertained from Root670's debugging data, all present (down to specific member names, and types).

So apparently, the fact that the music_info structures have entries that reference things like the title screen, how to play instruction screen, name entry screen, etc, is no accident - and they actually DO something.

Even the operator menu Sound Scale Test has a music_info struct, and you CAN change what is played from the chime to game music.

Because 2ndMIX and back was all analog audio, the indexes for what audio could be chosen could include the full BGM tracks for each song. I suspect that changed in 3rdMIX onwards BECAUSE of the use of mp3 encoding for the song BGMs.



Welp, I was WRONG - EVEN IN Digital 573 mixes, the full song tracks seem to be index in such a way where you can make the Sound Scale Check play full song BGMs!

So I was wrong - the playing of full BGMs through the Song Scale Check is doable in system573 Digital mixes too!

Here is it being done in Dancing Stage EuroMIX 2



the g_game_w structure which holds/tracks game information (player score, scoring, combo, diet info, relevant game flags, etc) is 701,844 bytes, or 701.844kb in the E3 demo of EXTREME - and 708,752 bytes, or 708.752kb in EXTREME U.S' retail release. I wonder how, or why, it gained that 6,908 bytes/6.908 kb of data.

Modifying the source code that came with PCSX2Dis to make things just a bit better - more space to make comments alongside each line, more space in the respective textboxes to allow bigger struct sizes and more struct data member definitions in a struct, bigger font size so one's eyes don't ache like hell from trying to read the text, and more than 10 breakpoints (seriously, that's not enough!) - maybe once I've gotten that done I can try to work on the stability issues it has.

Yes, PCSX2's debugger has improved a lot in recent years, but there is something I like about PCSX2Dis' UI, even if it means running an older version of PCSX2 alongside it.

So I had been looking at the DDR EXTREME 2 2005-06-222005-05-10 prototype and noticed that, unlike in the retail release, the DWARF debugging data had not been stripped.

Interestingly, I think this is the first PS2 DDR title whose debugging data leftover includes a massive amount of functions for edit mode, and handling SSQ format data (at least in the context of edit mode) - which is actually, IMO, rather monumental for ANY REing effort of Ps2 DDR titles.

Dwarf2CPP has been rather good at getting the data structures out of the DWARF data, along with the function declarations ... putting that stuff into GHidra will absolutely go a long way (even if in the context of edit mode)

https://media.discordapp.net/attachments/484898712397545483/892036255787941918/unknown.png?width=1251&height=678

STUPID QUESTION: why would an address be passed as a void pointer? Is it DWARF2CPP messing up with the parameters, or was it actually done that way on purpose? I'm so confused. (been a hectic while since I looked into any of this stuff)

Post #336 · Posted at 2021-09-27 01:41:52pm 3.4 weeks ago

Offline DMNBT
DMNBT Avatar Member+
68 Posts
Chile
Reg. 2014-06-15
Log in to send this user a private message
PlayStation Network: DMNBT-CL
Huh, the only context I've ever seen void* is when we used it as a sort of pointer to any type of data that then we'd have to parse to see which type it really was, but the context in which we used that trick doesn't seem to apply here as it seems to be expecting a concrete thing as opposed to a pointer to data of unknown type

Post #337 · Posted at 2021-10-20 03:09:50pm 1.3 days ago

Offline travelsonic
travelsonic Avatar Member
168 Posts
Not Set
Reg. 2006-10-18
Log in to send this user a private message
Steam: terminalcancer

Last updated: 2021-10-20 03:09pm
Quote: DMNBT
Huh, the only context I've ever seen void* is when we used it as a sort of pointer to any type of data that then we'd have to parse to see which type it really was

Same here - and it doesn't seem to match up with the coding styling I've observed from looking at the decoded DWARF debugging data from other DDR PS2 titles, and from looking at the PSX BM 5thMIX source code) in terms of how they handle passing addresses... then again, there ARE things D2CPP doesn't seem to do - like preserve variables and function parameters being declared with signed explicitly (signed int, signed short, etc instead of int, short, etc) and so on and so forth.

IF I can get an install of MetroWerks CodeWarrior (the one from the Internet Archive) installed and working, maybe I can run Root670's DWARF debugging data script to get a readout and see if this was accurate, or a foul up on Dwarf2CPP's part.

(Yes, I am aware that these mixes were coded in C, but IDK if that matters much)

Looking at debugging data for other functions that handle memory addresses, it seems that in all cases the addresses are passed as unsigned ints.... maybe Dwarf2cpp fucked up?

OK I am an idiot, looked more carefully and apparently it is something that is done, passing an address as a void ptr (at least in the BM 5th Source)

I just made a quick, probably very rough-around-the-edges, outline if you will of how the dance_level (your letter grade) is calculated https://pbs.twimg.com/media/FBpCJTOWYAAF5nL?format=png&name=medium

OK, so ... I now understand why the PERFECTs DP, and OKs DP multiplier multiplier are repeated in the game:

In calculating the percentage used to calculate your dance_level, it pulls from the data structure g_each_w where the song's total_notes - the total non-freeze steps, and total_freez - the total number of freeze steps, are stored, in order to calculate the maximum possible DP that a player can earn.

It multiplies total_notes by its respective multiplier, adds that to the product of total_freez * its respective multiplier.

But it makes me wonder, why not just use the existing multiplier constants for PERFECTs and OKs?
Register Log In Back To Forums

0 User(s) Viewing This Thread (Past 15 Minutes)

©2006-2021 Zenius -I- vanisher.com -5th style- IIPrivacy Policy
Web Server: 2% · Database: 4% · Server Time: 2021-10-21 23:27:42
This page took 0.02 seconds to execute.
Theme: starlight · Language: englishuk
Reset Theme & Language