Logo

Spreadsheet of step data for all songs and charts released in DanceDanceRevolution

Register Log In Back To Forums

Post #1 · Posted at 2022-12-10 06:29:52pm 1.2 years ago

Offline Landpaddle
Landpaddle Avatar Member
34 Posts
Not Set
Reg. 2022-11-16


Last updated: 2023-11-20 12:48pm
Link to spreadsheet:
https://docs.google.com/spreadsheets/d/10NT1VPYV8JHEpi68SBCTApOO5mWEjIAbJP3C5Csthyk/edit?usp=sharing

Published version:
https://docs.google.com/spreadsheets/d/e/2PACX-1vSYDo4PsjxMeiTGDoO62uoDzrCujw2RH8TF_PWRxmvp9yJOs_Y2lPnCk6Npndgm8_mOEqYoWBhcBJEj/pubhtml
(longer initial load and no sorting or filtering, but faster for string search once loaded)

The spreadsheet is best viewed on a desktop or laptop computer and may not include all rows when viewed on mobile devices.

What this is
A spreadsheet of all charts released in DanceDanceRevolution with song and step data for each chart
- The spreadsheet's contents are sourced from .sm file transcriptions of official data that was analyzed with StepMania 5.1-b2 using the theme Simply Love v5.1.2.
- Only step data for transcriptions of official charts is included.
- Songs and charts specific to console and spin-off releases are also included.

A resource for song data, step data, tech, and community data
- Song data: Title, Artist, Length (seconds), Song Folder
- Step data: Difficulty, Rating, Display and True BPM (min and max), Steps, Jumps, Holds (Freezes), Shock Arrows, Hands, Total Stream, Maximum Notes/Second, GROOVE RADAR values
- Tech data: Crossovers, Footswitches, Sideswitches, Jacks, Brackets
- Community data: "Real-world scoring difficulty" from https://ddrcommunity.com/ddr-a3-scoring-difficulty-tier-list/ (Skill Attack, 2022) and https://ddrcommunity.com/ddr-a3-difficulty-tier-list-provided-by-3icecream/ (Sanbai Ice Cream, 2023)

Sortable
- Each column can be filtered in ascending or descending order.
- Columns can be filtered to find charts with specific data, such as Basic charts, specific ratings, step count thresholds, songs with a specific BPM, etc. This can be combined with ascending and descending sorting.

Informational
- Song titles use their non-localized versions, English translations, romanized versions, and ASCII-only titles where applicable so that both Japanese and English-speaking users can easily find them. Alternate titles are separated with |
- Artist titles use the same format.
- For translations, Remywiki is the preferred resource, followed by .sm file naming if it differs.
- Column headers include notes with additional information about that column and outgoing links to informational posts and resources.

Computed
- Many computed columns are included that tally different categories of steps and tech, find the ratio of a particular step type or tech to total steps, and compute the step type/second.
- The "Estimate of relative difficulty" and "Alternate estimate of relative difficulty" use custom formulas to estimate the difficulties of songs using chart data.
- The "Estimated rating using MLR" columns estimate the rating based on a linear formula. The coefficients were generated in Microsoft Excel with the "Regression" tool from the Analysis ToolPak add-in. The constant was set to 0.

What this isn't
Flawless
- The information in this spreadsheet was entered manually. This means that there may be transcription errors.
- The Length (seconds) column is likely to differ slightly from official data. In cases where this is obviously wrong, it has been changed to match information from Remywiki.
- The difficulty estimate columns only provide a rough estimate. Real-world scoring difficulty is preferred.

100% Comprehensive
- Some songs and charts may have been excluded due to an incomplete record of community .sm transcriptions.
- Scoring difficulty estimates from DDR Community mainly include arcade releases and exclude brand new charts.
- Nonlinear multiple regression is not included.

Additional info
- Alternate spreadsheet formats, project scripts, and the multiple linear regression process are available here in a Google Drive folder: https://drive.google.com/drive/folders/1fRZPDVv_WbFC9ZC2LHao-ExqhU9DDD7i?usp=share_link
- A huge shout-out to Remywiki and DDR Community for additional song/chart information and scoring difficulty estimates, respectively.

Updates
2022 December 14
- Added GROOVE RADAR columns to the spreadsheet. These values were calculated with a script that was created by teleshoes and modified by razorblade. The script is available here: https://zenius-i-vanisher.com/v5.2/thread?threadid=11050 https://www.dropbox.com/s/6cy6gwvfxt2j7ps/DDR_Groove_Radar_Generator.zip?dl=0
- Added missing entries for Dancing Stage UNIVERSE and Dancing Stage UNIVERSE2.
- Moved the "Song Folder" column to the first column and froze it.
- Reduced the number of colored columns for first-load duration and performance reasons.
- Added the original Japanese titles and their romanizations to the "DDR with MARIO" folder.

2023 March 11
- Added full chart data for Double, Couple, Solo (6-panel), and 3-panel gameplay types.
- Added new A3 charts.
- Added stepchart info for several cut songs from 3rdMIX, 3rdMIX PLUS, 4thMIX PLUS, EuroMIX, and EuroMIX2 (credit: https://zenius-i-vanisher.com/v5.2/viewsimfilecategory.php?categoryid=1164).
- Updated the ULTRAMIX and UNIVERSE games' data with newer and more accurate info.

2023 March 26
- Added true minimum and maximum BPM values for all songs and charts in addition to display BPM
- Added BPM changes and stops for all songs and charts
- Added computed columns to show the difference between display BPM min/max, true BPM min/max, and display and true BPM
- Added computed columns for the Couples game type to show differences between player 1 and 2 steps
- Made additional updates and fixes

2023 September 1
- Added English transliterations for all song artists
- Added additional alternative 7-bit ASCII compatible song and artist titles for easier text searching and filtering
- Fixed various errors in song and artist titles
- Updated Google Drive backup with updated stepfiles, alternate spreadsheet formats, and JSON and Lua script output files: https://drive.google.com/drive/folders/1fRZPDVv_WbFC9ZC2LHao-ExqhU9DDD7i

2023 October 16
- Added official GROOVE RADAR data from Bemaniwiki for Single Play and Double Play charts in addition to the script-estimated values: https://bemaniwiki.com/?DanceDanceRevolution+A3
- Added real-world scoring data for rating 1-19 Single Play and Double Play charts from Sanbai Ice Cream (Version 5 Revision 1 data): https://3icecream.com/ddr/home - https://ddrcommunity.com/ddr-a3-difficulty-tier-list-provided-by-3icecream/ - https://www.reddit.com/r/DanceDanceRevolution/comments/176dlcy/comment/k4o81d8/?context=3
- Added computed columns to compare official and script-estimated GROOVE RADAR values
- Added alternate Google Sheets published website version

2023 November 20
- Updated Sanbai Ice Cream scoring difficulty data to Version 6 Revision 1 (2023 November 6)
- Added ARFF files (discrete and continuous) to Google Drive folder for difficulty estimation through machine learning in WEKA. The header information can be deleted to create text files or CSVs for other machine learning tools
- Added additional filter views to filter by X-rating charts, charts with official GROOVE RADAR values, and charts with Sanbai Ice Cream scoring data
- Updated alternate file formats, script output data, and extracted stepfiles

Post #2 · Posted at 2022-12-10 09:55:41pm 1.2 years ago

Offline forcednature
forcednature Avatar Member
1,009 Posts
Germany
Reg. 2012-06-12

""Captain uwu""
I like that it has like, *all* the songs. Like you said the difficulty calculations seem off, most of them seem to think that Lindwurm CSP is the hardest AC chart though that is obviously not true. But the actual database itself is very useful.

Maybe you could add groove radar data to the sheet? There's a script that does the proper ones for you.
https://i.imgur.com/SkGLw82.png

Post #3 · Posted at 2022-12-10 11:10:00pm 1.2 years ago

Offline Landpaddle
Landpaddle Avatar Member
34 Posts
Not Set
Reg. 2022-11-16

Quote: FoRCeDNaTuRe
Maybe you could add groove radar data to the sheet? There's a script that does the proper ones for you.

I can take a look into this. I can't promise that these will be added quickly, but this seems like a good thing to include. Thank you for the link to the script.

Post #4 · Posted at 2022-12-10 11:17:05pm 1.2 years ago

Offline Dancefreak
Dancefreak Avatar Member+
340 Posts
United Kingdom
Reg. 2006-08-12

Something I'd like to know is why Love is the power from 2nd mix ps2 is listed as having full charts, apparently the "Official" simfile on here does it too but I'm fairly sure those were fan made ? But other than that it's a pretty cool resource to look through and maybe have to quickly reference some song stats!

Post #5 · Posted at 2022-12-11 06:38:51pm 1.2 years ago

Offline Sigrev2
Sigrev2 Avatar Member+
4,159 Posts
United States
Reg. 2009-10-17

Nintendo Network ID: Sigrev2Nintendo Switch Friend Code: SW-2884-7660-37993DS Friend Code: 3883-7652-3160
"suffering from success"
this thing is absolutely mental. to whoever had the time and patience to put all of this together, you're not human - you're something more than that and i'm intimidated immensely.

Post #6 · Posted at 2022-12-11 06:45:40pm 1.2 years ago

Offline eataninja
eataninja Avatar Member
803 Posts
United States
Reg. 2011-05-05

Quote: Sigrev2
this thing is absolutely mental. to whoever had the time and patience to put all of this together, you're not human - you're something more than that and i'm intimidated immensely.

100% agreed.

I actually think the relative difficulty column is good, in the sense that it -seems- as accurate as a computer can get, and should highlight obvious discrepancies between one chart and others at its level. I kinda want to see how usable that data is, if I get the chance to play around with it later.

Also, if I could make -one- suggestion, I would maybe move folder info to the front, and then freeze that column and the difficulty level column. I know that reduces the amount of numeric data that is visible at once, but since there's so much data on the sheet, it makes things easier to find and compare with each other.

Post #7 · Posted at 2023-03-26 07:25:52am 1 year ago

Offline Landpaddle
Landpaddle Avatar Member
34 Posts
Not Set
Reg. 2022-11-16


Last updated: 2023-07-16 10:16pm
Currently working on adding computed Groove Radar values from the script by teleshoes and razorblade.

Quote: Dancefreak
Something I'd like to know is why Love is the power from 2nd mix ps2 is listed as having full charts, apparently the "Official" simfile on here does it too but I'm fairly sure those were fan made ? But other than that it's a pretty cool resource to look through and maybe have to quickly reference some song stats!

That is because I used Ziv simfile packs from the "View Charts" section. Any mistakes in these simfiles will be reflected in the chart data.

The only sections that were modified from Ziv file data were the duration field for the DanceDanceRevolution II (Wii) (North America) files with info from Remywiki since the duration of the full and short versions was shown as identical otherwise.

Please post any mistakes in simfiles you may have noticed in https://zenius-i-vanisher.com/v5.2/thread?threadid=5204

2023 March 26 edit: The extra Love is the Power charts have been removed now that the charts on Ziv have been updated to remove them as well.

Quote: FoRCeDNaTuRe
Like you said the difficulty calculations seem off, most of them seem to think that Lindwurm CSP is the hardest AC chart though that is obviously not true. But the actual database itself is very useful.

Quote: eataninja
I actually think the relative difficulty column is good, in the sense that it -seems- as accurate as a computer can get, and should highlight obvious discrepancies between one chart and others at its level.

If you have any advice or recommendations for changing the "Estimate of relative difficulty using weighted step data" or "Alternate estimate of relative difficulty" columns, please let me know. I am open to making any changes to the formulas that make these trial-and-error estimates more accurate. You are also welcome to request edit access to the document or copy this data into another spreadsheet to tinker.

For the MLR estimates, the dataset for these is in the Google Drive folder linked at the bottom of the first post. I highly encourage anyone more familiar with statistical modeling to make a nonlinear multiple regression estimate using the same data. NMLR is something I looked into but couldn't figure out, and it might make the best model outside of the trial-and-error custom formulas.

Quote: eataninja
Also, if I could make -one- suggestion, I would maybe move folder info to the front, and then freeze that column and the difficulty level column. I know that reduces the amount of numeric data that is visible at once, but since there's so much data on the sheet, it makes things easier to find and compare with each other.

I am not fully clear on this. What is the exact order you would like the frozen columns in from left to right? The current format for future reference is Title,Difficulty. I assume you are asking for one of the following:
Title,Difficulty,Song Folder
Title,Difficulty,Rating,Song Folder
Song Folder,Title,Difficulty
Song Folder,Title,Difficulty,Rating

2022 December 12 edit: I have merged the frozen columns into a Song Folder,Title,Difficulty format. The Song Folder values have been given common abbreviations to shorten the strings, and the key to these abbreviations is in a note on the column header. The frozen columns now make a unique key pairing of folder, title, and difficulty (assuming no duplicate titles in a folder). Let me know if you'd also like the Rating column frozen.

2023 March 11 update

- Added full chart data for Double, Couple, Solo (6-panel), and 3-panel gameplay types.
- Added new A3 charts.
- Added stepchart info for several cut songs from 3rdMIX, 3rdMIX PLUS, 4thMIX PLUS, EuroMIX, and EuroMIX2 (credit: https://zenius-i-vanisher.com/v5.2/viewsimfilecategory.php?categoryid=1164).
- Updated the data of charts from the ULTRAMIX and UNIVERSE series with newer and more accurate info.

2023 March 26 update

- Added true minimum and maximum BPM values for all songs and charts in addition to display BPM
- Added BPM changes and stops for all songs and charts
- Added computed columns to show the difference between display BPM min/max, true BPM min/max, and display and true BPM
- Added computed columns for the Couples game type to show differences between player 1 and 2 steps
- Made additional updates and fixes

Post #8 · Posted at 2023-04-20 05:36:24pm 11.4 months ago

Offline SpyHunter29
SpyHunter29 Avatar Member
330 Posts
United States
Reg. 2008-06-09

I wonder, would it be possible to record how many notes of each timing type (quarter notes, eights, sixteenths, and others) are present in each chart?
98% of teenagers do or have tried caving into peer pressure. If you're one of the 2% who hasn't, DO NOT copy & paste this in your signature.

Post #9 · Posted at 2023-11-20 12:48:43pm 4.2 months ago

Offline Landpaddle
Landpaddle Avatar Member
34 Posts
Not Set
Reg. 2022-11-16


Last updated: 2023-11-20 12:48pm
Quote: SpyHunter29
I wonder, would it be possible to record how many notes of each timing type (quarter notes, eights, sixteenths, and others) are present in each chart?

It's possible, but I'd realistically need a script to tally these. Manually counting all note timings across 20k+ charts would take over a year.

A better long-term solution would be automating the process via scripts and displaying the output on a website. Matt Greer made a Stepcharts website at https://ddr.stepcharts.com/ that is much more polished and performant than the spreadsheet, but it's missing a lot of information and is less up-to-date.

2023 September 1 update

- Added English transliterations for all song artists
- Added additional alternative 7-bit ASCII compatible song and artist titles for easier text searching and filtering
- Fixed various errors in song and artist titles
- Updated Google Drive backup with updated stepfiles, alternate spreadsheet formats, and JSON and Lua script output files: https://drive.google.com/drive/folders/1fRZPDVv_WbFC9ZC2LHao-ExqhU9DDD7i

2023 October 16 update

-Added official GROOVE RADAR data from Bemaniwiki for Single Play and Double Play charts in addition to the script-estimated values: https://bemaniwiki.com/?DanceDanceRevolution+A3
- Added real-world scoring data for rating 1-19 Single Play and Double Play charts from Sanbai Ice Cream (Version 5 Revision 1 data): https://3icecream.com/ddr/home - https://ddrcommunity.com/ddr-a3-difficulty-tier-list-provided-by-3icecream/ - https://www.reddit.com/r/DanceDanceRevolution/comments/176dlcy/comment/k4o81d8/?context=3
- Added computed columns to compare official and script-estimated GROOVE RADAR values
- Added alternate Google Sheets published website version

2023 November 20 update

- Updated Sanbai Ice Cream scoring difficulty data to Version 6 Revision 1 (2023 November 6)
- Added ARFF files (discrete and continuous) to Google Drive folder for difficulty estimation through machine learning in WEKA. The header information can be deleted to create text files or CSVs for other machine learning tools
- Added additional filter views to filter by X-rating charts, charts with official GROOVE RADAR values, and charts with Sanbai Ice Cream scoring data
- Updated alternate file formats, script output data, and extracted stepfiles
Register Log In Back To Forums

0 User(s) Viewing This Thread (Past 15 Minutes)

©2006-2024 Zenius -I- vanisher.com -5th style- IIPrivacy Policy
Web Server: 2% · Database: 8% · Server Time: 2024-03-28 09:03:16
This page took 0.009 seconds to execute.
Theme: starlight · Language: englishuk
Reset Theme & Language