Post #1 · Posted at 2020-05-11 11:38:00am 3.9 years ago
=== Stepmania Rainbow ===
Made with love in 2 days.
I was tired of having to import songs manually and discover titles by hand, so I made a tool that lets you scan your library and choose which songs you want to play easily
What is the purpose of this tool?
- Organise your Stepmania songs and give you information such as NAMES, ARTISTS, DIFFICULTIES on a App where you can search songs and import into your Stepmania Game easily.
- Use .zip packs instead of extracting them - The tool parses the .zip content recursively, which means that also song1/song2/song3 subfolders will be counted as 3 distinct songs/versions that you can try in your Stepmania - This reduced my Stepmania folder from around 150.000 files to 5000, all the packs I have.
- What if you have just single songs? zip them all in a zip and scan through the App! no need to unzip them - the tool organises the content for you
It currently handles my 1TB of Stepmania .zip files effortlessly :-) and I'm free to search packs and copy songs to my stepmania folder with a click.
The App started as a song analyser, but quite immediately I realised how tedious was to have so many slow files in my hard drive and to manipulate them and their subfolders. So I thought about analysing .zip files instead. A zip can contain around 10mb to 5gb of files in the packs you can find online, so it can have around 10.000 files, which as a .zip count as just 1 file, other than being reduced in side. This would solve the many-files/slow-hd problem and it will be the App effort to keep it organised for you. Then as a user, you just search the songs and decide which to play!
I've tested the App with a 100gb .zip file, and it is blazing fast to deep analyse it (30.000 songs), to search for them, and to extract them (less than a second per song in average from my use case).
Video on sample usage:
What technical features has the App?
- Keeping track of songs you choose: they get colored green in the App. Once you extract songs to your stepmania folder, the app colours them as green, so you can recognise previously exported musics in future searches.
- Search by name for individual songs/packs/artists/titles/difficulties and filter based on your difficulty preferences!
- Automatic recognition and skipping of already-scanned zip files (the app checks by .zip size in bytes). So you can add 1 file to your 5000 zip folders, to have 5001 files, and it will just deep scan the 1 file that before was not analysed.
- Zip size identification: if a zip is changed (updated), and you scan again a folder containing it, the app will clean the old data about that zip, and deep analyse it again.
How to use it?
- This tool supports .zip archives only! Song without zip have been disabled - just zip the pack and scan through the tool! (want to add songs that are not in a zip? zip them! )
1. Place all your zip packs in a folder (or .zip your packs and place in a folder)
2. Run the App
3. 'Choose /Songs Folder' - Select your /Stepmania/songs installation, be sure to select the /songs subfolder in your stepmania installation folder.
4. 'Save subfolder name' - put a subfolder name on the right field and then press this button to store it, all exported songs will be placed under StepmaniaFolder/Songs/subfolder-name that you chose.
5. 'Choose Search Folder' - Select the folder in your drive that contains packs in .zip format (as per point 1.)
6. 'Search Songs' - now the folder on point 5. will be analysed. When the analysis ends, songs will load in the App.
7. Remove the search filter 'linkin' at the top to what you want to search, or empty the field and press Search to show all your songs. The filter is put by default as if you have 100k songs, the application hangs or may be unresponsive.
----- one time setup above finishes here, the above will be remembered through app executions -----
8. Search for songs in the app, select them, then click 'Extract Selection to /Songs/subfolder-chosen' to copy all of them your Stepmania installation folder.
- If you want to analyse new folders with .zips to add new songs, repeat 5-6 as much as you prefer
9. Open Stepmania and play your songs! :-)
Disclaimer:
The app is free for everyone to use it.
The app is terribly ugly and it was my weekend project as I couldn't tolerate anymore navigating and trying songs one by one.
I hope you could understand, again, that it is very ugly - please don't use it if you find problems with it.
The App also is terribly unoptimised production wise, in fact, it weights over 100mb, and I would expect a clean production build to be around 10-20mb probably.
The app does its job and in future may be expanded, but I have close to no time apart from weekends in my life. :-)
FAQS:
- It is broken somehow, how to fix it?
Let's say that your app misbehaves or gets corrupted.
Close it, delete the stepmania-rainbow.sqlite3 database, and restart it. The DB will be recreated but empty, now you can add back your zips!
- The app hangs there and seems blocked!
I can reassure you it is probably not blocked - sqlite takes some time to add all your songs one by one, and in slower computers may appear to be hanging, but it is probably just processing and adding data.
It took about 30-60 minutes to add about 1tb of zips in my computer.
- What is the right way to use the search to search for songs?
Do not mix artist + song, search either song or artist or pack name, for example:
[V] good: linkin / linkin park / dylan / in the end / in the wind / DDR MIX
[X] bad: linkin park end / linkin in the end / dylan wind / DDR MIX Linkin
- I've added songs but they don't show up
Go up, empty the 'search' field, and click search - by default it filters with the keyword 'linkin' to prevent memory overuses in case of large datasets, so you'll need to change the search query to your desire each time you open the app.
Another option you could change is the "Should have at least 1 difficulty below ..." - It is by default 6 at every app start, but if you want harder songs, then change it to 999.
- What if you have just single songs and not a .zip file?
Zip the songs and scan through the App! no need to unzip them - the tool organises the content for you
- What if I want to add now new songs to the Zip that contain 3 my songs, but will contain 5 later on?
No prob - the tool understands if a zip changes, so go ahead and replace the old zip with a new one with the same name and scan the folder again, then the tool will 1) clean the old zip data and 2) reimport the new zip doing a deep scan of the new content.
- Does it support rar, 7zip or other compression tools?
Just zip files - I had about 20 .rar packs downloaded, I unrarred them and zipped back individually, then let the tool scan them
- Does it support other Operative Systems other than Windows x64?
Please share your will and given the time I'll try to make it available for other platforms
- I found bug x
Please report it with details, how did it happen? what did you do trigger it? Please be as detailed as possible, and also include a screenshot if reasonable.
- Sometimes the App doesn't seem to have loaded all the info at start
Just give it a search click and it will load the info
- Which stepmania file types are supported by the deep analysis?
sm and dwi only. These are moe than 99.9x% of around 200k songs, so I thought it was unnecessary to also support others, but for those that may be inetrested, here's a stat of some file types I found during my investigation:
{
sm: 127375,
ssc: 9195,
dwi: 62398,
ksf: 2225,
xstep: 241,
ds: 313,
dsx: 321,
bms: 1707,
aup: 4,
ksh: 2,
ffr: 1,
sma: 1566,
adwi: 4
}
Bear in mind that 9195 ssc doesn't mean that the tool can't process 9195 files, as all of them or quite all of them, come together with an .sm or .dwi, and the tool will find it and deep analyse it, leaving a very small amount without proper identification.
I really hope you may find it useful, this is my first ever contribution of this kind! :-)
Download links:
[v0.2 (10th May 2020)] - Extraction working
https://mega.nz/file/LgYnkSzK#cnFSwkgUausnLtG2bKi411m8jtjQgbehQbppw4Zcv8A
[v0.1 (10th May 2020)] - Bugged, 'Extract' button was not working
https://mega.nz/file/qkpkgaDI#V2VhxvYUvdj8EAAIQHdLCdiV0mZCvVxO370cIA9Tvio
Screenshot of this version:
Please do move this discussion if it is not in the right thread
---------------------
Updated to v0.2, fixing 'Extract' button not doing anything in v0.1
----------------------
Update: Added Video to show how the app works
Made with love in 2 days.
I was tired of having to import songs manually and discover titles by hand, so I made a tool that lets you scan your library and choose which songs you want to play easily
What is the purpose of this tool?
- Organise your Stepmania songs and give you information such as NAMES, ARTISTS, DIFFICULTIES on a App where you can search songs and import into your Stepmania Game easily.
- Use .zip packs instead of extracting them - The tool parses the .zip content recursively, which means that also song1/song2/song3 subfolders will be counted as 3 distinct songs/versions that you can try in your Stepmania - This reduced my Stepmania folder from around 150.000 files to 5000, all the packs I have.
- What if you have just single songs? zip them all in a zip and scan through the App! no need to unzip them - the tool organises the content for you
It currently handles my 1TB of Stepmania .zip files effortlessly :-) and I'm free to search packs and copy songs to my stepmania folder with a click.
The App started as a song analyser, but quite immediately I realised how tedious was to have so many slow files in my hard drive and to manipulate them and their subfolders. So I thought about analysing .zip files instead. A zip can contain around 10mb to 5gb of files in the packs you can find online, so it can have around 10.000 files, which as a .zip count as just 1 file, other than being reduced in side. This would solve the many-files/slow-hd problem and it will be the App effort to keep it organised for you. Then as a user, you just search the songs and decide which to play!
I've tested the App with a 100gb .zip file, and it is blazing fast to deep analyse it (30.000 songs), to search for them, and to extract them (less than a second per song in average from my use case).
Video on sample usage:
What technical features has the App?
- Keeping track of songs you choose: they get colored green in the App. Once you extract songs to your stepmania folder, the app colours them as green, so you can recognise previously exported musics in future searches.
- Search by name for individual songs/packs/artists/titles/difficulties and filter based on your difficulty preferences!
- Automatic recognition and skipping of already-scanned zip files (the app checks by .zip size in bytes). So you can add 1 file to your 5000 zip folders, to have 5001 files, and it will just deep scan the 1 file that before was not analysed.
- Zip size identification: if a zip is changed (updated), and you scan again a folder containing it, the app will clean the old data about that zip, and deep analyse it again.
How to use it?
- This tool supports .zip archives only! Song without zip have been disabled - just zip the pack and scan through the tool! (want to add songs that are not in a zip? zip them!
)1. Place all your zip packs in a folder (or .zip your packs and place in a folder)
2. Run the App
3. 'Choose /Songs Folder' - Select your /Stepmania/songs installation, be sure to select the /songs subfolder in your stepmania installation folder.
4. 'Save subfolder name' - put a subfolder name on the right field and then press this button to store it, all exported songs will be placed under StepmaniaFolder/Songs/subfolder-name that you chose.
5. 'Choose Search Folder' - Select the folder in your drive that contains packs in .zip format (as per point 1.)
6. 'Search Songs' - now the folder on point 5. will be analysed. When the analysis ends, songs will load in the App.
7. Remove the search filter 'linkin' at the top to what you want to search, or empty the field and press Search to show all your songs. The filter is put by default as if you have 100k songs, the application hangs or may be unresponsive.
----- one time setup above finishes here, the above will be remembered through app executions -----
8. Search for songs in the app, select them, then click 'Extract Selection to /Songs/subfolder-chosen' to copy all of them your Stepmania installation folder.
- If you want to analyse new folders with .zips to add new songs, repeat 5-6 as much as you prefer
9. Open Stepmania and play your songs! :-)
Disclaimer:
The app is free for everyone to use it.
The app is terribly ugly and it was my weekend project as I couldn't tolerate anymore navigating and trying songs one by one.
I hope you could understand, again, that it is very ugly - please don't use it if you find problems with it.
The App also is terribly unoptimised production wise, in fact, it weights over 100mb, and I would expect a clean production build to be around 10-20mb probably.
The app does its job and in future may be expanded, but I have close to no time apart from weekends in my life. :-)
FAQS:
- It is broken somehow, how to fix it?
Let's say that your app misbehaves or gets corrupted.
Close it, delete the stepmania-rainbow.sqlite3 database, and restart it. The DB will be recreated but empty, now you can add back your zips!
- The app hangs there and seems blocked!
I can reassure you it is probably not blocked - sqlite takes some time to add all your songs one by one, and in slower computers may appear to be hanging, but it is probably just processing and adding data.
It took about 30-60 minutes to add about 1tb of zips in my computer.
- What is the right way to use the search to search for songs?
Do not mix artist + song, search either song or artist or pack name, for example:
[V] good: linkin / linkin park / dylan / in the end / in the wind / DDR MIX
[X] bad: linkin park end / linkin in the end / dylan wind / DDR MIX Linkin
- I've added songs but they don't show up
Go up, empty the 'search' field, and click search - by default it filters with the keyword 'linkin' to prevent memory overuses in case of large datasets, so you'll need to change the search query to your desire each time you open the app.
Another option you could change is the "Should have at least 1 difficulty below ..." - It is by default 6 at every app start, but if you want harder songs, then change it to 999.
- What if you have just single songs and not a .zip file?
Zip the songs and scan through the App! no need to unzip them - the tool organises the content for you
- What if I want to add now new songs to the Zip that contain 3 my songs, but will contain 5 later on?
No prob - the tool understands if a zip changes, so go ahead and replace the old zip with a new one with the same name and scan the folder again, then the tool will 1) clean the old zip data and 2) reimport the new zip doing a deep scan of the new content.
- Does it support rar, 7zip or other compression tools?
Just zip files - I had about 20 .rar packs downloaded, I unrarred them and zipped back individually, then let the tool scan them
- Does it support other Operative Systems other than Windows x64?
Please share your will and given the time I'll try to make it available for other platforms
- I found bug x
Please report it with details, how did it happen? what did you do trigger it? Please be as detailed as possible, and also include a screenshot if reasonable.
- Sometimes the App doesn't seem to have loaded all the info at start
Just give it a search click and it will load the info
- Which stepmania file types are supported by the deep analysis?
sm and dwi only. These are moe than 99.9x% of around 200k songs, so I thought it was unnecessary to also support others, but for those that may be inetrested, here's a stat of some file types I found during my investigation:
{
sm: 127375,
ssc: 9195,
dwi: 62398,
ksf: 2225,
xstep: 241,
ds: 313,
dsx: 321,
bms: 1707,
aup: 4,
ksh: 2,
ffr: 1,
sma: 1566,
adwi: 4
}
Bear in mind that 9195 ssc doesn't mean that the tool can't process 9195 files, as all of them or quite all of them, come together with an .sm or .dwi, and the tool will find it and deep analyse it, leaving a very small amount without proper identification.
I really hope you may find it useful, this is my first ever contribution of this kind! :-)
Download links:
[v0.2 (10th May 2020)] - Extraction working
https://mega.nz/file/LgYnkSzK#cnFSwkgUausnLtG2bKi411m8jtjQgbehQbppw4Zcv8A
[v0.1 (10th May 2020)] - Bugged, 'Extract' button was not working
https://mega.nz/file/qkpkgaDI#V2VhxvYUvdj8EAAIQHdLCdiV0mZCvVxO370cIA9Tvio
Screenshot of this version:
Please do move this discussion if it is not in the right thread
---------------------
Updated to v0.2, fixing 'Extract' button not doing anything in v0.1
----------------------
Update: Added Video to show how the app works