File Ratings

[Zero]

I think that all judges and those who wish to input their opinions should use the same exact judging system to minimize confusions and to maintain structure. We could do a filerating system similar to FFR, using a 1 - 5 ranking because that seems to be the easiest (5 being the best possible score you can get)

Rating System
[X] - File does not have permission to be used, sync is totally off, the steps make no sense
[1] - The file needs to be completely redone and edited
[2] - The file needs significant work
[3] - The file has a possible chance of being accepted (borderline) with changes the file will most likely be accepted
[4] - The file is well done, but could use minor changes to become better
[5] - The file is excellent and is crafted very well - Extremely minor changes may be suggested

If you guys want to add anything to this just post, or if you want to spice up my ratings please do so.

[Nocturne]

Love the rating system. If there is a setlist of four main judges, what are we thinking the minimal score for acceptance is? Like 15 or 16? (Equates for a "4" rating average for every judge.)


Ex: Judge 1 - (5)
Judge 2 - (4)
Judge 3 - (3)
Judge 4 - (4)
Total Score - 16 (Acceptance score is 16)

File is passed :)

Or how about even a challenge of scores? Kinda like how you can challenge or question a play in sports, Maybe if there's a significantly different score than the others, that score can be dropped?


Ex: Judge 1 - (4)
Judge 2 - (5)
Judge 3 - (5)
Judge 4 - (1)
Total Score - 15 (Acceptance score is 16)

File would be Denied.


Judge 4 Scored much lower, that score is challenged or tossed out, leaving a score of 14. If the average score for an accepted file is 4 per judge (3 judges = 12 score for acceptance) than this file may be accepted if the lower score is challenged, or instead of just tossing out the lowest score, it can be replaced with a score from a "Back-Up" judge, which could just be a chosen simfiler that is not one of the main judges.

Just ideas here, throwing em out.

[Subin]

Love the rating system. If there is a setlist of four main judges, what are we thinking the minimal score for acceptance is? Like 15 or 16? (Equates for a "4" rating average for every judge.)



Or how about even a challenge of scores? Kinda like how you can challenge or question a play in sports, Maybe if there's a significantly different score than the others, that score can be dropped?



Judge 4 Scored much lower, that score is challenged or tossed out, leaving a score of 14. If the average score for an accepted file is 4 per judge (3 judges = 12 score for acceptance) than this file may be accepted if the lower score is challenged, or instead of just tossing out the lowest score, it can be replaced with a score from a "Back-Up" judge, which could just be a chosen simfiler that is not one of the main judges.

Just ideas here, throwing em out.


You'll never get perfect or near-perfect simfiles, so I don't really like the idea of having set judges to come up with a particular score in order for the song to make it in-game. I prefer, however, for people to comment on particular difficulties with reasons for and against, and a score to give a rough indication of how the simfile artist is doing.

These numbers should only be just that, numbers. It may indicate flaws, and incite simfile artists to develop and fix their crafting techniques, and shouldn't be entirely based on several figures that add up to mean, well, nothing really.

Just my thoughts.

[Vote4Nixon]

There will NOT be a file rating system similar to FFR to determine whether files get in or not. One of the good things we have going on in TS is lack of elitist bullcrap that frustrates up-and-coming sim authors and drives people away from submitting files. This is not the direction the site is going towards. We definitely want to promote quality for TS files, but this type of judging system is wrong.

For now, I'm content with having people review files before submission (ie if the sim author's intent was X, rate how well did they succeeded in executing X; or the sim author is new and doesn't understand how to properly sync notes and follow music, so guide him/her along those lines), and provide feedback accordingly. There's no need for a formal queue/batch system yet, and when the time comes to deal with mass submissions, we can discuss strategies (still NOT a queue/batch system).

[kisler]

We aren't aiming for perfection; we're aiming for satisfaction. Subjectivemanias makes perfection impossible anyway.

Also, this is fairly obvious - don't review your own files.

e: *reads above*

[Zero]

There will NOT be a file rating system similar to FFR to determine whether files get in or not. One of the good things we have going on in TS is lack of elitist bullcrap that frustrates up-and-coming sim authors and drives people away from submitting files. This is not the direction the site is going towards. We definitely want to promote quality for TS files, but this type of judging system is wrong.

For now, I'm content with having people review files before submission (ie if the sim author's intent was X, rate how well did they succeeded in executing X; or the sim author is new and doesn't understand how to properly sync notes and follow music, so guide him/her along those lines), and provide feedback accordingly. There's no need for a formal queue/batch system yet, and when the time comes to deal with mass submissions, we can discuss strategies (still NOT a queue/batch system).

Fair enough there will be no numbers - Just conversation about the file and overall consensus or whether or not its accepted or rejected (needs more work)

[Nocturne]

Fair enough there will be no numbers - Just conversation about the file and overall consensus or whether or not its accepted or rejected (needs more work)

sounds good.