Difficulty Scale and Balance
This doc has some suggestions about difficulty, set balance and on keeping a set fun, approachable, and challenging. What you're going to see here are not rules, just suggestions.
Note: be aware of Developers Code of Conduct
More details on these at the bottom.
In general there should be an steady sloping balance of difficulty. One should think of the overall difficulty of a set while building it. One should not make sets excessively difficult.
A good set balance on a 50 cheeves in an average set would have roughly: 4 freebies, 6 easy cheeves, 10 medium, 10 med-hard, 14 hard, and 6 very-hard, 0 kaizo.
A well balanced difficulty distribution has rewards for every type of player. It gives each skill level something to accomplish until mastery.
Achievements should not just be a check-mark of completing a game, (too easy) but create fun new memories for the game (diversity of challenges).
It leaves space for freebies that are featuring curious or fun elements of the game. It expects enough difficulty that once you've mastered the set you are a master of the game. But it doesn't ask too much from the player either.
It also shouldn't ask the player to be doing seemingly endless repetitive grinding to make a set seem harder. Doing the same task over and over isn't hard, it's typically dull. It should almost always avoid making marathon types with a hill at the end (ones that require long setup times with low or repetitive difficulty and then a spike of high difficulty at the end. Don't do it!).
Sets should be mastereable by a skilled player who gives a dedicated effort.
Sets should reflect the content of the game. Grinding cheeves in games that require grinding are reasonable, where in other places they would often be inappropriate.
Sets should also break out of just what is expected from a game, pushing the player to need to find new solutions for problems a veteran player never considered.
Set quantity should not have such hard rules. Quality is the key factor.
The set quantity strongly depends on how much fun and challenges the game can give. That's why the set developer must know the game very well.
Keep it fun, give it variety, challenge the player, give the player new things to do.
Some achievements can give much fun to the player when they are like a meta-game. Where the player is required to rethink the game entirely.
Some notable examples:
Fabulous ComboFabulous Combo (15) - it turns the game into a puzzle game, completing this can be really satifying
Swordless Adventurer (20) - with no sword you are forced to rethink the entire game. You can discover how useful all the subweapons are. Very fun!
There Are No Bullets in the Energy Zone (10) - pacifist challenges, when possible force a lot more agility from the player and break old habits. Very fun!
Definitions of Difficulty
0 - freebie
Takes almost no effort for basically all players to achieve this OR the player gets it without any conscious effort. The player gets these on accident and they pop up on the screen. Sometimes the player will have no idea why they earned them. Some freebies are good some are bad.
Examples: Starting the game, picking a character, collecting an easily accessible item, killing your first enemy. Stumbling on a cheeve that took no forethought but highlights elements of the game.
1 - Easy
Most players could get these on their first attempt, Some on their second or third attempt. Rarely players can mess it up.
Examples: Progress on easy stages, getting 100 coins, collecting a feather, collecting 20 rings. Getting the first upgrade, getting to a well known secret stage. Fighting a simple boss: most bosses in Super Mario World, completing early stages in Sonic.
2 - Medium
Many players could get this on their first attempt. Many could after a few attempts. Some will need to practice. Very few will struggle with them.
Examples: Getting all the colored switches in Super Mario World, collecting all of the items in a Castlevania stage, getting a bonus item in a more hidden area, like moons in SMW. Beating Contra with the Konami code.
3 - Medium-Hard
A low amount of players could get this on their first attempt. Some after a few attempts and most will take some effort and practice to complete these. A few will find them very challenging, and give up.
Examples: Getting some of the chaos emeralds in Sonic 2. Beating Super Mario World.
4 - Hard
Very few player can get these on their first attempt, perhaps if they already know the game (not really a first attempt is it?) Most can complete these with a solid effort, many attempts and some practice, and some need to practice other games first.
Examples: Getting all of the Chaos emeralds in Sonic 1,2,3. Beating Contra without the Konami code.
5 - Very Hard
Only the rarest of players can complete these on a first attempt. Most can complete it after many attempts, Many players will drop off here and won't be willing to complete the set unless they are completionists or fans of the game. Many are not able to complete this after very many attempts. Usually takes significant practice, high skills, experience, research, or guides to complete.
Examples: Beating Contra in one life. Escaping Zebes (Super Metroid) in under two minutes. Beating many of the Megaman bosses without taking damage. Many damagless stages, depending on the game.
6 - Super Hard
Most players are not able to complete this after dozens of attempts. Many players will never be able to complete it as it could require beyond normal reaction speeds. The top-most skilled players still consider these blisteringly hard. It may take marathon efforts to restart.
Examples: Completing a game without taking damage especially if the player has a lifebar, challenges that drastically restrict the player and require near frame perfect accuracy tor success. Beating contra twice in one life. Penniless plumber. Beating Megaman without taking damage.