Back in the early days of several franchises, the second game in a series tended to be radically different from the first, because developers had not quite pinned down what they wanted the series to become - take a look at Castlevania II or the western Super Mario Bros. 2 for example. Zelda II is another big name usually cited in lists like these, and probably the most radically different when compared to its predecessor.
The original Zelda was viewed entirely from a top-down perspective and featured a large overworld with a multitude of dungeons, in which Link could walk in four directions and (eventually) had several weapons at his disposal. For Zelda II, Nintendo threw all of this out of the window and started completely fresh. The overworld is still viewed from above, but it's more like a Dragon Quest game - You can only move onto set squares and there's various towns, caves and dungeons to enter. Wander around too long and (visible) enemies will pop up - collide with one of them and you'll be taken to a battle scene with the enemies in it as well as the location depending on the type of square you were on.
All these non-overworld scenes are where Zelda II really gets different - as soon as you enter one of them, the game becomes a 2D scroller in which you can only move left and right, as well as being able to jump at will, a rarity in Zelda games nowadays. Fighting enemies gives you experience points, and when you reach a certain amount, you'll level up and get to increase your strength, which increases your attack power, life (which is basically defence) or magic, which lowers the cost of spells. Surprisingly, despite there being experience points, you won't have to grind for hours on end, as clearing a dungeon will instantly get you to the next level no matter how much XP away it is. You'll also find experience bags in secret areas or as rare drops from enemies, which give you a big boost, meaning it never takes that long to get a level up.
These spells are also new to Zelda II and are learned in the game's various towns. They play quite a big role, as several of them are needed to advance - utility spells like a high jump and a "fairy" spell that lets you fly are a necessity to get past a few of the dungeons, while others, like the fireball spell, allow you to defeat otherwise invincible enemies. There's also a heal spell which you'll probably find yourself using the most.
All spells cost magic power. Your stock is displayed in a bar next to your health, which, just like the health bar, can be extended by locating special items - in this case, potions, rather than heart containers. Noticably, in return for getting magic, every single other form of attack Link enjoyed previously is gone: no bow, no bombs, no boomerang, no nothing!
Zelda II's a bit more straighforward than the first game, with a relatively simple overworld and several towns with people that provide clues - which mercifully aren't as vague as those in Castlevania II. That said, there's still a bit of trial and error involved when it comes to getting all the available health and magic extensions - some of them are hidden in completely non-descript squares of the overworld that look absolutely no different from others.
And believe us when we say you will want to get all of them - the original Zelda was quite difficult, but Zelda II is no slouch either, with massive dungeons filled to the brim with tough enemies and several tough bosses. There's also a bit of a platforming aspect to the game, and you'll find several dungeons with water and lava pits that will instantly kill you if you fall into them - and as you get knocked back a bit when hit by enemies, you'll want to be as careful around these as possible.
In terms of graphics and music Zelda II is about on the same level with its predecessor. There are lots of different locations due to overworld battle locations being based on what kind of square you were on, and the soundtrack has lots of catchy tunes. You might recognize the main dungeon theme as the "Temple" music from Super Smash Bros. Melee and Brawl.
Zelda II always seems to get a lot of hate for being so different from the rest of the series, but if you can see past the things it does differently there's actually a very addictive, engaging little game in there. Combat in sidescrolling form is a lot more entertaining than Link's hard-to-use overhead stabs from the first game, and all the new additions like the experience system and magic work surprisingly well. Back in the day, the game also seemed to be disliked for its difficulty, but hopefully the 3DS Virtual Console's save states will take care of that problem.