A megadungeon is both a campaign structure and a sort of exceptionally large and complex dungeon that a party may explore as part of a campaign using that structure. Large dungeons may be called megadungeons regardless of whether they are used in a megadungeon-style campaign.
The earliest ever campaigns of modern tabletop role-playing games used the megadungeon campaign structure, with Gary Gygax and other early GMs of Dungeons & Dragons running players through megadungeons like Castle Greyhawk.
In a campaign with a megadungeon structure, player characters will alternately explore the megadungeon and recuperate in a safe space. Because their actions will affect the contents and inhabitants of the megadungeon (e.g. killing monsters that block their way, solving puzzles or disarming traps), the party will be able to venture deeper and explore new areas every time they return to the dungeon.
Safe areas are often relatively easy to reach by backtracking, being either immediately outside the megadungeon or even within the megadungeon itself. This makes megadungeon campaigns suitable for episodic play, since player characters can return to safety at the end of each episode or session, and they therefore can be appropriate for open table play (i.e. with a variable group of players each session).