The envelope of experience is a phenomenon of immersion. When you play a role-playing game, the player is a person outside the game, while their player character is an imaginary person inside the game world. As a result, players experience things of both metagame interest as well as things inside the immersive experience. The envelope of experience describes the connectivity between the player and the character. Although the character exists within the player's mind, that persona is not interchangeable with the player.
One might visualize putting one's hand inside a glove. Your hand then animates and controls the glove. In this analogy, putting the hand inside the glove is entering the envelope of experience.
The reason this is significant is because irrespective of a player's orientation toward the game, they can make any decision for any reason. While the sensation of immersion is the way a person ejoys the imaginary challenges and conflits of role-playing, their true motivations and interests ultimately reside in their personality. Players use both game and non-game factors to make decisions.
In a degenerated form of role-playing, the player animates the character with no veil between their player motivations and the imaginary character's motivations. Although they are technically role-playing, their play is purely metagame in nature. In Forgespeak, this is called pawn stance. The player may view the game as a limitless sandbox, or they may be actively hostile to immersion. Without the envelope of experience, the immersive experience is inaccessible. Characters are reduced to avatars of the players, and the story is reduced to a series of mutually accepted propositions.