Historically, the Ferrymen, Far Shores and Hierarchy were all intended as mechanisms for achieving Transcendence. Charon's mandate from the Lady of Fate was to help the souls who ended up in the Underworld achieve Transcendence and move onwards. To that end, each organization served as a combination of spiritual therapy and group identity for a wraith who was hopefully making a temporary stay in the Underworld. Due to the presence of the Shadow and the ever-increasing danger of Oblivion, these organizations faltered in their missions. While the Ferrymen remain true to their charge, several Far Shores became corrupt kingdoms. This culminated in the Proclamation of Reason from Stygia, which declared Transcendence a myth.
Since that time, Stygia has officially disavowed Transcendence, with the majority of wraiths still officially seeking it being labeled Heretics. Most Stygians now view Transcendence as a myth, and the Far Shores and Ferrymen as sinister entities motivated by inscrutable goals.
At the same time, Oblivion's steady increase in power has made the extinction of wraiths an increasingly likely event. The Underworld was seemingly designed to promote Transcendence, even ugly events like Harrowings providing a therapeutic purpose, lessons in what mattered to the wraiths that underwent them. While Harrowings have become increasingly sadistic over the centuries, the shadow of their original purpose is still present in the rules that govern them - rules that Spectres chafe against, but cannot break.
Transcendence is fundamentally about accepting one's death. The things that keep a wraith bound to the underworld, Passions and Fetters, must be Resolved. Once everything is resolved, the wraith must seek some form of harmony with his Shadow through a Destruction Harrowing.
While Transcendence is a relatively well-defined process (in comparison to, say, Golconda or Ascension), there is still a degree of ineffability to the process. Traditionally, wraiths who Transcend leave the Underworld, but this is not necessarily so - Chan, the jester of Yu Huang, has elected to not Transcend and exists with his Hun in harmony with his P'o. Conversely, Charon's Transcendence didn't involve his Shadow in any major fashion.