His contract is hardly unreasonable. It's not even in the current top 10 highest paid TEs. In comparison, Aaron Hernandez (a TE#2) signed a contract extension in the offseason for five years at $40M. While he has nice stats and is an excellent player, he also plays in an inflated passing offense.
I don't know exactly what kind of expectations you have for a TE picked at 31, but other than his rookie season he's had 500+ yards per season in all but one, and he had at least 5 TDs in every season outside of his rookie year. He's currently on pace to finish with 884 yards and approxamately 5.5 TDs. He also played for the Bears who haven't had a good passing offense and use a heavy run game (he also had success in his limited role in Martz's offense).
You guys have to remember that Olsen was here during the Martz days. Martz' system doesn't use pass catching TEs only blocking ones. We got rid of him and brought back value bc if we would've kept him that year he would have walked at the end of the year to get a bigger contract.