Based on what?
Not only was Ortiz much older, he's also beaten nobody of note other than Bryant Jennings. Parker was world champion and had a win over Ruiz and other top 10 contenders.
It's pretty obvious that Ortiz was a lot of hype, he was hailed as the 'boogeyman' based off false rumours of fightings ducking him, and the fact he was an old Cuban southpaw.
Him being the number 4 guy is complete nonsense that can't be supported with anything tangible. Looking back, it's ok to say that we were wrong. He hadn't beaten the level of opposition required to have that status.