Fedor fought Arona in his 4th fight and Babalu in his 8th. Stipe's first 6 opponents don't even have Wiki pages....I've never heard of any of them. I completely agree Stipe had a tougher stretch, his last 7 opponents are all world class minus Arlovski who is shot to shit. Fedor never had a stretch like that. However, let's not forget the guy has faced Cro Cop, Big Nog x2, Coleman x2, Randleman, Werdum, Hunt, Sylvia, Arlovski, and Hendo. Those guys are all HOF type fighters.
As for the size thing, I would hope you can ignore that considering Fedor was undersized for a ton of his fights.
Tim Sylvia was UFC Heavyweight Champion the previous year. He was 263 lbs. for Fedor. After Fedor beat him, he took a year off and showed up well over 300 lbs. against Mercer and was a fat slob from then on.