Luckily I work in an industry that has such high talent competition that companies are using remote work as a bargaining chip.
As for productivity, my work has a huge "stop and go" pace to it. When work comes in, I have to move fast as others are depending on me to pass the work onto other teams. But when I'm all caught up, there isn't much to do.
I have to admit, when I was in the office, I spent WAAAAAAY more time on Sherdog, since cruising the internet is really the only thing to do on the downtime (that, or distract teammates when they have work <45>). Don't get me wrong, I still spend a good chunk of time on here now (if only to browse), but at least my options for how I spend my downtime is much greater now. As long as I'm equally productive when the work I do shows up, it's all good.