Tanker, I am not quite as big as you are (close, but not quite
). I too have had people tell me I needed a bigger bike. They were not trying to sell me anything either, just making conversation, I guess. Of course they were all riding big Harleys, dyna wde big in one case. One guy even told me I probably would not be able to do the dragon run, the bike probably would not take me up the mountain. That was 100% a comment on the bike, not my riding. My riding skills on the dragon would be another topic.
I tended to disagree with the "you need a bigger bike crowd" I really like the savage, it is small and light, nimble I would say. It will run me well over the speed limit, even the interstate limit. I have never had a problem with the bike being underpowered, which is not exactly your question.
I did get a set of Ed L's forward controls and they made the bike just right for me. I also like the simplicity of the bike. All in all, the savage is plenty of bike for me. I have no desire to get another, bigger bike. The savage fits me fine.