Wellllll.. you really shouldn't be shifting down to 1st at that speed.
What you looking to do?  Pop the clutch and use it for a brake?

Yes, my shift acts the same way.  It's nature's way of saying you are going to fast to put it in that gear.

The cam dogs won't engage at that speed, you feel them slipping over each other (that's your flutter sensation).

If you FORCED them to engage anyway you'd likely chip the leading edge of a dog or two -- not something you want to do.

You might want to stop abusing your first gear.  You engage it when you are nearly stopped and when you go again you shift out of it then stay out of it until you are nearly stopped again.

Dear Gutwrenchinggore:  (I never really figured that I would type those words together).

The 650cc single is a torque monster, and it has a lot more compression braking than your previous bike.  There is really no reason to rev this bike up for normal riding...let the torgue do the work.  Don't load it up with lots of throttle at low rpms if you feel the engine vibrating from being "lugged" - lugging is a term used to describe running an engine at low speeds and high throttle settings, and you can feel the engine shaking or straining as the pressures in the cylinder is high and the piston can't move down as fast as the expansion of the gases inside the cylinder.  Winding the engine out excessively or downshifting at high speeds is not necessary and not the reason we like owning a big single.    This is a very mildly tuned engine and will run just fine at low rpms.....just get used to riding it smoothly and work within it's limitations.  It is a darn fun little cruiser.....not a high revving sport bike.