I doubt if it's the handlebars.It's probably in your riser clamps.Bars steel,clamps aluminum.What I have one is take a hammer,and chisel and made some nicks in the bars(sorta made new splines)not huge just enough to make high spots for the risers to grab.
Harleys with apes are real good for moving around,and if you tighten them too much you strip the threads in the risers.