To remove old glue and heavier crap, a blue Scotch-Brite scour pad and gasoline for the 1st pass.  If you want to spend extra money, use Goof-Off or other solvent instead of gasoline.  A mask and rubber gloves might be a good idea.

For light basic cleaning, cleaning vinegar with a blue scotch-brite pad works well.  The vinegar (mildly acidic) will also remove light rust spots.  I’ve been known to soak paper towels with vinegar and wrap around the pipe and let sit for an hour and then use a spray bottle with vinegar and a cleaning pad to remove heavier rust spots.  In less conspicuous areas (for example, the inside facing portion of the header), I’ve used wadded up aluminum foil and vinegar to work on heavier rust spots.  The aluminum foil shouldn’t scratch the chrome unless you really apply a lot of elbow grease to it.  Just be careful and start off somewhere inconspicuous.  Don’t test your method (elbow grease strength) on the pretty part of the muffler until you get a feel for how it works.

Stay away from stiff wire brushes or getting heavy-handed with aluminum foil, and you should be okay.

For chrome, oven cleaner. Get the pipe warm/hot, protect everything from overspray (motor, painted pieces, etc.), then spray the oven cleaner on a damp, coarse terry cloth towel rag. Wear gloves.
Scrub the areas with the oven cleaner soaked rag vigorously, then wipe with another clean, damp towel.
If it doesn't work, repeat the process but spray the oven cleaner on the pipe, making sure you don't get any overspray on anything else.
This is also a great way to remove melted plastic from your pipe or header if you suck up a grocery bag while riding, or if your rain paints melt on the pipe.