To remove old glue and heavier crap, a blue Scotch-Brite scour pad and gasoline for the 1st pass.  If you want to spend extra money, use Goof-Off or other solvent instead of gasoline.  A mask and rubber gloves might be a good idea.

For light basic cleaning, cleaning vinegar with a blue scotch-brite pad works well.  The vinegar (mildly acidic) will also remove light rust spots.  I’ve been known to soak paper towels with vinegar and wrap around the pipe and let sit for an hour and then use a spray bottle with vinegar and a cleaning pad to remove heavier rust spots.  In less conspicuous areas (for example, the inside facing portion of the header), I’ve used wadded up aluminum foil and vinegar to work on heavier rust spots.  The aluminum foil shouldn’t scratch the chrome unless you really apply a lot of elbow grease to it.  Just be careful and start off somewhere inconspicuous.  Don’t test your method (elbow grease strength) on the pretty part of the muffler until you get a feel for how it works.

Stay away from stiff wire brushes or getting heavy-handed with aluminum foil, and you should be okay.