Porting and polishing will actually make it less tractable and user friendly at lower RPMs, if a little bit rippier in accelleration.
You need fuel atomization in a carbureted bike, and a polished intake port will actually cause the fuel to form microglobules (not a technical term-more of a description) This may cause a bit of a rough, uneven idle.
A mild sandblasting in the intake ports will enable atomization...it helps form microvortices... small tornados.......good for atomization and general clean running.
In the exhaust ports...polish away.
Match the intake manifold diameters to the intake and exhaust port openings on the head..... with the carbs off, but with the intake manifolds in place...slide your finger into the intake and exhaust ports... the diameters should match smoothly where the manifold meets the head... if you feel a step... smooth it away... match them up.
The SRX will benefit from less backpressure in a larger diameter header...or a more open muffler, but again... the benefits wiill all be in the upper RPM ranges.
If you have a stock muffler... remove it and cut about a 6" x 6" hole in the bottom with a very narrow kerf grinder. Peel the piece off (small welds retain it) and grind,cut, and drill holes in the eight miles of internal pipes and acoustical baffling. Reweld the square piece back on. You will have the attractive original muffler, a bit lighter,but it will breathe much better and sound like a thumper should.
If weight loss is the goal, hang the original muffler on your shop wall and go with a White Brothers or Supertrapp system... or maybe another that you like.
My eye sees the stock muffler as most beautiful,but it weighs about eleventeen tons and chokes it up a bit.
You make the call, based upon personal taste, time, and the thickness of your wallet.
