Aye beezer,
Maybe smooth it out so a sto remove a 'local hotspot', but no need to remove (extra compression!).
As for porting etc. I've only ever taken casting sprues, ridges etc out. Opening up ports can slow the gas flow, rather than speed it up, so you actually get less grunt.

If you keeping it on the road and not going to fit high lift, long duration cams, don't give yourself the grief. Clean up the rough edges, a less restictive, clean air filter, a free flowing exhaust with some back pressure and a check on the fuel mixture, should be plenty for years of reliable riding.

Good luck with the rebuild.
My regards, Bill.