Are you running through a pro logic/digital amp or just a tv? If so how are you connecting it all? Is the centre speaker turned up? Do other games seem to work ok? Also, iirc you can setup the PS2 sound output in the options.
If you have it set to output pro logic, and you only have stereo sound on your tv it sometimes does this. Change the sound settings and it should be OK.
If you are using a small TV with no stereo function, you need to switch the GT4 sounds function to mono in the sound menu. If you are not using a small TV. see above.