We bring fish back every trip. You'll have to pay for the extra luggage...up to 50# like someone said earlier.
Bring a roll of duct tape, don't tape the cooler until after customs has inspected it.
Also, some people forget that you can't put ice in the cooler! Just freeze the fish and get them home fast, or use ice packs to help keep them cold if you're travelling in particularly hot weather.
I just brought an ice-chest home a couple months ago on Alaska Airlines and I think it cost $20.
Also, I'm pretty sure that Alaska Airlines will waive the baggage fee for up to three bags if you are active duty military. You just have to show your ID.