RECREATIONAL fishermen across Victoria fear new coastal marine parks that ban recreational angling will devastate small coastal towns.

Port Fairy Angling Club, in the Victoria's southwest, has 200 members, many of who have expressed deep concerns.

Club member Russell Allardice said they were worried about the size of the proposed fishing restrictions around Lady Julia Percy Island off the coast of Port Fairy.

"While restrictions around the island won't affect those fishing here (on shore), those with boats may be affected, " Mr Allardice said.

"The concern is that the restricted area may grow and impact on us."

Fellow club anglers Rod Eccleston and Kirian Dalton said there was already adequate coastal protection.

"Victoria has enough marine parks, it doesn't need any more," Mr Dalton said.

Mr Eccleston said anglers were concerned increasing restrictions on coastal activities would "snowball until the recreational and commercial fishermen do not get a fair go".

Port Fairy Angling club president Matthew Dalton said fishing was a sport families could enjoy together.

"We don't want that to be taken away from families," Mr Dalton said.

Andrew Geary, 14, from Warrnambool said he liked to fish every week.

"It's a great way to get outside and have some fun," Andrew said.

Cranbourne angler Jarrod Day, who presents a radio show on fishing, said any plans to use marine parks to ban recreational fishermen was "absolutely absurd" and would face "huge protests if they go ahead".

"What impact on a seabed or seagrass does sitting in a boat on top of the water have anyway?," Mr Day said.

"Commercial fishing and drag nets are a different story, but the impact that some recreational fishermen would have is so small.

"If those parks go ahead it will financially ruin towns like Mallacoota that rely on tourism, and the financial impact all along Victoria's coastline would be as bad as the affect on the economies of the bushfire towns of Kinglake - they would be decimated."