I have the same problem. I use a 2" thick block of very hard wood (like purple heart) and a 5 lb mallet to tap it down. There is not much room to swing a hammer so I like the heavier tool. I just did this about 2 weeks ago and it probably took 50 blows but, I did not damage the PVC pipe.