2003 Chevy Silverado Differential Clunk

I have a clunk in my rear diff when i back up and shift to drive to go forward. If i back up and shift to nuetral then to drive instead I do not have a bang or clunk of any kind. And when i back up and shift to drive i am doing it very carefully and I am at a complete stop when shifting. The clunk is bad enough that the first time it happened I actually got out of the truck to see what I hit. Any ideas?
Thank you

Edit: This is in a 3/4 ton duramax