A few ideas.
1 Make sure that you are not inserting the bit too far into the collet. Some bits have a slightly rounded shoulder on the shank close to the cutting portion. Installing the bit too deeply causes the collet to tighten on this small area, not enough "bite" to hold the bit.
2 Make sure the shank is clean and rust free.
3 Make sure the collet is clean and rust free and there is no wood in the collet gaps.
4 check the nut that secures the collet for rust, etc. it may not be tightening fully. Same is true for the male threaded portion.
5 a worn collet may also cause this, try a new one.
Scot