Association with persons who are not spiritually advanced is forbidden. Lord Caitanya advised, asat-sanga-tyaga: one should avoid persons who are attached to the temporary. Asat is one who is too materially attached, who is not a devotee of the Lord and who is too attached to women or enjoyable material things.

Srila Prabhupada SB 3.29.18, Purport