After about a month and a half, my cosmos are huge but don't have any blossoms. I read that they don't bloom in over fertilised soil.
Is there any way to 'UN-fertilise' my soil?
Yes that is correct. Cosmos will not flower in over fertile soil. My feeling is that your nitrate levels are too high.
It may be a good idea to try to take them out and plant something else in that spot.
Alternatively after you have taken them out you could try mixing a quantity of washed river sand thru the soil in equal parts. 1 part soil to 1 part sand. This will change the existing structure of the soil and lower the available nutrients.