It will be alright for a while. If the card died without the fan then
there would be a lot of problems. Fans die how would you know it with the case
closed?
The card won't run very well without the fan though and it will probably
shorten its life if you did it all the time. The card also doesn't create as
much heat when its idle. So your not gonna hurt anything having the fan off for
a few minutes.