Does Barry Manilow know that you raid his wardrobe?
It's rare I do so, but I joined with another congregation for worship not too long ago. I wasn't wearing my customary "church clothes."
In conversation with a presumably well-meaning lady, she said, "God doesn't care what you wear." She was speaking in the context of what one wears when assembling to worship the true & living God.
I give her the benefit of the doubt as to her intentions, but I differ. While you'd be hard pressed to argue the Bible requires a suit or a fancy dress or even long pants on men folk, to say He's apathetic regarding attire in the assembly to to overstate the matter.
Modesty inclines us to not dress in such a way that draws attention to ourselves, especially when attention should be focused on God and our adoration & appreciation of Him.
Conventional wisdom inclines us to afford God at least the same respect we'd afford others we deem important by dressing in such a way that doesn't communicate our apathy. Again, that may not mean your "Sunday best" as other generations have put it, but I think it necessitates being "presentable."