Visual Studio takes too long to Attach to Unity

Visual Studio takes too long to "Attack to Unity" so that I can use breakpoints and step through code while playtesting. It didn't use to take so long while the project was in the earlier stages so I would like to ask if someone has any idea what might be slowing it down so much,be it script count, asset count or something else. It takes around 30-40 seconds from the moment I click on it until I can play the game.