Tech debt compounds. If you ignore it, shipping new features becomes impossibly slow...
Don't rewrite everything. Focus on the modules that change the most.