Don't pause yet, unless you've actually run the numbers. Whether you end up with an actual penalty or not depends on a number of inputs like each of your actual incomes, deductions, how your benefit packages, insurance, and other things are structured. It sounds like you haven't actually run the numbers yet, and are operating out of FUD. Talk to a tax expert, or do some research, before despairing.
Marriage penalty? My taxes went way down when I got married. Additionally, all of the income limits where benefits phase out go up when you're married, so for instance, you might qualify for a Roth IRA when you're married filing jointly, but not if you're single. In my experience, getting married is one of the best things you can do for your tax bill.
I have friends who "got married" in a legally unsanctioned way. No tax penalty, but they still feel like they expressed their commitment to one another.
And yes, obviously there are massive downsides to this approach.