So the "obvious" (but complex!) solution is a "hybrid cloud": use on-prem for the predictable, constant "DC component" of your demand, and use cloud for cyclical or unpredictable demand changes. That will keep peak demand decoupled from permanently provisioned capacity while saving on always-utilized capacity. Easier said than done, of course.
The route we ended up going, was hybrid cloud with a colocation for on-prem (Mainframes and a few servers), and then AWS for the cloud portion. Not sure on what the cost savings were. As I wrote the authentication service I knew what those numbers were, so I knew that on a normal day we would be sitting at 1% cpu usage basically all day except market open/close. And then fiscal quarter ends were a big bump. And then of course big news days. 99% of the time for 99% of the days were just nothing.
