I'm still doing that step manually as I figure out how I want to automate it. An easy way would be to emulate your load balancer's health check (or interface with your load balancer if you can), in our case this is AWS's ELB which just hits a /ping endpoint on the instance that gets handled reasonably deeply in our stack and puts that instance out of service after enough consecutive failures.
It helps to look at what your past failures have been when going live in production to know what to audit during incremental rollout. In our case, /ping has stopped responding before, but some other common cases were increased 500 response rates or severely decreased performance across average or peak response times. These can be used as metrics when we automate. It doesn't help much for infrequently-used features, but I think the main idea is to prevent all of your instances from going down/going haywire at once together, which is unlikely to be a problem caused by such features even if you rolled out to all at once.
Would be interested to hear if you get that working with Jenkins to handle incremental rollout, I haven't tried yet. I'm not even sure how this would be done in Fabric but that might be another option (and have Jenkins call Fabric and block on its completion).
It helps to look at what your past failures have been when going live in production to know what to audit during incremental rollout. In our case, /ping has stopped responding before, but some other common cases were increased 500 response rates or severely decreased performance across average or peak response times. These can be used as metrics when we automate. It doesn't help much for infrequently-used features, but I think the main idea is to prevent all of your instances from going down/going haywire at once together, which is unlikely to be a problem caused by such features even if you rolled out to all at once.
Would be interested to hear if you get that working with Jenkins to handle incremental rollout, I haven't tried yet. I'm not even sure how this would be done in Fabric but that might be another option (and have Jenkins call Fabric and block on its completion).