I’ve interrupted my Echo with “Alexa, stop” or “Alexa <repeated phrase more clearly or louder>” while it is speaking. So it doesn’t stop listening when speaking.
Stop listening only when saying "Alexa" not just when speaking in general. So to test you'd have to time saying Alexa at the same time it is saying Alexa