We do a lot of integration testing, more so than unit testing, and those tests, which randomly fail, are a real headache.
One thing I learned is that setting up tests correctly, independent of each other, is hard. It is even harder if databases, local and remote services are involved or if your software communicates with other software. You need to start those dependencies and take care of resetting their state, but there's always something: Services sometimes take longer to start, file handles not closing on time, code or applications which keeps running when another test fails... etc, etc...
There are obvious solutions: Mocking everything, removing global state, writing more robust test setup code... But who has time for this? Fixing things correctly can even take more time and usually does not guarantee that some new change in the future disregards your correct code...
>There are obvious solutions: Mocking everything, removing global state, writing more robust test setup code... But who has time for this?
I find that doing all of this tends to actually save time overall it's just that the up front investment is high and the payoff is realized over a long time.
Most software teams seem to prefer higher ongoing costs if it comes with quick wins to up front investment.
Those are the age-old arguments against TDD. Every team will have to analyze the value proposition in their context to see if the return is worth the investment.
>There are obvious solutions: Mocking everything, removing global state, writing more robust test setup code... But who has time for this?
If you do it from the beginning and structure your code in a testable way it doesn't take much time. It saved me a few time in my current company; make a small change -> turns out it breaks a feature from 3-4 years ago that no one even remember -> look at the tests -> understand the feature as well as why what you did broke it.
If you try to do it after X years of coding without thinking about tests you're doomed though.
We do a lot of integration testing, more so than unit testing, and those tests, which randomly fail, are a real headache.
One thing I learned is that setting up tests correctly, independent of each other, is hard. It is even harder if databases, local and remote services are involved or if your software communicates with other software. You need to start those dependencies and take care of resetting their state, but there's always something: Services sometimes take longer to start, file handles not closing on time, code or applications which keeps running when another test fails... etc, etc...
There are obvious solutions: Mocking everything, removing global state, writing more robust test setup code... But who has time for this? Fixing things correctly can even take more time and usually does not guarantee that some new change in the future disregards your correct code...