How to do integration testing on a dockerized Python microservice architecture?

Initially I hoped to find a good online training course for my team which goes deeply into testing (system, integration, and unit) of microservices. Since this is often done with docker containerization and Python thrives on the current deep learning hype I thought it would be easy to find something tailored to our needs… but no. I was very unsuccessful.

The conclusion from my google searches is that integration testing of dockerized microservices is not trivial. Many companies seem to skip it because of the associated cost. That is probably why there is a lack of information on the internet.

We found that Testcontainers is a useful and often recommended project that helps to perform integration tests on docker containers. While it was initially implemented for Java, there is also a Python port. Most material (Videos and articles) I can find goes into how to write tests with Testcontainers but I'm missing the more fundamental instructions on how to apply this in our CI.

Where are such integration tests located? An integration test can't be clearly assigned to one repository / service. Should they be located in an additional extra repository?

When should we run these? Obviously they are more expensive (execution time) than unit tests. Our unit tests run via hooks on every commit in the repository with the state of this commit. What frequency should we aim for? For every pull request onto our development branch (ignoring feature / bug / … branches)? Running nightly? …?

How to select the correct integration tests? When we make changes to one component then only the unit tests of this component are run. If we have a global set of defined integration tests then how are we selecting the integrations tests which are relevant for a certain component?

How to do versioning? We would have build a new docker image from the current code change and for all other necessary systems we would have used the current development docker image version from our docker registry. Is this the right approach? And how to proceed on production branch changes? Should we detect if the change was on the development or production branch and respectively load the docker image version? Also similarly for git tagged versions?

​

Maybe someone can share his/her experience.