If you write your code properly, you will have tests which break on every library incompatibility, and a tiny wrapper around your libraries. (If you do TDD properly, you will have it.)
In general, I'm not sure it's possible for all of these things to be true.
Many libraries are useful because of their side effects. Depending on what those side effects are, it may simply not be possible to integration test your entire system in a way that would satisfy all of the above claims.
The alternatives tend to involve replacing the parts of the code that cause real side effects with some sort of simulation. However, you're then no longer fully testing the real code that will be doing the job in production. If anything about your simulation is inaccurate, your tests may just give you a false sense of confidence.
All of this assumes you can sensibly write unit tests for whatever the library does anyway. This also is far from guaranteed. For example, what if the purpose of the library is to perform some complicated calculation, and you do not know in advance how to construct a reasonably representative set of inputs to use in a test suite or what the corresponding correct outputs would be?
In general, I'm not sure it's possible for all of these things to be true.
Many libraries are useful because of their side effects. Depending on what those side effects are, it may simply not be possible to integration test your entire system in a way that would satisfy all of the above claims.
The alternatives tend to involve replacing the parts of the code that cause real side effects with some sort of simulation. However, you're then no longer fully testing the real code that will be doing the job in production. If anything about your simulation is inaccurate, your tests may just give you a false sense of confidence.
All of this assumes you can sensibly write unit tests for whatever the library does anyway. This also is far from guaranteed. For example, what if the purpose of the library is to perform some complicated calculation, and you do not know in advance how to construct a reasonably representative set of inputs to use in a test suite or what the corresponding correct outputs would be?