It really comes down to tracking published vulnerabilities in your dependencies, and choosing trustworthy dependencies.

Nobody reviews third party source code for security issues (well, almost nobody). It would not be a productive use of time. There are almost certainly many other less expensive things you could do with that resource to improve security.