To be fair, the metrics you mentioned are objective. It's just debatable how relevant they are. Other metrics that are used are measures of coupling, cohesion, lines per method, methods per class, etc. There are probably more measures that are used. None of them are perfect, but they all help paint a picture that adds to our understanding.
To address your second point, it's actually not too difficult to isolate factors like TDD. The standard way is to have a control group and a test group. With a large enough of a sample size, you can determine statistical significance with standard tests.
Unfortunately, the test subjects are often university students, who are less experienced than professionals. The fact that data collected on students might not generalize to professionals is a threat to external validity, but should be made explicit in most papers. Most of the time, I think companies aren't very happy about having researchers use their engineers for experiments on the company's dime, but it does happen. So there are some papers out there reporting results with professionals.
To address your second point, it's actually not too difficult to isolate factors like TDD. The standard way is to have a control group and a test group. With a large enough of a sample size, you can determine statistical significance with standard tests.
Unfortunately, the test subjects are often university students, who are less experienced than professionals. The fact that data collected on students might not generalize to professionals is a threat to external validity, but should be made explicit in most papers. Most of the time, I think companies aren't very happy about having researchers use their engineers for experiments on the company's dime, but it does happen. So there are some papers out there reporting results with professionals.