As part of developing best practices in systematic review (SR) for toxicology and risk assessment, tools are being designed or refined to accommodate evidence bases which include many different types of studies (i.e., experimental animal, in vitro, and observational studies). This is necessary as such tools were originally designed for different evidence bases (i.e., clinical trials). In this case study, two tools available to appraise data quality are compared as a means of providing an understanding of the practical application of available approaches. Using the evidence base for TCE and CHDs, data were appraised using two tools: (1) the US EPA TSCA draft guidance on study quality evaluation SR principles, and (2) the NTP-OHAT Risk of Bias tool. Nine observational studies, 10 experimental animal studies, and 20 mechanistic studies were assessed representing a diverse range of study designs (in vivo, in vitro, and in ovo), durations, and exposure routes. The application of TSCA metrics to the TCE-CHD literature demonstrated some similarities with the NTP-OHAT criteria [e.g., none of the human studies were considered high quality (TSCA score <1.7 or OHAT Tier 1), and within the experimental animal studies, the inhalation animal studies were determined to be of higher quality than the oral studies]. However, there were more differences than similarities. Most notably, strict adherence to the TSCA metric criteria definitions resulted in designations of many human and mechanistic studies as “unacceptable” (and subsequently not considered in the risk assessment). A key difference between the tools and resulting quality characterizations is how the absence of reporting a metric is appraised. This was particularly impactful for several mechanistic studies, as several study design elements (e.g., cell plating and maintenance, cytotoxicity, etc.) were often not reported. Additional differences can be attributed to the thoroughness and specificity of the TSCA approach, as well as breadth of the TSCA criteria (includes multiple aspects of study validity), relative to NTP-OHAT. This case study demonstrates the need for continued refinement of critical appraisal tools to assess study quality.