Evaluation of mechanistic data in a systematic review is an element unique to the field of toxicology; methods established in evidence-based medicine are not sufficient to integrate this data stream. Smith et al., (2016) described an approach to organize mechanistic data via ten key characteristics of carcinogens (KCC). However, this approach does not incorporate data quality, directionality, and concordance with adverse outcomes – concepts key to use in risk assessment. Thus, a framework that integrates these elements is proposed; it has three components (reliability, strength, and activity) that are evaluated using an algorithm which provides a score for each KCC and subsequently categorization as weak/moderate/strong. Reliability scores provide a measure of study quality. Strength scores provide a measure of relevance for each model, which are used to characterize applicability to the evaluation of carcinogenicity in humans; this component considers both the number of models and assays. Activity scores account for active/inactive results. The algorithm allows for flexibility in component weighting, and the scoring approach allows for the incorporation of many study types, including high throughput screening data. Resulting data are then considered relative to animal and human evidence streams, and tumor responses. Application of this framework to multiple mechanistic datasets for chemicals associated with different types of cancers and in different models demonstrates that simple categorization of data by KCC are not alone sufficient, and that evaluation of complex and diverse data (i.e., endpoints measured at the organ, cellular, and molecular level) relative to tumors observed in other evidence streams is critical. The proposed framework provides a quantitative approach that accommodates data quality and relevance, thus increasing the utility of evaluating and integrating the identified KCC, as well as providing a transparent and reproducible process for assessment of mechanistic data in systematic reviews.