The Iterated Prisoner's Dilemma is a simultaneous two-player game. The actual values of payoffs take on different values in different research projects. In this study we perform an affine normalization of the payoff matrix and compare the agents that evolve when each of five different payoff matrices is used. Four of the matrices are chosen to lie near extremes of the normalized space, while the other lies in its center. The probability of cooperative behavior evolving is strongly influenced by the choice of payoff matrix. Fingerprinting of the evolved agents demonstrates a significant shift in the distribution of strategies that arise. A significant difference in pairwise competitive ability is also found between agents evolved with different payoff matrices. Placed into tournaments with agents evolved using all five payoff matrices, the players evolved with the central payoff matrix were found to have significantly better average tournament ranking. We conclude that the choice of payoff values is not a neutral choice and must be controlled for in any design of experiments.