Marcos Lopez de Prado (http://www.quantresearch.info/) has published extensively on the risks of backtesting, and how to detect or avoid it. This includes an online simulator of backtest-overfitting (http://datagrid.lbl.gov/backtest/).

Another result includes an estimate of the minimum length of the backtest that an investor should require given the number of trials attempted, to avoid selecting a strategy with a given in-sample SR during a given number of trials that has an expected out-of-sample SR of zero. This implies that, e.g., if only two years of daily backtest data is available no more than seven strategy variations should be tried, and if only five years of daily backtest data is available, no more than 45 strategy variations should be tried. See references for implementation details.

De Lopez Prado and Bailey (2014) also derive a deflated SR to compute the probability that the SR is statistically significant while controlling for the inflationary effect of multiple testing, non-normal returns, and shorter sample lengths (see the 03_multiple_testing subdirectory for the Python implementation of deflated_sharpe_ratio.py and references for the derivation of the related formulas).