Cochran joined the Cornell faculty in 1973 as an assistant professor and was promoted to full professor in 1986. He was a fellow at the Woodrow Wilson International Center for Scholars in Washington, D.C., and the Henry Luce senior fellow at the National Humanities Center in North Carolina. Cochran was appointed the Hu Shih Professor of Chinese History in July 2004.

In May 2010, the China and Asia-Pacific Studies (CAPS) program at Cornell established the Sherman Cochran Prize to honor Cochran, CAPS founding director, for his great contributions. He retired in 2012 but maintains an active research program.