SciNet: Lessons Learned from Building a Power-efficient Top-20 System and Data Centre

SciNet, one of seven regional HPC consortia operating under the Compute Canada umbrella, runs Canada's first and third fastest computers (as of June 2010) in a state-of-the-art, highly energy-efficient datacentre with a Power Usage Effectiveness (PUE) design-point of 1.16. Power efficiency, computational "bang for the buck" and system capability for a handful of flagship science projects were important criteria in choosing the nature of the computers and the data centre itself. Here we outline some of the lessons learned in putting together the systems and the data centre that hosts Canada's fastest computer to date.