Shove was born at Faversham, Kent,the son of Herbert Samuel Shove and Bertha Millen.
He was educated at Uppingham School and King's College, Cambridge, where he became a member of the Cambridge Apostles.
He married in 1915 Fredegond Maitland, daughter of historian Frederic William Maitland and his wife the playwright Florence Henrietta Fisher. In World War I he was a conscientious objector, like many others in the Bloomsbury Group, of which he was a member; he worked as a poultry keeper at Garsington, the home of Lady Ottoline Morrell.
His academic career was spent at King's College, Cambridge, becoming lecturer in 1923, Fellow in 1926, and Reader in 1945.
His younger brother was the Olympic rower Ralph Shove.
He died at Old Hunstanton and was buried at the Parish of the Ascension Burial Ground in Cambridge; his wife Fredegond was also interred in the same burial plot.