Allen was born near Kingston, Canada West (now incorporated into Ontario), the second son of Catharine Ann Grant and the Rev. Joseph Antisell Allen, a Protestant minister from Dublin, Ireland. His mother was a daughter of the fifth Baron of Longueuil. He was educated at home until, at age 13, he and his parents moved to the United States, then France and finally the United Kingdom. He was educated at King Edward's School in Birmingham and Merton College in Oxford, both in the United Kingdom. After graduation, Allen studied in France, taught at Brighton College in 1870–71 and in his mid-twenties became a professor at Queen's College, a black college in Jamaica.
Despite his religious father, Allen became an agnostic and a socialist. After leaving his professorship, in 1876 he returned to England, where he turned his talents to writing, gaining a reputation for his essays on science and for literary works. One of his early articles, 'Note-Deafness' (a description of what is now called amusia, published in 1878 in the learned journal Mind) is cited with approval in a recent book by Oliver Sacks.