This is probably the best historic image remix that I've seen to date on Toronto Before.

The castle-like building in this image is the Toronto Armories. The building was constructed in 1894 and was designed by architect Thomas Fuller. It served as a training centre for Canadian soldiers until the 1960s, when it was torn down to make room for the hideous Provincial Court House.

Image from Toronto Before.