Victoria College started in the 1830s with the help of Egerton Ryerson, and was located in the town of Cobourg, Ontario. Originally called the Upper Canada Academy, this name changed to Victoria College in 1841. Near the end of the century, the college was offered $200,000 in the will of William Gooderham if-and only if-it moved to Toronto. Another $200,000 was donated in the will of Hart Massey, who wanted the college kept in Cobourg but neglected to make his will conditional.

In 1892, the building known as "Old Vic" was built below Bloor Street, designed by the architect W.G. Storm. One of the influences on University College, Storm designed the building using the Richardsonian Romanesque style, borrowing much from the architect Henry Hobson Richardson. Several other buildings would be built to expand Victoria College such as Annesley Hall (built 1903), Burwash Hall (built 1913), and the Isabel Bader Theatre (built 2000).

Over the years, Victoria College has been associated with several important Canadian figures. As previously mentioned, Egerton Ryerson helped to found the college and became its first president. One of the most important figures in literary criticism, Northrop Frye not only graduated from Victoria College, but also worked as a professor, chair, principal, and chancellor. Popular Canadian poet, E.J. Pratt also graduated and taught for the college. Pioneer of women's rights in universities, Margaret Addison was dean of Annesley Hall in the early twentieth century. Famous students also include Prime Minister Lester B. Pearson, director Norman Jewison, poets Dennis Lee and Margaret Avison (not to be confused with Margaret Addison), and actor Bruce Gray. For more information on the history of Victoria College, click here.

Next to Victoria College is Emmanuel College, a collegiate gothic building facing the Royal Ontario Museum. In 1926, the majority of Knox College established its own organization known as Union College. This new college joined with the Faculty of Theology of Victoria University and was renamed Emmanuel College in 1928. Together, Victoria College and Emmanuel College comprise Victoria University. For more information on Emmanuel College, click here.

The Victoria University Bookstore is situated in the Southwest corner of Old Vic. This area of the building was once the office of the principal for Victoria College. Several paintings and sketches have been donated to the Vic Bookstore, including original paintings by Marc Chagall and Helen Kemp, and several portraits of authors such as Charles Baudelaire, Eugene Ionesco, and Samuel Beckett. For several years now, the Vic Bookstore has been serving the faculty and students of Victoria College, Emmanuel College, and other colleges in downtown Toronto. It emphasizes works concerning religion, philosophy, literary studies, and other language texts.