City Library, History