Dr. Alexander Milton Ross, who worked with the abolitionist John Brown, also wrote about a dinner he attended with President Lincoln:
"Assembled at the President's table were several prominent gentlemen, to whom Mr. Lincoln introduced me [Dr. Ross] as " a red-hot abolitionist from Canada." One of the guests, a prominent member of Congress (severely injured...) remarked, in a slurring manner, that he wished all the...[blacks]..of the United States would emigrate to Canada, as we Canadians were so fond of them. Mr. Lincoln said : " It would be all the better for the [blacks]...that's certain."
"Yes," I [Dr. Ross] replied, a little warmly, "it would be all the better...for, under our flag...[a black person]...is entitled to, and freely accorded every right and privilege enjoyed by native Canadians."
"Mr. Lincoln, in a jocular way, said to the member of Congress, " If you are not careful, you will bring on a war with Canada. I think we have got a big enough job on hand now." [Source]