Herbert Spencer wrote a letter to Japan's Baron Kanéko Kentarô in the late 19th century which included the following: "If the Chinese are allowed to settle extensively in America, they must either, if they remain unmixed, form a subject race standing in the position, if not of slaves, yet of a class approaching to slaves; or if they mix they must form a bad hybrid. In either case, supposing the immigration to be large, immense social mischief must arise, and eventually social disorganization. The same thing will happen if there should be any considerable mixture of European or American races with the Japanese. ... You see, therefore, that my advice is strongly conservative in all directions, and I end by saying as I began--keep other races at arm's length as much as possible."

I found this in the appendix of Lafcadio Hearn's Japan, An Attempt at Interpretation. You may be as surprised as I was at Hearn's take on this letter. If interested go here.