On January 1, 1900, the twentieth century does not begin. Rather, this New Year's Day begins the last year of the nineteenth century. The twentieth century begins on January 1, 1901.
What was called the Christian Era -- now called the Common Era to reflect the diversity of religions -- began in the year 1. Twenty centuries after the year 1 brings us to 1901, which puts the beginning of the twentieth century on January 1, 1901. This also means that twenty-first century actually begins on January 1, 2001.
A Kent, Washington, newspaper had this to say about when the century turned:
"Throughout christendom there has been a voluminous, fierce discussion as to when the new century begins -- whether it is this New Years [January 1, 1900] or next. The majority of the people of this country, as represented by publications and speakers, are agreed with the great majority elsewhere that the century does not begin until 1901. It seems plain that this is a logical deduction, for the Christian era started with the year 1, thereby being twelve months ahead on its first New Years. If the era had started with the year 0 then we would properly celebrate the new century this year. To avoid just such complications in the numerals of tickets all railroads start the numbering with 0 (zero) instead of with 1" (White River Journal).