The first section will link the Kenyan port of Mombasa to the capital, Nairobi, reducing the journey time from 15 hours to about four.
It is said to be the country's biggest infrastructure project since independence 50 years ago.
The cost of the railway will be $5.2bn (£3.2bn) - mostly funded by China.
Some Kenyans have complained that the contract was given to the Chinese state-owned China Road and Bridge Corporation (CRBC) without going to tender.
'Historic milestone'Continue reading the main story“Start Quote
Uhuru KenyattaThe project will define my legacy as president of Kenya”
Uhuru KenyattaViewpoint: Will corruption kill off Kenya's grand plans?Kenyan President Uhuru Kenyatta and his Chinese counterpart Xi Jinping agreed the deal in August in Beijing.
It is also hoped that the railway will reduce congestion in Mombasa, one of Africa's busiest ports.
The current railway network dates back to the colonial era.
After the Nairobi section is finished, with completion due in 2017, it will be extended through Uganda, with branch lines west to Kisangani in the Democratic Republic of Congo, south through Rwanda to Burundi and north to South Sudan.
Passenger trains will travel at a top speed of 120 km/h (75 mph), while freight trains will have a maximum speed of 80 km/h.
"What we are doing here today will most definitely transform... not only Kenya but the whole eastern African region," President Kenyatta told crowds at the ceremony, calling it an "historic milestone", the AFP news agency reports.
"As a result East Africa will become a competitive investment destination," the agency quoted him as saying.
Earlier, he said the railway was one of the flagship projects of his government's ambitious Vision 2030 initiative to improve much-neglected infrastructure.
"The project will define my legacy as president of Kenya and it is my personal desire that the implementation is done to the standard," he said in a statement on Wednesday after meeting CRBC officials.
He also thanked the Chinese government for the "massive infrastructural projects in the country", which "reduced the cost of doing business and created employment through increased investments".
Kenya would be working closely with Uganda and Rwanda during the first phase, which would boost ties between the neighbours, he added.
"Kenya will fully meet its obligations towards the project. I will personally oversee its implementation," Mr Kenyatta said.
