Mississippitoday.org – Jerry Mitchell – 2024-10-13 07:00:00. Construction began on the White House with the laying of the cornerstone. Black Americans, those enslaved and free, did most of the work on the foundations and the main residence. They quarried and cut the rough stone that became the walls of the White House. Historians say they also played a role in the carpentry, carting, rafting, plastering, glazing and painting.