Opinions?

Someone throws a White Elephant Xmas party, w/ 1:00pm start time. When the invitations go out, one of the invitees complains (not to the host) about the start time & says they are showing up at 8:00pm. By 6:00pm, everyone that is participating in the WE exchange is present, except for the 8pm arrivals. Texts & calls to the late arrivals go unanswered. At 7:45pm one of them finally answers the ph & says the other is on the way to pick 'em up & head over. They are told to hurry since the rest of the party is waiting. At 9:30pm, they haven't arrived yet so the party says "F it let's do it already." WE is started & done. About 10 minutes after it's done the 8pmers finally arrive. Of course they seem a lil' miffed when they realize it was done w/o them. The party awkwardly goes on. Everything settles down & after awhile everyone seems to have moved on and enjoy the rest of the night. Everyone eventually goes home w/o incident. My question: Do the late folks have a right to be upset, or did everyone have a right to go ahead w/ it after waiting? Unfortunately a white elephant party is one of the few parties that can be held up by tardiness...well besides surprise bdays and bachelor parties ("What time is the "entertainment" supposed to be here?").