In the lifetime of the Prophet (ﷺ) there was no wall around the Ka`ba and the people used to pray aroundthe Ka`ba till `Umar became the Caliph and he built the wall around it. 'Ubaidullah further said, "Itswall was low, so Ibn Az-Zubair built it."