Some parts of the Outer Fortress, which was built by the Roman Emperor Gordianus III in 242 AD and surrounds the city from the south and west, are still standing. The northern walls, which turn west from the Inner Castle, start from the entrance of the Grand Bazaar and reach the Düvenönü district. The parts of the old walls extending to the west from here and most of the walls that were made to turn south from here during the Byzantine period have not survived. However, a still standing section containing triangular Byzantine bastions close to Yoğunburç was restored. There were two gates in this part of the walls. One of them was Boyacı Gate in the east of Hatıroğlu Mosque, and the other was Kiçikapı in the east of Kayseri High School. There were two door openings, one in Sivas and the other in Yenikapı, on the walls reaching the Inner Castle from the eastern front. It is understood from the ruins here that the Inner Castle was surrounded by a wall extending to the northeast. Ok Burcu is also in this section. On these walls, there was also the famous "Square Gate" opening to the square. Only some parts of the outer walls, which have shelters, watchtowers and bastions, are still standing today.