Construction of the stone castle was started in 1420. The southwestern tower of 11,5 m in diameter was built during the construction of the castle. The thick brick walls of the tower evidence that gunpowder artillery was already used to besiege the castles at that time. Construction was terminated after the Treaty of Melno signed in 1422, according to which the Order recognised Sūduva to Lithuania. In the 19th century, the tower was 8-10 m high, only 2,5-3 m high wall survived. It is believed that massive oak beems were used for the construction of the tower, after rotting away of which the cavities prompting stories and legends about myterious caves had opened.