Kalpitiya Fort was built by the Dutch between 1667 and 1676. Kalpitiya was important when entering the adjacent bay Puttalam Lagoon. Kalpitiya was once a well-known trading center for passing Arab traders. In the mid-16th century, the Portuguese conquered the city and renamed it Cardiff Island. The surrounding Puttalam area is one of the major cinnamon growing areas in Sri Lanka. King Rajasinghe II of the Kandy Kingdom, the then ruler of Sri Lanka, turned to the Dutch to reclaim his land. However, the land was not returned to the king after the Dutch invasion. The Dutch built a canal from Puttalam to Colombo via Negombo to transport cinnamon from the area.

The fort is square in shape, about 4 m (13 ft) high, and the surrounding area is made of coral and limestone. It has a single entrance and in front of the lagoon, there is a pediment, above which is a belfry and it looks like the entrance to a church. Kalpitiya Fort has four bastions on each corner, each with its own guard post, with two smaller bastions facing the lagoon. The fort was captured by the British in 1795 and occupied until 1859 until the middle of the 19th century. During the decades-long civil war in Sri Lanka, the fort was used by the Sri Lanka Navy as a base for its training and operations. There were two tunnels away from the fort – one to the sea and the other to the Dutch Reformed Church 400 m (1,300 ft) outside the fort. These tunnels are blocked and inaccessible.