The North laid siege to the port early in the war and the city fell in 1862 but fighting continued for three more years.