The soldiers stationed along Mordor's border either died or returned to Gondor, where they were more needed. According to the official timeline from the novel version of The Lord of the Rings ...

While soldiers from Rhûn fight in Sauron's armies in "The Lord of the Rings," their land is a mystery ... But its western border is one continual run of coastland. This expansive body of water ...