We'd been thinking the same.
London is metropolitan and Seaford is sleepy- and yet they're only a few miles apart and have the same kind of shingle beach. I've no idea why they've developed along such very different lines.
Historically Seaford was the more important of the two. It was a Cinq Port when Brighton (then spelt Brighthelmstone) was just a fishing village.
I like Brighton, but we don't go there. We're no longer fond of bustle (not that we ever really were.) But the more we see of Seaford the more we love it.
And here's something Seaford has which Brighton doesn't- the great chalk outcrop of Seaford Head- most westerly of the white cliffs which the song says are of Dover but which are actually in Sussex about 70 miles away.