'Westminster' means the minster, or abbey church, to the west.
It was west of the City of London. The abbey was founded in the 10th century and rebuilt by Edward the Confessor, and it gave its name to the whole area, including the Houses of Parliament.