It was a process. They started by forbidding Jews from holding certain jobs like doctors, judges, lawyers and and also public jobs. They then passed the Nuremberg Laws that restricted citizenship to “full-blooded” Germans only and made it mandatory for Jews to wear specific marks on their clothes that marked them as Jewish (the yellow Star of David was one but there were others). Then they required Jews to live only in specific parts of cities, the ghettos. The last step just consisted of them coming and grabbing everyone from the ghettos and everyone that was wearing the marks, the groundwork had been already set.