On what grounds can my international passport be seized?

There is a difference between seizing a passport and withdrawing passport. While withdrawn can be permanent until you get another one, seizure is temporary and can be release at the appropriate time. Seizure applies mainly to restriction on the right of exit from Nigeria not movement within. A good example is the seizure of the passport of a criminal suspect, or convict, or his incarceration, although within limits and scope of the law.