First of all, you need to make sure you're using the right PostgreSQL 8.3 release --
ArcGIS 10.0 supports PG 8.3.8.
Second, you need to specify the port either by using the PGPORT environment variable,
or by editing the etc/dbinit.sde to contain "set PGPORT=5532". This should unclutter
your command-line down to the minimum (don't specify a '-i' at all -- all 'install' operations
are Direct Connect by necessity, and that '-i' doesn't accept the same parameters).
Note that you need one SDEHOME for each instance to make this work.
Third, starting at 10.0, ArcGIS could be used to create geodatabases, and at 10.1 became
the only way. You might want to start using the modern tools.
- V