I was able to get this to work using the 'sdetable -o create', 'sdelayer -o add', and 'sdelayer -o alter'. Ex:Create a table:sdetable -o create -t parcels -d "PIN string(20)" -D vector -u vector
Add a spatial column to the table including coordinate system:sdelayer -o add -l parcels,shape -e a -G 32614 -D vector -u vector
Alter the coordinate system:sdelayer -o alter -l parcels,shape -G 32618 -D vector -u vector
You should be able to alter the spatial index using the 'sdelayer -o alter' command:sdelayer -o alter -l parcels,shape -g GRID,300,1000,5000 -D vector -u vector
I am currently using ArcSDE 10 SP3 and SQL Server 2008.