Dim outputParameter As IGPParameterEdit3 = New GPParameterClass() outputParameter.DataType = CType(New GPSpatialReferenceType(), IGPDataType)
I got a reply from ESRI support on this question so I'm posting for others. It is possible to have a python spatial reference as an input type. If you don't set the value of the output parameter it will only accept integers, string names, the spatial reference as a string or a string path of a prj file. However, setting the value allows you to pass in a python spatial reference object and the parameter will behave in exactly the same way as the ESRI ones (see project in data management for example)
Dim outputParameter As IGPParameterEdit3 = New GPParameterClass()
outputParameter = New GPParameterClass()
outputParameter.DataType = DirectCast(New GPSpatialReferenceType(), IGPDataType)
outputParameter.Value = DirectCast(New GPSpatialReferenceClass(), IGPValue)
outputParameter.Direction = esriGPParameterDirection.esriGPParameterDirectionInput
outputParameter.DisplayName = "Output File Projection"
outputParameter.Name = "out_projection"
outputParameter.ParameterType = esriGPParameterType.esriGPParameterTypeRequired
Dim pParameters As IArray = New ArrayClass()
pParameters.Add(outputParameter)