Are you setting the output spatial reference on your spatial filter? If not, you'll get unexpected results if the spatial reference of your query geometry is different than the spatial reference of the features you are querying against. Also, isn't the spatial relationship you're looking for the same as "crosses"? If so, I would just set the spatial relationship property to esriSpatialRelCrosses and not use a custom filter.
esriSpatialRelCrosses
Returns a feature if the intersection of the interiors of the two shapes is not empty and has a lower dimension than the maximum dimension of the two shapes. Two lines that share an endpoint in common do not cross. Valid for Line/Line, Line/Area, Multi-point/Area, and Multi-point/Line shape type combinations.