The idea of using the ratio of the polygon's area to its perimeter can be helpful to some extent, but polygons with the same ratio may have very different sizes and the ratio won't reflect a narrow/length "branch" of a polygon.
Running the Buffer tool with a negative distance (a bit more than half of the average width of the linear polygons or branches) is more effective because a narrow or small portion of a polygon or entire polygon will not receive a buffer. If a buffer cannot be produced for an input polygon, the FID of the polygon will be reported by the buffer tool. You can use the input polygons to intersect the resulting buffer polygons. The input polygons that find no resulting buffer polygons are the linear or small polygons that you may exclude.
It is not easy to find and remove a linear branch of a polygon with existing tools. Perhaps a new tool needs to be developed.
Perhaps you can post an image or sample data to show the tree clusters and the linear polygons. Are the trees point or polygon features to begin with? If they are points, you can try the Aggreate Points tool (ArcGIS 10, arcinfo licensing). Linearly distributed points may not become polygons.
Regards,