topic Re: FGDB API is too slow in File Geodatabase API Questions

FGDB API is too slow

MuneemHabiba — Sat, 11 Dec 2021 12:54:44 GMT

I have read all feature classes and their associated geodata like Point,polyline,polygon data. API is taking too much time in reading these data.

My code is as follow:

Table testtable = geodatabase.OpenTable("\\sw_Manhole");

geoshape.Items = new List<ImpexItem>();
geoshape.LayerName = child;
bool checkFields = false;
int fieldCount = table.FieldInformation.Count;

foreach (Row row in table.Search("*", "", RowInstance.Recycle)) // reading coordinates
{
    ImpexItem layerItem = new ImpexItem();
    layerItem.PartList = new List<ImpexPart>();
    ImpexPart itemParts = new ImpexPart();
    itemParts.Coordinates = new List<PointD>();

    if (row.GetGeometry().shapeType.ToString() == "Point")
    {
        PointShapeBuffer geometry = row.GetGeometry();
        Esri.FileGDB.Point point = geometry.point;
        PointD itemPoints = new PointD();
        itemPoints.X = point.x;
        itemPoints.Y = point.y;
        itemParts.Coordinates.Add(itemPoints);
    }
    else
    {
        try
        {
            MultiPointShapeBuffer geometry = row.GetGeometry(); // if geometry type is polygon/polyline
            Esri.FileGDB.Point[] point = geometry.Points;
            for (int numpoints = 0; numpoints < point.Length; numpoints++)
            {
                PointD itemPoints = new PointD();
                itemPoints.X = point[numpoints].x;
                itemPoints.Y = point[numpoints].y;
                itemParts.Coordinates.Add(itemPoints);
            }
        }
        catch (Exception e)
        {
            Console.WriteLine("MultiPoint Exception" + e.Message);
        }
    }
    layerItem.PartList.Add(itemParts);

    geoshape.Items.Add(layerItem); // one layer data
}

for loop will run 300 times and its taking more then 4 minutes for reading this simple data.

Message was edited by: Vince Angelo (Formatted for legibility)

Re: FGDB API is too slow

VinceAngelo — Wed, 04 Nov 2015 20:00:46 GMT

You've left a great deal out of this post, making it almost impossible to assist:

On which operating system is the application deployed?
Is the source FGDB on a local or network disk?
- If local, what kind or disk, and what mean seek time?
- if networked, what networking protocol, and at what bandwidth?
How many total feature classes?
How many average features per feature class (rows per table)?
How many average vertices per feature?
What is the mean read time per feature?
What is the total size on disk of the geodatabase?
How do you define "too much time" (what is your requirement)?

In the future, please format code using the "Use advanced editor" option, selecting the code block, and choosing the ">>" icon to specify formatting with "Syntax Highlighting" or "C#" (or whatever is appropriate). It would also help to explicitly state you are using the 1.4 API with C# language and whatever IDE you are using.

- V

Re: FGDB API is too slow

MuneemHabiba — Thu, 05 Nov 2015 05:22:55 GMT

Windows 7 Operating system
Is the source FGDB on a local or network disk ? (Local)
- If local, what kind or disk, and what mean seek time?
- if networked, what networking protocol, and at what bandwidth?
How many total feature classes? (Right now i am testing only one)
How many average features per feature class (rows per table)? (310 rows per table)
How many average vertices per feature? (2)
What is the mean read time per feature? (time varies but on average its almost 30 sec)
What is the total size on disk of the geodatabase? (gdb attached)
How do you define "too much time" (what is your requirement)? it should be less than 2minute to read full gdb file as i have tested in postGIS and it is taking much less time

Re: FGDB API is too slow

VinceAngelo — Sat, 11 Dec 2021 12:54:47 GMT

Using a C++ app I had already created, I fetched every row of "\\sw_Manhole", and it took 7.3 MILLISECONDS to read 302 rows with 33 fields (OpenTable took 3.2ms, so the actual average read duration was 12.4 microseconds per feature). Each shape had one vertex (they're points, after all).

./fgdbtimer2 ../WS_Data.gdb "\sw_Manhole" SHAPE
Field 1: OBJECTID      OID
Field 2: ASSET_ID      NSTRING
Field 3: INSTALL_DATE  DATE
Field 4: RIM_ELEVATION  FLOAT64
Field 5: DEPTH  FLOAT64
Field 6: DIAMETER      FLOAT64
Field 7: COVER_TYPE    NSTRING
Field 8: VENT_TYPE      NSTRING
Field 9: DROP_STRUCTURE INT16
Field 10: CONTRACTNUM  NSTRING
Field 11: ASSET_STATUS  NSTRING
Field 12: OWNERSHIP    NSTRING
Field 13: MANAGEDBY    NSTRING
Field 14: UPDATETXID    INT32
Field 15: LASTEDIT      DATE
Field 16: WS_STREET    NSTRING
Field 17: WS_CROSS_STREET      NSTRING
Field 18: WS_MANUFACTURER      NSTRING
Field 19: WS_PROJECT_NAME      NSTRING
Field 20: WS_PROJECT_PHASE      NSTRING
Field 21: WS_ASSUMPTION_AREA    NSTRING
Field 22: WS_GPN        NSTRING
Field 23: WS_DEVELOPER  NSTRING
Field 24: WS_CONSULTANT NSTRING
Field 25: WS_FEATURE_CREATE    DATE
Field 26: WS_FEATURE_EDIT      DATE
Field 27: WS_LAST_EDITOR        NSTRING
Field 28: RJB_FLOW_TYPE INT16
Field 29: created_user  NSTRING
Field 30: created_date  DATE
Field 31: last_edited_user      NSTRING
Field 32: last_edited_date      DATE
Field 33: Shape SHAPE
  mode: SHAPE
  reads: 303
    v/f: 1
ms_desc: 0.153
ms_open: 3.27
  ColA: 3.423
ms_read: 3.769
ms_get: 0.122
  total: 7.314

Your application is sub-optimal for performance evaluation because of multiple GetGeometry invocations, but there's nothing obvious as to why my app should run nearly 33 thousand times faster. I recommend you:

Instrument your code with a millisecond resolution timer
Clean up the coding to be as efficient as possible
Re-run your evaluation

You can then report back on the timing for each function call.

- V

PS: MultiPointShapeBuffer is for multipoint shapes -- Lines and Polygons use MultiPartShapeBuffer

Re: FGDB API is too slow

PavelShkurko — Thu, 26 Nov 2015 08:44:29 GMT

Maybe the problem is in

try
{
...
}
catch (Exception e)
{
...
}

this feature works slow enough when you catch exceptions.

I tried same code - it works well. Check console output.