FileGDB crash windows 7 from worker thread

[MartynasKulvietis]

Hi,

I have FileGDB API crashing when called from a worker thread (no concurrent access to database). However, CreateGeodatabase function call crashes immediately (though db folder and files created). It's a shame, because I still have single-threaded access to database - just not the main thread!
Single isolated thread per DB connection is fine for me, however to avoid the crash or hang I'm forced to switch to main thread for calling Open/Create/Close functions. Crashing code is as simple as that:

    // called from a worker thread
    Geodatabase* pGeodatabase = new Geodatabase;   
    CreateGeodatabase(L"C:\\mydb.gdb", *pGeodatabase);

Any ideas here (I'm using latest FileGDB 1.2).

Martynas

[EricHallander]

I would like to piggyback onto this thread, and add some questions to the multi-threading.

When the README says "No multi-threading support", exactly what do you mean? This api should only ever be used in a single main thread environment, or that only a single thread should ever try to access an open FileGDB?

Our environment is a RHEL6.2 Linux installation, and locally we don't seem to have issues with a worker-thread approach to using the API calls for read access to a single open File GDB. However, a partner development group, running on the same RHEL6.1 VM environment is experiencing a core dump with a stack trace that shows up coring in the CloseGeoDatabase method.

#0  0x0557c478 in free () from /lib/libc.so.6
#1  0x036a131e in xmlResetError (err=0xac006b04) at error.c:871
#2  0x036a19e0 in xmlResetLastError () at error.c:895
#3  0x036b9379 in xmlCleanupParser () at parser.c:14074
#4  0x035a052a in UnInitializeXML() () from /h/FBCB2/cots/lib/libFileGDBAPI.so
#5  0x0346ce48 in FileGDBAPI::Geodatabase::CloseGeodatabase() ()
   from /h/FBCB2/cots/lib/libFileGDBAPI.so
#6  0x034740bf in FileGDBAPI::CloseGeodatabase(FileGDBAPI::Geodatabase&) ()
   from /h/FBCB2/cots/lib/libFileGDBAPI.so
#7  0x00a1dbec in vfsd::gst::vgp::tools::vfgdb::VFGDBDriver::CloseGeodatabase (
    this=0xa3dfbdd8)
    at ../../src/JBCDST_vgp/vfsd/gst/vgp/tools/vfgdb/VFGDBDriver.cpp:185
#8  0x00a1c9be in vfsd::gst::vgp::tools::vfgdb::VFGDBDriver::~VFGDBDriver (
    this=0xa3dfbdd8, __in_chrg=<value optimized out>)
    at ../../src/JBCDST_vgp/vfsd/gst/vgp/tools/vfgdb/VFGDBDriver.cpp:51
#9  0x00a117ad in vfsd::gst::vgp::tools::vnc::VncBuildMatrix::identifyManNetData (this=0xa3dfbfd0, fullPathToFGDB="/media/cdrecorder/Tiny.gdb", manNetInfo=...)
    at ../../src/JBCDST_vgp/vfsd/gst/vgp/tools/vnc/VncBuildMatrix.cpp:238
#10 0x00270788 in jbcp::dst::mannet::c_DstMapDataProcessor::getMapData (path=
    "/media/cdrecorder/Tiny.gdb", mapData=...)
    at ../../src/JBCDST_mannet/jbcp/dst/mannet/c_DstMapDataProcessor.cpp:70
#11 0x00275c21 in jbcp::dst::mannet::c_ManeuverNetCopyAgent::doCopy (
    this=0x962f530)
---Type <return> to continue, or q <return> to quit---
    at ../../src/JBCDST_mannet/jbcp/dst/mannet/c_ManeuverNetCopyAgent.cpp:141
#12 0x00279305 in boost::_mfi::mf0<void, jbcp::dst::mannet::c_ManeuverNetCopyAgent>::operator() (this=0x96b18f0, p=0x962f530)
    at /home/hendrixjl/alternate/builds/BUILD_core_1.1.0.31/BUILD_DIR/include/boost/bind/mem_fn_template.hpp:49
#13 0x002792a4 in boost::_bi::list1<boost::_bi::value<jbcp::dst::mannet::c_ManeuverNetCopyAgent*> >::operator()<boost::_mfi::mf0<void, jbcp::dst::mannet::c_ManeuverNetCopyAgent>, boost::_bi::list0> (this=0x96b18f8, f=..., a=...)
    at /home/hendrixjl/alternate/builds/BUILD_core_1.1.0.31/BUILD_DIR/include/boost/bind/bind.hpp:246
#14 0x00279264 in boost::_bi::bind_t<void, boost::_mfi::mf0<void, jbcp::dst::mannet::c_ManeuverNetCopyAgent>, boost::_bi::list1<boost::_bi::value<jbcp::dst::mannet::c_ManeuverNetCopyAgent*> > >::operator() (this=0x96b18f0)
    at /home/hendrixjl/alternate/builds/BUILD_core_1.1.0.31/BUILD_DIR/include/boost/bind/bind_template.hpp:20
#15 0x002791a8 in boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, jbcp::dst::mannet::c_ManeuverNetCopyAgent>, boost::_bi::list1<boost::_bi::value<jbcp::dst::mannet::c_ManeuverNetCopyAgent*> > > >::run (
    this=0x96b1820)
    at /home/hendrixjl/alternate/builds/BUILD_core_1.1.0.31/BUILD_DIR/include/boost/thread/detail/thread.hpp:56
#16 0x00840bf6 in thread_proxy () from /usr/lib/libboost_thread-mt.so.5
#17 0x00c57a09 in start_thread () from /lib/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#18 0x055e600e in clone () from /lib/libc.so.6

We are doing read access only from a single thread, so I fail to see how multi-threading could be to blame here.

Thanks
Eric

[LanceShipman]

Can you send a sample program that reproduces the problem. We have not been able to duplicate your issue.

[SalaheddinAl-Ajlouni]

Hi Lance,

I am have exactly the same problem mentioned here. I can send you sample application where the problem can be reproduced.

I realized that Geodatabase API C++ is not multi-thread. But as mentioned here if is called from a working thread, the second time for me it would crash. I always make sure that I close the database before starting the second call.

Interestingly, it does NOT crash if I used std:::async() but it does crash if I use AfxBeginThread().

Here is a code snippet.

struct data
{
    Geodatabase m_geodatabase;
    std::wstring m_name;
};


UINT create(LPVOID param)
{
    data *pObject  = reinterpret_cast<data*>(param);

    if (pObject == nullptr)
    {
        return 0;
    }

    // Create a new geodatabase in the current directory.
    fgdbError   hr;
    std::wstring     errorText;
    {
        try
        {
            if ((hr = CreateGeodatabase(pObject->m_name, pObject->m_geodatabase)) != S_OK)
            {
                ErrorInfo::GetErrorDescription(hr, errorText);
                return -1;
            }
        }
        catch (...)
        {

        }
    }


    CloseGeodatabase(pObject->m_geodatabase);
    delete pObject;
    pObject = nullptr;

    return 0;
}


void CTestGeoDlg::OnBnClickedButton1()
{
    data *d = new data;
    d->m_name = m_name1;

    // TODO: Add your control notification handler code here
    // create( d);
    std::async(create, d);
    // AfxBeginThread(create, d);
}


void CTestGeoDlg::OnBnClickedButton2()
{
    data *d = new data;
    d->m_name = m_name2;

    // TODO: Add your control notification handler code here
    // create( d);
    std::async(create, d);
    //AfxBeginThread(create, d);
}
{

m_name1 and m_name2 are different. The files get created but then crash.

[PaulAustin]

Lance,

I have found the solution to this issue. It's to do with how you access the libxml for the first time. If you access it from a thread other than the main thread then there can be issues unless xmlInitParser() has been called first. See the following web page.

http://www.xmlsoft.org/threads.html

Is there any chance that you could add a function (e.g. FileGDBAPI::Initialize) that applications could call from the main thread. This function would call xmlInitParser() and any other required initialization.

[PaulAustin]

Here is what I have found related to threading and the FGDB library. I'm using a Java wrapper.

1. Have a single synchronization monitor for the whole API. Use this if you create a geodatabase, close a geodatabase, add/change tables etc. The issue is the XML library which is not thread safe. In fact it's so thread dumb it shares memory when reading different XML files. FYI the mtkime and localtime function is also thread dumb if you use that to convert from a long time to a time_t instance.
2. Have a separate synchronization monitor for each GDB (if you use the same GDB in different threads). Use this when you do operations such as inserts, updates, deletes, queries, accessing enum rows and the row objects.
3. Avoid using ClearErrors, GetErrorRecordCount, GetErrorRecord. I'm constantly getting core dumps when calling these. I think there is something wrong with the pthread thread local variable code used there.

When using the thread monitor 1 I always synchronize on 2 first. That way you avoid deadlocking situations of holding the lock for 2 while getting 1.

FYI with the above strategy I've been able to develop multi-threaded applications with 20 threads that create 400 file GDB files.