topic Re: Any Plans for ArcGIS SDE Support for NoSQL Databases ? in Data Management Questions

Any Plans for ArcGIS SDE Support for NoSQL Databases ?

JamesFox1 — Wed, 12 Oct 2011 19:36:06 GMT

Does ESRI have plans for supporting any of NoSQL (non-SQL?) alternative database platforms and/or architectures like Cassandra, BigTable, Neo4J, HBase, Tokyo Cabinet,..etc that are gaining popularity these days in the large data web application sites like Facebook, Twitter and Google?

These Platforms appear to be well suited for flatter database designs (ie - web page history and such) with huge data throughput and performance requirements which also may not have as stringent transactional, business rule enforced or relational integrity requirements as with traditional hugely normalized databases in these high impact database areas.

The reason I ask is that many of of the SDE geodatabase designs I have seen so far appear to have a flatter design with fewer layers of normalization than that of the large traditional non-spatial RDBMS enterprise databases. Also with ArcGIS provding much of the data integrity through it's own functionality with domains, topologies and relationship classes I would guess these new up and coming platforms could be of major interest.

Sorry if I may seem a little ignorant in this area, but I have not yet attended any of the ESRI user conferences yet!

-Jim

Re: Any Plans for ArcGIS SDE Support for NoSQL Databases ?

VinceAngelo — Wed, 12 Oct 2011 22:10:25 GMT

Given that ArcSDE is dependent on a standards-conformant SQL engine to operate, it doesn't
seem likely.

The best way to get answers to questions that start "Does Esri have plans" is to contact Esri
directly, either through Tech Support or your local marketing representative. While Esri staff
often participate in these User Forums on technical issues, very few of us are empowered to
respond to future "plans" questions directly.

- V

Re: Any Plans for ArcGIS SDE Support for NoSQL Databases ?

JamesFox1 — Wed, 19 Oct 2011 15:24:40 GMT

Thanks Vince,

The point I am trying make is that the NoSQL movement is pushing the threshold ahead in cloud based data management and storage to support the exponentially increasing massive data requirements we are seeing in the online world today. Yes, these technologies do not currently address all the ACID properties that RDBMS products support.

Yes, these products are currently startup, incubator based, open source technologies and initiatives that are not currently standards based but I envision that these issues WILL be solved shortly and minimally these cloud based technologies can be in the short term integrated into portions of relational products that address certain performance critical areas such as data movement and caching. Oracle just announced BDA (Big Data Appliance) last week so this is already given.

I see GIS with immense amounts of data as having one of the biggest needs for this. Some of the data paradigms involved like graph databases are a perfect fit for GIS and especially ESRI products in general. I would like to see a large GIS data project with a Hadoop type of map-reduce solution for the data movement portion along with a traditional relational transaction/business logic architecture for a backend geodatabase. I am sure such a project may be going on a this very moment.

-Jim

Re: Any Plans for ArcGIS SDE Support for NoSQL Databases ?

VinceAngelo — Wed, 19 Oct 2011 16:16:05 GMT

My point is that ArcSDE is a very specific solution that is implemented in SQL databases.

If these new databases support a recognizable geometry type, the logical integration point
is not with an ArcSDE implementation, but with a QueryLayer driver. Perhaps the question
you should be asking (through your marketing rep or via ideas.arcgis.com) is whether Esri
has any plans to provide a DLL bridge mechanism to allow user-definable Query Layers,
which you could then customize to support NoSQL databases.

- V