roktech blog

Wait, there’s a difference between Loading and Appending data?

Posted by Emma Paz on Jun 8, 2015 11:27:54 AM

Recently, Jay came across what seemed likely to be a data-type discrepancy between an SDE Spatial View (SV) in SQL and its associated Feature Class (FC); the discrepancy was leading to a handful of unpopulated text fields (varchars and nvarchars).

 

On my quest to resolve the “data-type” issue, I took a few different courses of action and notes:

  • A simple export from the SV to a new FC also resulted in the same empty fields, even when accounting for the character-size difference between varchar and nvarchar..
    • This confirmed the varchar to nvarchar mismatch as insignificant--the fields should have come through regardless.
  • The fields could be captured in a FC of their own (after running a similar export as mentioned above), if they were the only ones selected from the SV.
    • The missing fields were originally “located” halfway through the entire dataset--indicating that the issue wasn’t a matter of attribute quantity either.
  • Exporting out to a regular file geodatabase FC worked (as opposed to exporting within the same SDE)--the data carried over and there were no missing fields. This was exciting in itself but the true test was to populate the data back into the live SDE FC.
    • First I loaded the data (right-click on a FC, select “Load Data” and progress through the field mapping screens, etc.) This was SUCCESSFUL and it seemed like the issue had been resolved!
    • The next step was to automate the update process by creating a “Truncate (Delete Features) and Append” model as the basis of a python script to run through Task Manager on a weekly basis. Unfortunately, this didn’t work, thus serving as a good reminder that ArcCatalog’s Load Data and Append tools (which we usually use interchangeably to update large amount of data) are actually different:
      • The Load Data tool is a Simple Data Loader, meaning that the data is not validated or processed as it loads--this method tends to run faster.
      • By contrast, the Append Geoprocessing Tool is a type of validating Object Data Loader, which is especially optimal when updating large quantities of features (either manually or through modeling) as well as when needing to customize “Environment Settings” like Output Coordinates, Map Extent, etc.

 

Final note: if you’re looking to load data into an existing FC while editing in ArcMap, there’s an additional Object Data Load method that you can read about here.

 

Happy mapping & data updating,

EP
 

Topics: arcGIS, ArcSDE, ArcSDE SQL, Data, Esri, GeoDatabase, GIS

Subscribe to Email Updates

Recent Posts

Posts by Topic

see all