Trouble accessing data in multi-dimensional JSON for NumPyArray to Feature

Question

This is more of a general Python question which will set me up to send this data from a NumPyArray to Feature Class. I am trying to parse through a web-service and retrieve certain records, however I am consistently receiving a KeyError for responses with more than one object. These records are returned in intervals, so sometimes I might receive one record and others I might receive 300. If I receive one record, the logic of my code works, if there are multiple items then the code doesn't work. Here is an example of an output with one object. I have posted to StackOverFlow and no luck, so this is cross-posted. { "status": { "code": 311, "message": "Service Request Successfully Queried.", "cause": "" }, "Response": { "NumOutputObjects": "1", "ListOfServiceRequest": { "ServiceRequest": [ { "AddressVerified": "Y", "SRNumber": "1-13967451", "SRType": "Service Not Complete", "CreatedDate": "05/08/2015 10:00:38", "UpdatedDate": "05/08/2015 10:00:49", "IntegrationId": "05082015100148678", "Status": "Open", "CreatedByUserLogin": "PROXYE", "UpdatedByUserLogin": "PROXYE", "Anonymous": "N", "Zipcode": "90032", "Latitude": "34.0843242531", "Longitude": "-118.171015007", "CustomerAccessNumber": "", "LADWPAccountNo": "", "NewContactFirstName": "jj", "NewContactLastName": "rambo", "NewContactPhone": "", "NewContactEmail": "", "ParentSRNumber": "1-10552271", "Priority": "Normal", "Language": "", "ReasonCode": "", "ServiceDate": "", "Source": "", "ClosedDate": "", "Email": "", "FirstName": "", "HomePhone": "", "LastName": "", "LoginUser": "", "ResolutionCode": "", "SRUnitNumber": "", "MobilOS": "", "SRAddress": "5163 E TEMPLETON ST, 90032", "SRAddressName": "", "SRAreaPlanningCommission": "East Los Angeles APC", "SRCommunityPoliceStation": "CENTRAL BUREAU", "SRCouncilDistrictMember": "Jose Huizar", "SRCouncilDistrictNo": "14", "SRDirection": "E", "SRNeighborhoodCouncilId": "48", "SRNeighborhoodCouncilName": "LA-32 NC", "SRStreetName": "TEMPLETON", "SRSuffix": "ST", "SRTBColumn": "F", "SRTBMapGridPage": "595", "SRTBRow": "6", "SRXCoordinate": "6509897", "SRYCoordinate": "1853117", "AssignTo": "NC", "Assignee": "NC eWaste Supervisor 01", "Owner": "BOS", "ParentSRStatus": "Open", "ParentSRType": "Electronic Waste", "ParentSRLinkDate": "05/08/2015 10:00:39", "ParentSRLinkUser": "PROXYE", "SRAreaPlanningCommissionId": "5", "SRCommunityPoliceStationAPREC": "HOLLENBECK", "SRCommunityPoliceStationPREC": "4", "SRCrossStreet": "", "ActionTaken": "", "SRCity": "", "RescheduleCounter": "", "SRHouseNumber": "5163", "SourceofRequestCouncil": "", "CCBPremiseType": "", "ContainerBlackCount": "", "ContainerBrownCount": "", "SRIntersectionDirection": "", "SRApproximateAddress": "N", "ContainerGreenCount": "", "OtherBureauName": "", "AssigneeName": "", "AssigneeOrganization": "E-Waste, NC", "AnotherBureauEmailId": "", "ListOfLa311BarricadeRemoval": {}, "ListOfLa311BulkyItem": {}, "ListOfLa311DeadAnimalRemoval": {}, "ListOfLa311GraffitiRemoval": {}, "ListOfLa311InformationOnly": {}, "ListOfLa311MultipleStreetlightIssue": {}, "ListOfLa311SingleStreetlightIssue": {}, "ListOfLa311SrPhotoId": { "La311SrPhotoId": [] }, "ListOfLa311BusPadLanding": {}, "ListOfLa311CurbRepair": {}, "ListOfLa311Flooding": {}, "ListOfLa311GeneralStreetInspection": {}, "ListOfLa311GuardWarningRailMaintenance": {}, "ListOfLa311GutterRepair": {}, "ListOfLa311LandMudSlide": {}, "ListOfLa311Pothole": {}, "ListOfLa311Resurfacing": {}, "ListOfLa311SidewalkRepair": {}, "ListOfLa311StreetSweeping": {}, "ListOfLa311BeesOrBeehive": {}, "ListOfLa311MedianIslandMaintenance": {}, "ListOfLa311OvergrownVegetationPlants": {}, "ListOfLa311PalmFrondsDown": {}, "ListOfLa311StreetTreeInspection": {}, "ListOfLa311StreetTreeViolations": {}, "ListOfLa311TreeEmergency": {}, "ListOfLa311TreeObstruction": {}, "ListOfLa311TreePermits": {}, "ListOfLa311BrushItemsPickup": {}, "ListOfLa311Containers": {}, "ListOfLa311ElectronicWaste": {}, "ListOfLa311IllegalDumpingPickup": {}, "ListOfLa311ManualPickup": {}, "ListOfLa311MetalHouseholdAppliancesPickup": {}, "ListOfLa311MoveInMoveOut": {}, "ListOfLa311HomelessEncampment": {}, "ListOfLa311IllegalAutoRepair": {}, "ListOfLa311IllegalConstruction": {}, "ListOfLa311IllegalConstructionFence": {}, "ListOfLa311IllegalDischargeOfWater": {}, "ListOfLa311IllegalDumpingInProgress": {}, "ListOfLa311IllegalExcavation": {}, "ListOfLa311IllegalSignRemoval": {}, "ListOfLa311IllegalVending": {}, "ListOfLa311LeafBlowerViolation": {}, "ListOfLa311NewsRackViolation": {}, "ListOfLa311Obstructions": {}, "ListOfLa311TablesAndChairsObstructing": {}, "ListOfLa311GisLayer": { "La311GisLayer": [ { "A_Call_No": "", "Area": "0", "Day": "MONDAY", "DirectionSuffix": "", "DistrictAbbr": "", "DistrictName": "NC", "DistrictNumber": "", "DistrictOffice": "", "Fraction": "", "R_Call_No": "", "SectionId": "", "ShortDay": "", "StreetFrom": "", "StreetTo": "", "StreetLightId": "", "StreetLightStatus": "", "Type": "GIS", "Y_Call_No": "", "Name": "05082015100148678100", "CommunityPlanningArea": "", "LastUpdatedBy": "", "BOSRadioHolderName": "" } ] }, "ListOfLa311ServiceRequestNotes": { "La311ServiceRequestNotes": [ { "CreatedDate": "05/08/2015 10:00:39", "Comment": "Materials have been out in a normal collection area, unsure why driver missed the e-waste items.", "CreatedByUser": "PROXYE", "IsSrNoAvailable": "", "CommentType": "External", "Notification": "N", "FeedbackSRType": "", "IntegrationId": "050820151001486782", "Date1": "", "Date2": "", "Date3": "", "Text1": "", "AnotherBureau": "", "EmailAddress": "", "ListOfLa311SrNotesAuditTrail": {} }, { "CreatedDate": "05/08/2015 10:00:39", "Comment": "", "CreatedByUser": "PROXYE", "IsSrNoAvailable": "", "CommentType": "Address Comments", "Notification": "N", "FeedbackSRType": "", "IntegrationId": "050820151001486781", "Date1": "", "Date2": "", "Date3": "", "Text1": "", "AnotherBureau": "", "EmailAddress": "", "ListOfLa311SrNotesAuditTrail": {} } ] }, "ListOfLa311SubscribeDuplicateSr": {}, "ListOfChildServiceRequest": {}, "ListOfLa311BillingCsscAdjustment": {}, "ListOfLa311BillingEccAdjustment": {}, "ListOfLa311BillingRsscAdjustment": {}, "ListOfLa311BillingRsscExemption": {}, "ListOfLa311SanitationBillingBif": {}, "ListOfLa311SanitationBillingCssc": {}, "ListOfLa311SanitationBillingEcc": {}, "ListOfLa311SanitationBillingLifeline": {}, "ListOfLa311SanitationBillingRssc": {}, "ListOfLa311SanitationBillingSrf": {}, "ListOfLa311DocumentLog": {}, "ListOfAuditTrailItem2": { "AuditTrailItem2": [ { "Date": "05/08/2015 10:00:49", "EmployeeLogin": "SADMIN", "Field": "Assignee", "NewValue": "NC eWaste Supervisor 01", "OldValue": "" } ] }, "ListOfLa311GenericBc": { "La311GenericBc": [ { "ATTRIB_08": "", "NAME": "05082015100148678100", "PAR_ROW_ID": "1-8BDCR", "ROW_ID": "1-8BOCG", "TYPE": "GIS", "ATTRIB_16": "", "ListOfLa311GenericbcAuditTrail": {} }, { "ATTRIB_08": "", "NAME": "05082015100148678", "PAR_ROW_ID": "1-8BDCR", "ROW_ID": "1-8BOCJ", "TYPE": "Service Not Complete", "ATTRIB_16": "", "ListOfLa311GenericbcAuditTrail": {} } ] }, "ListOfLa311ServiceNotComplete": { "La311ServiceNotComplete": [ { "ContainerLocation": "", "ContainerType": "", "DriverFirstName": "", "DriverLastName": "", "MissedCollectionService": "Electronic Waste", "OtherServiceMissedReason": "", "ServiceDateRendered": "", "ServiceMissedReason": "I'm not sure", "TruckNo": "", "Type": "Service Not Complete", "WireBasketLocation": "", "LastUpdatedBy": "", "Name": "05082015100148678" } ] }, "ListOfLa311Other": {}, "ListOfLa311WeedAbatementForPrivateParcels": {}, "ListOfLa311SanitationBillingInquiry": {} } ] } } } The code is below; data2 = jsonpickle.decode((f2.read()))Start = datetime.datetime.now()data2 = jsonpickle.encode(data2)url2 = "myURL"headers2 = {'Content-type': 'text/plain', 'Accept': '/'}r2 = requests.post(url2, data=data2, headers=headers2)decoded2 = json.loads(r2.text)try: r2except requests.exceptions.ConnectTimeout as e: print "Too slow Mojo!"items = []for sr in decoded2['Response']['ListOfServiceRequest']['ServiceRequest']: SRAddress = sr['SRAddress'] Latitude = sr['Latitude'] Longitude = sr['Longitude'] ReasonCode = sr['ReasonCode'] SRNumber = sr['SRNumber'] FirstName = sr['FirstName'] LastName = sr['LastName'] ResolutionCode = sr['ResolutionCode'] HomePhone = sr['HomePhone'] CreatedDate = sr['CreatedDate'] UpdatedDate = sr['UpdatedDate'] CreatedDate = datetime.datetime.strptime(CreatedDate, "%m/%d/%Y %H:%M:%S") UpdatedDate = datetime.datetime.strptime(UpdatedDate, "%m/%d/%Y %H:%M:%S") print SRAddress print SRNumberItemInfo = " "for ew in sr["ListOfLa311ServiceRequestNotes"][u"La311ServiceRequestNotes"]:Comment = ew['Comment']print Comment Output Materials have been out in a normal collection area, unsure why driver missed the e-waste items. If I use the logic above for a response with more than one object returned I receive a KeyError value and am unable to access the array that I want to parse. Example of code with multiple objects returned; Output when I use; I receive a key error if I attempt to do something along the lines of CommodityType = sr['ListOfLa311ElectronicWaste']['ElectronicWasteType'] below for sr in decoded2['Response']['ListOfServiceRequest']['ServiceRequest']: CommodityType = sr['ListOfLa311ElectronicWaste'] # ItemType = sr['ElectronicWestType'] # DriverFirstName = sr ['DriverFirstName'] # DriverLastName = sr ['DriverLastName'] # ItemCount = sr['ItemCount'] # ItemInfo += '{0}, {1}, '.format(ItemType, ItemCount) # ParentNumber = sr['Name'] # print CommodityType {u'La311ElectronicWaste': [{u'IllegallyDumped': u'N', u'OtherElectronicWestType': u'hash', u'ItemCount': u'5', u'Name': u'6a31f058-ece1-4e7d-b682-7d9052a512f4', u'MobileHomeSpace': u'', u'DriverLastName': u'', u'ActiveStatus': u'Y', u'DriverFirstName': u'', u'LastUpdatedBy': u'', u'GatedCommunityMultifamilyDwelling': u'Outside the main gate', u'IllegalDumpCollectionLoc': u'', u'ElectronicWestType': u'Other', u'CollectionLocation': u'Gated Community', u'Type': u'Electronic Waste', u'ServiceDateRendered': u'', u'TruckNo': u''}]} {u'La311ElectronicWaste': [{u'IllegallyDumped': u'Y', u'OtherElectronicWestType': u'', u'ItemCount': u'5', u'Name': u'3f4d9d20-a712-4be3-822f-e6a45219c1cf', u'MobileHomeSpace': u'', u'DriverLastName': u'', u'ActiveStatus': u'Y', u'DriverFirstName': u'', u'LastUpdatedBy': u'', u'GatedCommunityMultifamilyDwelling': u'', u'IllegalDumpCollectionLoc': u'Cul De Sac', u'ElectronicWestType': u'Electronic Equipment', u'CollectionLocation': u'Alley', u'Type': u'Electronic Waste', u'ServiceDateRendered': u'', u'TruckNo': u''}]}

JeffWard · Answer

Your dictionary doesn't need the : in your key reference:

for manualcount in Manual["La311ManualPickup"]:

enterprisemaps-gwest · Answer

I was able to solve this with an inner loop of those items. Now I have another issue where the inner loop is not writing to the db.  The inner loops read as;

for sr in Manual:
   for manualcount in Manual["La311ManualPickup":]:
   countmanual = manualcount['ItemCount']
print countmanual

CarlSunderman · Answer

You could try using a getcount and a if result >1, that should handle if it returns more than one