Wednesday, August 3, 2016

Integrating UAV/drone remote data acquisition with Salesforce to enhance logistics

Late last year I did a brief interview at the Auckland Salesforce APAC tour event and talked about capturing sensor data from quadcopters in flight and the potential of integrating this data with Salesforce.

One of the first scenarios that I explored was one that is topical here in New Zealand.

New Zealand has no native terrestrial mammals (except for bats and seals).
[Biodiversity of New Zealand and DOC Native animals]

If something is running around in the bush and it isn't an insect or bird (or a very confused native bat or seal) then in is an introduced species.

A number of introduced species, such as rats, possums, and mustelids survive by predation of native species. And if the they aren't directly eating the natives they are competing for the same food and resources. Possums and feral cats can also spread diseases. All in all they are unwelcome visitors in the local ecosystem. Monitoring and trapping programs are used by various conservation groups to help control the spread and population of introduced species.

A large number of traps and monitoring stations are deployed out in the wild and on farms in locations where it can be time consuming to check them; either because of their remoteness and distribution over many hectares and/or because of their numerousness. Staff or volunteers need to physically visit each site regularly to check if the trap needs any attention. The labour involved in checking and maintaining the traps can be a significant percentage of the overall cost [Source] and a limiting factor in how many traps can be deployed.

Decline (attenuation) in radio signal strength through
forest with increasing distance from a transmitter of
four frequencies compared to transmission through
free space (i.e. with no vegetation). [Source]

There are existing solutions that utilize a sensor and wireless connection on each trap. These sensors rely of having either an internet, cellular, or satellite link to communicate the trap status back. Transmitting through dense forest also reduces radio signal strength.

Which brings me back to Salesforce and UAVs. Can I cut the cost of the equipment deployed with each trap by keeping the sensor and wireless link very basic and then relying on UAV flights in the area to collect the data periodically? The end goal is to allow trap checkers to focus their attention where they will be most productive, and to expand the area of operation.

High level plan:

  • Put a small short range transmitter on each trap that sends out a periodic signal when the trap needs attention. Ideally the sensor on each trap will be very basic and will be able to transmit for a sufficient period of time once triggered. The transmission range should be able to reach a height above the canopy where the UAV can pass by.
  • Record the geolocation where each trap is deployed using a record in Salesforce.
  • Periodically dispatch a UAV to fly a circuit of the traps in the area. The paths for these flights can be determined from the geolocation data in Salesforce.
  • The UAV will carry the receiver and small computer (Raspberry Pi, Ardunio or similar) to capture the signal data plus additional telemetry (GPS location at time of signal).
  • When the UAV has an internet connection relay the collected data back into Salesforce.
  • Use the collected data to notify which traps need attention.
  • Run analytics over the gathered data to identify gains, such as finding areas that would benefit from having more traps deployed.

There are a lot of moving parts here (some more literally than others). So before I get too far ahead of myself we'll start with some of the basics. I'll cover off all the parts in a number of subsequent blogs posts as there is lots to cover.

Trap tracking in Salesforce

A good place to start will be how the trap records are stored in Salesforce. In the simplest case this can just be a custom record with fields to include the applicable details, such as when and where the trap was deployed. A geolocation Compound field is particularly useful for the latter part as it brings native support for calculations of distances around a latitude and longitude pair.

I'll take a slight detour here from the immediate scenario above to explore something similar but still cover a number of important points. Another introduced pest species here in Nelson is the Great white butterfly. The key difference here is that much of the trapping for the great white butterfly is occurring in a suburban environment around residential addresses. This allows the use of the new automatic geocoding for addresses that became available in Summer '16.

Before the auto geocoding will occur you need to review and activate the clean rules. I activated them for Leads as a starting point.
Setup > Administer > Administration > Clean > Clean Rules

I also found it useful to add additional formula fields for the geolocation fields (latitude, longitude and accuracy) as they can't otherwise be directly exposed on the page layout.

Now with just the street address details for the properties of interest entered against Leads in Salesforce a SOQL query can be used to find the points I need to fly to within my operational area. Using the GEOLOCATION function to define the takeoff point and the DISTANCE function to search for sites of interest within the operating area.

SELECT Id, LastName, Latitude, Longitude
WHERE DISTANCE(Address, GEOLOCATION(-41.264268, 173.291987), 'KM') < 2

Unfortunately the native Visualforce mapping controls aren't available in developer edition orgs. Instead I'll export the locations of interest in the Keyhole Markup Language (KML) used by Google Earth. I was going to use the Google Earth API to embed it in a web page, but it was deprecated by Google (Boo!).

A Visualforce page can be used to generate the KML file from the SOQL query. This is a minimal initial version. I'll parameterize the origin location for the search and likely use it as the starting point for the flight.

Visualforce to generate KML

Controller to generate KML

That's sufficient to export the KML file into Google Earth for the points of interest.

Visualforce for Google Maps loading KML

An additional Visualforce page can be setup to embed a Google Map and directly load the KML file in. I've kept the Google API key in a custom setting. The KML file needed to be publicly accessible to the Google Mapping servers, so I set up Sites to expose it and allowed the Public Access Settings profile access to Leads and the Geolocation fields. It appears Google was caching the KML content. Adding a random query string was sufficent to get it updating.

There is still lots to explore here, with the next pressing part being finding a route between all the sites that need to be flown. If you've ever done any computer algorithms or AI course you'll know this as the Travelling Salesman Problem - how to find the optimal (or close to optimal) path that visits every node.

I'll come back to that shortly, as it is way too interesting not to try something like a genetic algorithm or nearest neighbor algorithm in Apex to look for some solutions.

In the meantime...


There are laws and regulations on where and when you can fly a UAV, quadcopter, kite, helium balloon on a really long string, etc...
You will need to educate yourself about the laws and regulations that may be applicable in your country, state, province or locality.

Within New Zealand the Airshare website is a good starting point for the rules defined by the Civil Aviation Authority.