Pages

Friday, September 22, 2017

Teaching an Old Log Parser New Tricks - FIT sfdx CLI plugin

I've currently got two problems with the prototype FitDX Apex debug log parser I released a couple of weeks ago.

  1. It comes from the FuseIT SFDC Explorer source and is excessively large for what it does. It's got a lot of baggage for features it doesn't currently expose.
  2. To be a native SFDX CLI plugin I need to be working from node.js.

Enter Edge.js and it's quote from "What problems does Edge.js solve?"

Ah, whatever problem you have. If you have this problem, this solves it.

--Scott Hanselman (@shanselman)

As promised, it does what it says on the box and helps solve both problems by connecting the .NET world with node.js - in process.

In my case I've got a .NET library that has many years of work put into it to parse Apex debug logs from a raw string. It's also still actively developed and utilized in the FuseIT SFDC Explorer UI. Between that and my lack of experience with node.js I wasn't in a hurry to rewrite it all so it could be used as a native SFDX CLI plugin. While I still needed to reduce the .NET DLL down to just the required code for log parsing I don't need to rewrite it all in JavaScript. Bonus!

This seems like a good place for a quick acknowledgement to @Amorelandra for pointing me in the right direction with node.js. Thanks again!

With the edge module the node.js code becomes the plumbing to pass off input from the heroku plugins to my happy place (the .NET code). And it is all transparent to the user (assuming the OS specific edge.js requirements are meet). A big bonus being a plugin is that it gives me a prebuilt framework for gathering all the inputs in a consistent way.

I used Wade Wegner's SFDX plugin as a boiler plate for integrating with the cli, chopped most of it out, and kept the command to pull the latest debug log from an org. This debug log is then feed directly into the .NET library to use the existing FuseIT parser.

Enough patting myself of the back for getting this all working. How does it work in practice?

Summary

Like with fitdx, this will give you a count for each event type that appears in the log.

>sfdx fit:apex:log:latest -u <targetusername> --summary
41.0 : 41.0 APEX_CODE,DEBUG;APEX_PROFILING,NONE;CALLOUT,NONE;DB,INFO;SYSTEM,INFO;VALIDATION,INFO;VISUALFORCE,NONE;WAVE,INFO;WORKFLOW,INFO
INFO;WORKFLOW,INFO
CODE_UNIT_FINISHED : 2
CODE_UNIT_STARTED : 11
CUMULATIVE_LIMIT_USAGE : 18
CUMULATIVE_LIMIT_USAGE_END : 18
DML_BEGIN : 22
DML_END : 22
ENTERING_MANAGED_PKG : 768
LIMIT_USAGE_FOR_NS : 54
SOQL_EXECUTE_BEGIN : 25
SOQL_EXECUTE_END : 25
USER_DEBUG : 2
USER_INFO : 1

768 ENTERING_MANAGED_PKG events! Story of my life.

Debug only

>sfdx fit:apex:log:latest -u  --debugOnly
28.0 APEX_CODE,DEBUG;APEX_PROFILING,INFO;CALLOUT,INFO;DB,INFO;SYSTEM,DEBUG;VALIDATION,INFO;VISUALFORCE,INFO;WAVE,INFO;WORKFLOW,INFO
18:56:59.180 (1181147014)|USER_DEBUG|[4]|ERROR|UserInfo.getSessionId(): 00D700000000001!ApexTestSession
18:56:59.416 (1416651819)|USER_DEBUG|[4]|ERROR|UserInfo.getSessionId(): 00D700000000001!ApexTestSession

Filtering

>sfdx fit:apex:log:latest -u  --filter USER_INFO,CODE_UNIT_STARTED
28.0 APEX_CODE,DEBUG;APEX_PROFILING,INFO;CALLOUT,INFO;DB,INFO;SYSTEM,DEBUG;VALIDATION,INFO;VISUALFORCE,INFO;WAVE,INFO;WORKFLOW,INFO
18:56:58.400 (4013418)|USER_INFO|[EXTERNAL]|005700000000001|daniel.ballinger@example.com|Pacific Standard Time|GMT-07:00
18:56:58.979 (979316201)|CODE_UNIT_STARTED|[EXTERNAL]|01q70000000HFV6|DFB.TestAccountBeforeUpdate1 on Account trigger event BeforeInsert for [new]
18:56:58.982 (982430091)|CODE_UNIT_STARTED|[EXTERNAL]|01q70000000HFVB|DFB.TestAccountBeforeUpdate2 on Account trigger event BeforeInsert for [new]
18:56:58.985 (985467868)|CODE_UNIT_STARTED|[EXTERNAL]|01q70000000blnO|DFB.AccountTrigger on Account trigger event BeforeInsert for [new]
18:56:59.108 (1108633716)|CODE_UNIT_STARTED|[EXTERNAL]|01q70000000Ti5b|DFB.ToolingTest on Account trigger event BeforeUpdate for [0010g00001YxxQ5]
18:56:59.180 (1180950117)|CODE_UNIT_STARTED|[EXTERNAL]|01q70000000H1vs|DFB.RestrictContactByName on Contact trigger event BeforeInsert for [new]

Format Shifting

>sfdx fit:apex:log:latest -u  --json
>sfdx fit:apex:log:latest -u  --csv
>sfdx fit:apex:log:latest -u  --human

The first two should still be fairly self explanatory and the latter is still intended to make reading the log easier for those that run on digesting food rather than electricity.

Installation

Most of the steps are covered in the Github repo. I found the most challenging part here to be piggybacking off the existing install of node.js and npm that lives under sfdx.

Your results may vary. It is likely safer to install your own instance of node.js. The standard disclaimer applies.

One more thing

Friday, September 8, 2017

Back to the command line - FitDx Apex debug log parser

Love it or loathe it, there are times in a developers day where you are going to find yourself at the command line. I'll leave the GUI vs command line debate to others to sort out.

My general position is they both have their strengths and weaknesses and we as developers should try and use the right tool for the job. Very much like clicks and code with Salesforce. Use them together where possible to get the most done in the least amount of effort.

Tooling shouldn't be a zero sum game.

Debug logs

Salesforce Debug logs have been a pet project area for me for some time now E.g. 2011, 2014, 2017. They form such a fundamental part of the development process but don't get a lot of love. Let me paint you a picture for a typical day in development ...

Earlier in the day something broke badly in production and you’ve only just been provided a raw 2MB text version of the debug log to diagnose the problem from. What do you do?

You can’t use any of the tooling that the Developer Console provides for log inspection on a text file. There is no way of opening it. So there will be no stack tree or execution overview to help you process a file approximately the same size as a 300 page ebook. I say approximately here because there are so many factors that could affect the size in bytes for a book. The general point is that a 2MB text file contains a lot of information that you aren't going to read in a few minutes.

The interactive debugger isn't any help for a production fault and you don't even know the steps to reproduce the problem let alone if the same thing would happen in a sandbox.

You could start combing the log file with a variety of text processing tools, or, or...

It's normally about this point that I'd direct you to the FuseIT SFDC Explorer debug log tooling which gives you:

  • (somewhat retro) GUI tooling for grouping and filtering events by category.
  • Helps highlight important events such as fatal errors and failed validations.
  • And, my current favorite, gives you a timeline view to make more sense of execution time measured in nanoseconds.

But I'm not going to cover any of that in this post. Instead we're going to cover something more experimental than the software that's been in perpetual beta for who knows how many years.

Fit DX

The FuseIT DX CLI, or FitDx in usage, is a proof of concept that I could take the debug log parser from the explorer product mentioned above and apply it directly to debug logs at the command line. I'm just putting it out there to see if there is any interest in such a thing.

You can get it right now as part of the zip download for the FuseIT SFDC Explorer.

There are certainly areas for improvement. First and foremost is the executable size. It's swollen up a fair bit from features that it doesn't expose. I'll look at whipping those out in a future release and should result in a significantly smaller package.

But enough about how big FitDx is. What's more important is how you use it.

Summary Command

If you ask sfdx for a debug log, that's exactly what you'll get. The complete, raw, unabridged debug log dumped to the command line. An experienced command line person would at this stage type out some grep regex wizardry to just show them what they wanted to see. Such is the power of the command line, but it isn't always clear where to start.

I wanted something simpler that would give you a very quick overview of what is in the log.

>sfdx force:apex:log:get -i 07L7000004G0U8kEAF -u DeveloperOrg | fitdx --summary

28.0 : 28.0 APEX_CODE,FINE;APEX_PROFILING,FINEST;CALLOUT,ERROR;DB,FINEST;SYSTEM,FINE;VALIDATION,DEBUG;VISUALFORCE,FINER;WORKFLOW,INFO
CODE_UNIT_FINISHED : 14
CODE_UNIT_STARTED : 14
CONSTRUCTOR_ENTRY : 14
CONSTRUCTOR_EXIT : 14
CUMULATIVE_LIMIT_USAGE : 14
CUMULATIVE_LIMIT_USAGE_END : 14
CUMULATIVE_PROFILING : 4
CUMULATIVE_PROFILING_BEGIN : 1
CUMULATIVE_PROFILING_END : 1
ENTERING_MANAGED_PKG : 184
EXCEPTION_THROWN : 2
EXECUTION_FINISHED : 14
EXECUTION_STARTED : 14
FATAL_ERROR : 4
LIMIT_USAGE_FOR_NS : 14
METHOD_ENTRY : 73
METHOD_EXIT : 73
SYSTEM_CONSTRUCTOR_ENTRY : 33
SYSTEM_CONSTRUCTOR_EXIT : 33
SYSTEM_METHOD_ENTRY : 165
SYSTEM_METHOD_EXIT : 165
TOTAL_EMAIL_RECIPIENTS_QUEUED : 14
USER_INFO : 14

The summary will currently give you a count for the events that are covered in the piped input. In this case I can see that there were 4 FATAL_ERROR events

Filtering

Now that I know I what I'm looking for I want to see it just those events of interest. The --filter command accepts a comma separated list of event types that you want to see. Everything else gets dropped. There is also the --debugOnly option which is a preset to filter for USER_DEBUG only.

>sfdx force:apex:log:get -i 07L7000004G0U8kEAF -u DeveloperOrg | fitdx --filter FATAL_ERROR
28.0 APEX_CODE,FINE;APEX_PROFILING,FINEST;CALLOUT,ERROR;DB,FINEST;SYSTEM,FINE;VALIDATION,DEBUG;VISUALFORCE,FINER;WORKFLOW,INFO
20:25:09.695 (698828931)|FATAL_ERROR|System.AssertException: Assertion Failed: Expected: {"filters":{"footer":{"settings":{"enable":"1","text/plain":"You can haz footers!"}}}}, Actual: {"filters":{"footer":{"settings":{"text/plain":"You can haz footers!","enable":"1"}}}}

Class.DFB.SmtpapiTest.setSetFilters: line 116, column 1
20:25:09.695 (698839161)|FATAL_ERROR|System.AssertException: Assertion Failed: Expected: {"filters":{"footer":{"settings":{"enable":"1","text/plain":"You can haz footers!"}}}}, Actual: {"filters":{"footer":{"settings":{"text/plain":"You can haz footers!","enable":"1"}}}}

Class.DFB.SmtpapiTest.setSetFilters: line 116, column 1
20:25:10.160 (1162726841)|FATAL_ERROR|System.AssertException: Assertion Failed: Expected: {"section":{"set_section_key":"set_section_value","set_section_key_2":"set_section_value_2"}}, Actual: {"section":{"set_section_key_2":"set_section_value_2","set_section_key":"set_section_value"}}

Class.DFB.SmtpapiTest.testAddSection: line 80, column 1
20:25:10.160 (1162743640)|FATAL_ERROR|System.AssertException: Assertion Failed: Expected: {"section":{"set_section_key":"set_section_value","set_section_key_2":"set_section_value_2"}}, Actual: {"section":{"set_section_key_2":"set_section_value_2","set_section_key":"set_section_value"}}

Class.DFB.SmtpapiTest.testAddSection: line 80, column 1

Format Shifting

The final set of alpha features are around changing how the debug log is presented. Options include --json, --csv, and --human. The first two should be fairly self explanatory on what the output will be. The human option was just an idea to use more column like alignment rather than vertical bars (|) to separate the elements.

In hindsight I should probably allow for multiple options at once so you can specify both the required filters and output format in one command. For the time being you can just pipe the filtered output back in.

>sfdx force:apex:log:get -i 07L7000004G0U8kEAF -u DeveloperOrg | fitdx --filter EXCEPTION_THROWN | fitdx --human
28.0                           APEX_CODE,FINE;APEX_PROFILING,FINEST;CALLOUT,ERROR;DB,FINEST;SYSTEM,FINE;VALIDATION,DEBUG;VISUALFORCE,FINER;WORKFLOW,INFO
20:25:09.695    EXCEPTION_THROWN              [116]System.AssertException: Assertion Failed: Expected: {"filters":{"footer":{"settings":{"enable":"1","text/plain":"You can haz footers!"}}}}, Actual: {"filters":{"footer":{"settings":{"text/plain":"You can haz footers!","enable":"1"}}}}
20:25:10.160    EXCEPTION_THROWN               [80]System.AssertException: Assertion Failed: Expected: {"section":{"set_section_key":"set_section_value","set_section_key_2":"set_section_value_2"}}, Actual: {"section":{"set_section_key_2":"set_section_value_2","set_section_key":"set_section_value"}}

The Forward Looking Statements

This is the part where I promise you the world with all the cool things I could potentially do in the future. Things like:

  • Direct integration as a SFDX plugin via a node.js to .NET library
  • Expose other features from the FuseIT SFDC Explorer at the command line, such as the alternative version of WSDL2Apex.
  • I've got a crazy idea for extra things like pulling apex class and trigger definitions directly from a Salesforce StackExchange question or answer and then load them directly into a newly created scratch burner org. The process could then also be reversed with the contents of the scratch org created into the required markdown. This could greatly speed up the process of asking and answering questions on the SFSE.

Friday, August 18, 2017

FuseIT SFDC Explorer 3.6.17184.1 and 3.7.17230.1 - Summer '17

The latest v3.6 release of the FuseIT SFDC Explorer is out and contains a couple of new features around Apex Debug logs.

Aaaaaannnd... because I got distracted before publishing this post, 3.7 is also out. So it's now a twofer release post.

Improvements to log parsing

Carrying on from the previous release, I've made a number of modifications to the debug log viewer.

All the calculations for duration's are now done of the underlying nanosecond ticks. This helps prevent compounding errors due to the lack of definition in C# Timespans.

The improved duration calculation is particularly important for the new grouping functionality. Take the two examples below. With the standard log you can get 200 CODE_UNIT_STARTED events for the validation of each record in a transaction (plus the corresponding 200 CODE_UNIT_FINISHED events). With the new "Group Recurring Events" button in the tool bar any repeating events of the same type will be collapsed into a single node with a summary.

Before

After

While still on the topic of the Apex Log Tree View, DML operations now have a basic icons to indicate which variant of Insert, Update, Upsert, or Delete operation they are. Trigger, Validation, and Workflow have consistent color coding between the Tree View and the Timeline.

Auto build a Metadata package

There is a new option on the Metadata Deploy tab under "Deploy Selected Files" called "Compare File Hashes". As per the name, it will do a CRC32 diff of the body of Apex classes, triggers and visualforce pages between the local file system and the connected Salesforce org. If anything is different it gets added to the current package ready for deployment.

If it turns out the local file is the one that should be updated there is a context menu option to update the local file.

Colored logging

Both the Metadata Deployment results and test method error results now include some basic color.

Salesforce DX CLI integration

If you have the Salesforce DX CLI installed (and in the path) you can use it to manage org logins. It might take a moment after pressing "Refresh Orgs" for the list to come back. Once it does, just select an org and press login.

Other changes 3.6

  • Log Tree view. Color code Trigger, Validation, and Workflow events. Open debug log from File. Toggle Timeline and Treeview display.
  • Highlight FLOW_ELEMENT_ERROR messages in the log.
  • Handle log highlighting for Fine, Finer, and Finest logging levels. Provide links for ids for the active org.
  • Optionally display scale on the log. Highlight VALIDATION_FAIL events.
  • Highlight Exception and skipped/maxlog events in the Log Timeline.
  • When parsing a log, identify the log levels applied (if present) in the footer.
  • Attempt to determine if an ApexClass contains tests. Use this to drive direct running tests cases from the class.
  • Improve performance of building the EntityTree from sObject describe results. Especially for orgs with a large number of custom objects and settings.

Other changes 3.7

  • Add optional allowExistingSObjectsWithoutId="true" to the binding configuration element to allow sObjects to be created with a null Id. Typically this isn't allowed as the ID is used to control insert/update operations and to identify relationship types. This setting can be used for more basic SOQL queries where the results won't be subsequently used for DML.
  • Highlight CALLOUT_REQUESTs in the timeline as they often take up large chunks of time.
  • Fix for updating available metadata types with Session change.
  • Improve SessionId validation pre login
  • When exporting results, give the user the choice of folder to export to.
  • Handle SOQL Aggregate Results that return only a single value
  • Improve IDEWorkspace integration. Allow for switching of workspaces.
  • Wsdl2Apex: Handle case where the Schema TargetNamespace is missing
  • Perform Metadata Deploy Hash on Apex Pages. Option to refresh local metadata from Server.

Tuesday, June 13, 2017

Importing the Salesforce Summer 17 (v40.0) Tooling and Metadata APIs to .NET

The Salesforce Summer '17 release includes the return of an old friend in both the Tooling and Metadata APIs. After refreshing to the latest WSDLs and making sure the project builds it starts producing errors like the following with calling the API:

error CS0029: Cannot implicitly convert type 'XYZ.SalesforceConnector.SalesforceTooling.NavigationMenuItem' to 'XYZ.SalesforceConnector.SalesforceTooling.NavigationMenuItem[]'

I say it's an old friend, because I've seen this before with the Partner API when it transitioned to Winter '15 (v32.0) for the QuickActionLayoutItem. It's also cropped up with Winter '13 (v29.0). Thankfully the same fix that was applied in those cases will also work here.

The problem is the navigationLinkSet property that gets generated with a multidimensional array return type. The type for the corresponding XmlArrayItemAttribute is incorrect.

Before Fix

/// 
[System.Xml.Serialization.XmlArrayItemAttribute("navigationMenuItem", typeof(NavigationMenuItem), IsNullable=false)]
public NavigationMenuItem[][] navigationLinkSet {
    get {
        return this.navigationLinkSetField;
    }
    set {
        this.navigationLinkSetField = value;
    }
}

After Fix

/// 
[System.Xml.Serialization.XmlArrayItemAttribute("navigationMenuItem", typeof(NavigationMenuItem[]), IsNullable=false)]
public NavigationMenuItem[][] navigationLinkSet {
    get {
        return this.navigationLinkSetField;
    }
    set {
        this.navigationLinkSetField = value;
    }
}

The same fix needs to be applied to both the Tooling API and the Metadata API.

Monday, June 12, 2017

Controlling analog PWM servos from a Raspberry Pi

As part of a project I'm working on I needed a way to generate motion from my Raspberry Pi. It needed to be a precise variable motion rather than the all or nothing type motion of a straight DC motor. I opted to go with PWM (Pulse Width Modulation) based servos as I'm familiar with them and have some on hand for RC planes. Stepper motors or a DC motor with a rotary encoder were other options, but I have neither in my spare parts bin.

So PWM servos it is. Now, how to control them from the Pi? Getting the Pi to feed the hard real-time response requirements of a servo motor directly can be difficult, so I opted instead to offload the task to an Adafruit 16-Channel PWM / Servo HAT.

Some assembly required

Software

Because I'm a .NET person I used Windows IoT to program the Pi. Adafruit even provide the AdafruitClassLibrary for Windows IoT. The rough code, less exception handling.

var i2cSettings = new Windows.Devices.I2c.I2cConnectionSettings(0x40);

var gpio = Windows.Devices.Gpio.GpioController.GetDefault();

Pca9685 hat = new Pca9685(0x40);
await hat.InitPCA9685Async();

hat.SetPWMFrequency(50);

ushort servoMinPulseLength = 150;
ushort servoMaxPulseLength = 600;

hat.SetAllPWM(0, servoMinPulseLength);

           
while (true)
{
    hat.SetAllPWM(0, servoMinPulseLength);

    await System.Threading.Tasks.Task.Delay(500);

    hat.SetAllPWM(0, servoMaxPulseLength);

    await System.Threading.Tasks.Task.Delay(500);
}

The exception handling turned out to be one of the harder parts of getting this to work. I kept getting debug messages like Value cannot be null.:I2C Write Exception: {0}. Not very helpful. Once I found the corresponding Github project and worked directly from the source the problem was easier to isolate. They are catching the majority of the exceptions in their code and dumping them out to the debug log and then carrying on. This was causing all sorts of problems, with the first occuring in the `InitPCA9685Async` call. The ultimate resolution was to up the Microsoft.NETCore.UniversalWindowsPlatform NuGet package to at least 5.2.2.

After that I just needed to set sensible values for the PWMFrequency and minimum and maximum pulse lengths. Most of the demo code I've seen has a default frequency of 1000. That seemed way to high and the servos became a jittery mess. The general consensus for an analog RC servo is somewhere in the 30Hz to 60Hz range. Some good reading on the difference between PPM and PMW.

The Result

All going well I'll have more posts soon to bring this together with the other parts I'm working on.

Friday, April 7, 2017

Steps required to support POSTing multipart/form-data Content-Type from Apex

I came across an interesting challenge when transferring example images to the Einstein Predictive Vision Service (PVS). As part of PVS you can upload example images to train an image classifier. As a gross generalization - here is a picture of a cat. If you get another image like that tell me it is probably a cat.

What follows are the lengths I had to go to in order to be able to call this web service method from Apex.

In their documentation MetaMind were kind enough to provide an example cURL command to submit the example image.

curl -X POST -H "Authorization: Bearer <TOKEN>" -H "Cache-Control: no-cache" 
 -H "Content-Type: multipart/form-data"
 -F "name=77880132.jpg" -F "labelId=614" -F "data=@C:\Mountains vs Beach\Beaches\77880132.jpg" 
 https://api.metamind.io/v1/vision/datasets/57/examples

It always struck me as odd that specifying the API usage via a specific command line tool has somehow become the de facto standard. It's like describing how to use a REST API is so difficult that it is easiest to just rely on the conventions of a known command line tool. Nothing against cURL, but it is hiding some of the mechanics of how that API call is working. In particular with this example is how the -F is composing the multipart/form-data and how the @ in the command will be substituted with the actual file from that path.

Back to uploading example images to PVS. Performing this API call from Apex proves to be a bit of a challenge as there isn't currently direct native support for a multipart form callouts from Apex. Please pause your blog reading at this point and consider voting for the idea - Image upload using multipart/form-data.

My first naive attempt at replicating the cURL call from Apex using the supplied HttpFormBuilder class failed miserably. The binary blob data that I was extracting from an image stored in a ContentVersion record wasn't being accepted by MetaMind.

    public ExampleResponse addExample(string access_token, string datasetId, string labelId, string filename,
        blob exampleImageData) {
        
        string postUrl = CREATE + '/' + datasetId + '/examples';
        
        string contentType = HttpFormBuilder.GetContentType();
        System.assertEquals('multipart/form-data', contentType);
        
        //  Compose the form
        string form64 = '';

        form64 += HttpFormBuilder.WriteBoundary();
        form64 += HttpFormBuilder.WriteBodyParameter('labelId', EncodingUtil.urlEncode(labelId, 'UTF-8'));
        form64 += HttpFormBuilder.WriteBoundary();
        form64 += HttpFormBuilder.WriteBodyParameter('name', EncodingUtil.urlEncode(filename, 'UTF-8'));
        form64 += HttpFormBuilder.WriteBoundary();
        form64 += HttpFormBuilder.WriteBodyParameter('data', EncodingUtil.base64Encode(exampleImageData));
        form64 += HttpFormBuilder.WriteBoundary(HttpFormBuilder.EndingType.CrLf);
        
        blob formBlob = EncodingUtil.base64Decode(form64);
        string contentLength = string.valueOf(formBlob.size());
        //  Compose the http request
        HttpRequest httpRequest = new HttpRequest();

        httpRequest.setBodyAsBlob(formBlob);
        httpRequest.setHeader('Connection', 'keep-alive');
        httpRequest.setHeader('Content-Length', contentLength);
        httpRequest.setHeader('Content-Type', contentType);
        httpRequest.setMethod('POST');
        httpRequest.setTimeout(120000);
        httpRequest.setHeader('Authorization','Bearer ' + access_token);
        httpRequest.setEndpoint(postUrl);

        //...
    }

Where did I go wrong? Well, unlike a prediction there isn't a variant of this call API that accepts the Base64 encoded image. Using HttpFormBuilder.WriteBodyParameter with the Base64 encoded blob to write the data wasn't going to work. MetaMind want the actual bytes and associated Content-Type header. Here is how the working request appears in Postman:

POST /v1/vision/datasets/1001419/examples HTTP/1.1
Host: api.metamind.io
Authorization: Bearer 12346539689dae622cbd91d9d4880b3314bfb747
Cache-Control: no-cache

----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="data"; filename="ItsAFrog.png"
Content-Type: image/png

<bytes of file go here>
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="labelId"

8588
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="name"

itafrog.png
----WebKitFormBoundaryE19zNvXGzXaLvS5C

An alternative method was needed to HttpFormBuilder.WriteBodyParameter. One that would set the correct Content-Disposition and Content-Type headers for the binary data and then also correctly append the bytes from the image. That last point is really important. Correcting the headers wasn't very difficult, but with the binary file data I found the last few bytes were getting corrupted on the file.

Thankfully Enrico Murru had already covered a lot of the details around how to get proper multipart form POSTs working in Apex, so full credit to him for the blog post POST Mutipart/form-data with HttpRequest. This was then later refined by Grant Wickman in an answer to Post multipart without Base64 Encoding the body. The more I researched why HttpFormBuilder wasn't working the more I found MetaMind had based HttpFormBuilder partially on Enrico and Grant's work but hadn't taking it through to completion. For instance, the EndingType enum has the following comment indicating they were clearly thinking about it, just never applied it.

Helper enum indicating how a file's base64 padding was replaced.

To get it working I needed to finish applying Enrico and Grant's solutions.

It's all about the Base(64 encoding)

The smallest unit of storage on a file is a (octet) byte, which is composed of 8 bits. In contrast, Base64 encoding represents 6-bits with each character. So 4 Base64 characters can represent 3 bytes of input data. That ratio of 4:3 is very important with Base64 encoding. If the number of input bytes to be encoded is divisible by 3 then everything is fine as all 4 Base64 characters will represent meaningful input.

The problem occurs if the input byte length isn't divisible by 3. In that case the Base64 encoding process would normally append a padding symbol (=) or two to the end of the encoding to indicate which were the valid bytes in the last 3 and which were padding to get to the correct grouping of 3 bytes to 4 characters.

The following table shows how the padding needs to be applied if there are only two or one bytes to be encoded. You can see how some of the bits in the 6-Bit groupings no longer represent actual data when encoding only one or two input bytes.

Data Bytes In Binary Form 1 1 0 1 0 1 0 0 0 0 1 0 0 1 1 1 1 1 1 1 0 1 1 1
Data Rearranged Into 6-Bit Groups 1 1 0 1 0 1 0 0 0 0 1 0 0 1 1 1 1 1 1 1 0 1 1 1
6-Bit Groups in Decimal Form     5 3           2         3 1         5 5    
Groups Converted to ASCII Characters       1           C           f           3    
Data Bytes In Binary Form 1 1 0 1 0 1 0 0 0 0 1 0 0 1 1 1                
Data Rearranged Into 6-Bit Groups 1 1 0 1 0 1 0 0 0 0 1 0 0 1 1 1 0 0            
6-Bit Groups in Decimal Form     5 3           2         2 8                
Groups Converted to ASCII Characters       1           C           c           =    
Data Bytes In Binary Form 1 1 0 1 0 1 0 0                                
Data Rearranged Into 6-Bit Groups 1 1 0 1 0 1 0 0 0 0 0 0                        
6-Bit Groups in Decimal Form     5 3           0                            
Groups Converted to ASCII Characters       1           A           =           =    

Enrico's method to build up the full multipart form submission from Apex is to first build each part individually using Base64 encoding and then concatenate those parts together. That final Base64 string is then converted back to a Blob to become the body of the HTTP POST request.

The challenge with this method is you can't have intermediate padding characters. The Apex Base64 decoding process is going to ignore those intermediate padding characters. This in turn causes an incorrect mapping between the 4 Base64 characters and the 3 bytes they are supposed to represent. Here is an example using some anonymous Apex. Note how the decoding the concatenated values gives the same result with or without the internal padding:

String encodedA = EncodingUtil.base64Encode(Blob.valueOf('A'));
String encodedB = EncodingUtil.base64Encode(Blob.valueOf('BC'));
System.debug(encodedA);
System.debug(encodedB);
System.debug(EncodingUtil.base64Decode(encodedA).toString());
System.debug(EncodingUtil.base64Decode(encodedB).toString());
System.debug(EncodingUtil.base64Decode(encodedA + encodedB).toString());
System.debug(EncodingUtil.base64Decode('QQ' + 'QkM=').toString());

Output

20:17:50.65 (69059707)|USER_DEBUG|[3]|DEBUG|QQ==
20:17:50.65 (69127131)|USER_DEBUG|[4]|DEBUG|QkM=
20:17:50.65 (69373165)|USER_DEBUG|[5]|DEBUG|A
20:17:50.65 (69500491)|USER_DEBUG|[6]|DEBUG|BC
20:17:50.65 (69636544)|USER_DEBUG|[7]|DEBUG|A$
20:17:50.65 (69759126)|USER_DEBUG|[8]|DEBUG|A$

For encoding text this problem is bypassed by appending additional whitespace characters to the base text until the Base64 encoded representation no longer requires padding. Without the padding it is then possible to concatenate the two base 64 encoded values and then later decode the complete string back to the Blob sans additional bytes. This however doesn't work with a binary file like an image. You can't just go on appending additional bytes on the end without corrupting the file.

Instead the solution presented by Grant Wickman is to borrow one or both of the CR (\r) or LF (\n) characters that separate the end of the file bytes from the multipart boundary.

By shuffling these legitimate characters onto the end of the files bytes the need for padding can be removed. The padding on the boundary can then be adjusted as required using the white-space technique where it won't affect the integrity of the data.

There is a certain elegance to it as a solution. Note how the first 4 bits of the CR align with the last 6 bits of the second Base64 character so they don't need to be changed. And then again, if only switching out one padding = for a LF the first two bits again align.

I find that table very satisfying. I think I might print it out and keep it at my desk like one of those inspirational posters.

Example of additional HttpFormBuilder method

What did we learn?

  1. That rogue = padding characters in the middle of a Base64 string will corrupt the data.
  2. If the Base64 encoding for the file ends with "==", crop those characters off and replace with 0K to represent a CRLF. Then don't prepend the following boundary footer with those characters.
  3. If the Base64 encoding for the file ends with "=", crop that character off and replace with N to represent a CR. Then only prepend the following boundary footer a LF.
  4. That @muenzpraeger has a more refined Apex project for working with the Einstein Predictive Vision Service. I've submitted my changes there.
  5. The O'Reilly Book covers featuring pictures of animals are done in a woodcut/hedcut style.
  6. Base64 encoding examples can be inspirational and look great at your desk.

Friday, March 24, 2017

The Mad Catter - Salesforce Predictive Vision Services

Disclaimer

No animals were harmed in the creation of this blog post or the associated presentation. The standard disclaimer applies.

I've got a problem. A cat problem to be precise. While I'm more of a dog person, I don't particularly mind cats. However, recently there has been a bit of an influx of them with the neighbors. There are at least a half dozen of them that roam freely around the neighborhood. Not the end of the world, but they have a nasty habit of leaving presents on the lawn for the kids to find. Things like the following:

In short, they poop everywhere and leave the remains of birds lying around. Things that aren't so great to step on or for the kids to find when playing in the garden.

I spoke with the immediate neighbor who owns two of the cats, and he suggested spraying them with water to deter them away. While that did indeed prove to be a very effective, amusing, and satisfying approach to move them on it required 24 hour vigilance as they just kept coming back.

Get a Dog

My wife keeps saying we should just get a dog. I guess getting a dog to chase the cats away is an option. But it seems like training a dog to use the hose might be more trouble in the long run.

Technology to the Rescue

Thankfully I found a handy device the attaches to the end of the house and activates a sprinkler head when a built in motion detector is set off.

Perfect! Great! Cat comes into range, then a sudden noise and spray of water sends them off towards someone else's lawn to do their cat business. Problem solved and I can go back to doing more fun activities.

Except there was one small problem. The PIR motion sensor didn't particularly care what was moving in front of it. Cats, birds, the kids on their way to school, a courier with a parcel for me, a tree in the wind, the mother in law. It would spray them all regardless of whether I wanted it to or not.

Salesforce Predictive Vision Service

Technology wasn't solving my problem. I needed to use more of it!

I recalled a recent presentation by the Salesforce Developers team - Build Smarter Apps with New Predictive Vision Service. The short version of that presentation is you can train a deep learning image classifier with a set of training images. Then when you give it a new image it will give you probabilities about what is likely in the picture. I've create a quick start unmanaged package for it to save you going through most of the install steps.

To make this work I needed a large collection of cat images to train the dataset from a create a model. Luckily for me, providing pictures of cats is something that the internet excels at.

The second challenge with using the predictive vision services is managing how many images I am going to send through to the service. If I just point a web camera out the window it could be capturing 30+ frames per second. Not really practical to send off each frame to the service for identification when there might be nothing of interest happening 99% of the time.

Motion detection

I had a few options here.

Option One would to to stick with the basic PIR motion sensor, but it would still be generating a ton of false positives that would need to pass through the image recognition. A simple cool down timer would help, but the image captured immediately after the first motion is detected would likely get something as it is just entering the frame.

I figure since I'm going to need a camera to capture the initial image I might as well get some usage out of it in detecting the motion. Because of the initial processing step I can exclude motion from certain areas, such as a driveway or tree that often moves in the wind. There can also be a slight delay after the motion is detected and before the prediction image is captured. This gives the subject time to move into the image.

The prototype solution looks like this:

  1. A webcam that can be pointed at the area to be monitored.
  2. Motion Detection software to process the video feed and determine where the movement is and the magnitude. The magnitude is useful, as particularly small subjects like birds can be completely ignored.
  3. The ability for that software to pass frames of interest off to the Salesforce Predictive Vision Service. This is a simple REST POST request using an access token.
  4. If the probability from the frame indicates a Cat is present, send a signal to the Raspberry Pi.
  5. On the signal, the Raspberry Pi activates the GPIO pin connected to a relay.
  6. When activated, the relay supplies power to the existing automated sprinkler, which activates on initial power on when the sensitivity is set to maximum. Another option here is directly connecting a solenoid water value to the hose line.

When all put together the end result looks something like this:

The Einstein bust with terminator-esque glowing red eyes was part of the presentation I gave on this topic.

While filming that video I inadvertently live tested it on myself as well. An aging fitting on the hose connector to the sprinkler had come loose outside at the tap. So I went out to fix that, restored the water pressure to the sprinkler, then walked back to the laptop to resume the test. Only when I checked the motion detection screen did I realize it had captured my image passing in front of the sprinkler. Thankfully the predictive vision services came back indicating I didn't resemble a cat and the sprinkler didn't activate. Success!

Refinements

It occured to me that there were further improvements that could be made.

The first and easiest change I made is to activate on things other than cats. It can be equally selective to activate on a wandering neighbors dog, squirrels, general wildlife, etc...

I needed a way to deal with unfortunate false positives, such as a person wearing something with a picture of a cat on it. These can partially be avoided by looking at all the probabilities that Einstein is returning and having thresholds against each label. I.e. Activate on any Cat prediction above 50% unless there is any prediction indicating a person in the field of view. Images kept from the activations could also be used to further refine the examples in the dataset.

These first two refinements are actually present in the video above. When using the general image classifier it typically identifies the stuffed cat as a teddy bear. So in the small section to the bottom right of the app you can mark labels to activate on and labels that will override and prevent the activation.

Other changes I might consider making:

The motion sensor could be maintained and introduced as the step prior to activating the video feed. This would increase the time between the target entering the area and the sprinkler activating, but would save a lot of endless processing loops looking at an unchanging image.

If I forgo some of the more processing intensive motion tracking the whole solution could be moved onto the Raspberry Pi. This would make it a much more economical solution.

However, another option with the motion detection still in place would be to crop the frame image to just the area where the motion was detected. This should lead to much higher prediction accuracy as it is only focusing on the moving subject.

When real world testing commences with live subjects I'll need to add a video capture option to cover the time from just before the sprinkler is activated till just after it switches off. I think the results will be worth the extra effort.

I have a range of other devices that could easily be activated via the relay attached to the Raspberry Pi. One such device is an ultrasonic pest repeller. Perhaps combined with a temperature sensor as a slightly kinder deterrent on cold nights.

User Group Presentation

I gave a talk to the Sydney developer user group on this project. The slides, as they were:


I still feel the need to settle on a name for the project. Options include:

  • The Mad Catter (after the elusive Catter Trailhead badge)
  • The Einstein Cannon
  • The Cattinator (After the general them of the presentation.)

See also:

Gallery

Thursday, March 2, 2017

Salesforce SOAP Callout debugging trickery

Here's a handy practice when making SOAP callouts from Salesforce and handling errors.

When a Callout goes pear-shaped and you get an exception, keep track of the request parameters by doing a JSON serialize and keeping the result in a custom object.

Then in the dev packaging org you can rehydrate the same request by deserializing the JSON from the custom object and making the same callout. Because you are now in a dev org you can see the raw SOAP message in the CALLOUT_REQUEST logging.

string jsonData = [Select ReferenceData__c from Error_Details__c where ID = 'a084000000w6ReO'].ReferenceData__c;

SoapWebService.Order order = (SoapWebService.Order)JSON.deserialize(jsonData, SoapWebService.Order.class);

SoapWebService.ServiceCredential credential = new SoapWebService.ServiceCredential();

SoapWebService.BasicHttpBinding_IConnectNS service = new SoapWebService.BasicHttpBinding_IConnectNS();
service.UpdateOrder(credential, order);

From there you can take the raw SOAP request over to something like SOAP UI to debug it further.

See also:

Friday, February 10, 2017

Visualforce Quick Start with the Salesforce Predictive Vision Services

Salesforce recently released the Salesforce Predictive Vision Services Pilot. You can watch the corresponding webinar.

I went through the Apex Quick Start steps and thought I could simplify them a bit. At the end of the process you should have the basic Apex and a visualforce page to test image predictions against.

Steps

  1. Sign up for a predictive services account using a Developer Org. The instructions here are fairly straight forward.
    Go to https://metamind.io/ and use the Free Sign Up link. OAuth to your dev org. Download the resulting predictive_services.pem file that contains your private key and make note of the "you've signed up with" email address. You will need the file later and the email address if your org users email address differs.
    • Note: the signup is associated with the Users Email address, not the username. So you might get conflicts between multiple dev orgs sharing the same email address.
  2. Upload your predictive_services.pem private key to the same developer org into Files and title it 'predictive_services'. This title is used to get the details of the private key by the Apex Code.
  3. Install the unmanaged package that I created (Requires Spring '17).
    I've pulled the required parts together from https://github.com/salesforceidentity/jwt and https://github.com/MetaMind/apex-utils. I've also made some modifications to the Visualforce page and corresponding controller to give more flexibility defining the image URL.
  4. Browse to the PredictService Visualforce Tab.
  5. Press the [Vision From Url] button.
  6. Examine the predictions against the General Image Model Class List.
  7. Change the Image URL to something publicly accessible and repeat the previous couple of steps as required.