Salesforce Glossary - An unofficial list of Salesforce terms and acronyms

This list is by no means exhaustive. It's more a random collection to terms and acryoums that get thrown around with Salesforce work that might not always be readily apparent. There is an official Glossary provided by Salesforce.

Where terms may have alternative meanings I've focused on the Salesforce usage.

I've been slowly compiling this over time with the intention of publishing it when it reaches some addiquate level of completness. However, it has become apparent that this will be more of a living blog post that I'll update over time. Please let me know if there are gaps or mistakes that need to be corrected.

A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z

A

AE

"Account Executive" - Sales point of contact at Salesforce that handles the client relationship. Sells product licenses.

ACV

Annualized Contractual Value

Apex

Apex is a strongly typed, object-oriented programming language that allows developers to execute flow and transaction control statements on Salesforce servers in conjunction with calls to the API. Using syntax that looks like Java and acts like database stored procedures, Apex enables developers to add business logic to most system events, including button clicks, related record updates, and Visualforce pages. Apex code can be initiated by Web service requests and from triggers on objects.

Source

Aura

Aura Components were the predecessor to Lightning Web Components before more of the implementation stack was supported by web standard. See Introducing Lightning Web Components

AmpScript

The proprietary scripting language for Marketing Cloud

Aloha Apps

Older terminology for App Exchange apps that didn't count towards your orgs limits. Source

APAC

Asia Pacific & Australia regions

AMER

US, Canada & Latin America regions

B

Bulkified / Bulkification

Code and processes that are designed to work on multiple records at once rather than one at a time (I.e. in bulk). This is particularly relavant to SOQL queries and DML operations where doing operations on records individually may lead to limits being hit.

Black Tab

An internal tab available to Salesforce employees and support that can toggle specific features and sdjust limits in customer orgs. See What is the Black Tab?

C

CMDT

Custom Metadata Type

CPQ

(Configure, Price, Quote) - formerly Steelbrick. This has now been rebranded Revenue Cloud.

CFP

Call for Presenters

COA

Partner Channel Order App - "enables you to create, submit, and track orders with Salesforce"

CLI

[SFDX] Command Line Interface

Comfortable Shoes

The battle cry of attending an inperson Dreamforce event where you will be doing a lot of walking.

Code Coverage

"Code coverage indicates how many executable lines of code in your classes and triggers have been exercised by test methods." - source

CIM

Cloud Information Model

Community Clould

Rebranded Experience Cloud

D

Daryl Shaber

"We Are Daryl Shaber" - Reference

Deployment Fish

The result of a bug in the changeset deployment monitoring UI that would stretch the circular control into veriaous shapes. Often fish like in nature.

DML

Data Manipulation Language. The Create, Update, and Delete parts of the database CRUD operations. Retrieve is handled via SOQL queries.

Dwarf

The occasional Oracle error code that leaks out of Salesforce will include references to one of the seven dwarfs. See Salesforce, Oracle and the Seven Dwarfs.

D&B

"Dun & Bradstreet" - D&B Company Fields

Dreamforce

Annual Salesforce user conference.

DRAES

Designing Record Access for Enterprise Scale. Reference

E

ESAPI

Enterprise Security API

Evergreen

Internal project code name for Salesforce Functions

Environment Hub

"allows partners to view, connect, create, and log in to multiple Salesforce organizations from one location" - Reference

EMEA

Europe, Middle East & Africa regions

F

[Salesforce] Functions

Previously known as Evergreen. The ability to write code that integrates with Salesforce data and events in a serverless environment.

Falcon

Salesforce CLI plugin built for AppExchange Partners Reference

Field Service Lightning

Rebranded Field Service

G

GACK

An internal Salesforce error. What Is A Gack?

GA

Generally Available - The point where a new Salesforce feature has passed the preview and beta phases and is ready for use.

GUS

Grand Unified System - Internal Salesforce tool that handles tickets, user stories, bugs, builds and releases. Meet GUS: The Grand Unified System That Keeps Salesforce Agile

GS0

Pre-release org instance for testing before a release reaches sandboxes.

H

Happy Soup

A phrase coined by Wade Wegner at TrailheadDX 2017 to describe how metadata was deployed to a production org without any strongly defined boundaries between applications/components.

Hyperforce

The capability to deploy Salesforce on major public cloud platforms like Azure, AWS, Google, and Alibaba. This helps address data residency and complicance requirements. Reference
Introducing Hyperforce - General Information & FAQ/dd>

I

IaaS

Infrastructure as a Service

ID

A Salesforce record identifier. Either 15 or 18 characters long - What are Salesforce ID's composed of?

ISV

Independent software vendor

J

Jorje

Internal codename for the Apex Language Server

K

Keyprefix

The first 3 characters of a Salesforce ID. These can be used to determine the sObject type.

L

LMA

License Management App - Used by ISV Partners to manage applications

LWC

Lightning Web Component

LWR

Lightning Web Runtime

Limits

"Because Apex runs in a multitenant environment, the Apex runtime engine strictly enforces limits so that runaway Apex code or processes don’t monopolize shared resources."

Source

M

Metadata

"Metadata are the assets in your org (assets being objects, tabs, classes, etc). These metadata assets are also called metadata components."

Source

Marketing Cloud

"Salesforce Marketing Cloud is a provider of digital marketing automation and analytics software and services."

Source

MSA

Master Subscription Agreement - Reference

N

O

OWD

Organization-Wide Defaults

OCLIF

Open CLI Framework

OLI

OpportunityLineItem / Opportunity Product

Org62

The Salesforce Org that they use internally to run Salesforce.

P

PaaS

Platform as a Service

Pod

A collection of Salesforce servers that is required to run the complete application. A.K.A Instance

PAM

Partner Account Manager

Polymorphic Lookup

A releationship field that can point to more than one entity.

PBO

Partner Business Org - a production Salesforce enviroment for Salesforce Partners - Docs

PDO

Product Development Outsourcers

Q

Quiddity

PHILOSOPHY "the inherent nature or essence of someone or something."

Exposed in Apex to represent the type of a transaction. Docs

R

S

SaaS

Software as a Service

Safe Harbor

The precursor to the forward-looking statement

S2S

Salesforce to Salesforce

SFDC

Salesforce Dot Com

SFDX

Salesforce DX (Developer Experience)

SOQL

Salesforce Object Query Language. A varient of SQL specifically for Salesforce that is used to query Salesforce data. Reference

SLDS

Salesforce Lightning Design System https://lightningdesignsystem.com/

Brand and product design guidelines, Design Patterns and Component Blueprints for creating unified UI in the Salesforce ecosystem.

SFDO

Salesforce.org

sControl

Deprecated customization control for Salesforce that was superseded by Visualforce pages. Docs

SObject

An abstract base class for Salesforce record types that are stored in the database. Docs

Stealbrick / CPQ

Stealbrick Quote Quickly (SBQQ)

Source of truth

The data source that contains the authoriative data about a whole object.

SI

Systems Integrators

Security Token

A case-sensitive alhanumberic key that is appended to the users password when authenticating against the Salesforce API. Docs

Site Switching

Each Salesforce instance is built and maintained in two geographically separate locations. An instance is actively served from one location (the active site) with transactions replicating in near real-time to the other completely redundant location (the ready site). A site switch means the locations of an instance’s active and ready sites are swapped, making the ready site the new active site and vice versa -- the instance name does not change.

Reference

T

Tableau CRM

Formerly Einstein Analytics

TDTM

Table-Driven Trigger Management - "a tool to managed apex triggers". Source

U

Upsert

A database operation that will adjust between an insert and update depending on the record in question. Reference

Universal Containers

A fictional shipping container company referenced by Salesforce, especially in various certification exam questions and Trailhead modules.

V

Visualforce

Visualforce is a framework that allows developers to build sophisticated, custom user interfaces that can be hosted natively on the Lightning platform. The Visualforce framework includes a tag-based markup language, similar to HTML, and a set of server-side “standard controllers” that make basic database operations, such as queries and saves, very simple to perform.

Source

Vlocity

Rebranded Salesforce Industries

W

WSDL2Apex

A Salesforce tool for generating stub Apex classes from a SOAP WSDL. Reference

X

Y

Z

#

1GP

First-Generation Managed Packaging - a Zip file of metadata components that can be deployed to an org via the Metadata API. They typically form an application or piece of functionality. For distribution via the app exchange they are created via a packaging org. Docs

2GP

Second-Generation Managed Packaging - Salesforce CLI (rather than Org) based packaging. Docs and Comparison of 2GP and 1GP Managed Packages

Speeding up Salesforce unit testing performance

For many years I've had this thorn in my side with Apex test cases. It goes by the name of "Disable Parallel Apex Testing" and since Spring `13 (v27.0) it has needed to be enabled constantly else I'd get a UNABLE_TO_LOCK_ROW error due to the custom hierarchy settings that get updated in the test cases.

Leap forward to Summer/Winter `18 (v42.0/v43.0) and I'm still tangling with this. That's five years of waiting for all the test classes to run sequentially one after another. That's about the right amount of procrastination to finally fix this problem! So hold on, were going to raise a support case and keep at this until the bitter end¹.

What follows is a true story. The namespaces and class names have been changed to protect the innocent, but the snippets of messages with support are real.

Step 1 - Turning off "Disable Parallel Apex Testing"

This was simple enough. Uncheck a checkbox, run all the tests, and... Uh oh.

Could not run tests on class 01p400000000001 because: connection was cancelled here

With that setting checked all my tests were passing. Now some of my previously passing test classes are falling over when run in parallel. Worse still, the specific test classes to fail were intermittent. It wasn't just one specific class causing a problem. Any one of a dozen or so classes could fail in this manner and they would change from run to run.

Step 2 - Winter `18 @isTest isParallel annotation to the rescue

Winter 18 added the new @isTest(isParallel=true) annotation to:

indicate test classes that can run in parallel and aren’t restricted by the default limits on the number of concurrent tests.

That's great, except I don't want to modify 90% of my test classes to deal with the 10% that are having issues. No, I'll just use @isTest(isParallel=false) and explicitly exclude the problem cases... except that doesn't work. At least not at the time of writing. Please vote for the idea Parallel Tests Option (isParallel) on the @IsTest Annotation to exclude tests to make this a viable approach.

Step 3 - Once more unto the support case, dear friends, once more

[Day 1 - 2018-05-09] Desperate times. Let's raise a support case to see if they can isolate the underlying issue.

[Day 6 - 2018-05-14] After a bit of back and forth with tier 2 to establish how to reproduce the problem (press the run all test cases in the org) the initial advice back was:

The root cause of this issue?
Answer: - Parallel test execution is the root cause of this issue.

Solution?
Answer: - Disable Parallel test execution.

Question: - Why do you need to Disable Parallel Apex Testing?
Answer: - As per the salesforce document, Tests that are started from the Salesforce user interface (including the Developer Console) run in parallel. Parallel test execution can speed up test run time. Sometimes, parallel test execution results in data contention issues, and you can turn off parallel execution in those cases.

Huh. Well, yes I knew that already (as per the case description when I raised it). That would work, but it's been five years of this situation, so lets push a bit harder. SHIBBOLEET!

Step 4 - How bad is the problem empirically?

Before responding with a kiwi yeah-nah to support I timed the total test run time for both parallel and synchronous test execution:

Synchronous test run is ~12% slower, but I could easily lose that time needing to watch for the parallel failures and rerun them.

— Daniel Ballinger 🦈 (@FishOfPrey) May 15, 2018

As it stood, the parallel test execution was marginally faster as long as I reran any initially failed test cases immediately after the first run completed. That was an interesting result, and I think I can explain the similar timing between the two later on.

The general consensus on twitter was that updates to custom hierarchy settings were probably to blame for the contention and subsequent timeouts.

Step 5 - Isolating Custom Hierarchy Settings using the Stubbing API

I use a number of hierarchy custom settings to toggle various functions in the app. All interactions with those settings from Apex are done via a single class. This allows for sensible defaults etc...

Those with particular feelings on how mock testing should be performed or a sensitive testing disposition may want to look away now...

To prevent any potentially blocking DML operations on the custom hierarchy settings I've injected a StubProvider when in a testing context. The StubProvider prevents any DML occuring when altering the settings in Apex tests. This isn't the typical usage for a test mocking framework, but it serves my needs here to help avoid database locking issues.

Here is a shortened version of how it looks:

And after all that the result on the asynchronous test execution was..... not much really. The tests were marginally faster, but the underlying problem with the test classes having the connection closed remains. At least I can rule out DML on the custom settings as being the problem.

Step 6 - Back to the drawing board

Blaming the problem on the custom hierarchy settings made sense from a historical perspective, but it doesn't appear to be the source of my challenges.

I went back and had a closer look at the debug logs from the tests in the run. That's when I saw it:

"What?", you may ask, "Am I currently looking at?"

That colorful image is a selection of debug log timelines from various parallel test runs. Ignoring the majority of the markings, the important thing is a purple line representing a DML operation that is taking more and more of each transaction. Right up until the point that Salesforce starts terminating it. If we drill into one of those logs we can see it is a DML operation to insert 6 records taking 3 minutes 52 seconds (for 6 records!):

14:14:15.0 (325578801)|DML_BEGIN|[992]|Op:Insert|Type:DFB__OpportunityPriceType__c|Rows:6
14:18:07.376 (232376043557)|DML_END|[992]

Step 7 - Returning to the support case with proof

[Day 9 - 2018-05-17] If that isn't a smoking gun then I don't know what is. Let's take this new evidence back to the support case.

Me to support:

I've just noticed something else odd in the debug logs for the running tests. Inserts to DFB__OpportunityPriceType__c for 6 rows are taking an excessive amount of time.

I'm seeing times for the CommonTestSetup.createPriceTypeMappings() method of between 20 and 45 seconds to complete.
Attached Log1.txt - Look for the lines:
16:18:58.0 (150146460)|DML_BEGIN|[941]|Op:Insert|Type:DFB__OpportunityPriceType__c|Rows:6
16:19:19.214 (21214892238)|DML_END|[941]

It isn't clear to me why the inserts for those records are taking so long. The fields on it are [reasonably] basic and it is only 6 rows.
The only thing I can think of is that all the test classes are trying to use the same records and they all have the same indexed values in the PriceTypeID__c field.

A bit more context as you dear reader can't see into the org like support can.

I have a custom object DFB__OpportunityPriceType__c with a unique external id field. That field is indexed. Just about every single test class I have requires data in this custom object as it contains configuration that gets linked to from Opportunity records. I thought I was being clever and used a @TestSetup method to insert 6 of these records once for all the other test methods to use. There are no triggers, workflow outbound messages or other automatons hanging off this custom object. None of the test classes use SeeAllData=true.

Step 8 - More steps in the Support dance

[Day 13 - 2018-05-21] Support responds:

I reviewed your case and ran the test classes to troubleshoot the issue further. I noticed the error in the server log "RunningForTooLongException: connection was canceled here".
The error is thrown because - for asynchronous Apex tests, any test method execution has a timeout limit. Once an Apex test run exceeds 6 minutes, an internal process "kills" it.

After reviewing the logs, it seems that this particular method is taking time to execute when all the tests classes are run asynchronously - CommonTestSetup.minimalSetupWithSettings();

Can you please try to refactor this method and see if it can be optimized?
I see there are many methods being called from this function.

Another frustrating response. They pointed out the method that gets called directly in the @testSetup method. I'd already identified the inner method from that one and even the line in question. That it is a RunningForTooLongException is new information, but I'm no closer to what is blocking those records from inserting.

[Day 15 - 2018-05-23] We try a GoToMeeting. I work through with Tier 2 support in ensuring the external ID's that get assigned to DFB__OpportunityPriceType__c records in the test cases are unique per @testSetup. It didn't seem to help. Beyond that, I think support has a good grasp of the problem now.

[Day 20 - 2018-05-28] The support case has been escalated to Tier 3.

[Day 30 - 2018-06-07] Tier 3 responds:

- When 'Disable Parallel Apex Testing' checkbox is checked in Developer Console.
Result: Running Perfect

- When 'Disable Parallel Apex Testing' checkbox is unchecked in Developer Console.
Result: Exception is thrown

- In Serial Mode, it's working perfectly because it's running in Sync mode and no data contention issue.
However, in Parallel mode, multiple classes and trigger have been invoked in the same context/transaction where performing DML Operation and also making callouts and waiting for a response in Async mode. This may cause some data contention. Also since there is a lot of custom code involved it becomes very difficult for us to analyze the scenario.

- This is WAD as per Salesforce documentation. If Apex Class will take more than 6 minutes of time to complete the transaction, the connection will be closed by Salesforce DB.

- Test classes are run in serial mode during deployment. This error will not occur when you are trying to deploy your package.

Possible workarounds -

- Try to modify the code and remove that many dependencies.
- Else run in Serial Mode as a Workaround.

This seems distinctly like they put it in the too hard basket and are trying to avoid investigating the actual issue. I'll push back to see if I can get a better answer.

Also, I'm not sure what they mean by "making callouts and waiting for a response in Async mode". This is all happening in a test context. There are no callouts or waiting for async responses. I'm not going to pursue that as I don't want to get distracted.

[Day 35 - 2018-06-12] Update from support.

My T3 has officially logged an investigation with our RnD team on this issue. My RnD team is currently reviewing the case.

[Day 36 - 2018-06-13] An update from R&D, as communicated via Tier 2.

I have received an update from the DB team on this case.

As per the DB team, it seems that the parallel sessions are working on common recordsets and causing the wait events.
Because of this waitime the DB team is high and eventually the classes are getting timed out.

May be this explains why the insertion to DFB__OpportunityPriceType__c is taking time. We cannot exactly comment on the common recordsets but DFB__OpportunityPriceType__c records is one of the potential candidate.

Solution -
Have the parallel sessions work on different recordset/dependencies to avoid this situation.

So it appears that all the test cases are trying to work with the same record sets and they are all blocking until they can get exclusive access to those records. They either stack up and run one after another or wait so long they time out. This explains why the parallel test performance is so similar to the serial performance.

I'm fairly certain the DFB__OpportunityPriceType__c records are unique between @testSteup executions, so it must be something else...

Step 9 - Time to regroup

[Day 42 - 2018-06-19] Much to my shame I've closed the case out. I need time to regroup and revisit this from a new approach. I'm certainly not giving up on it yet and have a few ideas to try:

• Try adjusting the Logging levels used when capturing the logs. Particularly around the Database logging levels. Also try capturing no logs at all.
• Push the entire packages metadata into a Scratch Org and and retry the tests. The dev org doesn't have much data loaded, but the scratch org can provide a completely empty environment to ensure it isn't a data siloing issue.
• If it is reproducable in the scratch org, start hacking parts out until it becomes easier to reproduce.
• Remove the creation of the DFB__OpportunityPriceType__c records from the @testSetup. Only create them when required in the individual test methods.
• Simplify the test cases where possible. The challenge is almost all of them work with Opportunities and OpportunityLineItems at some level. Which means a large overhead to construct all the dependent records as well.

---

Footnotes

1. The bitter end may or may not occur within the timeframe of this blog post.

Salesforce Log Categories and Events by Level - Revisited

Way back in June 2014 I posted a table to logging events by level and category - Salesforce Log Categories and Events by Level.

I was never really happy with the table layout trying to squeeze that much data in. Also, new logging levels keep getting added and several have been shuffled around recently with respect to the level they occur at.

Here is a hopefully simpler revised attempt using lists. It is compiled directly from the Debug Log Levels detail page, so it should be eaiser to keep up to date.

Similar data can be found in the Debug Log Levels documentation. I found at the time of publishing that I had several in my list that didn't appear on that page, such as CALLOUT_REQUEST_PREPARE, USER_DEBUG_WARN, and DUPLICATE_DETECTION_MATCH_INVOCATION_DETAILS.

The levels are cumulative. So everything that appears at the ERROR level will appear at all the lower levels as well. Everything at the WARN level will appear at INFO, DEBUG, FINE, ... and so on.

• Updated for Summer '18 - v43.0

Steps required to support POSTing multipart/form-data Content-Type from Apex

I came across an interesting challenge when transferring example images to the Einstein Predictive Vision Service (PVS). As part of PVS you can upload example images to train an image classifier. As a gross generalization - here is a picture of a cat. If you get another image like that tell me it is probably a cat.

What follows are the lengths I had to go to in order to be able to call this web service method from Apex.

In their documentation MetaMind were kind enough to provide an example cURL command to submit the example image.

curl -X POST -H "Authorization: Bearer <TOKEN>" -H "Cache-Control: no-cache" 
 -H "Content-Type: multipart/form-data"
 -F "name=77880132.jpg" -F "labelId=614" -F "data=@C:\Mountains vs Beach\Beaches\77880132.jpg" 
 https://api.metamind.io/v1/vision/datasets/57/examples

It always struck me as odd that specifying the API usage via a specific command line tool has somehow become the de facto standard. It's like describing how to use a REST API is so difficult that it is easiest to just rely on the conventions of a known command line tool. Nothing against cURL, but it is hiding some of the mechanics of how that API call is working. In particular with this example is how the -F is composing the multipart/form-data and how the @ in the command will be substituted with the actual file from that path.

Back to uploading example images to PVS. Performing this API call from Apex proves to be a bit of a challenge as there isn't currently direct native support for a multipart form callouts from Apex. Please pause your blog reading at this point and consider voting for the idea - Image upload using multipart/form-data.

My first naive attempt at replicating the cURL call from Apex using the supplied HttpFormBuilder class failed miserably. The binary blob data that I was extracting from an image stored in a ContentVersion record wasn't being accepted by MetaMind.

    public ExampleResponse addExample(string access_token, string datasetId, string labelId, string filename,
        blob exampleImageData) {
        
        string postUrl = CREATE + '/' + datasetId + '/examples';
        
        string contentType = HttpFormBuilder.GetContentType();
        System.assertEquals('multipart/form-data', contentType);
        
        //  Compose the form
        string form64 = '';

        form64 += HttpFormBuilder.WriteBoundary();
        form64 += HttpFormBuilder.WriteBodyParameter('labelId', EncodingUtil.urlEncode(labelId, 'UTF-8'));
        form64 += HttpFormBuilder.WriteBoundary();
        form64 += HttpFormBuilder.WriteBodyParameter('name', EncodingUtil.urlEncode(filename, 'UTF-8'));
        form64 += HttpFormBuilder.WriteBoundary();
        form64 += HttpFormBuilder.WriteBodyParameter('data', EncodingUtil.base64Encode(exampleImageData));
        form64 += HttpFormBuilder.WriteBoundary(HttpFormBuilder.EndingType.CrLf);
        
        blob formBlob = EncodingUtil.base64Decode(form64);
        string contentLength = string.valueOf(formBlob.size());
        //  Compose the http request
        HttpRequest httpRequest = new HttpRequest();

        httpRequest.setBodyAsBlob(formBlob);
        httpRequest.setHeader('Connection', 'keep-alive');
        httpRequest.setHeader('Content-Length', contentLength);
        httpRequest.setHeader('Content-Type', contentType);
        httpRequest.setMethod('POST');
        httpRequest.setTimeout(120000);
        httpRequest.setHeader('Authorization','Bearer ' + access_token);
        httpRequest.setEndpoint(postUrl);

        //...
    }

Where did I go wrong? Well, unlike a prediction there isn't a variant of this call API that accepts the Base64 encoded image. Using HttpFormBuilder.WriteBodyParameter with the Base64 encoded blob to write the data wasn't going to work. MetaMind want the actual bytes and associated Content-Type header. Here is how the working request appears in Postman:

POST /v1/vision/datasets/1001419/examples HTTP/1.1
Host: api.metamind.io
Authorization: Bearer 12346539689dae622cbd91d9d4880b3314bfb747
Cache-Control: no-cache

----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="data"; filename="ItsAFrog.png"
Content-Type: image/png

<bytes of file go here>
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="labelId"

8588
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="name"

itafrog.png
----WebKitFormBoundaryE19zNvXGzXaLvS5C

An alternative method was needed to HttpFormBuilder.WriteBodyParameter. One that would set the correct Content-Disposition and Content-Type headers for the binary data and then also correctly append the bytes from the image. That last point is really important. Correcting the headers wasn't very difficult, but with the binary file data I found the last few bytes were getting corrupted on the file.

Thankfully Enrico Murru had already covered a lot of the details around how to get proper multipart form POSTs working in Apex, so full credit to him for the blog post POST Mutipart/form-data with HttpRequest. This was then later refined by Grant Wickman in an answer to Post multipart without Base64 Encoding the body. The more I researched why HttpFormBuilder wasn't working the more I found MetaMind had based HttpFormBuilder partially on Enrico and Grant's work but hadn't taking it through to completion. For instance, the EndingType enum has the following comment indicating they were clearly thinking about it, just never applied it.

Helper enum indicating how a file's base64 padding was replaced.

To get it working I needed to finish applying Enrico and Grant's solutions.

It's all about the Base(64 encoding)

The smallest unit of storage on a file is a (octet) byte, which is composed of 8 bits. In contrast, Base64 encoding represents 6-bits with each character. So 4 Base64 characters can represent 3 bytes of input data. That ratio of 4:3 is very important with Base64 encoding. If the number of input bytes to be encoded is divisible by 3 then everything is fine as all 4 Base64 characters will represent meaningful input.

The problem occurs if the input byte length isn't divisible by 3. In that case the Base64 encoding process would normally append a padding symbol (=) or two to the end of the encoding to indicate which were the valid bytes in the last 3 and which were padding to get to the correct grouping of 3 bytes to 4 characters.

The following table shows how the padding needs to be applied if there are only two or one bytes to be encoded. You can see how some of the bits in the 6-Bit groupings no longer represent actual data when encoding only one or two input bytes.

Data Bytes In Binary Form	1	1	0	1	0	1	0	0	0	0	1	0	0	1	1	1	1	1	1	1	0	1	1	1
Data Rearranged Into 6-Bit Groups	1	1	0	1	0	1	0	0	0	0	1	0	0	1	1	1	1	1	1	1	0	1	1	1
6-Bit Groups in Decimal Form			5	3						2					3	1					5	5
Groups Converted to ASCII Characters				1						C						f						3
Data Bytes In Binary Form	1	1	0	1	0	1	0	0	0	0	1	0	0	1	1	1
Data Rearranged Into 6-Bit Groups	1	1	0	1	0	1	0	0	0	0	1	0	0	1	1	1	0	0
6-Bit Groups in Decimal Form			5	3						2					2	8
Groups Converted to ASCII Characters				1						C						c						=
Data Bytes In Binary Form	1	1	0	1	0	1	0	0
Data Rearranged Into 6-Bit Groups	1	1	0	1	0	1	0	0	0	0	0	0
6-Bit Groups in Decimal Form			5	3						0
Groups Converted to ASCII Characters				1						A						=						=

Enrico's method to build up the full multipart form submission from Apex is to first build each part individually using Base64 encoding and then concatenate those parts together. That final Base64 string is then converted back to a Blob to become the body of the HTTP POST request.

The challenge with this method is you can't have intermediate padding characters. The Apex Base64 decoding process is going to ignore those intermediate padding characters. This in turn causes an incorrect mapping between the 4 Base64 characters and the 3 bytes they are supposed to represent. Here is an example using some anonymous Apex. Note how the decoding the concatenated values gives the same result with or without the internal padding:

String encodedA = EncodingUtil.base64Encode(Blob.valueOf('A'));
String encodedB = EncodingUtil.base64Encode(Blob.valueOf('BC'));
System.debug(encodedA);
System.debug(encodedB);
System.debug(EncodingUtil.base64Decode(encodedA).toString());
System.debug(EncodingUtil.base64Decode(encodedB).toString());
System.debug(EncodingUtil.base64Decode(encodedA + encodedB).toString());
System.debug(EncodingUtil.base64Decode('QQ' + 'QkM=').toString());

Output

20:17:50.65 (69059707)|USER_DEBUG|[3]|DEBUG|QQ==
20:17:50.65 (69127131)|USER_DEBUG|[4]|DEBUG|QkM=
20:17:50.65 (69373165)|USER_DEBUG|[5]|DEBUG|A
20:17:50.65 (69500491)|USER_DEBUG|[6]|DEBUG|BC
20:17:50.65 (69636544)|USER_DEBUG|[7]|DEBUG|A$
20:17:50.65 (69759126)|USER_DEBUG|[8]|DEBUG|A$

For encoding text this problem is bypassed by appending additional whitespace characters to the base text until the Base64 encoded representation no longer requires padding. Without the padding it is then possible to concatenate the two base 64 encoded values and then later decode the complete string back to the Blob sans additional bytes. This however doesn't work with a binary file like an image. You can't just go on appending additional bytes on the end without corrupting the file.

Instead the solution presented by Grant Wickman is to borrow one or both of the CR (\r) or LF (\n) characters that separate the end of the file bytes from the multipart boundary.

By shuffling these legitimate characters onto the end of the files bytes the need for padding can be removed. The padding on the boundary can then be adjusted as required using the white-space technique where it won't affect the integrity of the data.

There is a certain elegance to it as a solution. Note how the first 4 bits of the CR align with the last 6 bits of the second Base64 character so they don't need to be changed. And then again, if only switching out one padding = for a LF the first two bits again align.

I find that table very satisfying. I think I might print it out and keep it at my desk like one of those inspirational posters.

Example of additional HttpFormBuilder method

What did we learn?

1. That rogue = padding characters in the middle of a Base64 string will corrupt the data.
2. If the Base64 encoding for the file ends with "==", crop those characters off and replace with 0K to represent a CRLF. Then don't prepend the following boundary footer with those characters.
3. If the Base64 encoding for the file ends with "=", crop that character off and replace with N to represent a CR. Then only prepend the following boundary footer a LF.
4. That @muenzpraeger has a more refined Apex project for working with the Einstein Predictive Vision Service. I've submitted my changes there.
5. 
   The O'Reilly Book covers featuring pictures of animals are done in a woodcut/hedcut style.
6. 
   Base64 encoding examples can be inspirational and look great at your desk.

The Mad Catter - Salesforce Predictive Vision Services

Disclaimer

No animals were harmed in the creation of this blog post or the associated presentation. The standard disclaimer applies.

I've got a problem. A cat problem to be precise. While I'm more of a dog person, I don't particularly mind cats. However, recently there has been a bit of an influx of them with the neighbors. There are at least a half dozen of them that roam freely around the neighborhood. Not the end of the world, but they have a nasty habit of leaving presents on the lawn for the kids to find. Things like the following:

In short, they poop everywhere and leave the remains of birds lying around. Things that aren't so great to step on or for the kids to find when playing in the garden.

I spoke with the immediate neighbor who owns two of the cats, and he suggested spraying them with water to deter them away. While that did indeed prove to be a very effective, amusing, and satisfying approach to move them on it required 24 hour vigilance as they just kept coming back.

Get a Dog

My wife keeps saying we should just get a dog. I guess getting a dog to chase the cats away is an option. But it seems like training a dog to use the hose might be more trouble in the long run.

Technology to the Rescue

Thankfully I found a handy device the attaches to the end of the house and activates a sprinkler head when a built in motion detector is set off.

Perfect! Great! Cat comes into range, then a sudden noise and spray of water sends them off towards someone else's lawn to do their cat business. Problem solved and I can go back to doing more fun activities.

Except there was one small problem. The PIR motion sensor didn't particularly care what was moving in front of it. Cats, birds, the kids on their way to school, a courier with a parcel for me, a tree in the wind, the mother in law. It would spray them all regardless of whether I wanted it to or not.

Salesforce Predictive Vision Service

Technology wasn't solving my problem. I needed to use more of it!

I recalled a recent presentation by the Salesforce Developers team - Build Smarter Apps with New Predictive Vision Service. The short version of that presentation is you can train a deep learning image classifier with a set of training images. Then when you give it a new image it will give you probabilities about what is likely in the picture. I've create a quick start unmanaged package for it to save you going through most of the install steps.

To make this work I needed a large collection of cat images to train the dataset from a create a model. Luckily for me, providing pictures of cats is something that the internet excels at.

The second challenge with using the predictive vision services is managing how many images I am going to send through to the service. If I just point a web camera out the window it could be capturing 30+ frames per second. Not really practical to send off each frame to the service for identification when there might be nothing of interest happening 99% of the time.

Motion detection

I had a few options here.

Option One would to to stick with the basic PIR motion sensor, but it would still be generating a ton of false positives that would need to pass through the image recognition. A simple cool down timer would help, but the image captured immediately after the first motion is detected would likely get something as it is just entering the frame.

I figure since I'm going to need a camera to capture the initial image I might as well get some usage out of it in detecting the motion. Because of the initial processing step I can exclude motion from certain areas, such as a driveway or tree that often moves in the wind. There can also be a slight delay after the motion is detected and before the prediction image is captured. This gives the subject time to move into the image.

The prototype solution looks like this:

1. 
   A webcam that can be pointed at the area to be monitored.
2. 
   Motion Detection software to process the video feed and determine where the movement is and the magnitude. The magnitude is useful, as particularly small subjects like birds can be completely ignored.
3. 
   The ability for that software to pass frames of interest off to the Salesforce Predictive Vision Service. This is a simple REST POST request using an access token.
4. 
   If the probability from the frame indicates a Cat is present, send a signal to the Raspberry Pi.
5. On the signal, the Raspberry Pi activates the GPIO pin connected to a relay.
6. 
   When activated, the relay supplies power to the existing automated sprinkler, which activates on initial power on when the sensitivity is set to maximum. Another option here is directly connecting a solenoid water value to the hose line.

When all put together the end result looks something like this:

The Einstein bust with terminator-esque glowing red eyes was part of the presentation I gave on this topic.

While filming that video I inadvertently live tested it on myself as well. An aging fitting on the hose connector to the sprinkler had come loose outside at the tap. So I went out to fix that, restored the water pressure to the sprinkler, then walked back to the laptop to resume the test. Only when I checked the motion detection screen did I realize it had captured my image passing in front of the sprinkler. Thankfully the predictive vision services came back indicating I didn't resemble a cat and the sprinkler didn't activate. Success!

Refinements

It occured to me that there were further improvements that could be made.

The first and easiest change I made is to activate on things other than cats. It can be equally selective to activate on a wandering neighbors dog, squirrels, general wildlife, etc...

I needed a way to deal with unfortunate false positives, such as a person wearing something with a picture of a cat on it. These can partially be avoided by looking at all the probabilities that Einstein is returning and having thresholds against each label. I.e. Activate on any Cat prediction above 50% unless there is any prediction indicating a person in the field of view. Images kept from the activations could also be used to further refine the examples in the dataset.

These first two refinements are actually present in the video above. When using the general image classifier it typically identifies the stuffed cat as a teddy bear. So in the small section to the bottom right of the app you can mark labels to activate on and labels that will override and prevent the activation.

Other changes I might consider making:

The motion sensor could be maintained and introduced as the step prior to activating the video feed. This would increase the time between the target entering the area and the sprinkler activating, but would save a lot of endless processing loops looking at an unchanging image.

If I forgo some of the more processing intensive motion tracking the whole solution could be moved onto the Raspberry Pi. This would make it a much more economical solution.

However, another option with the motion detection still in place would be to crop the frame image to just the area where the motion was detected. This should lead to much higher prediction accuracy as it is only focusing on the moving subject.

When real world testing commences with live subjects I'll need to add a video capture option to cover the time from just before the sprinkler is activated till just after it switches off. I think the results will be worth the extra effort.

I have a range of other devices that could easily be activated via the relay attached to the Raspberry Pi. One such device is an ultrasonic pest repeller. Perhaps combined with a temperature sensor as a slightly kinder deterrent on cold nights.

User Group Presentation

I gave a talk to the Sydney developer user group on this project. The slides, as they were:

---

I still feel the need to settle on a name for the project. Options include:

• The Mad Catter (after the elusive Catter Trailhead badge)
• The Einstein Cannon
• The Cattinator (After the general them of the presentation.)

---

See also:

• Einstein Vision Demo of the General Image Classifier
• Trailhead Quick Start: Predictive Vision Service
• Introducing Einstein Vision: Your Apps, Now Powered by Salesforce Einstein
• Java based wrapper for using the Predictive Vision Services - salesforce-einstein-vision-java
• Inspirational video dealing with cats spraying cars

Gallery