Monday, July 27, 2020

FuseIT SFDC Explorer 3.15.20209.3 - Summer '20

Another roundup of some of the changes to the FuseIT SFDC Explorer since the 3.13.20008.1 release.

Display the primary sObject type for a SOQL query in the executed units

When looking at the "Show Executed Units" for a debug log that includes SOQL queries there is a new "SOQL Primary sObject" column. This makes it easier to sort the SOQL queries in the transaction by the sObject they were operating over.

There is also a new context menu option in the executed units area to filter the main log view to just the selected events.

Other changes include tracking Flow executed units and SOQL Total Rows metrics from debug logs.

Add the MetadataComponentDependency API calls via the Bulk API 2.0

Previously the Metadata Dependencies tab was limited by the underlying MetadataComponentDependency API only supporting queries over 2,000 records in a single query. Switching to the bulk API expands this out to 100,000 dependency records.

There is also a new experimental graph format called "Microsoft Automatic Graph". This will use a built in graph layout tool. It is no where near as powerful as Gephi, but it is built in and allows you to gradually build the graph up. E.g. try starting with a single ID against "Where is it used" and then using the context menus to expand the graph incrementally.

Rebuilt login and authentication controls.

The various login and authentication controls that are displayed on app startup have a more consistent UI. For example, they have a consistent Login button.

Other changes 3.15

  • Update SalesforceSession so that all APIs are working against v49.0 - Summer 20
  • Apex logging - add SOQL_EXECUTE_EXPLAIN Log event
  • Update SOQLResult column names when the defaultNamespace is defined. Includes first level relationship fields.
  • SFDCExplorer: Executed Units Context Menu to set Apex Log filter
  • SFDCExplorer: Show a message if there are issues showing an individual entity
  • SFDCExplorer: Support for flow log message types. Guard against debug logs with recursive search depth that is too great.
  • SFDCExplorer: Exception Handling when showing reports.
  • wsdl2apex: Warn if the schemaTypeName is missing.
  • SFDCExplorer: Expand Graph search ability
  • SFDCExplorer: Format VF_ log events in the timeline Handle deeply nested log structures to prevent stack overflows.
  • SFDCExplorer: Format byte size of log. Handling clicking a record ID in the log that doesn't exist in the current org. Option to format Callout requests in the debug log.
  • SFDCExplorer: Ensure the popup window that shows SOQL sub query results has a unique name so that the caching key is unique.
  • Tooling API. Check access to the DebugLevel metadata before attempting to query it.
  • SFDCExplorer: always prompt for directory to save metadata to.
  • Update All API versions to Spring 2020 (v48.0)
  • SFDCExplorer: Update .NET Framework version to 4.7.2 and add support for scaling to high DPI monitors.
  • SFDCExplorer: Update to cs31623 to allow for variation in the instanceUrl provided by the SFDX CLI
  • SFDCExplorer: Auto expand the test class search node when searching. Detect test classes in the queued state
  • SFDCExplorer: Export the results of a Bulk CSV load. This is useful if there where errors with the operation on individual records.
  • SFDCExplorer: Log metrics for callouts
  • Tooling API: Correct updating the log levels on TraceFlags

Wednesday, May 20, 2020

Dreamforce 2019 Presentation - Reducing the Cost of Enforcing CRUD and FLS in the ESAPI

At Dreamforce this last year (2019) Chris Peterson and I gave a theater presentation on combining the new Apex Security.stripInaccessible method with the existing ESAPI library for enforcing Create, Retrieve, Update, Delete (CRUD) and field level security (FLS) in Apex.

Unfortunately the Dreamforce theater sessions weren't recorded in 2019 as they were in previous years 🤦‍♂️. To make up for that, below are the session slides. I'll also expand on the key points in this blog post. These will be my words rather than Chris's, although I'll try and cover most of the same content.

What is the ESAPI?

The ES stands for Enterprise Security. And the API... Well, hopefully you know what an API is.

The Salesforce ESAPI is a port of a Java library created by OWASP (Open Web Application Security Project).

To address why you would want it in your Salesforce org, here is a quote from the OWASP ESAPI project page:

The ESAPI libraries are designed to make it easier for programmers to retrofit security into existing applications. The ESAPI libraries also serve as a solid foundation for new development.

These seem like noble goals. We want to improve the security of existing applications and implement newer applications with the same level of security from the start.

The three core areas that the Salesforce ESAPI addresses are:

  1. Input Validation - is a given string a valid date? Is it a valid credit card number? A valid redirection URL?
    • E.g. ESAPI.validator.SFDC_isValidDate
  2. Output Encoding - is is safe to render the content back to the users browser via HTML?
    • E.g. ESAPI.encoder.SFDC_URLENCODE
  3. Access Control - enforce the built in access control mechanisms: CRUD, FLS, and Sharing.
    • insertAsUser/updateAsUser
    • DML on a limited set of fields
    • Override sharing for a single DML operation

(Re) Introducing the ESAPI

Salesforce originally released their version of the ESAPI library in 2010. In 2016 they added the new ESAPI.encoder.SFDC_BASE64_URLENCODE method. Other than that it was stalled for any maintenance or new development.

In 2019 Chris Peterson and Jake Meredith from Salesforce took ownership of the Github repo. Even better, they are accepting pull requests.

One of the first steps in rejuvenating the repo was to increase the built in code coverage from 54% to 93% on the security specific test methods. And, perhaps more importantly, add a number of meaningful assertions and negative test cases along the way. Overall project test coverage is now up by 39%.

One of the particular challenges with this was scripting out of the box test cases using only the built in Profiles and sObjects. The test cases needed to be portable to any org, so they couldn't rely on a specific custom Profile existing. At the time the "Read Only" profile was the most restrictive system profile available. Going forward I might revisit this with the still to GA Minimum Access profile.

With the better test coverage in place it was then possible to overhaul how the field and object level security was enforced. More on that later...

A Recap of the Security.stripInaccessible() method

The Spring '20 release included the GA version of the new Security.stripInaccessible method.

This new method provides a streamlined way to check both field and object level data permissions for the following access types:

  • AccessType.READABLE - Check the fields of an sObject for read access.
  • AccessType.CREATABLE - Check the fields of an sObject for create access.
  • AccessType.UPDATABLE - Check the fields of an sObject for update access.
  • AccessType.UPSERTABLE - Check the fields of an sObject for insert and update access.

The SObjectAccessDecision Object

After calling stringInaccessible() an SObjectAccessDecision object is returned. This provides a number of helpful methods.

getRecords() provides a new List<SObject> that has all the inaccessible fields removed. These are also detached from the source sObjects.

Two additional methods getModifiedIndexes() and getRemovedFields() provide details about which records and specific fields were modified.

How it works in practice to enforce security requirements

Using the newer stringInaccessible method has a number of advantages. It will cover all possible sObject field types. It will check relationship fields, including nested relationships. Sub-queries and polymorphic lookups are also covered.

The example in the image above shows a new Opportunity for "Appy's App" that was generated in the trusted system mode. As such, it could set the custom Standing__c and Value__c fields. This Opportunity is then passed through stringInaccessible with the AccessType.CREATABLE parameter. This does all the hard work for us of checking the users Profiles, Permission Sets, the Permission Set groups, the muting permission sets, etc... The output Opportunity has the fields that the user doesn't have create access to completely removed. The stripped fields aren't null: they're really undefined. This is important when it comes to subsequent DML as we don't want to inadvertently clear fields out.

Other advantages:

  • It's particularly useful when handling untrusted input (like from JavaScript controllers)
  • It's also great for gracefully degrading UI experiences like SFDC does natively

Example walk through code the changes to the ESAPI

The video shows the core structural changes that were made to the ESAPI methods to use the new methods.

Measuring the Performance changes - Methodology

Beyond enforcing the security requirements the next important consideration is changes to CPU Limit usage and Heap usage. Usually you trade more CPU for less Heap or vice versa. If we can cut both down we are doing well!

The general goal is always to see equal of better performance while enforcing the same security requirements. There may be some tradeoffs in performance made, but the security can't be compromised on. The coverage and assertions from the automated tests ensure we are still enforcing the same security requirements.

The performance differences in terms of Apex limits were measured using Test harness classes and Adrian Larsons LimitsProfiler framework. Performing multiple runs between the current/baseline implementation and the new version using the stripping methods allows the relative changes in limits to be measured.

During the testing it was important to have the debug logging completely off as it would otherwise affect the outcomes.

The testing below was done in ALL_OR_NONE mode. This is more demanding than the alternative BEST_EFFORT mode as it requires per field level checks.

To allow for the limits testing framework to repeatedly insert multiple records Savepoints were used as part of the setup and teardown steps. This prevented hitting the storage limits while still measuring the performance differences in the ESAPI.

Measured results for bulk inserting Contacts

The performance difference can vary greatly based on scenario:

  • There is negligible performance difference when checking object CRUD permissions. Note, the Apex ESAPI is still enforcing against the sObject Schema.SObjectType until 224 (Spring `20) with the fix.
  • There is a significant performance difference if there are a large number of requested fields that aren’t set on all the sObjects.

Measured improvements on 25 iterations inserting 200 Contacts. 33 Standard fields

  • 25% Less CPU usage
  • 18.6% Less Heap usage

Links and Resource

Thursday, February 13, 2020

FuseIT SFDC Explorer 3.13.20008.1 - Winter '20

Another roundup of some of the changes to the FuseIT SFDC Explorer since the 3.12.19175.1 release.

Redesigned Apex Tests selector

The Apex test selection control has been reworked to include a top level node for the last known state of each test method. This allows you to monitor the accumulated test results from a single location. If you redeploy one of the test cases in question it will reset to an unknown status.

Updates to the Debug Log Executed Units

In the last release I put in the first version of the Executed Units control. This release builds on that.

It is now possible to mix and match the different log types. So you can focus on, for example, just the SOQL queries.

The same toolstrip includes a button to export the executed units data.

Smaller generated Wsdl2Apex classes

If you've ever looked closely at the Apex classes that Wsdl2Apex generates you'll notice a recurring string - the namespace of the the root element. Depending on the WSDL it's not uncommon for this same string to be repeated thousands of times in the generated Apex class. This update will now generate a single static final to handle the repeated usages.

New controls for exploring Execution overlays, including heap dumps

The developer console currently provides a viewer for execution overlay output created by a ApexExecutionOverlayAction record. This includes the Heap Dump, which has all sorts of interesting details about the data stored in heap/memory for the transaction at the the point of time the overlay was captured.

Right now it mostly just replicates the existing developer console functionality and allows browsing the stored objects by type.

Other changes 3.13

  • [cs32197] SFDCExplorer: Option to save a debug log directly to disk.
  • [cs32152] SFDCExplorer: Auto add metadata files when adding package components. Update display of Metadata Deploy results.
  • [cs32150] SFDCExplorer: Allow the target save directory to be specified when doing an unpackaged metadata export (if the working directory isn't accessible)
  • [cs32149] SFDCExplorer: Use internal list of known key prefixes with the Entity Explorer
  • [cs32148] SFDCExplorer: Option on running SOQL queries to include deleted rows
  • [cs32003] Update to Winter '20 release. v47.0
  • [cs31838] Wsdl2Apex: Handle an operation binding with no output. Raising a warning for a missing portType rather than an exception.
  • [cs31820] SFDCExplorer: Hide JavaScript errors during OAuth login