Thursday, June 11, 2015

The Holy Grail of Apex Testing in Salesforce

The quest has been long and difficult, but I think I've finally found it. The Holy Grail of testing Apex in Salesforce. (Skip directly to the end result)

Many would consider the introduction of the @isTest annotation in the Spring 12 release to be one of the most important releases for Apex testing. Or even the subsequent ability to create Price Book Entries when using @isTest(SeeAllData=true) in Summer 14. Freeing the majority of remaining cases that still needed to access live data. There has also been the shift from synchronous test execution to asynchronous test execution.

Those are indeed note worthy, but I think I've found something more useful.

Origins

It seems like a long time ago, but the idea exchange tells me it was "about" 2 years ago (in the frustrating way it refuses to show actual specific dates on anything). It's like there is some psychology at play where not having the specific date doesn't make it feel as old. I digress.

I had the idea that if anonymous apex could be run in a testing context then it could be used to run individual test methods from the suite of tests that an Apex Class contains. Feel free to take a minute and vote for it, I'll wait - Run anonymous apex as if it were a test case.

The end game was to be able to run a single test method of interest without having to manipulate the Apex class to do it. Sure you can comment out the @IsTest annotations or testMethod keywords on the tests that aren't of interest, but that is just an extra step towards yak shaving and away from what you want to be doing. Running a single test method, checking the test result and inspecting the resulting debug log.

There would also be other side benefits to having anonymous apex code able to run in a testing context. Isolation from live data, testing out code before actually using it, callout mocks...

As with most good ideas that are worth having once, someone else had also had the idea. Perhaps more succinctly towards the actual goal. Allow single test method execution from an Apex Class. Vote for that one too.

Dreamforce 2014

No great quest would be complete without a great journey, so in 2014 I voyaged 11,000 km (6800 miles) from Nelson, New Zealand to Dreamforce in San Francisco to pitch my idea to any Salesforce employee I could find. I also found time for a side quest of presenting at said conference.

The Meet the Developers session was a great place to raise the idea in front of those who might make it happen. I also managed to talk directly to Josh Kaplan about it in the halls.

The Product Managers that say Ni No!!

I demand a 2,500 voting point threshold. Ni!!

As you may have already read on my idea, it was largely hacked to bits with the suggestion to cause a rollback by throwing an error or using an assertion. True, in most cases this would rollback the transaction. It does however miss the main goal of running an individual test method. There are other limitations of it as well. For instance, callouts can't be rolled back.

More to the point, as per the response in the meet the developers session and the response in the other idea, there is a backlog item where Salesforce will add native support for running single test methods. Long term this makes more sense for the platform rather than hacking it in by using anonymous Apex.

Still. "You make me sad"

Not to be deterred by such a small flesh wound, I kept at the end game goal, looking for viable alternatives.

Then an accumulation of the work I'd been doing with various parts of Salesforce gave me an idea. It all came together to make something quite useful.

The Holy Hand Grenade

At this stage I'm not sure I want to expose how it works internally. A determined developer could probably root it out once they realize it is possible.
You can see details from how it works in Using the Salesforce Metadata API to run a single test method in isolation. It is also available in the FuseIT SFDC Explorer version 2.7 onward.

You can select an individual test method to run in isolation via a context menu on both the Apex Tests and Apex Test Queued tabs. Then you can see the results of that test and review the resulting Apex log. This will be done in isolation of any other test methods in the class. Be gone "MAXIMUM DEBUG LOG SIZE REACHED"!. No modifications to the Apex test class or org in geneneral are made as part of this process.

As with all such things, there are some requirements. The Apex class containing the test methods and the methods to be targeted both need to be public. Also, I haven't checked it with Apex classes using the @testSetup annotation, but I suspect it won't work. Static constructors should be OK.


UPDATE: Summer '15

So, it turns out in the Summer '15 (v34.0) of the Tooling REST API it is possible to POST JSON to the /runTestsAsynchronous/ that specifies both the Apex class Ids of the test classes and the test method names as well. This comes back to the feature being integrated into the platform overall.

It isn't currently apparent if the SOAP version of the Tooling API has the same capability. It does have the runTestsAsynchronous() method that takes a string parameter. The example in ApexTestQueueIte shows it being passed a comma seperated list of Apex Class Ids.

Further details