Chris Peterson, Director of Product Management for Apex at Salesforce, joins us this week to discuss Spring ‘20 Apex. Chris’s day-to-day is often complex; it includes approving increases in Apex limits, sometimes putting out developmental “fires,” and overseeing Apex product development. 

In today’s episode, we highlight some key elements about Spring ‘20 Apex, including Apex Finalizer.  We also discuss some of the security enhancements coming, including the addition of WITH_SECURITY_ENFORCED for SOQL and an Apex method called stripInaccessible. Join us for an informational and interesting conversation about some of the new features coming to Apex in Spring ‘20. 

Show Highlights:

  • Why Chris believes it’s important to take action quickly 
  • How important Hammer is to the ongoing stability and development process of Apex
  • Apex Finalizer – what it is and how developers can use it
  • One private API Chris added: logging 
  • The Apex Finalizer pilot phase
  • A security enhancement for SOQL 
  • The stripInaccessible() security feature is a “Swiss Army Knife” method Chris always wished existed
  • Apex Describe performance issues addressed

Resources:

Shout Out:

Episode Transcript

Chris: And it’s like me and a couple of other people from the apex engineering team or the other end of the table, where I’m bending their ear about feature requests and bugs that I had had a hard time getting justified.

Josh: That is Chris Peterson, Director of Product Management for Apex at Salesforce. I’m Josh Birk, Developer Evangelist at Salesforce and here on the Salesforce Developer Podcast, you’ll hear stories and insights from developers for developers. In that clip, Chris is describing a dinner he called “a please stop calling Parker Harris and come talk to us” dinner where as a customer he got to talk directly to the internal engineering team at Salesforce. Those conversations would later lead to his hire as a product manager, eventually landing on Apex. Now his day to day is complex and often hard to nail down but one part of it approving increases in Apex limits serves as an example.

Chris: Three quarters of the time, nothing contentious, easy to approve. I just worked through the queue a quarter of the time, I think get very sad when I see this. And then we have to go and have a harder conversation. I feel like that’s kind of a microcosm of the job is three quarters of the time, it’s great. And then a quarter of the time kind of unpredictably something could catch fire. If you don’t take the right steps pretty urgently.

Josh: The this that Chris is referring to is actually a villain that has shown up on this podcast Previously, we are talking about the infamous i++ style of unit test, which is basically nothing but garbage

Chris: which works great until you need an increase in code coverage size. And then you have a major refactoring project that is a hard blocker to you to play your new code. So yeah, we definitely do notice Please don’t do that. Yes, you’re flying blind.

Josh: And again, we can refer people back to Roopa’s episode too, which please, please stop tormenting the hammer team with your unit tests.

Chris: Yeah, and I mean, the other thing is, it’s really hard for me to overstate how important hammer is to the ongoing stability and iterative development process of Apex Language, if we didn’t have hammer, there’s a lot of things that we’d be afraid to try. There are definitely times where we put something into the release and go. Alright, we’ll have to see how about the hammer fall out is maybe it’ll end up being a critical update, maybe nobody depends on it. Let’s go find out. And all of the speaks to how important is that your unit test, they’re helping you as a developer, they’re helping your applications, they’re helping your users, but they’re also helping Salesforce have that confidence level for putting up their new releases. Once we put Apex out there in a new release. And it’s actually gone live on customers, it’s really hard to go back and change something. We can’t go undo a bug fix without a lot of potential disruption. And so hammer is this like vital lifeblood of our ability to move forward and we need it. We can’t just give that up. It would slow down the evolution of language substantially if we did.

Josh: Okay, so you could put me, Chris  and Roopa, and probably a bunch of people on the mic to talk about unit testing in general. But that’s actually not the point of this episode. This episode, I really wanted to highlight a few key things about Spring 20. And some of the new things that we’re seeing in Apex. And one of the neat things that we’re going to get is something called Apex finalizer.

Chris: So finalizer is our new Apex interface called finalizer, a new static method and system called attached finalizer. And a new modification to the core platforms behavior for queable classes. So what you do is you implement your own finalizer class that obviously implements the finalizer interface, either queable job you call attach finalizer and pass in your instance of it. And what that does is it records that object as the callback for when this code completes or fails. So if the queable gacks, if it fails with an unhandled exception, or if it completes successfully, your finalizer is invoked. And so that’s already something fairly novel. We don’t have a lot of on failure callbacks in Apex.

Josh: Now there’s a lot of things on the platform and in Apex that do very specific things like for instance may be creating and sending email. But that is not the design goal for finalizers.

Chris: Well, I tried to design them as something of a Swiss Army knife, so that they can solve a huge number of problems. And I’ve probably only thought of a third of the use cases that they’re actually gonna end up covering once they’re out in the wild. It was specifically designed to be a lower level API and not particularly prescriptive so that the blossoming level of open source around Apex can start to fill in the gaps with more prescriptive frameworks. And I know Kevin Foreman’s actually promised he’s gonna have a blog post out next week, well as when we’re recording this. So probably will be out by the time you hear this, where he’s updating his Apex promises API to use finalizer. So that if a promise chain fails, it can either in cue the next promise or handle an error call back. And so the type of use case I’m most excited about isn’t necessarily that you’ll write your own code that implements by analyzers. It’s the frameworks that gonna pop up around five analyzers that let the framework do something that seems like magic and would have been really hard to do reliably before. Now, developers right now are kind of in a world where they’re using things like platform events, and custom objects in order to kind of apply this layer of transactional functionality in in that sense, you’re always going to have to be juggling your resources versus that functionality. And the design of Apex finalizer really takes that to heart. And what they do that’s really interesting is all of the state of the finalizer object at the end of the queable success or failure gets serialized out and then deserialized into a brand new Apex context with a brand new database context. And so what that means is everything other than the finalizer instance itself is gone. So if you hit a heap limit error in your cubicle job, your finalizer can still run assuming that the finalizer object itself was not 100% of your heap utilization. It also means that you can do things like take a custom object that you’re using for logs and buffer them in memory on the finalizer object. And then in the finalizer execution, do the DML. Which means if you’re doing things with call outs, and would like to log the state of a series of columns that have happened in the same killable job, that was always a challenge before, because you would have to either do DML or hope that you got to the end, and then could do DML there properly, which if you hit the hundred cumulative limit on call outs could be an issue or even if you just had some sort of other unhandled exception. And so with finalizers, you can buffer all those log messages in memory on the finalizer object itself as a member variable. And then when it gets serialized and deserialized into its new context, do the DML there. And because it’s a clean database transaction, if you’ve done set up the DML beforehand, and the queable, you can do data DML in the finalizer. So because everything is clean and fresh, you really don’t have any of the constraints of the parent transaction, you can do whatever you need to i

Josh: Interestingly, the origins of this functionality goes all the way back to when Chris first started at the company. He needed better logging.

Chris: Everything about AppExchange is public API’s. It’s Apex, its Visualforce, the stuff you know and love that and there’s like three private API’s and like there around license management and a couple of other things like that. And then the one I added, which was logging so that I could write to the core application logs without accounting DML operation, very first thing I did, because I needed it for intensive workloads. And now that I’m in a position where I can solve the problem, not just for me selfishly, but for the entire customer base, I want to do extend that same capability of robust logging that doesn’t kill your DML elements as something that was more freeform and could be used for other things, but still gave you that capability because it’s something that I know everybody needs at the end of the day.

Josh: One thing to note about Apex finalizer is is that they are still in a pilot phase, but we are trying something a little novel with that pilot, although it’s a different approach for a pilot.

Chris: So what we did is we made it an open pilot using something that I don’t believe has been done before, but it’s a pilot that contains no legal paperwork and no nomination process. And so how it works is it’s only available in scratch orgs, which means we can make breaking changes the API and not break your org because we know it’s not in production. So one of the key things about that is we can actually react to your feedback with API changes, which was something we wanted to make sure that we could do, because again, I can’t imagine that I’ll think of every finalizer use case out there. I’m sure somebody can come up with some feedback that isn’t just a feature request, but kind of a foundational aspect of it being modified slightly. And we wanted to keep the door open for that. Because what’s the point of a pilot when you don’t have user feedback coming in? That’s actionable.

Josh: Well, that all boils down to the fact that if you want to start tinkering around with Apex finalzers, you’re going to want to Spring 20 hub org, you’re going to want to start spitting up some scratch org and then kick the tires of it and definitely get Chris and team your feedback. Now moving on to our next topic, which is a pair of features. We have some security enhancements when it comes to Spring 20 and Apex. And the first one is an update to SOQL which is getting a WITH SECURITY ENFORCED clause.

Chris: So it fully enforces feel that object level security for everything returned from the software. And I’ll put the emphasis on the word returned, because by design, it doesn’t enforce security on the were an ORDER BY clause. We did that for a couple of reasons. One is because this is the first in a series of features, so we at some point are going to do true user mode queries where it’s not executed in system mode, like apex. But there’s a couple of things that have to get done first, one of those is critical update we put out in this release called us improve schema caching, which solves some internal caching issues that made it so that switching between user and system mode could do strange things. And I’m not going to get into it here because they’re sufficiently obscure that they aren’t interesting. They’re just annoying.

Josh: Now, ladies and gentlemen, one thing I have learned over the years of working with engineering teams and product teams and product managers is when they tell you that something is just plain annoying, and not all that interesting – believe them. In this case, there is a lot of stuff that has been going on with platform caching and things that work behind the scenes that Chris’s engineers have been spending a lot of time fixing.

Chris: And so this critical update is going to be great because it’s setting us up for user mode. Now, in the meantime, things like the with security clause to cycle, they’re going to help streamline security, while also setting us up for success once true user mode is actually in place. Now one of the things that went into the design of the with security clause is that the security clause is going to affect the select statement. It doesn’t affect things like the where statement, and that’s by design. And when we looked into how people were writing queries, a lot of the time they were excluding data with their WHERE clause that they’d want excluded whether or not the user could see the field. So hide everything that has the is confidential checkbox set, but you don’t necessarily want to show these confidential checkbox to your users. And so we tried to build something that would be a drop in replacement for all the field and object level checks in the from and the SELECT clause but not necessarily constrained. What you can do in the where clause because be prescriptive there was a little bit dangerous, we could make it gracefully degrade and just pretend that those clauses weren’t there if they referenced fields you didn’t have access to. But that gets really dangerous for the use case of excluding data.

Josh: Also not covered with the with security clause is the order by statement. And again, that’s by design. We don’t want to disrupt your ad hoc security system. But Chris notes that, you know, you should think about the fact that this kind of stuff is still visible, you might expose hints, you know, if you order by an opportunity value, you need to be careful about that.

Chris: Make sure that the user actually is allowed to know the relative order of opportunity value, but that’s a whole different category and a much lower severity. And also something that doesn’t change radically when you’re doing dynamic SOQL. Usually not dramatically adjusting the where clause, like you might be with the SELECT clause. So that was the design philosophy there. And I talked about it more on release readiness live, and I think the developer docs are really good for it, but I’m sure Josh will put my Twitter handle in the podcast notes.

Josh: If you’d like to reach out with questions, you better believe that we will be linking to Chris’s Twitter. account and probably a few other things related to Chris in the episode in theshow notes. And we will also be linking to the release readiness live video if you would like to catch up with that. Now one of the big goals of putting in a clause like this is to reduce the amount of code required that you would need right now in order to try to duplicate that functionality, which if you go back to that video for release readiness live, you’ll see Chris’s pointing to like six or seven lines of code just for described functionality to try to duplicate the limitation of data that’s going back to the user. Similarly to this, we also have a new feature in a Apex called strip inaccessible, which is doing something kind of similar for DML.

Chris: stripInAccessable is the Swiss Army Knife security method that I always wished existed and myself and a number of other people have tried to implement and past lives and done mostly well enough but usually with an edge case somewhere not working right or performance being suboptimal. So what it does is it takes an arbitrary list of sobjects in memory. It doesn’t care how you got them, they could be coming from a query, they could be coming from JSON that was passed when Apex REST service, you could have constructed the manually in Apex, it doesn’t care you have a list of sobjects, however they got there, you tell it what type of access check you would like to perform. So accessible, creatable updatable observable, and it takes that input looks at every field that’s populated on every one of those s objects, including sub queries, and including cross object fields. And then make sure that the running user has access to it for the defined operation. And if they don’t, the result is a cloned set of objects with all of those fields that they don’t have access to stripped out. And so the literal only constraint on it is it doesn’t support aggregate result because aggregate result doesn’t track the aggregation fields back to their original source field like that fidelity is lost once an aggregate result is generated by the query so there’s no way to tie it back to a concrete field

Josh: Chris does a demonstration of this work where he’s got a lightning web components, and we’re sending JSON up to the apex controller, and that Jason is describing an account with custom fields. And within the Apex class, they’re checking for all the fields that you know, should be coming from the form. But if there is a field that, for instance, in the runtime of JavaScript, you just add a new field to the JSON before it goes up to the Apex controller, and the programmer couldn’t have known that that field was going to be added, you can have six, seven lines of field level security checks, but you’re not going to know necessarily which fields to be looking for. And so once again, Chris is trying to kind of create a Swiss Army knife here, which gives you a one stop shop to just make sure that you can stop that from happening.

Chris: And that if the users not supposed to have access to the field, or whatever level of you know, create edit access, or if you do want to fine tune it, you still have that ability, you may not be able to create a case as the site guest user, but if you go through defines knowledge wizard, we may want to actually create a case on your behalf at the end of that process. So that type of thing was something that we needed to not make harder and ideally treated as a first class use case. And so I think that’s what we did with stripInaccessible is if you want to gracefully degrade, great, if you want to just tell if there’s any access check violations, it does that too. And it also gives you back a little bit of debug information about what fields and objects were stripped off the input. So you can actually do system debug rights so that if you’re debugging production issues around permissions, you can see what stripping accessible is catching.

Josh: With every new release and with new features it’s really needs to be in a developer’s mindset. What do I do with these new shiny toys? So I apply it to my old code. And there’s an old adage, if it’s not broken, don’t fix it. But we’re talking security here. And when discussing this with Chris, his answer was fairly nuanced.

Chris: So with security enforced, if you already have a framework, if you’re using like the FF led framework, I think ff-lib has DML manager as API I actually took over as a maintainer for because maintainer left, if you’re using a framework, or you’ve written your own, or you have some sort of system that works well for you, and is provably robust, I don’t think that there’s a material value add in switching from that system to with security enforced in your legacy code, it’s ultimately going to be doing the same thing, does it make a nice opportunistic refactor and next time you’re in there and are going to be retesting it? Absolutely less code to maintain is less code to maintain. But if you have a solution that works for you, I don’t think there’s a compelling reason to go and rewrite it today. For stripping accessible though, I think the number of edge cases and different attack models that it covers is sufficiently novel that at least in the case of Apex rest controllers, LWC, and aura controllers, and JavaScript promoting and visual force controllers. You should take a very serious look at retrofitting it right away. Because the deserialization attack I demo on release readiness is one that well possible to defend against historically isn’t something that I’ve seen a lot of awareness around and a lot of effective countermeasures against, and so any place That you have a potentially malicious client sending you data. So an attacker can use pearl to send deliberately malformed data or Apex REST services. They can use burp suite to intercept the communications between your browser and the apex controller in LWC. And or any place that isn’t like traditional Visualforce, where you have you state with encryption, that makes it hard to tamper with is definitely a place where you should do a serious audit for deserialization based attacks and probably retrofit because at the end of the day, even if you are defending against this model effectively today stripInaccessible is probably going to do it with less CPU time.

Josh: Before we go Chris also wanted to give a shout out to a blog post that he’s got describing some new features and functionality when it comes to described as the complexity of the Salesforce schema has grown.

Chris: We’ve seen a handful of complaints around Apex described performance and so it’s something that we’ve started working with the an internal team called schema services the actually saved people behind entity interfaces because they own the underlying describe API’s, internally that we expose through effects but I did just put out a blog post that talks about some of the things that the apex team did, to try and defer calculation of the heavier elements of describes. So I’ll have Josh put a link to that blog post in the description here. But especially for ISP partners, but anybody who’s had CPU performance issues with describes, it’s going to be really relevant to and some of the ways of opting into the performance benefits might not be as intuitive as you would expect. So take a look at that if you’ve ever had complaints from me about the scribe performance, because there are some actionable things in there. And the other thing that’s really interesting is there’s a whole feature called the metadata catalog that is a s object based API for inspecting different aspects of your schema. And I didn’t realize this until I talked to the team that owns it. And I got a tweet pointing out that nobody else did either. If you ever tried to figure out what the sharing model for an object is, that wasn’t ever in Apex describes, but it is in the metadata catalog, and it even breaks down internal versus external sharing models. So the full granularity that you get on that sharing settings page that you’ve probably wanted to get that programmatically before is in that metadata catalog.

Josh: So both performance improvements for described that may be actionable, and a new thing that you probably didn’t know about for inspecting the schema, that is definitely worth taking a look at. Finally, I did ask Chris about one of his favorite non technical hobbies and as something that’s becoming a trend on the podcast, it is about food.

ChrisThe previous owner was nice enough to leave behind a Wi Fi enabled pellet smoker because they were moving to a different continent and didn’t want to take it with them. And also I live near Costco. So I’ve taken up smoking pork ribs quite a bit since I have to put it to good use. And that’s become my go to way of bribing people to do things for me. So I just had my dad come over and help wall mounted TV, because I’m terrible at stud finding, in exchange for some ribs.

Josh: And that is our show. Thank you for listening and a huge thanks to Chris for all of the hard work on the Apex features coming into Spring ‘20. If you want to learn more about this podcast, head on over to developer.salesforce.com/podcast where you can hear old episodes, and also links to your favorite podcast subscriptions. Thanks again and I’ll talk to you next week.

 Get notified of new episodes with the new Salesforce Developers Slack app.