The Apex Interactive Debugger

You asked for it. We announced it. You waited for it. We delivered it. I am pleased to announce that the Apex Interactive Debugger is now generally available!

Sweet. What does it do?

The interactive debugger is an addition to our existing debugging suite. It does exactly what you’d expect an interactive debugging tool to do. It allows you to set breakpoints throughout your code, in the cloud, on our multitenant architecture. It stops requests at these breakpoints. When it stops, you can inspect the transaction state. You have full stack information. You have full variable information. You can control the transaction, stepping into and out, and running to the next breakpoint.

Didn’t you demo this a year ago? What took so long?

Despite what I said just now, this is not your average interactive debugging tool. Salesforce is a cloud-based multitenant system, which presents multiple challenges. The threads you want to inspect are stopped on a different computer than the one you are using to debug. Routing your subsequent step/run requests to the appropriate stopped thread, on the appropriate app server, is complicated. It’s complicated enough that we are the first to ever try such a thing! We also had to ensure service protection, meaning automated monitoring tools and “panic button” capability in case anything ever goes squirrely.

Thank you for keeping us safe. How do I use this tool?

Just use the standard debugging tools in the Eclipse IDE. We adhered to the “principle of least surprise,” so the buttons and commands in Eclipse will do the same thing they’d do if you were debugging a Java application locally. The step buttons step. The variable pane shows the variables. Double-clicking the gutter sends us breakpoints. If you have used a debugger before, it will feel very similar.

Eclipse provided us with a fully-featured debugging UI, which allowed us to focus on the underlying connectivity instead of the user experience. And we have a IDE plugin already, which you probably know about (and which you probably have strong opinions about (hopefully good (and if not (or if so), please join the open-source development project!))).

Will I be able to debug in other IDEs?

Yes, but not just yet. There are three parts to the debugger: the cloud-based multitenant routing system previously mentioned; the client application where you interact with and operate the debugging process (currently only Eclipse); and the API to pass information between that client and the application server where your thread is stopped. My friends who have built an IDE plug-in are frequently asking when we can make the API part public. (Note: building a IDE plug-in is an ideal way to become my friend. Highly recommended.) They will have access soon and will hopefully incorporate the interactive debugging capability to their IDE tools.

I love the Developer Console. Can I debug there?

Aww, shucks. *blush* Alas, you will not be able to use the interactive debugger in the Developer Console. That would require building out a brand-new UI, which we do not plan to do.

For now, the team is focused on finishing out the debugger functionality. (Click to see a live construction cam of the team building the debugger!) We are designing ways to set method breakpoints and exception breakpoints. We have a plan to offer statement evaluation (“eval”, to those in-the-know), which would enable conditional breakpoints.

We are also focused on capacity management. Capacity is the critical target for us, one which we’re constantly monitoring. We want as many of you to be able to use this as possible. We’ll be releasing a few new capabilities in the coming release to reduce unnecessary use, which should permit more legitimate use. As an example, you’ll be able to whitelist different users or entry points, so that breakpoints in common code are ignored when hit by other users or cron jobs you’re not trying to debug.

Tell me how this capacity management works….

One part of managing capacity is the fact that this is an add-on product that must be purchased, and there are a limited number of spaces we can sell.

I heard you were now charging for all debugging. That’s CRUEL!

It’s also FALSE.

We are only charging for the interactive debugger. Debug logs and the nifty Developer Console parsed log viewer, checkpoints, and heap dumps are still free as they have always been.

I’m a retired developer myself, so I know that efficiency and frugality are part of the mindset. I assure you, though, that you are very happy that we are charging for this. The alternative was the common Salesforce pattern in managing multitenant capacity: LIMITS.

Ah. Good point. I don’t know which I like less: limits, or paying for things.

We asked ourselves that question, too. We determined that limits would have made this particular feature difficult to use. None of the types of limits that work for the rest of our platform work with the particular usage patterns of an interactive debugger.

Typical transaction timeouts wouldn’t work. Imagine if you had two minutes for a stopped thread for debugging. You’d have a clock counting down in your head every time you used the debugger. You’d be rushing to get stuff done, and not free to sit and stare at the code and the transaction state and try to unwind how in the name of all things holy did sCountryCode get set to ‘Never’?!?

Typical usage counting, like API calls per day, also doesn’t work for debugging. You’d accidentally set a breakpoint in the wrong spot, and it would get hit by a bunch of threads, and you’d be out of “credits” for the day.

Due to the finite capacity, we’d need a queue to get going. When you stop at a breakpoint, you consume a thread and a database connection. Normal transactions also consume one of each, but they’re finished within milliseconds. Your debugging threads will live for several minutes, which means they have an outsized impact on capacity. In effect, stopping at a breakpoint reduces our service capacity by one thread/connection.

We can absorb some amount of this capacity reduction without impacting service quality. Once we’ve reached that amount, we can’t let any more debugging sessions in the door. So you’d be waiting for an opportunity rather than solving problems.

OK, I see that charging some amount keeps the service available.  How does it work?

The unit being sold is not per-user, and it’s not per-org. You can purchase debugger sessions, which are shared across all users and all sandboxes from your parent org.

If your organization has purchased debugging capability, all of the sandboxes spawned from it are enabled for debugging. The number of sessions you purchase represents the number of your sandboxes that can be debugging at the same time.

Think of this like a phone line in your house, from back when you would have had a phone line in your house. There were many telephones, and they could all make calls, assuming nobody was on the line. However, if Mom needed to call someone, she had to wait until you were done, because you were having a VERY IMPORTANT CONVERSATION and you would be off IN A MINUTE and STOP SHOUTING I’M ON THE PHONE and then you’d stretch the cord around the dresser and all the way into your closet so she’d stop bothering you and

What were we talking about?

Oh, right, sorry about that. The debugger sessions you purchase are like your phone line, which only one of you could use at one time. If it got ugly, you paid for a second phone line.

We are going to provide visibility in the parent org as to which sandbox(es) are engaged at any point in time, along with the user doing the debugging. This will allow the admin to contact them and ask them nicely to get off the phone. Er, debugger. We’ll also have a less-polite “Kill Session” button in case someone goes rogue.

Mom would have loved that button.

Yes, she would have. Fortunately for me, I wasn’t building technology back then, only abusing it.

What other capacity work are you’re doing?

(Warning: the following paragraphs may contain forward-looking statements. If there are small children in the room, you may want to ask them to leave, lest they make purchasing decisions based on anything but currently available software.)

I mentioned the white-list idea before. There will also be an “are you still there” pop-up if you are idle for a few minutes, similar to the one you get at your online banking website. This will let us terminate sessions for people who have stopped a thread and stopped paying attention, which will free up threads for you (because you’d never do such a thing!). We’re also tweaking our load-balancing algorithms to attempt to maximize how many threads we can stop at the same time.

If we get all this right, we ought to have capacity for every “serious” debugging session request. We won’t be able to scale it to the looky-loo use case, though, so we’ll always probably have a nominal charge to restrict use to those of you who need it. (If you’re reading this, you are in that group.)

You said “sandbox” a lot of times. What about debugging my production org?

Currently, interactive debugging is only available in sandbox orgs. This has to do with the number of threads taken out of the capacity pool that we can absorb without impacting the service. In sandbox pods, that number is sufficient to offer an interactive debugging service. In production pods, it’s pretty much zero.

We are working on a way to offer occasional debugging sessions in production, but we must ensure that such a thing will not impact production system operation.

How will I debug in my DE org?

You will not be able to debug in current DE orgs, since these are not on sandbox hardware. You’ll need to use the sandbox orgs that are a part of your company’s org (or your client’s org, if you’re doing project work).

Does this mean ISVs cannot use the debugger?

Not at all! ISVs do some of the most debug-worthy coding on the platform, so we made sure they could use the product.

Hey! What are you trying to say about ISV code?

I’m just saying it’s complex, that’s all! If an ISV purchases the debugger, they will get sandbox environments provisioned, where they can develop and debug and share sessions like customer orgs.

I’ve wanted a sandbox on my DE org for years, now I can finally get one?!

Yes, if you purchase the debugger.

How will ISVs debug their code in subscriber orgs?

Ah, you have observed that what I’ve described allows ISVs to debug their application in isolation, but not as a part of a subscriber org. Good catch.

(Warning: more forward-looking statements.)

We are going to allow debugging to occur in a way similar to the current “login-as” functionality. When subscribers debug, ISV-managed code and variables will be removed from the variables and stack information just like they have been in debug logs forever. ISVs can request permission to log in to the subscriber org, which will unblock the managed stack and variables when running the debugger. This is similar to how debug logs are made available today.

OK, so now I need to act fast and purchase the interactive debugger. 

That’s right. As I mentioned, there are limited licenses available, and when they are gone, you’ll have to wait in a queue until more become available. Contact your sales representative for information on how you can use the project. Act Now!

Remember: we can only support a finite number of sessions, and we’ll be selling them to the first people who buy them. Once we run out, we will queue up requests until we’re able to safely allow more customers to purchase the tool. You’ll end up like “Josh with Cable” instead of like “Josh with DirecTV”. Don’t be like that Josh.


tagged , , , , Bookmark the permalink. Trackbacks are closed, but you can post a comment.
  • MikeBZ

    The key question is always… how much is it?

  • ram

    Why can’t Salesforce just release a SDK like AWS ?

    • Ammar

      yes. I agree with you.
      since we have to pay for the Service, why not provide better tools for development and debugging.? and free

  • keylabs keyabs

    nice post thank you </a

  • Derek Marshburns

    Thanks a lot, this really is a truly awsome article! I think it could also be useful for everyone to know how and where to fill a form online. BTW, there is an online service through which you can fill out a a form, the fillable blank is here


  • Abhilash K

    As always great article by Josh.
    My question is, for such a basic, long awaited feature why would you charge? Don’t you love your developers?

  • Nilesh Dethe

    Josh, This is very nice and useful article. Being developer and decision maker this article answers all my questions. Thank You!!