Raj Advani is the Senior Director of Product Management here at Salesforce. Raj has held a variety of different roles at the company, beginning with an Engagement Manager when he started 17 years ago. He now owns many of the event driven architectures for Salesforce, including Change Data Capture and Platform Events. 

In this episode, we have a conversation with Raj about various integration solutions on the platform. We discuss what Change Data Capture and Platform Events are, the advantages of a Pub/Sub API, and more!

Show Highlights:

  • Some of the internal features using Platform Events.
  • Moving toward a bus centric world in Platform Events.
  • The filtering conditions in Change Data Capture.
  • The benefits of event driven architecture.
  • The options for a distinct integrations from Salesforce.
  • How Pub/Sub API plays into the event architecture.

Links:

Episode Transcript

Raj Advani:
I’m fascinated by how they work, how to program in them. I started as a developer back way back in the day, even before with going into college, and then just worked my way through until I reached to Salesforce.

Josh Birk:
That is Raj Advani, Senior Director of Product Management here at Salesforce. I’m Josh Birk, your host with the Salesforce Developer Podcast, and here on the podcast you’ll hear stories and insights from developers, for developers. Today, we’re going to sit down and talk with Raj about various Salesforce integration solutions that we have on the platform, but before we get started, we’re going to pick up right where we left off about how he got into Salesforce.

Raj Advani:
Oh, that’s a great question. So I was working for a company that got acquired by Dell, and my manager came over to Salesforce and called me the next day and said, “Hey, why don’t you come over to Salesforce?” And I had knew nothing about the company. This was in the infancy of cloud, and it was very interesting to come over. So I really just came over based on I was following my manager, which I think a lot of employees end up doing these days.

Josh Birk:
Yes, indeed. I have multiple people who have been my manager multiple times. It’s kind of embarrassing.

Raj Advani:
Right.

Josh Birk:
So follow up question. When you joined Salesforce, did you know what CRM stood for?

Raj Advani:
I did not. I learned all of that maybe-

Josh Birk:
So common.

Raj Advani:
… two days, I learned all of that maybe two days before my first interview at Salesforce.

Josh Birk:
Yeah, yeah. My first interview in this ecosystem with my good friend Reid Carlberg, also multiple times my manager, and he was like … Well, the question was something like, “Well, what do you think about CRM solutions?” And I’m like, “Back up, buddy. I have no idea what you’re talking about right now.”

Raj Advani:
And so you know at Salesforce, at least back in the day, they sent you through a two week bootcamp.

Josh Birk:
Right. Right.

Raj Advani:
The first week bootcamp was more on the product as a whole, and then the second week for the technical folks was a little bit more technical. And I remember sitting in the second week of bootcamp, because I was coming as a developer and now I was going to be a consultant, and thinking to myself, “This really just looks like a UI on top of the database.” Little did I know how deep it got. When I walked into that class, that’s the first thing I remember thinking.

Josh Birk:
And I think that’s part of the magic of it is, and like back when I did workshops, people would be like, “Well, what stack does this run on?” And I’m like, “My answer to you is that it’s Oracle and Java, but it’s not recognizable.” If you looked at what’s actually making this thing run, it’s not being done like any normal relational database that you’re thinking about. And I think once people understand the power of the multi-tenancy and the metadata structure, that really kind of drives home the charm of Salesforce.

Raj Advani:
Correct. And especially back in the day, right? That took me the longest time to wrap my own head around on the fact that, who cares what the stack is running on that? That’s the great part is the customer doesn’t need to care anymore.

Josh Birk:
Yeah, exactly. Speaking of the old days, exactly how long have you been at Salesforce?

Raj Advani:
So I am coming up on, in August it’ll be 17 years.

Josh Birk:
Wow. So you’ve been with the company like, oh, about three-fourths of its lifespan. That’s pretty awesome.

Raj Advani:
Yeah, it’s been … It’s home. It really feels like home. I tell some of my other friends who are also in the tech industry that, at this point, I’m part of the furniture, so I’m not going anywhere.

Josh Birk:
Exactly. How many hats have you worn over that time?

Raj Advani:
Oh, that’s another interesting story. So I’m, Josh, I’m one of the type of person that I like to learn, I continuously like to grow. I’m not as motivated by titles as I am about really learning more and more, so I’ve worn maybe seven or eight hats here at Salesforce.

Josh Birk:
Gotcha.

Raj Advani:
I started as an engagement manager, then I became a technical architect, then I became a program architect, then I became a CSM, and then moved over to product, because my first life at Salesforce was always in services and I was only helping one customer at a time, and product’s then what felt like a natural progression, I can help more than one customer at a time with what I was working on.

Josh Birk:
Nice. How would you describe your current job?

Raj Advani:
My current job is probably my most ideal job that I’ve had at Salesforce.

Josh Birk:
Nice.

Raj Advani:
I am now the product manager of a couple of very interesting products here at Salesforce. I own Change Data Capture and Platform Events. Those are the two main products I own. There’s also a bunch of sub-products under that, Push Topics, Streaming API, Outbound Messaging and so on, but I own that whole space, so a lot of the event-driven architecture space for Salesforce is what I own these days. And why I say it’s my ideal job is because I’m one of the few product managers here that I have a tremendous install base of external customers as well as internal customers. You’d be surprised how many other internal features use Platform Events and Change Data Capture under the covers.

Josh Birk:
Okay. So, that’s fascinating. I found out we did that kind of stuff early on in my consultancy career. I found out there was like a secret API for enabling the mobile interfaces, and my favorite was the years that we spent testing the tooling API by basically running developer console on it.

Raj Advani:
Right.

Josh Birk:
Are there other examples you can say publicly that we’re doing that kind of thing?

Raj Advani:
Absolutely. So Einstein uses Platform Events behind the scenes to help generate some of their inquiries. They use Platform Events as their messaging service. Change Data Capture is used in many areas. In fact, one of the areas that it’s going to be used in is this upcoming realtime or low latency sync for core tenants with Genie. A lot of that is built as Change Data Capture as the backbone, which is what my team is working on now, which is a fantastic merger, if you’d say, of two teams. So I have the Genie team, who’s acting as my customer, telling me what they need. They need to be able to get data out of Salesforce and into Genie in a real-time manner. Well, instead of building a new bridge, why not use Change Data Capture? That’s what it’s there for. So we’re actually using Change Data Capture to be able to sync data from Core into Genie.

Josh Birk:
So I think that is kind of maybe the answer to some of the core of the next questions I was going to ask, because I’m somebody who evangelized the Streaming API a bunch, and it feels like we’re moving more into a, and tell me if I’m characterizing this wrong, like a more Bus-centric world around Platform Events and CDC, and my question was kind of like why and what are the advantages of, but I feel like you kind of just answered that. Would the old Streaming API be able to withhold the customer demands of people like Genie?

Raj Advani:
No. I think Change Data Capture and Platform Events really are the more scalable features or products built on top of Streaming API, and that’s the way I would characterize it.
But going a little deeper into your question, when you talk about that we’re moving more towards a Bus-centric, that is definitely the case both with our internal products as well as our external products and what we are telling customers. When I started 16 years ago, Salesforce was so new that when customers would build an integration, they would just build a bridge. I always put it back into layman’s terms of I’m building a bridge from Salesforce to your third-party system. Well then as that customer’s ecosystem grew, they ended up with hundreds of bridges. And now I go back to those same customers and they say, “Hey, Raj, I don’t have the budget to maintain these thousands of bridges, all this infrastructure. I need something better.” And that’s where I come in now with the Bus.
So instead of building all these bridges, you’re just building these connectors to the Bus, but everything is going through that one main artery, and now you just have these little capillaries, if you’d like to call them, that are offloading that data where you need it to go, and it’s a far more robust and scalable system, especially for those customers that have been with us for a while.

Josh Birk:
Right. People used to ask me like, “Do you integrate with X, Y and Z?” And my answer was like, “Well, can they use a REST API? Then the answer is yes, because we’re an API focused company.” But the move from an API focus to a Bus-centric focus allows the developers to … Again, tell me if I’m mis-characterizing this. It allows developers to respond to events as they’re having to build those discrete integrations. Is that about right?

Raj Advani:
Absolutely correct. It allows developers to develop. Instead of having to fix and maintain something that they’ve already developed, it allows them to go on and develop what they need to and not worry about what they’ve built in the past. The Event Bus takes care of it for them.

Josh Birk:
Okay. And I feel like we’ve defined, Platform Events and Event Bus, I think we’re defining that pretty well. Change Data Capture Events, like the name kind of describes it, but tell me exactly what the Change Data Capture Event Relay, I believe that’s the full title, tell me exactly what that does.

Raj Advani:
Sure. So let’s talk back in its infancy. We started with Platform Events, and Platform Events is, as it says, right? When you do something in Salesforce and it meets a certain criteria, you can create this event, and this event can be whatever you want it, the payload of that event can be whatever you want it to be.
Well, we noticed a lot of our customers were using Platform Events to build Change Data Capture, meaning they were using Salesforce as their system of record, and when data changed in Salesforce, they needed to make sure their downstream systems got that change and were in sync, the data was in sync with Salesforce. And we noticed so many customers were using Platform Events to build Change Data Capture, we decided why not make it easier for them and just build it for them themselves? So we built Change Data Capture, which sits on top of Platform Events.

Josh Birk:
Got it.

Raj Advani:
And what Change Data Capture does is exact exactly what the name says. When you make a change in Salesforce, in any object, it tracks that change, and then puts that change into an event or a payload, and then puts it on the Event Bus. And then all your downstream systems can listen to the Event Bus and capture that change. So, if you change an address in Salesforce, and you may have an order management system or a fulfillment system that needs to get that address change, it’ll just be listening to the Event Bus, get an event that the address is changed in Salesforce, and then change in those downstream systems.

Josh Birk:
Got it. How discreet and fine-tuned can we get with this? What are some of the filtering conditions that say this is a change in my data that is worthy enough for my downstream systems to know about it?

Raj Advani:
So the beauty of Change Data Capture is we capture every change, so there’s no setup saying I only want to capture this field or this field. We capture it all and send it all.

Josh Birk:
Okay.

Raj Advani:
Now, when your third-party system is listening, your client is listening for it, we let the customer at the client side decide what they want to do with it. If they only care about field A, B, and C, that’s great. If they care about all the fields, that’s great. We give them everything, and we give the customer the flexibility to decide what fields they want. There’s no extra cost based on the number of fields being sent.

Josh Birk:
Got it. And so I think that gets to, when we start looking at different use cases, there’s some commonality, like near real time frequency is almost always something people are asking for, if not real time. And then the capacity to work with different vendors and different parties and different systems, it seems like a lot of the changes and use cases really kind of come to breadth and width, and so like if I have a complex robust data structure on Salesforce, I’m using Salesforce as my source of truth, how is it … I think you’ve just kind of help describe it, but walk me through a good integration system for that that’s using CDC so that if I have 20, 30 different vendors that are listening to me, they can all get the right solution done.

Raj Advani:
Right. So that’s the beauty of the event-driven architecture. Let’s say you have 25 systems that need the same data that Salesforce has and needs to stay in sync with Salesforce. Before, as we talked about, you’d have to build 25 individual bridges. Now, you have 25 systems each having their own client, which think of it just as a listener, each having a listener that’s connected to the Event Bus listening for the events, and then they decide, do I need to act on this event or not?
And so when you make a change in Salesforce, I change the address of customer ACME, that change is created as an event, as a payload, put onto the Event Bus. All those systems can listen. They get that change and then they decide, “Oh, I don’t care. ACME doesn’t apply to me, so I don’t need to do anything.” Or systems one, two, and three may say, “ACME applies to me, let me go ahead and change their address.”

Josh Birk:
Got it. Got it. How does that compare, like if I need to do a more discreet integration from Salesforce out, what are some of my options there?

Raj Advani:
Great. So Change Data Capture is really focused more towards all data. Then you have Platform Events, which is really more system-to-system communication. I wouldn’t say it’s really data synchronization. Again, you can make it data synchronization, but Platform Events is far more flexible in the fact that it’s any way you want two systems to communicate.
And so you would set up your event to trigger based on any condition that occurs in Salesforce. You may change a field, you may change an opportunity to close one, and then you have control of what you want to put into that payload. I can put certain fields into their payload. I can put certain messages into the payload. And then again, very similar to Change Data Capture, that would go on to the Event Bus. The listeners would be listening. But the difference here is, once those listeners get that payload, it necessarily might not be for data synchronization, it might just be, oh, those systems now need to run their own set of processes. They just need to be notified that this triggering event happened in Salesforce.

Josh Birk:
So we have kind of two layers here. We’ve got CDC, which is sort of the constant clarion of this is the current state of the Salesforce data, and then we can dial that down to a more discreet action, and in Apex or in Flow, construct a Platform Event, which is what I think certain third-party agencies might need to know about in the format that I want them to see it.

Raj Advani:
Exactly. That’s exactly the best way to characterize it. And then the fact that you can do it through Apex or Flow, right? We’re trying to hit both our customer segments, both the pro-code developers as well as our citizen admins or citizen developers, and we want to make sure … One of my mantras and platform is I want to make sure the platform is available to everybody, not just our hardcore Apex developers. I want to make sure the power of the platform can be used by our citizen developers.

Josh Birk:
Well, and that’s nice because it’s like there’s nothing … I’m a huge Apex fan, don’t get me wrong, but there’s nothing specialized about Apex for creating something like a Platform Event because a Platform Event is basically just another data object. And so, on almost a technical level, it’s like saying, “Well, Flow wouldn’t be able to create a contact.” Well, of course Flow would be able to create a contact, so why shouldn’t it be able to create something like a Platform Event?

Raj Advani:
Absolutely. That’s how I think of it as well. A Platform Event to me is nothing more than another object.

Josh Birk:
Yeah. Now, there’s one we have not talked about so far, which is relatively new, the Pub/Sub API, how does that play into this architecture?

Raj Advani:
So the Pub/Sub API, and I get this question from our solution engineers still quite frequently, and I think we need to do a better job of messaging it internally as well, is when I talk about those listeners that I’ve been talking about, these third-party systems will have these listeners, that’s based on this CometD protocol, and CometD is a transport mechanism for events on how events get to these third-party systems. CometD is, I don’t want to say outdated, but-

Josh Birk:
Well …

Raj Advani:
… it’s been around for a long time, and there are some new and fancy ways of now getting events to their destination, as well as interacting with the Event Bus. So publishing events onto the Event Bus, as well as subscribing to the events, and that’s where Pub/Sub API comes into play.
So Pub/Sub API is our new foray into that space where you can use the Pub/Sub API to interact with the event and to interact with both Change Data Capture and Platform Events. So, they go hand-in-hand. You would use the Pub/Sub API to publish and subscribe events, but the mechanism of those events would be created by either Change Data Capture or Platform Events.

Josh Birk:
Yeah. I mean, for those listening at home, when we say CometD is an old protocol, I commonly used it with a jQuery plugin, so that’s dating. It’s all pretty well I think.

Raj Advani:
Right.

Josh Birk:
Yeah.

Raj Advani:
But one of the things I want to call out, because I know as soon as this podcast hits, I’m going to have customers tweeting me and so on, “CometD is not going away, is it?”

Josh Birk:
Right.

Raj Advani:
CometD is not going away. Like I mentioned, I have a huge install base of customers, over a hundred thousand orgs that are using Platform Events and Change Data Capture today on CometD. It’s not going away. Just like we do everything at Salesforce, we want to give you multiple tools to solve a problem and then let you decide based on your expertise and what you’re comfortable with which tool meets your need.

Josh Birk:
Yeah. CometD still works. It’s never going to stop working, and there wouldn’t be any reason for us to turn off that faucet. I have an episode on Pub/Sub, but can you kind of give me a high level description of some of the advantages of Pub/Sub over CometD?

Raj Advani:
Yeah. I think some of the advantages are, first, it’s built on gRPC, so it’s a little bit more of a newer medium. You get better control with flow, not Flow the product, flow control. So right now when CometD, if you publish 10,000 events, you don’t have a way of controlling how many events you want to go to the client at one time. It’s a lot more cumbersome with the batches, how many batches events go to a client at a time. You get better flow control with Pub/Sub API. You also have, it’s a easier interact when you want to publish an event onto Salesforce. So if you want to take a third-party system and put an event onto the Event Bus, you have a much easier time doing that with Pub/Sub API than you would with CometD.

Josh Birk:
Nice. So we’ve talked about a couple of use cases. One we haven’t talked about is data virtualization, where I can work with data that’s off platform, but it’s like on platform. What kind of solutions do we have like that at Salesforce?

Raj Advani:
So the big one we have there, that’s another fantastic question, is the use of Salesforce Connect, right? That’s that data virtualization, when you want to use external data within Salesforce but without having to copy it onto the platform. And so it’s essentially like a window into a third-party system, where you’ll be able to see the data, but it’s not residing within Salesforce.
And so I mean, Salesforce Connect would be the way to go there. It’s fantastic if you have, as I mentioned in the past, order fulfillment data that’s stored outside of Salesforce. The goal really of Salesforce Connect is really to have … It’s like a two-prong goal. You want to avoid the swivel chair, so you want to avoid customers having to go through two different systems. They can at least look at the data within multiple systems, or from multiple systems within Salesforce.
So the key there is we want to keep eyes on Salesforce and eyes in Salesforce, and also it’s fantastic because there’s a lot of companies out there, especially with security concerns, they don’t want copies of their data floating around all over the place, and this way you can have your data securely stored in one location and just visible through Salesforce.

Josh Birk:
Yeah. Now we’ve been talking at very high levels because it would probably be about a four-hour episode if we started to try to get into custom solutions for custom problem sort of things. What are some resources you recommend for somebody who might be listening to this and they’re like, “Hey, I’ve got a upcoming integration, I want to make sure I’m using latest and greatest with Salesforce”?

Raj Advani:
So there’s a couple of fantastic integration guides out there that myself and my team and other PMs spent considerable amount of hours putting together, and I don’t know how to get the links out to your group, but if they were to search for data integration guide on Salesforce, you’d get a fantastic guide. It’s kind of like a flow chart that goes through helping you make this decision tree.

Josh Birk:
Got it.

Raj Advani:
These are my requirements, this is what I’m looking to do, I need the data stored here, or I don’t mind multiple copies of the data, and kind of walks you through every single option that we have for integration.
Now, if you are more focused on event-driven integrations, which I really hope most folks who are listening to this are, search for event-driven architecture guide, and that’ll give you a more in-depth version of the first guide that I talked about, but all of the different ways you can do all the integrations at Salesforce, but going through the event-driven architecture paradigm.

Josh Birk:
Yeah. And tell me, Raj, do you have anything in the pipeline that people can get their hands on?

Raj Advani:
There’s going to be a fantastic pilot coming out for our Genie customers or anybody considering Genie. In the second half of this year, we’re going to have a great pilot I kind of talked about with that low latency data replication from Core to Genie. Keep an eye out for that. If you have any interest in that, you can tweet me, and I can get you added to that pilot. We’re definitely looking for more customers there. And if you have any other questions around event-driven architecture, Pub/Sub API, Platform Events, Change Data Capture, or Genie, please feel free to reach out to me.
One of the things I love about my job is being able to talk to customers, whether they’re existing customers, angry customers, or potential customers. Any customer is always somebody I’m willing to talk to and wanting to talk to, to understand how I can help them.

Josh Birk:
Awesome. Anything else on the roadmap that you want to give a shout-out to?

Raj Advani:
Both my teams are really focused on Genie right now. We have a couple of cool GAs coming out for Platform Events. One is published callbacks, which we’ve been demoing at TDX and Dreamforce for the past year. I’m really excited that it’s finally getting to GA. And the other one is a lot of my customers have had some pain points around understanding how they use Platform Events, so getting an idea of where they are with limits, how they’re using them, who are their top clients and top listeners, and that’s also coming into GA in the next release. So, it should be a win for both, for my customers there.

Josh Birk:
And that’s our show. Now, before we go, I did ask after Raj’s favorite non-technical hobby, and it turns out it’s something that’s a little more than a spectator sport for him.

Raj Advani:
So, I was born and raised in Canada, and I got three kids now, and they all play ice hockey.

Josh Birk:
Nice.

Raj Advani:
So I feel like my favorite hobby these days is coaching them, although they’re getting a little better than me and I’m getting older, so I don’t know how much longer that can go.

Josh Birk:
Got it.

Raj Advani:
But if I’m not at work, I’m usually usually at an ice rink, either watching them or on the ice participating with them.

Josh Birk:
Got it.

Raj Advani:
So that keeps me pretty busy, besides all of the Salesforce stuff.

Josh Birk:
I want to thank Raj for the great conversation and information, as always. I want to thank you for listening. Now, if you want to learn more about this show, head on over to developer.salesforce.com/podcast where you can hear old episodes, see the show notes, and have links to your favorite podcast service. Thanks again, everybody, and I’ll talk to you next week.

 Get notified of new episodes with the new Salesforce Developers Slack app.