Apex Code Development You need to sign in to do that
Don't have an account?
nwing Callout Limits within Batch Apex?
Does the below limit....
'A single Apex transaction can make a maximum of 10 callouts to an HTTP request or an API call.'
Apply within a batch transaction? For instance if I can write the batch to iterate on 10 records at a time, will I be able to get by this limit? Really unfortunate if not.
Thanks for any direction.
I just asked this the other day (and got the answer).
It's per each call of execute() in the batch, so for each "batch", you can call up to 10 HTTP callouts.
If the callout can handle multiple records at a time, do so, and batch up to 10 callouts per batch, otherwise, set the batch scope to 1, so each apex batch handles one record at a time.
http://blog.paulmcgurn.com/2010/08/creating-batch-apex-to-process-many.html
All Answers
I just asked this the other day (and got the answer).
It's per each call of execute() in the batch, so for each "batch", you can call up to 10 HTTP callouts.
If the callout can handle multiple records at a time, do so, and batch up to 10 callouts per batch, otherwise, set the batch scope to 1, so each apex batch handles one record at a time.
http://blog.paulmcgurn.com/2010/08/creating-batch-apex-to-process-many.html
So, if I understand correctly, each scheduled batch run can do at most 10 callouts ( my present callout code makes a callout for each record, and I believe that is the constraint I have, though I did not write the receiving end of it.).
Which means I will need to schedule the job repeatedly and NOT be able to have it run, for instance, one job a night and get everything?.... There may be a couple thousand records that need to have a field updated.
Since I already have the class written that does the callout (via a an after update trigger) can I just setup the scheduler to use that? I have not used batch apex yet so the complexities are as of yet unfamilier.
What is the basic setup to schedule a batch apex job to trigger a class, or can I even do it that way? Ideally I would like it to repeatedly run until the query returns no more records, with perhaps an overall limit of 100 runs or something in case there is a problem.
Not sure how to get started, so any additional help would be great.
a batch is a set of executions, where you pass in a subset of the overall batch, and execute code on it.
so in your instance, you could either:
1. kick off a batch to iterate through all records, one at a time (setting batch scope to 1)
2. kick off a batch to iterate through 10 records at a time, and in the execute method, you loop through those 10 records to do a callout for each one
both achieve the same thing, but #1 is cleaner, because if one api call fails, that single batch failed, vs. having 1 in 10 calls fail to fail the whole batch.
i hope that clears it up for you.
While going through the documentation, I noticed this comment.....
"The start, execute and finish methods can implement only one callout in each method."
This is even worse that the normal limit of 10. How do you get past this.......? It kind of negates the advantage of a 'batch'.
Oh,
so I could loop outside of the execute method? thereby doing however many records there happen to be with one scheduled batch?
the start() method gets your data set, and the execute method goes through it. if you set the scope for execute to 1, it'll just iterate through the whole set, one at a time. that's what we did.
I have a functional class that I used the apex scheduler to initiate, but I am getting the 'more than one callout' fail. If I post what I have would you be able to comment? I followed most of your example otherwise, but I am not entirely clear on how to put it all together, particularly with the apex code scheduler itself.
i wasn't aware of batch apex only allowing 1 callout per execution, but that's probably what you're hitting. post your batch apex code, and some more specifics around what each part of the start() and execute() methods do, and I'll see if I can get you going.
I know using batch apex to do one record at a time may seem inefficient (it is), but, it allows you to get your batch completely done, regardless of size, while adhering to SF's resource limits.
Here is what I have in the class. As I mentioned I am using the scheduler in the management interface to trigger it, but am guessing I will need to implement the coded scheduler instead. When I tried to write it to accomodate that scenario per your suggestions I got some save errors so I went this route to see what would happen. Obviously it is not quite working.
Thanks a ton.
I am doing some formatting on the response to accomodate the fact that the callout was originally designed as a backend for a visualforce display that was not originally used to update an actual field...... which is what I am doing now. I was guessing that I just needed to use the coded scheduler, but when I originally tried to set that up I had save issues, but if that is all that is left and I just need a little help to write that, much appreciated.
global class ClosestTowerBatch_cls implements Database.Batchable<Lead>, Database.AllowsCallouts{ Datetime myDate = datetime.newinstance(2010,1,1); Lead[] LeadsToUpdate = [SELECT id, lastname, street, city, state, postalcode, Closest_Tower__c, Closest_Tower_Distance__c FROM Lead WHERE Closest_Tower__c = Null AND street != null AND (NOT street LIKE 'PO%') AND city != null AND (state = 'AZ' OR state = 'NV') AND IsConverted = False AND LastModifiedDate >= :myDate limit 500]; global Iterable<Lead> start(database.batchablecontext BC){ return (LeadsToUpdate); } global void execute(Database.BatchableContext BC, List<Lead> scope){ String EmailStreets = ''; List<Lead> accsToUpdate = new List<Lead>(); for(Lead a : scope){ String str = a.street; String cit = a.city; String sta = a.state; String zip = a.postalcode; String lid = a.id; EmailStreets = EmailStreets + ' - ' + lid + ' - ' + str; if(str.startsWith('PO') == False) { str = str.trim(); str = str.replaceAll(' ',' '); str = str.replaceAll(' ','%20'); Integer spot = str.IndexOf('#'); if(spot != -1) { str = str.substring(0,spot); } else{} cit = cit.trim(); cit = cit.replaceAll(' ','%20'); // Instantiate a new http object Http h = new Http(); // Instantiate a new HTTP request, specify the method (GET) as well as the endpoint HttpRequest req = new HttpRequest(); req.setMethod('GET'); req.setEndpoint('https://secure.booboo.com/salesforce/webservice.php?command=get_pops_by_address&street=' + str + '&city=' + cit + '&state=' + sta + '&postcode=' + zip); String username = 'booboo'; String password = 'picinicbasket'; Blob headerValue = Blob.valueOf(username + ':' + password); String authorizationHeader = 'BASIC ' +EncodingUtil.base64Encode(headerValue); req.setHeader('Authorization', authorizationHeader); String TowerStuff; HttpResponse res; // Send the request, and return a response try{ //HttpResponse res = h.send(req); }catch(System.CalloutException e){ Messaging.SingleEmailMessage mail = new Messaging.SingleEmailMessage(); String[] toAddresses = new String[] {'booboo@basket.net'}; mail.setToAddresses(toAddresses); mail.setReplyTo('booboo@basket.net'); mail.setSenderDisplayName('Apex Code Callout'); mail.setSubject('Error'); mail.setBccSender(false); mail.setPlainTextBody(EmailStreets); Messaging.sendEmail(new Messaging.SingleEmailMessage[] { mail }); TowerStuff = 'OOOOO'; EmailStreets = ''; } if(TowerStuff != 'OOOOO'){ TowerStuff = res.getBody(); } String str2 = TowerStuff.trim(); Double NoMiles = 0; Integer spot2 = str2.IndexOf(' '); if(spot2 != -1) { Integer miles = str2.IndexOf('miles'); Integer colon = str2.IndexOf(':'); NoMiles = Double.valueOf(str2.substring(colon+2, miles-1)); str2 = str2.substring(0,spot2); } else{} a.Closest_Tower__c = str2; a.Closest_Tower_Distance__c = NoMiles; } accsToUpdate.add(a); }//for lead list loop update accsToUpdate; }//execute loop global void finish(Database.BatchableContext info){ }//global void finish loop }//global class loopwhat is the code you use to execute this batch? that's where you define the size of the recrd set you pass in to each execute() call.
In my sample (assuming yours is in your scheduler class, which is fine to do via the GUI)
the "1", is the record set size, so I'm issuing one callout per record, because I'm only passing 1 record to each call of execute(). It's still treated as a List<SObject>, but the list size is 1 (to adhere to the callout limit per execute.
That help?
the code posted is all I have put together at this point. So, I need to create two more classes? One for the scheduler and one for the call (which you posted below?)
The GUI scheduler only allows one execute per day. So if I need to do this several times ( I need to have it set to do at least a thousand or so records or else its not much use) I need to make the scheduler class, correct?..... which I believe is where I had the problem.
How many actual classes do you have setup?
start with getting the batch working, then move on to the Scheduler.
Try executing your batch via the Execute Anonymous part of Eclipse, or the System Log in the web interface, with the scope set to 1, like in my previous example. If that works, you can move on to putting that code into a class that implements Schedulable, and then finally, executing that in the Execute Anonymous or System Log to run it more often than once per day.
I have two that are specific to the batch apex and scheduler, one for each. The rest are the actually worker classes that do the heavy lifting.
One class implements Batchable (to do the batch apex), and the other implements Schedulable (to allow for the batch apex to be called on a schedule).
I then call the Schedulable class directly from Execute Anonymous in Eclipse, to put it on an hourly schedule. But like I said, start with getting the batch working. The rest depends on that.
By 'getting the batch working' you mean get the class that I have posted functional?
yes.
execute this in the system log or execute anonymous, and tell me if it processes the whole batch, vs. just the first record and then erroring.
ClosestTowerBatch_cls batchapex = new ClosestTowerBatch_cls(); id batchprocessid = Database.executebatch(batchapex,1); system.debug('Process ID: ' + batchprocessid);Notice i'm passing "1" as the scope size (second line, second param of executebatch)
wow.....
it is cranking through the 500. I did not think it woudl work like that..... meaning the scope setting actually controlling the list execution like that........ nice though....... even though I do not entirely understand it at this point, I will definately dig into that.
Its got 500 to go.....one about every 4 seconds it looks like.
think of it like this
start() in your batch apex class will gather all the records the batch (as a whole) is going to process.
executebatch() (in your manual execution, soon to be in your scheduler) takes two params, your batch apex, and scope. When you set scope to 1, you're telling execute() in your class itself to only take 1 object at a time from the start() collection that grabbed the whole record set.
It's a REALLY convoluted way to implement a batch processor, but it has to be done this way for SF's callout limitations. If your webservice could support more than one record at a time, you could pass in a bigger scope than 1, and modify the code inside execute() to not use a for loop, but that's out of my knowledge.
paul-Imi
Thank You VERY much for the direction. It is working as well as I could've hoped for and I do not think I could've figured it out from the documentation alone. Very interesting to say the least how it all works together.
Hi Paul thanks for the direction.
However I wonder does it means I can only process 24 callouts a day? Assuming that I can only process 1 callout per batch and schedule the batch hourly? Can I process about 500 callouts per day using batch apex? Please advise. Thanks :)
Regards,
dfebria
it's one callout per "chunk" of the batch, not the whole batch. so if the batch has 5000 records, and you set the scope at 1, it'd go through 5000 callouts.
I am still not clear how this should work, since there is a limit of 5 concurrent batch jobs and we create a batch job per record as you suggest in order to be under 1 callout limit.
Thanks
5 total batch jobs, not iterations in a single batch.
a batch can iterate through 50 million records, either in chunks (default is 200), or one at a time (if you set scope to = 1, which is what i outlined.
When you submit the batch job, the platform will pull a collection of the records involved in your query, and then iterate through that collection, processing records in groups the size you set the scope to. If scope is 1, it will do one at a time, and thus each iteration can get 1 callout. The documentation is flakey on these governor limits, but I can tell you for a fact that it works. We have Twitter, Youtube, and Lithium API callout batch jobs that run on an Apex Scheduler, which uses this implementation.
You're not creating 1 batch job per record, your creating one batch job of all the records you want to touch, and setting the scope of that job to be equal to 1, so the batch job will only process one record at a time.
Can you tell me why I'm still getting the error message about too many callouts.
global class FedExBatchPackageDelivered implements Database.Batchable, Database.AllowsCallouts{ List OppsToUpdate = [Select FedEx_Tracking_Number__c, Delivery_Status__c From Opportunity Where FedEx_Tracking_Number__c != null And Delivery_Status__c IN ('Package Received by Carrier', 'Label Printed')]; global Iterable start(database.batchablecontext BC){ return (OppsToUpdate); } global void execute(Database.BatchableContext BC, List scope){ List accsToUpdate = new List(); string TrackingNumber; for(Opportunity op: OppsToUpdate) { TrackingNumber = op.FedEx_Tracking_Number__c; Map<String, String> TrackingData = new Map<String, String>{'StatusCode' => '', 'ActualDeliveryTime' => '', 'EstimatedDeliveryTime' => ''}; Http h = new Http(); HttpRequest req = new HttpRequest(); req.setMethod('POST'); req.setHeader('content-type', 'image/gif'); req.setHeader('Content-Length','1024'); req.setHeader('Host','gatewaybetawsbeta.fedex.com'); req.setHeader('Connection','keep-alive'); req.setHeader('Port','443'); req.setEndpoint('https://wsbeta.fedex.com:443/web-services/track'); req.setBody('w3KHdCRwsUjM2jHHGycKFwUPjenJfmLOXPsPb8rS9510087682118509785*** Track Request v5 using APEX ***trck500' + trackingNumber +'TRACKING_NUMBER_OR_DOORTAG'); Http http = new Http(); HttpResponse res = h.send(req); XmlStreamReader reader = res.getXmlStreamReader(); TrackingData = getTrackingStatus(reader); accsToUpdate.add(setDeliveryPackageDelievered(op, TrackingData)); } update accsToUpdate; }@Hyzer: have you put the batch size to 1 as explained in the post?
1) If I make 3 separate http call-outs in the execute method with batch size set to 1, will that work ?
2) Can't I set the batch size to 10 with just 1 http call out in execute method? That will be 10 callouts per batch or just 1 callout?
Please clarify. Thanks much.
Cheers!