The Spring '11 release introduced bulk queries as a pilot* feature of the Force.com Bulk API 21.0. With this feature, you can submit SOQL queries to Salesforce, run them asynchronously, and then download all your results once the query is completed.
This is a huge win for any developers working with long-running queries or queries with large result sets. You no longer have to leave a connection open while the query is being executed or call queryMore() a zillion times to retrieve the full result set! You can even monitor the progress of the query in Salesforce and download the results in your browser. As a bonus, both CSV and XML formats are supported.
How do you get your hands on this? *Since this feature is still in pilot, first you will need to contact your Salesforce representative to have it enabled for your organization. Next, you have a few choices depending on your needs:
- Workbench: a web-based suite of API tools for admins and developers, which includes Bulk API Query support out of the box along with a ton of other utilities.
http://wiki.apexdevnet.com/index.php/Workbench - PHP Bulk API Client: the standalone version of the same Bulk API client used in Workbench, but designed to be included in your own PHP applications. This includes sample code and examples.
http://wiki.developerforce.com/index.php/Workbench#Standalone_PHP_Bulk_API_Client - Roll your own: the Bulk API has an easy REST-based interface that you can call directly from cURL or using other REST clients. See the Bulk API documentation for details.
http://www.salesforce.com/us/developer/docs/api_asynchpre/index.htm