# Large-Sized Partners

Large partners have large subscriber bases and many managed packages. A large partner’s total daily data usage is more than 20 GB. Sometimes a large partner’s data from just one managed package is larger than the 20-GB daily limit. Also, large partners often must create a smaller time range for each query to complete in under the 15-minute processing time limit.

| Required Editions                                                                        |
| ---------------------------------------------------------------------------------------- |
| Available in: both Salesforce Classic and Lightning Experience                           |
| Available in: **Enterprise**, **Performance**, **Unlimited**, and **Developer** Editions |

Large partners frequently create one query per managed package per 12, 6, or 1-hour increments throughout a 24-hour period. How frequently you schedule your queries really depends on your data volume.

We recommend that you use a combination of queries and multiple catch-up queries for all data types

| Data Type               | How to Get Started                                                                                                                                                                                                                    | How to Schedule Catch-Up Queries                                                                                                                                                                                                                                                                                                                         |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Subscriber Snapshots    | One daily query per package.                                                                                                                                                                                                          | <ul><li>One daily query per package.</li><li>Set `AvailableSince` to the day and time your last regular query ran.</li><li>Set `StartTime` to 30 days ago.</li><li>Omit `EndTime`.</li><li>Each day advance `StartTime` and `AvailableSince` by 1 day.</li></ul>                                                                                         |
| Package Usage Summaries | One daily query per package.                                                                                                                                                                                                          | <ul><li>One daily catch-up query per package.</li><li>Set `AvailableSince` to the day and time your last regular query ran.</li><li>Set `StartTime` to the first of the previous month.</li><li>Omit `EndTime`.</li><li>Each day advance `AvailableSince` by 1 day.</li><li>Each month advance `StartTime` to the first of the previous month.</li></ul> |
| Package Usage Logs      | <ul><li>To retrieve all your data, create multiple segmented daily, automated App Analytics queries spread throughout the day.</li><li>Break up your requests by managed package and by time increments throughout the day.</li></ul> | Create two levels of catch-up queries per day. <ul><li>Create one catch-up query per package that sweeps data from 2 days ago.</li><li>Create a second catch-up query that sweeps data from 3 to 30 days ago.</li><li>Each day advance `StartTime`, `EndTime`, and `AvailableSince` by 1 day.</li></ul>                                                  |

## Example

Your customers use your package on all Salesforce instances around the world, and your managed packages produce significant amounts of data. You schedule queries to run at the same time, each covering a 12-hour period, and you create a layered catch-up query plan to capture data from all instances.

In this example, we show two of your dozens of managed packages.

- On March 31 at `18:00 UTC`, run your regular package usage log queries.

  Package 1

  ```nolang
  sf data create record data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-30T00:00:00
  EndTime=2021-03-30T12:00:00
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-30T12:00:00
  EndTime=2021-03-31T00:00:00
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  Package 2

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-30T00:00:00
  EndTime=2021-03-30T12:00:00
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-30T12:00:00
  EndTime=2021-03-31T00:00:00
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

- On April 1 at `18:00 UTC`, run your regular and catch-up package usage log queries.

  ![Package log file regular and catch-up queries timeline on April 1.](https://a.sfdcstatic.com/developer-website/sfdocs/platform-pkg-exchange/media/pkg2\_dev/app_analytics_catchup_large_today.png)

  A. Package Usage Log Regular Queries

  Package 1

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-31T00:00:00Z
  EndTime=2021-03-31T12:00:00Z
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-31T12:00:00Z
  EndTime=2021-04-01T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  Package 2

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-31T00:00:00Z
  EndTime=2021-03-31T12:00:00Z
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-31T12:00:00Z
  EndTime=2021-04-01T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  B. Package Usage Log 2 Days Ago Catch-Up Queries

  Package 1

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-30T00:00:00Z
  EndTime=2021-03-31T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy
  AvailableSince=2020-03-31T18:00:00Z"
  ```

  Package 2

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-30T00:00:00Z
  EndTime=2021-03-31T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy
  AvailableSince=2020-03-31T18:00:00Z"
  ```

  C. Package Usage Log From 3 to 30 Days Ago Catch-Up Queries

  Package 1

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-01T00:00:00Z
  EndTime=2021-03-30T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy
  AvailableSince=2020-03-31T18:00:00Z"
  ```

  Package 2

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-01T00:00:00Z
  EndTime=2021-03-30T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy
  AvailableSince=2020-03-31T18:00:00Z"
  ```

- On April 2 onwards, run your regular and your catch-up package usage log queries, advancing the dates by 1 day.

  ![Package log file regular and catch-up queries timeline on April 2.](https://a.sfdcstatic.com/developer-website/sfdocs/platform-pkg-exchange/media/pkg2\_dev/app_analytics_catchup_large_tomorrow.png)

  A. Package Usage Log Regular Queries

  Package 1

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-04-01T00:00:00Z
  EndTime=2021-04-01T12:00:00Z
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-04-01T12:00:00Z
  EndTime=2021-04-02T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  Package 2

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-04-01T00:00:00Z
  EndTime=2021-04-01T12:00:00Z
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-04-01T12:00:00Z
  EndTime=2021-04-02T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy"
  ```

  B. Package Usage Log 2 Days Ago Catch-Up Queries

  Package 1

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-31T00:00:00Z
  EndTime=2021-04-01T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy
  AvailableSince=2020-04-01T18:00:00Z”
  ```

  Package 2

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-31T00:00:00Z
  EndTime=2021-04-01T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy
  AvailableSince=2020-04-01T18:00:00Z"
  ```

  C. Package Usage Log From 3 to 30 Days Ago Catch-Up Queries

  Package 1

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-02T00:00:00Z
  EndTime=2021-03-31T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0336XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy
  AvailableSince=2020-04-01T18:00:00Z"
  ```

  Package 2

  ```nolang
  sf data create record
  --sobjecttype AppAnalyticsQueryRequest
  --values "StartTime=2021-03-02T00:00:00Z
  EndTime=2021-03-31T00:00:00Z
  DataType=PackageUsageLog
  PackageIds=0337XXXXXXXXXX
  FileType=parquet
  FileCompression=snappy
  AvailableSince=2020-04-01T18:00:00Z"
  ```