Merge pull request #349 from ballerina-platform/csverror

Add support for pagination

README.md

@@ -114,7 +114,7 @@ bal run
 
 The `salesforce` connector provides practical examples illustrating usage in various scenarios. Explore these examples below, covering use cases like creating sObjects, retrieving records, and executing bulk operations.
 
-1. [Salesforce REST API use cases](https://github.com/ballerina-platform/module-ballerinax-sfdc/tree/master/examples/rest_api_usecases) - How to employ REST API of Salesforce to carryout varies tasks.
+1. [Salesforce REST API use cases](https://github.com/ballerina-platform/module-ballerinax-sfdc/tree/master/examples/rest_api_usecases) - How to employ REST API of Salesforce to carryout various tasks.
 
 2. [Salesforce Bulk API use cases](https://github.com/ballerina-platform/module-ballerinax-sfdc/tree/master/examples/bulk_api_usecases) - How to employ Bulk API of Salesforce to execute Bulk jobs.
 

ballerina/Ballerina.toml

@@ -2,7 +2,7 @@
 distribution = "2201.8.0"
 org = "ballerinax"
 name = "salesforce"
-version = "8.0.1"
+version = "8.0.2"
 export = ["salesforce", "salesforce.bulk", "salesforce.soap"]
 license= ["Apache-2.0"]
 authors = ["Ballerina"]
@@ -17,10 +17,10 @@ observabilityIncluded = true
 graalvmCompatible = true
 
 [[platform.java17.dependency]]
-path = "../native/build/libs/salesforce-native-8.0.1.jar"
+path = "../native/build/libs/salesforce-native-8.0.2-SNAPSHOT.jar"
 groupId = "io.ballerinax"
 artifactId = "salesforce"
-version = "8.0.1"
+version = "8.0.2-SNAPSHOT"
 
 [[platform.java17.dependency]]
 groupId = "com.opencsv"

ballerina/Dependencies.toml

@@ -377,7 +377,7 @@ modules = [
 [[package]]
 org = "ballerinax"
 name = "salesforce"
-version = "8.0.1"
+version = "8.0.2"
 dependencies = [
 	{org = "ballerina", name = "http"},
 	{org = "ballerina", name = "io"},

ballerina/Module.md

@@ -16,16 +16,16 @@ Ballerina Salesforce connector supports [Salesforce v59.0 REST API](https://deve
 
    <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/create-connected-apps.png alt="Create Connected Apps" style="border:1px solid #000000; width:50%">
 
-    - Here we will be using https://test.salesforce.com as we are using sandbox enviorenment. Users can use https://login.salesforce.com for normal usage.
+    - Here we will be using https://test.salesforce.com as we are using sandbox environment. Users can use https://login.salesforce.com for normal usage.
 
     <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/create_connected%20_app.png alt="Create Connected Apps" style="border:1px solid #000000; width:100%">
 
-4. After the creation user can get consumer key and secret through clicking on the `Manage Consume Details` button.
+4. After the creation user can get consumer key and secret through clicking on the `Manage Consumer Details` button.
 
    <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/crdentials.png alt="Consumer Secrets" style="border:1px solid #000000; width:100%">
 
 5. Next step would be to get the token.
-    - Log in to salesforce in your prefered browser and enter the following url.
+    - Log in to salesforce in your preferred browser and enter the following url.
   ```
   https://<YOUR_INSTANCE>.salesforce.com/services/oauth2/authorize?response_type=code&client_id=<CONSUMER_KEY>&redirect_uri=<REDIRECT_URL>
   ```
@@ -71,7 +71,7 @@ salesforce:ConnectionConfig config = {
     }
 };
 
-salesforce:Client salesforce = new(config);
+salesforce:Client salesforce = check new (config);
 ```
 
 #### Step 3: Invoke connector operation
@@ -99,7 +99,7 @@ bal run
 
 The `salesforce` connector provides practical examples illustrating usage in various scenarios. Explore these examples below, covering use cases like creating sObjects, retrieving records, and executing bulk operations.
 
-1. [Salesforce REST API use cases](https://github.com/ballerina-platform/module-ballerinax-sfdc/tree/master/examples/rest_api_usecases) - How to employ REST API of Salesforce to carryout varies tasks.
+1. [Salesforce REST API use cases](https://github.com/ballerina-platform/module-ballerinax-sfdc/tree/master/examples/rest_api_usecases) - How to employ REST API of Salesforce to carryout various tasks.
 
 2. [Salesforce Bulk API use cases](https://github.com/ballerina-platform/module-ballerinax-sfdc/tree/master/examples/bulk_api_usecases) - How to employ Bulk API of Salesforce to execute Bulk jobs.
 

ballerina/Package.md

@@ -16,16 +16,16 @@ Ballerina Salesforce connector supports [Salesforce v59.0 REST API](https://deve
 
    <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/create-connected-apps.png alt="Create Connected Apps" width="50%" style="border:1px solid #000000">
 
-    - Here we will be using https://test.salesforce.com as we are using sandbox enviorenment. Users can use https://login.salesforce.com for normal usage.
+    - Here we will be using https://test.salesforce.com as we are using sandbox environment. Users can use https://login.salesforce.com for normal usage.
 
     <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/create_connected%20_app.png alt="Create Connected Apps" width="100%" style="border:1px solid #000000">
 
-4. After the creation user can get consumer key and secret through clicking on the `Manage Consume Details` button.
+4. After the creation user can get consumer key and secret through clicking on the `Manage Consumer Details` button.
 
    <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/crdentials.png alt="Consumer Secrets" width="100%" style="border:1px solid #000000">
 
 5. Next step would be to get the token.
-    - Log in to salesforce in your prefered browser and enter the following url.
+    - Log in to salesforce in your preferred browser and enter the following url.
   ```https://<YOUR_INSTANCE>.salesforce.com/services/oauth2/authorize?response_type=code&client_id=<CONSUMER_KEY>&redirect_uri=<REDIRECT_URL>```
    - Allow access if an alert pops up and the browser will be redirected to a Url like follows.
 
@@ -65,7 +65,7 @@ salesforce:ConnectionConfig config = {
     }
 };
 
-salesforce:Client salesforce = new(config);
+salesforce:Client salesforce = check new (config);
 ```
 
 #### Step 3: Invoke connector operation
@@ -97,7 +97,7 @@ The `salesforce` integration samples illustrate its usage in various integration
 
 The `salesforce` connector provides practical examples illustrating usage in various scenarios. Explore these examples below, covering use cases like creating sObjects, retrieving records, and executing bulk operations.
 
-1. [Salesforce REST API use cases](https://github.com/ballerina-platform/module-ballerinax-sfdc/tree/master/examples/rest_api_usecases) - How to employ REST API of Salesforce to carryout varies tasks.
+1. [Salesforce REST API use cases](https://github.com/ballerina-platform/module-ballerinax-sfdc/tree/master/examples/rest_api_usecases) - How to employ REST API of Salesforce to carryout various tasks.
 
 2. [Salesforce Bulk API use cases](https://github.com/ballerina-platform/module-ballerinax-sfdc/tree/master/examples/bulk_api_usecases) - How to employ Bulk API of Salesforce to execute Bulk jobs.
 

ballerina/client.bal

@@ -27,6 +27,7 @@ import ballerinax/salesforce.utils;
 
 public isolated client class Client {
     private final http:Client salesforceClient;
+    private map<string> sfLocators = {};
 
     # Initializes the connector. During initialization you can pass either http:BearerTokenConfig if you have a bearer
     # token or http:OAuth2RefreshTokenGrantConfig if you have Oauth tokens.
@@ -90,7 +91,7 @@ public isolated client class Client {
     //Describe Organization
     # Lists summary details about each REST API version available.
     #
-    # + return - List of `Version` if successful. Else, the occured `error`
+    # + return - List of `Version` if successful. Else, the occurred `error`
     isolated remote function getApiVersions() returns Version[]|error {
         string path = utils:prepareUrl([BASE_PATH]);
         return check self.salesforceClient->get(path);
@@ -400,10 +401,10 @@ public isolated client class Client {
         return check self.salesforceClient->get(path);
     }
 
-    # Executes up to 25 subrequests in a single request.
+    # Executes up to 25 sub-requests in a single request.
     #
     # + batchRequests - A record containing all the requests
-    # + haltOnError - If true, the request halts when an error occurs on an individual subrequest
+    # + haltOnError - If true, the request halts when an error occurs on an individual sub-request
     # + return - `BatchResult` if successful or else `error`
     isolated remote function batch(Subrequest[] batchRequests, boolean haltOnError = false) returns BatchResult|error {
         string path = utils:prepareUrl([API_BASE_PATH, COMPOSITE, BATCH]);
@@ -651,18 +652,57 @@ public isolated client class Client {
 
     }
 
-    # Get query job results
+    # Get bulk query job results
+    # 
     # + bulkJobId - Id of the bulk job
-    # + return - string[][] if successful else `error`
-    isolated remote function getQueryResult(string bulkJobId)
-            returns string[][]|error {
-        string path = utils:prepareUrl([API_BASE_PATH, JOBS, QUERY, bulkJobId, RESULT]);
+    # + maxRecords - The maximum number of records to retrieve per set of results for the query
+    # + return - The resulting string[][] if successful else `error`
+    isolated remote function getQueryResult(string bulkJobId, int? maxRecords = ()) returns string[][]|error {
+
+        string path = "";
+        string batchingParams = "";
+
+        if maxRecords != () {
+            lock {
+                if self.sfLocators.hasKey(bulkJobId) {
+                    string locator = self.sfLocators.get(bulkJobId);
+                    if locator is "null" {
+                        return [];
+                    } 
+                    batchingParams = string `results?maxRecords=${maxRecords}&locator=${locator}`;
+                } else {
+                    batchingParams = string `results?maxRecords=${maxRecords}`;
+                }
+            }
+            path = utils:prepareUrl([API_BASE_PATH, JOBS, QUERY, bulkJobId, batchingParams]);
+            // Max records value default, we might not know when the locator comes
+        } else {
+            lock {
+                if self.sfLocators.hasKey(bulkJobId) {
+                    string locator = self.sfLocators.get(bulkJobId);
+                    if locator is "null" {
+                        return [];
+                    } 
+                    batchingParams = string `results?locator=${locator}`;
+                    path = utils:prepareUrl([API_BASE_PATH, JOBS, QUERY, bulkJobId, batchingParams]);
+                } else {
+                    path = utils:prepareUrl([API_BASE_PATH, JOBS, QUERY, bulkJobId, RESULT]);
+                }
+            }
+        } 
+
         http:Response response = check self.salesforceClient->get(path);
         if response.statusCode == 200 {
             string textPayload = check response.getTextPayload();
             if textPayload == "" {
                 return [];
             }
+            lock {
+                string|http:HeaderNotFoundError locatorValue = response.getHeader("sforce-locator");
+                if locatorValue is string {
+                    self.sfLocators[bulkJobId] = locatorValue;
+                } // header not found error ignored 
+            }
             string[][] result = check parseCsvString(textPayload);
             return result;
         } else {

ballerina/modules/bulk/Module.md

@@ -15,16 +15,16 @@ This module supports [Salesforce Bulk API v1](https://developer.salesforce.com/d
 
    <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/create-connected-apps.png alt="Create Connected Apps" width="50%" style="border:1px solid #000000">
 
-    - Here we will be using https://test.salesforce.com as we are using sandbox enviorenment. Users can use https://login.salesforce.com for normal usage.
+    - Here we will be using https://test.salesforce.com as we are using sandbox environment. Users can use https://login.salesforce.com for normal usage.
 
     <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/create_connected%20_app.png alt="Create Connected Apps" width="100%" style="border:1px solid #000000">
 
-4. After the creation user can get consumer key and secret through clicking on the `Manage Consume Details` button.
+4. After the creation user can get consumer key and secret through clicking on the `Manage Consumer Details` button.
 
    <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/crdentials.png alt="Consumer Secrets" width="100%" style="border:1px solid #000000">
 
 5. Next step would be to get the token.
-    - Log in to salesforce in your prefered browser and enter the following url.
+    - Log in to salesforce in your preferred browser and enter the following url.
   `https://<YOUR_INSTANCE>.salesforce.com/services/oauth2/authorize?response_type=code&client_id=<CONSUMER_KEY>&redirect_uri=<REDIRECT_URL>`
    - Allow access if an alert pops up and the browser will be redirected to a Url like follows.
   `https://login.salesforce.com/?code=<ENCODED_CODE>`
@@ -64,7 +64,7 @@ bulk:ConnectionConfig sfConfig = {
     }
 };
 
-bulk:Client bulkClient = new (sfConfig);
+bulk:Client bulkClient = check new (sfConfig);
 ```
 
 ### Step 3: Invoke  connector operation

ballerina/modules/soap/Module.md

@@ -15,16 +15,16 @@ This module supports [Salesforce v48.0 SOAP API Enterprise WDSL](https://develop
 
    <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/create-connected-apps.png alt="Create Connected Apps" width="50%" style="border:1px solid #000000">
 
-    - Here we will be using https://test.salesforce.com as we are using sandbox enviorenment. Users can use https://login.salesforce.com for normal usage.
+    - Here we will be using https://test.salesforce.com as we are using sandbox environment. Users can use https://login.salesforce.com for normal usage.
 
     <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/create_connected%20_app.png alt="Create Connected Apps" width="100%" style="border:1px solid #000000">
 
-4. After the creation user can get consumer key and secret through clicking on the `Manage Consume Details` button.
+4. After the creation user can get consumer key and secret through clicking on the `Manage Consumer Details` button.
 
    <img src=https://raw.githubusercontent.com/ballerina-platform/module-ballerinax-sfdc/master/docs/setup/resources/crdentials.png alt="Consumer Secrets" width="100%" style="border:1px solid #000000">
 
 5. Next step would be to get the token.
-    - Log in to salesforce in your prefered browser and enter the following url.
+    - Log in to salesforce in your preferred browser and enter the following url.
   `https://<YOUR_INSTANCE>.salesforce.com/services/oauth2/authorize?response_type=code&client_id=<CONSUMER_KEY>&redirect_uri=<REDIRECT_URL>`
    - Allow access if an alert pops up and the browser will be redirected to a Url like follows.
   `https://login.salesforce.com/?code=<ENCODED_CODE>`
@@ -61,7 +61,7 @@ soap:ConnectionConfig config = {
     }
 };
 
-soap:Client salesforce = new(sfConfig);
+soap:Client salesforce = check new (config);
 ```
 
 ### Step 3: Invoke connector operation