API requests calculation on a bulk upsert costing 6 calls Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsHow to perform upsert in Salesforce Bulk APIProcess Builder Error with Bulk APIExporting entire database via APISalesforce Bulk API upsert - Error: ENTITY_IS_DELETEDUpserting objects with many to many relationship using apiWhat are Best Practices for Upserting Objects with Foreign Key Members (i.e. References to Other Objects)How to connect to salesforce instanceUpdating record from Ruby on Rails to SalesforceHow do I get the record ID's for a bulk data insert using sfdx CLI?Can the Bulk API solve the CPU Time Limit Issue I am having

Why did the IBM 650 use bi-quinary?

How to find all the available tools in macOS terminal?

Bonus calculation: Am I making a mountain out of a molehill?

Java 8 stream max() function argument type Comparator vs Comparable

Is it true to say that an hosting provider's DNS server is what links the entire hosting environment to ICANN?

Is there a service that would inform me whenever a new direct route is scheduled from a given airport?

G-Code for resetting to 100% speed

What do you call a plan that's an alternative plan in case your initial plan fails?

How to motivate offshore teams and trust them to deliver?

Storing hydrofluoric acid before the invention of plastics

Why aren't air breathing engines used as small first stages

How can players work together to take actions that are otherwise impossible?

Is above average number of years spent on PhD considered a red flag in future academia or industry positions?

List *all* the tuples!

Difference between these two cards?

Is 1 ppb equal to 1 μg/kg?

Proof involving the spectral radius and Jordan Canonical form

What is the musical term for a note that continously plays through a melody?

What makes black pepper strong or mild?

Why one of virtual NICs called bond0?

What is the longest distance a 13th-level monk can jump while attacking on the same turn?

How do I stop a creek from eroding my steep embankment?

Why is "Captain Marvel" translated as male in Portugal?

Antler Helmet: Can it work?



API requests calculation on a bulk upsert costing 6 calls



Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsHow to perform upsert in Salesforce Bulk APIProcess Builder Error with Bulk APIExporting entire database via APISalesforce Bulk API upsert - Error: ENTITY_IS_DELETEDUpserting objects with many to many relationship using apiWhat are Best Practices for Upserting Objects with Foreign Key Members (i.e. References to Other Objects)How to connect to salesforce instanceUpdating record from Ruby on Rails to SalesforceHow do I get the record ID's for a bulk data insert using sfdx CLI?Can the Bulk API solve the CPU Time Limit Issue I am having



.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








1















I am using the simple-salesforce Python package to upsert records into our Salesforce instance. I am finding that each bulk API call counts as 6 API requests. Is this correct? Is there a way to reduce this whilst performing this operation.



The function I am using to perform the upsert:



def bulk_upsert(self, object_name, data_set, identity_field):


upsert_response = SFBulkType(object_name, self.bulk_con.connect.bulk_url,
self.bulk_con.connect.headers, self.bulk_con.connect.session).upsert(data_set, identity_field)

return upsert_response


The object name is the custom object created in Salesforce for our instance, the dataset is a list of dictionaries containing 200 records, the identity field is the external id for the Salesforce object.



I establish the Salesforce connection (sf) and then loop through requested_chunks which is a list of the data to upsert broken into 200 records each.



for request in request_chunks:

upsert_response = sf.bulk_upsert(sf_object, request, sf_identity)


I am finding that each time through this loop it is costing 6 credits. I have done a comparison with the integration recommended by simple-salesforce and that also costs 6 calls per bulk upsert. Any thoughts?










share|improve this question







New contributor




James Wellington is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.


























    1















    I am using the simple-salesforce Python package to upsert records into our Salesforce instance. I am finding that each bulk API call counts as 6 API requests. Is this correct? Is there a way to reduce this whilst performing this operation.



    The function I am using to perform the upsert:



    def bulk_upsert(self, object_name, data_set, identity_field):


    upsert_response = SFBulkType(object_name, self.bulk_con.connect.bulk_url,
    self.bulk_con.connect.headers, self.bulk_con.connect.session).upsert(data_set, identity_field)

    return upsert_response


    The object name is the custom object created in Salesforce for our instance, the dataset is a list of dictionaries containing 200 records, the identity field is the external id for the Salesforce object.



    I establish the Salesforce connection (sf) and then loop through requested_chunks which is a list of the data to upsert broken into 200 records each.



    for request in request_chunks:

    upsert_response = sf.bulk_upsert(sf_object, request, sf_identity)


    I am finding that each time through this loop it is costing 6 credits. I have done a comparison with the integration recommended by simple-salesforce and that also costs 6 calls per bulk upsert. Any thoughts?










    share|improve this question







    New contributor




    James Wellington is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.






















      1












      1








      1








      I am using the simple-salesforce Python package to upsert records into our Salesforce instance. I am finding that each bulk API call counts as 6 API requests. Is this correct? Is there a way to reduce this whilst performing this operation.



      The function I am using to perform the upsert:



      def bulk_upsert(self, object_name, data_set, identity_field):


      upsert_response = SFBulkType(object_name, self.bulk_con.connect.bulk_url,
      self.bulk_con.connect.headers, self.bulk_con.connect.session).upsert(data_set, identity_field)

      return upsert_response


      The object name is the custom object created in Salesforce for our instance, the dataset is a list of dictionaries containing 200 records, the identity field is the external id for the Salesforce object.



      I establish the Salesforce connection (sf) and then loop through requested_chunks which is a list of the data to upsert broken into 200 records each.



      for request in request_chunks:

      upsert_response = sf.bulk_upsert(sf_object, request, sf_identity)


      I am finding that each time through this loop it is costing 6 credits. I have done a comparison with the integration recommended by simple-salesforce and that also costs 6 calls per bulk upsert. Any thoughts?










      share|improve this question







      New contributor




      James Wellington is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.












      I am using the simple-salesforce Python package to upsert records into our Salesforce instance. I am finding that each bulk API call counts as 6 API requests. Is this correct? Is there a way to reduce this whilst performing this operation.



      The function I am using to perform the upsert:



      def bulk_upsert(self, object_name, data_set, identity_field):


      upsert_response = SFBulkType(object_name, self.bulk_con.connect.bulk_url,
      self.bulk_con.connect.headers, self.bulk_con.connect.session).upsert(data_set, identity_field)

      return upsert_response


      The object name is the custom object created in Salesforce for our instance, the dataset is a list of dictionaries containing 200 records, the identity field is the external id for the Salesforce object.



      I establish the Salesforce connection (sf) and then loop through requested_chunks which is a list of the data to upsert broken into 200 records each.



      for request in request_chunks:

      upsert_response = sf.bulk_upsert(sf_object, request, sf_identity)


      I am finding that each time through this loop it is costing 6 credits. I have done a comparison with the integration recommended by simple-salesforce and that also costs 6 calls per bulk upsert. Any thoughts?







      bulk-api httprequest upsert






      share|improve this question







      New contributor




      James Wellington is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|improve this question







      New contributor




      James Wellington is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|improve this question




      share|improve this question






      New contributor




      James Wellington is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked 12 hours ago









      James WellingtonJames Wellington

      61




      61




      New contributor




      James Wellington is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      James Wellington is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      James Wellington is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.




















          2 Answers
          2






          active

          oldest

          votes


















          1














          Is your python lib uses Bulk API or simple record insert/update?
          If it uses Salesforce Bulk API it needs 1 request to create Bulk Job, 1 request to upload records, 1 request to mark job as posted.
          And needs to poll request to track job status if records are processed






          share|improve this answer






























            1














            The Bulk API is designed for bulk uploads of data. If you're importing less than 1,000 records, you're definitely wasting API calls. Six calls sounds about right; it definitely requires at least three just to initialize, upload, and start the Bulk API call, plus more calls to wait for completion. Use the normal upsert API call if you want to upload smaller batches of data (e.g. 200 records at a time).






            share|improve this answer























            • Will this cost 1 API call per record? So a batch of 400 records which was split into two API requests to the bulk API previously, would then be 400 separate upsert calls?

              – James Wellington
              11 hours ago











            • I can increase the batch size from 200 to circa 2000 without coming into problems with total characters. I went with 200 as assumed it wouldn't change the API number of calls. The documentation states that the API splits batches into 200 records behind the scenes.

              – James Wellington
              11 hours ago











            • @JamesWellington The synchronous upsert call can do 200 records API call. The Bulk API can process up to 10,000 records per file, and millions of records per day. The Bulk API will indeed split the file in to batches of 200. If you use the bulk API, there's really no reason to send less than 10,000 records per file. You'll want to read the documentation for more information.

              – sfdcfox
              11 hours ago











            Your Answer








            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "459"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader:
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            ,
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );






            James Wellington is a new contributor. Be nice, and check out our Code of Conduct.









            draft saved

            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f257890%2fapi-requests-calculation-on-a-bulk-upsert-costing-6-calls%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            1














            Is your python lib uses Bulk API or simple record insert/update?
            If it uses Salesforce Bulk API it needs 1 request to create Bulk Job, 1 request to upload records, 1 request to mark job as posted.
            And needs to poll request to track job status if records are processed






            share|improve this answer



























              1














              Is your python lib uses Bulk API or simple record insert/update?
              If it uses Salesforce Bulk API it needs 1 request to create Bulk Job, 1 request to upload records, 1 request to mark job as posted.
              And needs to poll request to track job status if records are processed






              share|improve this answer

























                1












                1








                1







                Is your python lib uses Bulk API or simple record insert/update?
                If it uses Salesforce Bulk API it needs 1 request to create Bulk Job, 1 request to upload records, 1 request to mark job as posted.
                And needs to poll request to track job status if records are processed






                share|improve this answer













                Is your python lib uses Bulk API or simple record insert/update?
                If it uses Salesforce Bulk API it needs 1 request to create Bulk Job, 1 request to upload records, 1 request to mark job as posted.
                And needs to poll request to track job status if records are processed







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered 12 hours ago









                pklochkovpklochkov

                478412




                478412























                    1














                    The Bulk API is designed for bulk uploads of data. If you're importing less than 1,000 records, you're definitely wasting API calls. Six calls sounds about right; it definitely requires at least three just to initialize, upload, and start the Bulk API call, plus more calls to wait for completion. Use the normal upsert API call if you want to upload smaller batches of data (e.g. 200 records at a time).






                    share|improve this answer























                    • Will this cost 1 API call per record? So a batch of 400 records which was split into two API requests to the bulk API previously, would then be 400 separate upsert calls?

                      – James Wellington
                      11 hours ago











                    • I can increase the batch size from 200 to circa 2000 without coming into problems with total characters. I went with 200 as assumed it wouldn't change the API number of calls. The documentation states that the API splits batches into 200 records behind the scenes.

                      – James Wellington
                      11 hours ago











                    • @JamesWellington The synchronous upsert call can do 200 records API call. The Bulk API can process up to 10,000 records per file, and millions of records per day. The Bulk API will indeed split the file in to batches of 200. If you use the bulk API, there's really no reason to send less than 10,000 records per file. You'll want to read the documentation for more information.

                      – sfdcfox
                      11 hours ago















                    1














                    The Bulk API is designed for bulk uploads of data. If you're importing less than 1,000 records, you're definitely wasting API calls. Six calls sounds about right; it definitely requires at least three just to initialize, upload, and start the Bulk API call, plus more calls to wait for completion. Use the normal upsert API call if you want to upload smaller batches of data (e.g. 200 records at a time).






                    share|improve this answer























                    • Will this cost 1 API call per record? So a batch of 400 records which was split into two API requests to the bulk API previously, would then be 400 separate upsert calls?

                      – James Wellington
                      11 hours ago











                    • I can increase the batch size from 200 to circa 2000 without coming into problems with total characters. I went with 200 as assumed it wouldn't change the API number of calls. The documentation states that the API splits batches into 200 records behind the scenes.

                      – James Wellington
                      11 hours ago











                    • @JamesWellington The synchronous upsert call can do 200 records API call. The Bulk API can process up to 10,000 records per file, and millions of records per day. The Bulk API will indeed split the file in to batches of 200. If you use the bulk API, there's really no reason to send less than 10,000 records per file. You'll want to read the documentation for more information.

                      – sfdcfox
                      11 hours ago













                    1












                    1








                    1







                    The Bulk API is designed for bulk uploads of data. If you're importing less than 1,000 records, you're definitely wasting API calls. Six calls sounds about right; it definitely requires at least three just to initialize, upload, and start the Bulk API call, plus more calls to wait for completion. Use the normal upsert API call if you want to upload smaller batches of data (e.g. 200 records at a time).






                    share|improve this answer













                    The Bulk API is designed for bulk uploads of data. If you're importing less than 1,000 records, you're definitely wasting API calls. Six calls sounds about right; it definitely requires at least three just to initialize, upload, and start the Bulk API call, plus more calls to wait for completion. Use the normal upsert API call if you want to upload smaller batches of data (e.g. 200 records at a time).







                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered 12 hours ago









                    sfdcfoxsfdcfox

                    265k13212459




                    265k13212459












                    • Will this cost 1 API call per record? So a batch of 400 records which was split into two API requests to the bulk API previously, would then be 400 separate upsert calls?

                      – James Wellington
                      11 hours ago











                    • I can increase the batch size from 200 to circa 2000 without coming into problems with total characters. I went with 200 as assumed it wouldn't change the API number of calls. The documentation states that the API splits batches into 200 records behind the scenes.

                      – James Wellington
                      11 hours ago











                    • @JamesWellington The synchronous upsert call can do 200 records API call. The Bulk API can process up to 10,000 records per file, and millions of records per day. The Bulk API will indeed split the file in to batches of 200. If you use the bulk API, there's really no reason to send less than 10,000 records per file. You'll want to read the documentation for more information.

                      – sfdcfox
                      11 hours ago

















                    • Will this cost 1 API call per record? So a batch of 400 records which was split into two API requests to the bulk API previously, would then be 400 separate upsert calls?

                      – James Wellington
                      11 hours ago











                    • I can increase the batch size from 200 to circa 2000 without coming into problems with total characters. I went with 200 as assumed it wouldn't change the API number of calls. The documentation states that the API splits batches into 200 records behind the scenes.

                      – James Wellington
                      11 hours ago











                    • @JamesWellington The synchronous upsert call can do 200 records API call. The Bulk API can process up to 10,000 records per file, and millions of records per day. The Bulk API will indeed split the file in to batches of 200. If you use the bulk API, there's really no reason to send less than 10,000 records per file. You'll want to read the documentation for more information.

                      – sfdcfox
                      11 hours ago
















                    Will this cost 1 API call per record? So a batch of 400 records which was split into two API requests to the bulk API previously, would then be 400 separate upsert calls?

                    – James Wellington
                    11 hours ago





                    Will this cost 1 API call per record? So a batch of 400 records which was split into two API requests to the bulk API previously, would then be 400 separate upsert calls?

                    – James Wellington
                    11 hours ago













                    I can increase the batch size from 200 to circa 2000 without coming into problems with total characters. I went with 200 as assumed it wouldn't change the API number of calls. The documentation states that the API splits batches into 200 records behind the scenes.

                    – James Wellington
                    11 hours ago





                    I can increase the batch size from 200 to circa 2000 without coming into problems with total characters. I went with 200 as assumed it wouldn't change the API number of calls. The documentation states that the API splits batches into 200 records behind the scenes.

                    – James Wellington
                    11 hours ago













                    @JamesWellington The synchronous upsert call can do 200 records API call. The Bulk API can process up to 10,000 records per file, and millions of records per day. The Bulk API will indeed split the file in to batches of 200. If you use the bulk API, there's really no reason to send less than 10,000 records per file. You'll want to read the documentation for more information.

                    – sfdcfox
                    11 hours ago





                    @JamesWellington The synchronous upsert call can do 200 records API call. The Bulk API can process up to 10,000 records per file, and millions of records per day. The Bulk API will indeed split the file in to batches of 200. If you use the bulk API, there's really no reason to send less than 10,000 records per file. You'll want to read the documentation for more information.

                    – sfdcfox
                    11 hours ago










                    James Wellington is a new contributor. Be nice, and check out our Code of Conduct.









                    draft saved

                    draft discarded


















                    James Wellington is a new contributor. Be nice, and check out our Code of Conduct.












                    James Wellington is a new contributor. Be nice, and check out our Code of Conduct.











                    James Wellington is a new contributor. Be nice, and check out our Code of Conduct.














                    Thanks for contributing an answer to Salesforce Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid


                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.

                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f257890%2fapi-requests-calculation-on-a-bulk-upsert-costing-6-calls%23new-answer', 'question_page');

                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -bulk-api, httprequest, upsert

                    Popular posts from this blog

                    Creating 100m^2 grid automatically using QGIS?Creating grid constrained within polygon in QGIS?Createing polygon layer from point data using QGIS?Creating vector grid using QGIS?Creating grid polygons from coordinates using R or PythonCreating grid from spatio temporal point data?Creating fields in attributes table using other layers using QGISCreate .shp vector grid in QGISQGIS Creating 4km point grid within polygonsCreate a vector grid over a raster layerVector Grid Creates just one grid

                    Word for a person who has no opinion about whether god existsWord for having a definite opinion while simultaneously withholding judgment?What's the opposite of “newcomer? Is ”veteran" OK?What do you call an “atheist” who might believe in an afterlife?What's a word for someone who wants to voice opinions but not have them challenged?Word for someone who dismisses contrary opinions as irrational?Somone who thinks they are overly special/out of the ordinaryIs there a word, phrase or idiom for “a person who is incapable of thinking about the future”?The belief that a god is human-likeA word for a non-famous person/thing you have heard a lot aboutAdjective for a person who enjoys taking care of their appearance

                    Can I redirect output to a log file and background a process at the same time?2019 Community Moderator ElectionUnable to write to file in shell script when running command in backgroundredirect and log script outputhow to properly log the output of a console program that frequently updates “parts” of the screen, resulting in a messy log file?How can I redirect the output of a child process?grep script - output lines at the same time into echoHow to run process in background and get its pid to create log file nameHow to redirect output to a log from expect commandHow to clear a redirect log file content in shell?Log background jobs started in all the manually started shells?Why do `jobs` and `dirs` run in command subsitution, process substitution, pipeline, and background jobs output the same as in original shell?How to redirect output to file to STDOUT?