+ Reply to Thread
Results 1 to 2 of 2
This is a discussion on PHP bug 43038 reported as Bogus within the Salesforce PHP Tutorials forums, part of the Salesforce category; Well after I sent the package which reproduced the error that I
  1. #1
    mike's Avatar
    mike is offline Administrator
    Join Date
    May 2007
    Location
    Wylie, Texas
    Posts
    607
    Blog Entries
    16

    PHP bug 43038 reported as Bogus

    Well after I sent the package which reproduced the error that I was getting, Dmitry sent me an email about my bug and his findings:

    Hi Mike,

    I've looked into your example and I don't see any PHP bug here.
    Your application tries to construct huge result set and as result it outs of memory.
    May be some old versions required less memory overhead and was able to pass the example with the same memory but it is not a bug.

    I suggest re-implement the application a little bit to improve memory usage.

    In function get_records() you collect all records into one array and then insert each of them into database.

    But you don't need to collect all of them.

    You are able to retrieve a portion of data then insert each record from this portion and then select the next portion...

    Using this approach you'll need memory only for one portion of data.

    While I do not agree that we should have to change our applications based on new overhead by a scripting language, I do agree that I needed to find a way to fix this by not processing so much data into an array. So I started today by changing my process of capturing everything into one LARGE php array and then processing the data/loading the data into my local database, I split it up in chunks of 500 to 1000 records a time and then move onto the next set of data. Doing it this way REALLY speeds up the script.

    If I process 1000 datasets at a time, the runtime of my Product2 data replication script decreases from 45 minutes to 6 minutes, on average
    So I consider this issues closed!

    Once I fine tune the scripts and make sure that I am not running into any other problems, I will show you some examples that can vastly improve on your data processing against the API of Salesforce.com

  2. #2
    jarrettcoggin is offline Junior Member
    Join Date
    Jul 2009
    Posts
    14
    I'll definitely be on the lookout for this. I also agree that we should not have to change due to overhead. Luckily, I'm just beginning to implement our systems tie in to SF using the API, so anything that helps reduce overhead will help greatly.

+ Reply to Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

SEO by vBSEO 3.5.2