javascript - The best performant way to push items into array? -


in website have many arrays data. example: vertices array, colors array, sizes array...

i'm working big amounts of items. tens of millions.

before adding data arrays need process it. until now, did in main thread , made website freeze x seconds. froze because of processing , because of adding processed data arrays.

today 'moved' (did lot of work) processing web workers, processed data being added in main thread. managed save freezing time of processing not of adding.

the adding done array.push() or array.splice().

i've read articles how array works, , found out when add item array, array being copied new place in memory array.length + 1 size , there adding value. makes data pushing slow.

i read typed array faster. need know size of array, don't know, , creating big typed array counter , managing adding items in middle(and not end of array) lot of code change, don't want @ time.

so, question, have typedarray return web worker, , need put regular array. best performant way it? (today i'm running in loop , pushing 1 after other)

edit

example how website work: client add count of items, lets 100000. items raw data being collected , send worker. worker processing information , sending processed data typed-array (for using transferable objects). in main thread adding processed data arrays - end or in specific index. 2nd round. client add 100000 items. sending worker , result being added main thread arrays. 3nd round can 10 items, 4nd round 10000, 5nd round can remove indices 10-2000, ...

did more research using comments , thought direction.

i've tried using typedarray.set method , discovered very fast.

10 million of items using sets took 0.004 seconds, compares array.push 0.866 seconds. separated the 10 millions 10 arrays make sure set method not working faster when starting index 0.

this way think implement insertatindex of own using typedarray, pushing items forward , setting new one\s in right index.

in addition, can use typedarray.subarray fetch sub data according real amount of data in array (which not copying data) - useful uploading data buffer (webgl)

i said want work regular arrays performance boost don't think other wise. , not work, when i'm wrapping mynewtypedarray typedarray push, splice, own implementation.

hope info helped anyone

var maxcount = 10000000;  var = new float32array(maxcount);  var asimple = [];    var arrays = [];  var div = 10;  var arraylen = maxcount / div;  (var arraysidx = 0; arraysidx < div; arraysidx++) {    var b = new float32array(arraylen);    (var = 0; < b.length; i++) {      b[i] = * (arraysidx + 1);    }    arrays.push(b);  }    var timebefore = new date().gettime();    (var currarrayidx = 0; currarrayidx < arrays.length; currarrayidx++) {    a.set(arrays[currarrayidx], currarrayidx * arraylen);  }  var timeafter = new date().gettime();  good.innerhtml = (timeafter - timebefore) / 1000 + " sec.\n";  timebefore = new date().gettime();  (var currarrayidx = 0; currarrayidx < arrays.length; currarrayidx++) {    (var = 0; < arraylen; i++) {      asimple.push(arrays[currarrayidx][i]);    }  }    timeafter = new date().gettime();  bad.innerhtml = (timeafter - timebefore) / 1000 + " sec.\n";
using set of typedarray:  <div id='good' style='background-color:lightgreen'>working...</div>  using push of array:  <div id='bad' style='background-color:red'>working...</div>


Comments

Popular posts from this blog

android - MPAndroidChart - How to add Annotations or images to the chart -

javascript - Add class to another page attribute using URL id - Jquery -

firefox - Where is 'webgl.osmesalib' parameter? -