![]() ![]() In addition, unintended purging delay caused by application inactivity is avoided with background threads. The weights of the model when working on the CPU are in RAM and the work can be accelerated by a little tuning.Įnabling jemalloc background threads generally improves the tail latency for application threads, since unused memory purging is shifted to the dedicated background threads. ![]() sudo apt-get install -y libjemalloc-devĬOMMANDLINE_ARGS="-precision autocast -use-cpu all -no-half -opt-channelslast -skip-torch-cuda-test -enable-insecure-extension-access"Ĭould you please explain a bit what options does and how to tweak for other CPU capabilities?.I added the following lines to the end of the webui-user.sh file:Įxport LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so:$LD_PRELOADĮxport MALLOC_CONF="oversize_threshold:1,background_thread:true,metadata_thp:auto,dirty_decay_ms: 60000,muzzy_decay_ms:60000"Įxport LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libiomp5.so:$LD_PRELOAD Xeon E3 1265L v3 (16Gb, 4 core) speed up from 10s/it to 8s/it.Using some system optimizations, borrowed from HuggingFace, it turned out to increase the speed of work from 1.25x to 1.5x. I have searched the existing issues and checked the recent builds/commits. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |