Docker sidekiq much slower than native OS - multithreading

Question

I've been running my application using docker compose for a while now. One of the heaviest parts of the application are the background tasks.
I noticed that most of my background tasks (running with sidekiq) were running much slower than one of my colleagues computer (not using docker).
Using docker, same background task runs in 40 seconds. On native OS it runs in 12 seconds. I tried this myself, on my machine and run it on native OS and I could confirm that it's much faster.
Docker info:
Containers: 14
Running: 4
Paused: 0
Stopped: 10
Images: 42
Server Version: 17.12.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 89623f28b87a6004d4b785663257362d1658a729
runc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.60-linuxkit-aufs
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 7.786GiB
Name: linuxkit-025000000001
ID: CFFM:EFLI:4A5K:XTPG:E27S:KXJT:26SS:ZAPE:ZAFW:3BRM:E6YK:MVAA
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 76
Goroutines: 129
System Time: 2018-02-09T14:13:44.910242335Z
EventsListeners: 3
HTTP Proxy: docker.for.mac.http.internal:3128
HTTPS Proxy: docker.for.mac.http.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Docker compose:
version: '3.4'
services:
sidekiq-1:
build: .
command: bundle exec sidekiq -c 4 -L log/sidekiq-1.log
tty: true
stdin_open: true
volumes:
- '.:/app'
environment:
- DATABASE_URL=postgres://username#postgres/database
- REDIS_URL=redis://redis:6379
sidekiq-2:
build: .
command: bundle exec sidekiq -c 4 -L log/sidekiq-2.log
tty: true
stdin_open: true
volumes:
- '.:/app'
environment:
- DATABASE_URL=postgres://username#postgres/database
- REDIS_URL=redis://redis:6379
I'm a bit lost in regards to what might be happening.
One of the things I noticed is that even though I have allocated 8 cores to docker, only 4 threads run at the same time on sidekiq and CPU usage using docker stats never goes above 80% for these 2 containers.
Any help appreciated.

Answer 1

Docker for Mac has known performance issues for certain workloads that are filesystem-intensive. See here and here for official info. Mounted volumes with Mac tend to be the worst. I've seen similar performance hits when mounting a mid-sized Django + node project and trying to get the runserver command to be responsive (spoiler, it isn't very in this case, too much fs overhead).
Something you can try, instead of mounting the whole app directory, mount as little as possible. Hard to say how helpful that would be not knowing what the project looks like. You should also be able to increase performance by not using a bind mount; COPY your files in via Dockerfile, then use a named volume to persist them. That puts a little bit of a damper on your development workflow, but I think it would significantly speed up the sidekiq performance.