Skaffold syncs files but pod doesn't refresh - node.js

Question

Kubernetes newbie here.
I have some strange Skaffold/Kubernetes behavior. I'm working in Google Cloud but I've changed to the local environment just for test and it's the same. So probably it's me how's doing something wrong. The problem is that though I see Skaffold syncing changes these changes aren't reflected. All the files inside the pods are the old ones.
Skaffold.yaml:
apiVersion: skaffold/v2alpha3
kind: Config
deploy:
kubectl:
manifests:
- ./infra/k8s/*
build:
# local:
# push: false
googleCloudBuild:
projectId: ts-maps-286111
artifacts:
- image: us.gcr.io/ts-maps-286111/auth
context: auth
docker:
dockerfile: Dockerfile
sync:
manual:
- src: 'src/**/*.ts'
dest: .
- image: us.gcr.io/ts-maps-286111/client
context: client
docker:
dockerfile: Dockerfile
sync:
manual:
- src: '**/*.js'
dest: .
- image: us.gcr.io/ts-maps-286111/tickets
context: tickets
docker:
dockerfile: Dockerfile
sync:
manual:
- src: 'src/**/*.ts'
dest: .
- image: us.gcr.io/ts-maps-286111/orders
context: orders
docker:
dockerfile: Dockerfile
sync:
manual:
- src: 'src/**/*.ts'
dest: .
- image: us.gcr.io/ts-maps-286111/expiration
context: expiration
docker:
dockerfile: Dockerfile
sync:
manual:
- src: 'src/**/*.ts'
dest: .
When a file inside one of the directories is changed I see following logs:
time="2020-09-05T01:24:06+03:00" level=debug msg="Change detected notify.Write: \"F:\\projects\\lrn_microservices\\complex\\expiration\\src\\index.ts\""
time="2020-09-05T01:24:06+03:00" level=debug msg="Change detected notify.Write: \"F:\\projects\\lrn_microservices\\complex\\expiration\\src\\index.ts\""
time="2020-09-05T01:24:06+03:00" level=debug msg="Change detected notify.Write: \"F:\\projects\\lrn_microservices\\complex\\expiration\\src\\index.ts\""
time="2020-09-05T01:24:06+03:00" level=debug msg="Change detected notify.Write: \"F:\\projects\\lrn_microservices\\complex\\expiration\\src\\index.ts\""
time="2020-09-05T01:24:06+03:00" level=debug msg="Change detected notify.Write: \"F:\\projects\\lrn_microservices\\complex\\expiration\\src\\index.ts\""
time="2020-09-05T01:24:06+03:00" level=debug msg="Change detected notify.Write: \"F:\\projects\\lrn_microservices\\complex\\expiration\\src\\index.ts\""
time="2020-09-05T01:24:07+03:00" level=debug msg="Found dependencies for dockerfile: [{package.json /app true} {. /app true}]"
time="2020-09-05T01:24:07+03:00" level=debug msg="Skipping excluded path: node_modules"
time="2020-09-05T01:24:07+03:00" level=debug msg="Found dependencies for dockerfile: [{package.json /app true} {. /app true}]"
time="2020-09-05T01:24:07+03:00" level=debug msg="Skipping excluded path: .next"
time="2020-09-05T01:24:07+03:00" level=debug msg="Skipping excluded path: node_modules"
time="2020-09-05T01:24:07+03:00" level=debug msg="Found dependencies for dockerfile: [{package.json /app true} {. /app true}]"
time="2020-09-05T01:24:07+03:00" level=debug msg="Skipping excluded path: node_modules"
time="2020-09-05T01:24:07+03:00" level=debug msg="Found dependencies for dockerfile: [{package.json /app true} {. /app true}]"
time="2020-09-05T01:24:07+03:00" level=debug msg="Skipping excluded path: node_modules"
time="2020-09-05T01:24:07+03:00" level=debug msg="Found dependencies for dockerfile: [{package.json /app true} {. /app true}]"
time="2020-09-05T01:24:07+03:00" level=debug msg="Skipping excluded path: node_modules"
time="2020-09-05T01:24:07+03:00" level=info msg="files modified: [expiration\\src\\index.ts]"
Syncing 1 files for us.gcr.io/ts-maps-286111/expiration:2aae7ff-dirty#sha256:2e31caedf3d9b2bcb2ea5693f8e22478a9d6caa21d1a478df5ff8ebcf562573e
time="2020-09-05T01:24:07+03:00" level=info msg="Copying files: map[expiration\\src\\index.ts:[/app/src/index.ts]] to us.gcr.io/ts-maps-286111/expiration:2aae7ff-dirty#sha256:2e31caedf3d9b2bcb2ea5693f8e22478a9d6caa21d1a478df5ff8ebcf562573e"
time="2020-09-05T01:24:07+03:00" level=debug msg="getting client config for kubeContext: ``"
time="2020-09-05T01:24:07+03:00" level=debug msg="Running command: [kubectl --context gke_ts-maps-286111_europe-west3-a_ticketing-dev exec expiration-depl-5cb997d597-p49lv --namespace default -c expiration -i -- tar xmf - -C / --no-same-owner]"
time="2020-09-05T01:24:09+03:00" level=debug msg="Command output: [], stderr: tar: removing leading '/' from member names\n"
Watching for changes...
time="2020-09-05T01:24:11+03:00" level=info msg="Streaming logs from pod: expiration-depl-5cb997d597-p49lv container: expiration"
time="2020-09-05T01:24:11+03:00" level=debug msg="Running command: [kubectl --context gke_ts-maps-286111_europe-west3-a_ticketing-dev logs --since=114s -f expiration-depl-5cb997d597-p49lv -c expiration --namespace default]"
[expiration]
[expiration] > expiration#1.0.0 start /app
[expiration] > ts-node-dev --watch src src/index.ts
[expiration]
[expiration] ts-node-dev ver. 1.0.0-pre.62 (using ts-node ver. 8.10.2, typescript ver. 3.9.7)
[expiration] starting expiration!kdd
[expiration] Connected to NATS!
NodeJS server inside the pod restarts. Sometimes I see this line, sometimes not, the result overall is the same
[expiration] [INFO] 22:23:42 Restarting: src/index.ts has been modified
But there are no changes made. If I cat the changed file inside a pod it's the old version, if I delete a pod it starts again with an old version.
My folder structure:
+---auth
| \---src
| +---models
| +---routes
| | \---__test__
| +---services
| \---test
+---client
| +---.next
| | +---cache
| | | \---next-babel-loader
| | +---server
| | | \---pages
| | | +---auth
| | | \---next
| | | \---dist
| | | \---pages
| | \---static
| | +---chunks
| | | \---pages
| | | +---auth
| | | \---next
| | | \---dist
| | | \---pages
| | +---development
| | \---webpack
| | \---pages
| | \---auth
| +---api
| +---components
| +---hooks
| \---pages
| \---auth
+---common
| +---build
| | +---errors
| | +---events
| | | \---types
| | \---middlewares
| \---src
| +---errors
| +---events
| | \---types
| \---middlewares
+---config
+---expiration
| \---src
| +---events
| | +---listeners
| | \---publishers
| +---queue
| \---__mocks__
+---infra
| \---k8s
+---orders
| \---src
| +---events
| | +---listeners
| | | \---__test__
| | \---publishers
| +---models
| +---routes
| | \---__test__
| +---test
| \---__mocks__
+---payment
\---tickets
\---src
+---events
| +---listeners
| | \---__test__
| \---publishers
+---models
| \---__test__
+---routes
| \---__test__
+---test
\---__mocks__
Would be grateful for any help!

Answer 1

What worked for me was using --poll flag with ts-node-dev.
My script looks like this
"start" : "ts-node-dev --respawn --poll --inspect --exit-child src/index.ts

Answer 2

For your start script try to add --poll. For example if you start script is "start" : "nodemon src/index.js", change it to "nodemon --poll src/index.js"

Answer 3

It looks like no major error in the logs, my guess is that the files are actually being put in another directory. You can try running in the container.
find / -name "index.ts"
to see if that didn't land somewhere else.
Another thing to check is the WORKDIR value in your container(s). Check what directory you land on when you run:
kubectl exec -it -c <container-name> <pod-name>
✌️