upload egg file raise an DistributionNotFound exception for requirements package - python-3.x

Question

Description
When I tried to deploy an egg file to the scrapyd server within a docker container, it always returned a DistributionNotFound error for each external package.
I tried both with curl and scrapyd-deploy
Steps to Reproduce
python setup.py bdist_uberegg -r requirements.txt
curl http://localhost:13334/addversion.json -F project=pr_reviews -F version=1.0.0 -F egg=.\dist\project-1.0-py3.10.egg
or
scrapyd-deploy --include-dependencies
Actual behavior: [What actually happens]
$ curl http://localhost:13334/addversion.json -F project=pr_reviews -F version=re1 -F egg=#".\dist\project-1.0-py3.10.egg"
{"node_name": "e9e273febed8", "status": "error", "message": "/usr/local/lib/python3.9/dist-packages/scrapy/utils/project.py:81: ScrapyDeprecationWarning: Use of environment variables prefixed with SCRAPY_ to override settings is deprecated. The following environment variables are currently defined: EGG_VERSION\n warnings.warn(\nTraceback (most recent call last):\n File \"/usr/lib/python3.9/runpy.py\", line 197, in _run_module_as_main\n return _run_code(code, main_globals, None,\n File \"/usr/lib/python3.9/runpy.py\", line 87, in _run_code\n exec(code, run_globals)\n File \"/usr/local/lib/python3.9/dist-packages/scrapyd/runner.py\", line 46, in <module>\n main()\n File \"/usr/local/lib/python3.9/dist-packages/scrapyd/runner.py\", line 43, in main\n execute()\n File \"/usr/local/lib/python3.9/dist-packages/scrapy/cmdline.py\", line 144, in execute\n cmd.crawler_process = CrawlerProcess(settings)\n File \"/usr/local/lib/python3.9/dist-packages/scrapy/crawler.py\", line 290, in __init__\n super().__init__(settings)\n File \"/usr/local/lib/python3.9/dist-packages/scrapy/crawler.py\", line 167, in __init__\n self.spider_loader = self._get_spider_loader(settings)\n
File \"/usr/local/lib/python3.9/dist-packages/scrapy/crawler.py\", line 161, in _get_spider_loader\n return loader_cls.from_settings(settings.frozencopy())\n File \"/usr/local/lib/python3.9/dist-packages/scrapy/spiderloader.py\", line 67, in from_settings\n return cls(settings)\n File \"/usr/local/lib/python3.9/dist-packages/scrapy/spiderloader.py\", line 24, in __init__\n self._load_all_spiders()\n File \"/usr/local/lib/python3.9/dist-packages/scrapy/spiderloader.py\", line 51, in _load_all_spiders\n for module in walk_modules(name):\n File \"/usr/local/lib/python3.9/dist-packages/scrapy/utils/misc.py\", line 88, in walk_modules\n submod = import_module(fullpath)\n File \"/usr/lib/python3.9/importlib/__init__.py\", line 127, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n File \"<frozen importlib._bootstrap>\", line 1030, in _gcd_import\n File \"<frozen importlib._bootstrap>\", line 1007, in _find_and_load\n File \"<frozen importlib._bootstrap>\", line 986, in _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line 664, in _load_unlocked\n File \"<frozen importlib._bootstrap>\", line 627, in _load_backward_compatible\n File \"<frozen zipimport>\", line 259, in load_module\n File \"/tmp/pr_reviews-re1-vukl7x51.egg/pr_reviews/spiders/ReviewCrawler.py\", line 2, in <module>\n File \"<frozen importlib._bootstrap>\", line 1007, in _find_and_load\n File \"<frozen importlib._bootstrap>\", line 986, in _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line 664, in _load_unlocked\n File \"<frozen importlib._bootstrap>\", line 627, in _load_backward_compatible\n File \"<frozen zipimport>\", line 259, in load_module\n File \"/tmp/pr_reviews-re1-vukl7x51.egg/pr_reviews/items.py\", line 4, in <module>\n File \"<frozen importlib._bootstrap>\", line 1007, in _find_and_load\n File \"<frozen importlib._bootstrap>\", line 986, in _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line 664, in _load_unlocked\n File \"<frozen importlib._bootstrap>\", line 627, in _load_backward_compatible\n File \"<frozen zipimport>\", line 259, in load_module\n File \"/tmp/pr_reviews-re1-vukl7x51.egg/pycountry/__init__.py\", line 13, in <module>\n File \"/usr/local/lib/python3.9/dist-packages/pkg_resources/__init__.py\", line 478, in get_distribution\n dist = get_provider(dist)\n File \"/usr/local/lib/python3.9/dist-packages/pkg_resources/__init__.py\", line 354, in get_provider\n return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]\n File \"/usr/local/lib/python3.9/dist-packages/pkg_resources/__init__.py\", line 909, in require\n needed = self.resolve(parse_requirements(requirements))\n File \"/usr/local/lib/python3.9/dist-packages/pkg_resources/__init__.py\", line 795, in resolve\n raise DistributionNotFound(req, requirers)\npkg_resources.DistributionNotFound: The 'pycountry' distribution was not found and is required by the application\n"}
setup.py
from setuptools import setup, find_packages
setup(
name='project',
version='1.0',
packages=find_packages(),
install_requires=[
"itemadapter",
"langdetect",
"pycountry",
"scrapy",
"setuptools",
"scrapy_zyte_smartproxy",
"scrapy_user_agents"
],
entry_points={'scrapy': ['settings = pr_reviews.settings']},
)
requirements.txt
itemadapter==0.6.0
langdetect==1.0.9
pycountry==22.3.5
scrapy==2.6.1
setuptools==63.1.0
scrapy_zyte_smartproxy==2.1.0
scrapy_user_agents==0.1.1
Scrapd server
FROM ubuntu:20.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update -qq \
&& apt-get install -y tini git python3.9 python3.9-dev python3.9-distutils curl python3.9-venv build-essential libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev nginx apache2-utils \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
RUN python3.9 get-pip.py
ADD requirements.txt /
RUN python3.9 -m pip install -r /requirements.txt
RUN python3.9 -m pip install -e git+https://github.com/necrophcodr/chaperone.git#egg=chaperone
RUN mkdir /etc/chaperone.d
COPY ./scrapyd.conf /etc/scrapyd/
VOLUME /etc/scrapyd/ /var/lib/scrapyd/
EXPOSE 6800
ENTRYPOINT ["tini", "--"]
CMD ["scrapyd", "--pidfile="]