Update GitLab installation after upgrading the box - gitlab

I'm trying to update my gitlab installation. So I simply follows that documentation :
https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/update/patch_versions.md
However, when executing the following command I get that error :
$ bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse]" RAILS_ENV=production --trace
** Invoke gitlab:workhorse:install (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute gitlab:workhorse:install
rake aborted!
Gitlab::TaskFailedError: Gitlab::TaskFailedError
/home/git/gitlab/lib/tasks/gitlab/task_helpers.rb:87:in `run_command!'
/home/git/gitlab/lib/tasks/gitlab/workhorse.rake:19:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/workhorse.rake:18:in `chdir'
/home/git/gitlab/lib/tasks/gitlab/workhorse.rake:18:in `block (3 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/home/git/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/bin/rake:33:in `<top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:23:in `load'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:23:in `<top (required)>'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/cli/exec.rb:74:in `load'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/cli/exec.rb:74:in `kernel_load'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/cli/exec.rb:27:in `run'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/cli.rb:332:in `exec'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/cli.rb:20:in `dispatch'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/cli.rb:11:in `start'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/exe/bundle:34:in `block in <top (required)>'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/friendly_errors.rb:100:in `with_friendly_errors'
/home/git/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/exe/bundle:26:in `<top (required)>'
/home/git/.rvm/gems/ruby-2.3.1/bin/bundle:23:in `load'
/home/git/.rvm/gems/ruby-2.3.1/bin/bundle:23:in `<main>'
/home/git/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `eval'
/home/git/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => gitlab:workhorse:install
I'd like to give some context. I just updated my box so packages are up-to-date but I don't know what the implication with Gitlab.
Do you know how to fix that ?

Related

GitLab reports an error : PG::DuplicateSchema: ERROR: schema "gitlab_partitions_dynamic" already exists

Installed by GitLab Docker Compose and upgraded according to the official documentation
docker compose pull
docker compose up -d
The gitlab container cannot be started, the detailed error log is as follows:
...
StandardError: An error has occurred, this and all later migrations canceled:
PG::DuplicateSchema: ERROR: schema "gitlab_partitions_dynamic" already exists
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20181228175414_init_schema.rb:7:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:265:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:264:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:63:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
ActiveRecord::StatementInvalid: PG::DuplicateSchema: ERROR: schema "gitlab_partitions_dynamic" already exists
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20181228175414_init_schema.rb:7:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:265:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:264:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:63:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
PG::DuplicateSchema: ERROR: schema "gitlab_partitions_dynamic" already exists
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20181228175414_init_schema.rb:7:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:265:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:264:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:63:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
STDERR:
---- End output of "bash" "/tmp/chef-script20221119-30-1pungg2" ----
Ran "bash" "/tmp/chef-script20221119-30-1pungg2" returned 1
I tried to stop the gitlab container and restart it, still not working, how can I fix this, thanks!

gitlab.com/gitlab-org/labkit#v1.14.0: Get "https://proxy.golang.org/gitlab.com/gitlab-org/labkit/#v/v1.14.0.zip": dial tcp: i/o timeout

Due to some reason, I have to install GitLab from source code.
When I install 'gitlab-shell' follow the guide:
# Run the installation task for gitlab-shell:
sudo -u git -H bundle exec rake gitlab:shell:install RAILS_ENV=production
I got results:
client/gitlabnet.go:14:2: github.com/golang-jwt/jwt/v4#v4.4.1: Get "https://proxy.golang.org/github.com/golang-jwt/jwt/v4/#v/v4.4.1.zip": dial tcp 142.251.43.17:443: i/o timeout
internal/gitaly/gitaly.go:8:2: github.com/grpc-ecosystem/go-grpc-middleware#v1.3.0: Get "https://proxy.golang.org/github.com/grpc-ecosystem/go-grpc-middleware/#v/v1.3.0.zip": dial tcp 142.251.43.17:443: i/o timeout
internal/gitaly/gitaly.go:9:2: github.com/grpc-ecosystem/go-grpc-prometheus#v1.2.0: Get "https://proxy.golang.org/github.com/grpc-ecosystem/go-grpc-prometheus/#v/v1.2.0.zip": dial tcp 142.251.43.17:443: i/o timeout
internal/metrics/metrics.go:6:2: github.com/prometheus/client_golang#v1.12.1: Get "https://proxy.golang.org/github.com/prometheus/client_golang/#v/v1.12.1.zip": dial tcp 142.251.43.17:443: i/o timeout
……
make: *** [Makefile:57: bin/gitlab-shell] Error 1
rake aborted!
command failed: make build
/home/git/gitlab/lib/tasks/gitlab/shell.rake:37:in `block (5 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/shell.rake:35:in `each'
/home/git/gitlab/lib/tasks/gitlab/shell.rake:35:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/shell.rake:20:in `chdir'
/home/git/gitlab/lib/tasks/gitlab/shell.rake:20:in `block (3 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/sentry-ruby-core-5.1.1/lib/sentry/rake.rb:26:in `execute'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/cli/exec.rb:58:in `load'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/cli/exec.rb:58:in `kernel_load'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/cli/exec.rb:23:in `run'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/cli.rb:484:in `exec'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/cli.rb:31:in `dispatch'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/cli.rb:25:in `start'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/exe/bundle:48:in `block in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/bundler-2.3.6/exe/bundle:36:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
/usr/local/bin/ruby_executable_hooks:22:in `eval'
/usr/local/bin/ruby_executable_hooks:22:in `<main>'
Tasks: TOP => gitlab:shell:install
(See full trace by running task with --trace)
I have modified my GO111MODULE=on and GOPROXY=https://goproxy.cn,direct, but it still shows ... Get "https://proxy.golang.org/... ... i/o timeout, so what should I do or where should I modify the source code? Or are there actually other issues here that need to be addressed?

To ]8;;https://github.com/Shopify/shopify-cli/issues/new\submit an issue]8;;\ include the stack trace

Issue summary
I am getting this error whenever I try to start the development server
Expected behavior
Shopify node serve should run the development server
Actual behavior
What actually happens?
C:\Users\HP-PC\Desktop\myfirsttest>shopify node serve
X An unexpected error occured.
To ]8;;https://github.com/Shopify/shopify-cli/issues/new\submit an issue]8;;\ include the stack trace.
C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/process_supervision.rb:40:in `Integer': invalid value for Integer(): "\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000" (ArgumentError)
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/process_supervision.rb:40:in `for_ident'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/process_supervision.rb:127:in `running?'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/process_supervision.rb:62:in `start'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/tunnel.rb:189:in `start_ngrok'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/tunnel.rb:68:in `start'
from C:/Ruby30-x64/lib/ruby/3.0.0/forwardable.rb:238:in `start'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/project_types/node/commands/serve.rb:16:in `call'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/sub_command.rb:13:in `call'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/command.rb:19:in `call'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/core/executor.rb:17:in `block (2 levels) in call'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/executor.rb:37:in `block (2 levels) in with_logging'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-ui/lib/cli/ui/stdout_router.rb:169:in `with_id'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/executor.rb:36:in `block in with_logging'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-ui/lib/cli/ui.rb:176:in `log_output_to'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/executor.rb:35:in `with_logging'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/core/executor.rb:16:in `block in call'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/executor.rb:45:in `block (2 levels) in with_traps'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/executor.rb:51:in `twrap'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/executor.rb:44:in `block in with_traps'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/executor.rb:51:in `twrap'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/executor.rb:43:in `with_traps'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/core/executor.rb:15:in `call'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/core/entry_point.rb:24:in `block in call'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/core/monorail.rb:30:in `log'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/lib/shopify_cli/core/entry_point.rb:23:in `call'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/bin/shopify:37:in `block (2 levels) in <top (required)>'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/error_handler.rb:75:in `handle_abort'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/vendor/deps/cli-kit/lib/cli/kit/error_handler.rb:21:in `call'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/bin/shopify:36:in `block in <top (required)>'
from C:/Ruby30-x64/lib/ruby/gems/3.0.0/gems/shopify-cli-2.6.6/bin/shopify:47:in `<top (required)>'
from C:/Ruby30-x64/bin/shopify:23:in `load'
from C:/Ruby30-x64/bin/shopify:23:in `<main>'

Gitlab-ce backup restore failing for certain repos

I'm trying to set up a perpetually 1 day old clone of a gitlab server. The gitlab version on both machines 12.4.1-ce.0. There's a daily crontab for the following:
gitlab-rake GZIP_RSYNCABLE=yes gitlab:backup:create
Then it transfers the backup to my clone machine and it runs
gitlab-ctl stop unicorn && gitlab-ctl stop sidekiq && gitlab-rake gitlab:backup:restore BACKUP=most_recent force=yes --trace && gitlab-ctl restart
All of the infrastructure there works and I'm happy with it, but then during the restore, many many repos are failing to restore, throwing one of two messages for one of two types of restores:
repos larger than ~800MB
Error: 4:Deadline Exceeded
all wikis
Error 13:CreateRepositoryFromBundle: cmd wait failed: exit status 128
Searching around online said that this is probably a gitaly timeout issue, so I changed the default timeout from 55s to 475s, the fast from 10s to 30s, and the medium from 30s to 115s, but that made no difference.
At the end of the restore, I get this message:
2019-11-20 12:46:14 -0500 -- Restoring lfs objects ...
2019-11-20 12:46:14 -0500 -- done
** Invoke gitlab:backup:registry:restore (first_time)
** Invoke gitlab_environment
** Execute gitlab:backup:registry:restore
2019-11-20 12:46:14 -0500 -- Restoring container registry images ...
rake aborted!
Backup::Error: Backup failed
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/files.rb:81:in `run_pipeline!'
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/files.rb:44:in `restore'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:234:in `block (4 levels) in <top (required)>'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `block in execute'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `execute'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:214:in `block in invoke_with_call_chain'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:183:in `invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:71:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `block in execute'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `execute'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:214:in `block in invoke_with_call_chain'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:183:in `invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:160:in `invoke_task'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:116:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:116:in `block in top_level'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:125:in `run_with_threads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:110:in `top_level'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:83:in `block in run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:80:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
/opt/gitlab/embedded/bin/rake:23:in `load'
/opt/gitlab/embedded/bin/rake:23:in `<top (required)>'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `load'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `kernel_load'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:28:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:463:in `exec'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:27:in `dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:18:in `start'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:30:in `block in <top (required)>'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:22:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:backup:registry:restore
I'm not even sure if that's because of the same problem, or if it's entirely unrelated. Anyone know what's wrong, or where I can start looking next?

Updating failure - Gitlab CE 7.0.0 - Ubuntu 12.04 - Package installation

Upgrading Gitlab CE 7.0.0 to latest version using the package at https://packages.gitlab.com/gitlab/gitlab-ce/packages/ubuntu/precise/gitlab-ce_8.12.4-ce.0_amd64.deb failed.
I have followed this guide.
Please see below the log entries below after runnnig sudo gitlab-rake gitlab:check SANITIZE=true.
Checking GitLab Shell ...
GitLab Shell version >= 3.6.1 ? ... OK (3.6.1)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ... rake aborted!
ActiveRecord::StatementInvalid: PG::UndefinedColumn: ERROR: column projects.pending_delete does not exist
LINE 1: SELECT COUNT(*) FROM "projects" WHERE "projects"."pending_de...
^
: SELECT COUNT(*) FROM "projects" WHERE "projects"."pending_delete" = 'f'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `async_exec'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `block in exec_no_cache'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `exec_no_cache'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:584:in `execute_and_clear'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:160:in `exec_query'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract/database_statements.rb:356:in `select'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract/database_statements.rb:32:in `select_all'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract/query_cache.rb:70:in `select_all'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/calculations.rb:270:in `execute_simple_calculation'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/calculations.rb:227:in `perform_calculation'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/calculations.rb:133:in `calculate'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/calculations.rb:48:in `count'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/querying.rb:13:in `count'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/check.rake:469:in `check_repos_hooks_directory_is_link'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/check.rake:348:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
PG::UndefinedColumn: ERROR: column projects.pending_delete does not exist
LINE 1: SELECT COUNT(*) FROM "projects" WHERE "projects"."pending_de...
^
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `async_exec'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `block in exec_no_cache'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `exec_no_cache'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:584:in `execute_and_clear'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:160:in `exec_query'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract/database_statements.rb:356:in `select'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract/database_statements.rb:32:in `select_all'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract/query_cache.rb:70:in `select_all'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/calculations.rb:270:in `execute_simple_calculation'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/calculations.rb:227:in `perform_calculation'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/calculations.rb:133:in `calculate'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/calculations.rb:48:in `count'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/querying.rb:13:in `count'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/check.rake:469:in `check_repos_hooks_directory_is_link'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/check.rake:348:in `block (3 levels) in <top (required)>'
Here is the environment info run with sudo gitlab-rake gitlab:env:info
System: Ubuntu 12.04
Current User: git
Using RVM: no
Ruby Version: 2.3.1p112
Gem Version: 2.6.6
Bundler Version:1.13.1
Rake Version: 10.5.0
Sidekiq Version:4.1.4
rake aborted!
ActiveRecord::UnknownAttributeError: unknown attribute 'pending_delete' for Project.
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:59:in `rescue in _assign_attribute'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:54:in `_assign_attribute'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:41:in `block in assign_attributes'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:35:in `each'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:35:in `assign_attributes'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/attr_encrypted-3.0.3/lib/attr_encrypted/adapters/active_record.rb:28:in `perform_attribute_assignment'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/attr_encrypted-3.0.3/lib/attr_encrypted/adapters/active_record.rb:36:in `assign_attributes'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/association.rb:169:in `initialize_attributes'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/association.rb:251:in `block in build_record'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/state_machines-activerecord-0.4.0/lib/state_machines/integrations/active_record.rb:465:in `block in initialize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/core.rb:283:in `initialize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/default_value_for-3.0.2/lib/default_value_for.rb:142:in `initialize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/state_machines-activerecord-0.4.0/lib/state_machines/integrations/active_record.rb:463:in `initialize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/inheritance.rb:61:in `new'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/inheritance.rb:61:in `new'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/reflection.rb:141:in `build_association'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/association.rb:250:in `build_record'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/collection_association.rb:146:in `build'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/collection_proxy.rb:259:in `build'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/info.rake:33:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
NoMethodError: undefined method `pending_delete=' for #<Project:0x00000007337b90>
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activemodel-4.2.7.1/lib/active_model/attribute_methods.rb:433:in `method_missing'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:54:in `public_send'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:54:in `_assign_attribute'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:41:in `block in assign_attributes'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:35:in `each'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/attribute_assignment.rb:35:in `assign_attributes'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/attr_encrypted-3.0.3/lib/attr_encrypted/adapters/active_record.rb:28:in `perform_attribute_assignment'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/attr_encrypted-3.0.3/lib/attr_encrypted/adapters/active_record.rb:36:in `assign_attributes'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/association.rb:169:in `initialize_attributes'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/association.rb:251:in `block in build_record'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/state_machines-activerecord-0.4.0/lib/state_machines/integrations/active_record.rb:465:in `block in initialize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/core.rb:283:in `initialize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/default_value_for-3.0.2/lib/default_value_for.rb:142:in `initialize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/state_machines-activerecord-0.4.0/lib/state_machines/integrations/active_record.rb:463:in `initialize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/inheritance.rb:61:in `new'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/inheritance.rb:61:in `new'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/reflection.rb:141:in `build_association'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/association.rb:250:in `build_record'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/collection_association.rb:146:in `build'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/associations/collection_proxy.rb:259:in `build'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/info.rake:33:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
Tasks: TOP => gitlab:env:info
Following steps helped me complete this upgrade:
Upgrade to 7.10 using the package at https://packages.gitlab.com/gitlab/gitlab-ce/packages/el/6/gitlab-ce-7.10.0~omnibus-1.x86_64.rpm
Then update to latest version: curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash followed by sudo apt-get install gitlab-ce

Resources