Gitlab Enterprise Edition HTTPS - gitlab

I'm setting up gitlab-ee with out a liscense. I want to to have https but its giving me this error
Recipe: letsencrypt::http_authorization
* letsencrypt_certificate[gitlab.example.com] action create
* acme_certificate[staging] action create
* file[gitlab.example.com SSL key] action create_if_missing (up to date)
================================================================================
Error executing action `create` on resource 'acme_certificate[staging]'
================================================================================
Acme::Client::Error::Malformed
------------------------------
Method not allowed
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb:77:in `block in class_from_file'
Resource Declaration:
---------------------
suppressed sensitive resource output
Compiled Resource:
------------------
suppressed sensitive resource output
System Info:
------------
chef_version=14.13.11
platform=ubuntu
platform_version=18.04
ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client
================================================================================
Error executing action `create` on resource 'letsencrypt_certificate[gitlab.example.com]'
================================================================================
Acme::Client::Error::Malformed
------------------------------
acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 25) had an error: Acme::Client::Error::Malformed: Method not allowed
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb:77:in `block in class_from_file'
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/recipes/http_authorization.rb
5: letsencrypt_certificate site do
6: crt node['gitlab']['nginx']['ssl_certificate']
7: key node['gitlab']['nginx']['ssl_certificate_key']
8: notifies :run, "execute[reload nginx]", :immediate
9: notifies :run, 'ruby_block[display_le_message]'
10: only_if { omnibus_helper.service_up?('nginx') }
11: end
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/recipes/http_authorization.rb:5:in `from_file'
letsencrypt_certificate("gitlab.example.com") do
action [:create]
default_guard_interpreter :default
declared_type :letsencrypt_certificate
cookbook_name "letsencrypt"
recipe_name "http_authorization"
crt "/etc/gitlab/ssl/gitlab.example.com.crt"
key "/etc/gitlab/ssl/gitlab.example.com.key"
alt_names []
cn "gitlab.example.com"
only_if { #code block }
end
System Info:
------------
chef_version=14.13.11
platform=ubuntu
platform_version=18.04
ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client
Running handlers:
There was an error running gitlab-ctl reconfigure:
letsencrypt_certificate[gitlab.example.com] (letsencrypt::http_authorization line 5) had an error: Acme::Client::Error::Malformed: acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 25) had an error: Acme::Client::Error::Malformed: Method not allowed
I dont see why its making it fail the creation of certifications. Its creating them and storing them in /etc/gitlab/sslanyone able to help?

Just for posterity, to complete #Rafael Pardini's answer, the mentioned solution boils down to simply commenting the acme_certificate 'staging' block (from do to end) inside the /opt/gitlab/embedded/cookbooks/letsencrypt/resources/certificate.rb file.

Related

GitLab upgrade from 9.5.10 to 10.8.7 unsuccessful

After installing the package for 10.8.7, which is the next supported upgrade path per https://docs.gitlab.com/ee/update/#upgrade-paths, gitlab-ctl reconfigure fails with:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab-ee/recipes/default.rb
================================================================================
NoMethodError
-------------
undefined method `[]' for nil:NilClass
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/secrets_helper.rb:81:in `write_to_gitlab_secrets'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/gitlab.rb:186:in `generate_secrets'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/gitlab.rb:259:in `generate_config'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:26:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:26:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab-ee/recipes/default.rb:20:in `from_file'
Relevant File Content:
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/secrets_helper.rb:
74: },
75: 'registry' => {
76: 'http_secret' => Gitlab['registry']['http_secret'],
77: 'internal_certificate' => Gitlab['registry']['internal_certificate'],
78: 'internal_key' => Gitlab['registry']['internal_key']
79: },
80: 'letsencrypt' => {
81>> 'auto_enabled' => Gitlab['letsencrypt']['auto_enabled']
82: },
83: 'mattermost' => {
84: 'email_invite_salt' => Gitlab['mattermost']['email_invite_salt'],
85: 'file_public_link_salt' => Gitlab['mattermost']['file_public_link_salt'],
86: 'sql_at_rest_encrypt_key' => Gitlab['mattermost']['sql_at_rest_encrypt_key']
87: },
88: 'postgresql' => {
89: 'internal_certificate' => Gitlab['postgresql']['internal_certificate'],
90: 'internal_key' => Gitlab['postgresql']['internal_key']
Seeing the error, I attempted to add the following to /etc/gitlab/gitlab-secrets.json:
"letsencrypt": {
"auto_enabled": false
},
Now I see a warning along with the error:
Ignoring section letsencrypt in /etc/gitlab/gitlab-secrets.json, does not exist in gitlab.rb
So I added the following to /etc/gitlab/gitlab.rb:
letsencrypt['auto_enabled'] = false
But now I get a different error when running gitlab-ctl reconfigure:
*ERROR*: Encountered unsupported config key 'letsencrypt' in /etc/gitlab/gitlab.rb.
================================================================================
ERROR: Encountered unsupported config key 'letsencrypt' in /etc/gitlab/gitlab.rb.
================================================================================
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab-ee/recipes/default.rb
================================================================================
NoMethodError
-------------
undefined method `[]=' for nil:NilClass
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/settings_helper.rb:100:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:28:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/settings_helper.rb:100:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/gitlab.rb:125:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:28:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/gitlab.rb:125:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:23:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:26:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab-ee/recipes/default.rb:20:in `from_file'
Relevant File Content:
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/settings_helper.rb:
93: def ee_attribute(name, **config)
94: config = { ee: true }.merge(config)
95: attribute(name, **config)
96: end
97:
98: def from_file(_file_path)
99: # Allow auto mash creation during from_file call
100>> Gitlab::ConfigMash.auto_vivify { super }
101: end
102:
103: def method_missing(method_name, *arguments) # rubocop:disable Style/MethodMissing
104: # Give better message for NilClass errors
105: # If there are no arguments passed, this is a 'GET' call, and if
106: # there is no matching key in the configuration, then it has not been set (not even to nil)
107: # and we will output a nicer error above the exception
108: if arguments.length.zero? && !configuration.key?(method_name)
109: breaktxt = '=' * 80
Additionally, I get the following errors if I try to run gitlab-rake cache:clear:
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/elastic.rake:61: warning: already initialized constant INDEXABLE_CLASSES
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/tasks/gitlab/elastic.rake:61: warning: previous definition of INDEXABLE_CLASSES was here
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab.rb:31: warning: already initialized constant Gitlab::VERSION
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/2_app.rb:6: warning: previous definition of VERSION was here
WARNING: Unable to check/update clone_url_prefix for Geo: undefined method `connected?' for Gitlab::Geo:Module
Did you mean? concern
rake aborted!
NameError: uninitialized constant API::Entities::ProjectPushRule
/opt/gitlab/embedded/service/gitlab-rails/lib/api/project_push_rule.rb:28:in `block (2 levels) in <class:ProjectPushRule>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/project_push_rule.rb:27:in `block in <class:ProjectPushRule>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/project_push_rule.rb:10:in `<class:ProjectPushRule>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/project_push_rule.rb:2:in `<module:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/project_push_rule.rb:1:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/v3/project_git_hook.rb:23:in `block (2 levels) in <class:ProjectGitHook>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/v3/project_git_hook.rb:22:in `block in <class:ProjectGitHook>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/v3/project_git_hook.rb:13:in `<class:ProjectGitHook>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/v3/project_git_hook.rb:3:in `<module:V3>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/v3/project_git_hook.rb:2:in `<module:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/v3/project_git_hook.rb:1:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:66:in `block in <class:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:25:in `<class:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:2:in `<module:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:1:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/rack_attack.rb:12:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:11:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => cache:clear => cache:clear:redis => environment
(See full trace by running task with --trace)
In my case, I was able to successfully upgrade to 10.8.7 by selecting an arbitrary intermediate version (in my case, 10.5.8). Not sure why directly upgrading as per the upgrade path did not work. I will continue my journey to GitLab 14 now!

Jenkins console shows permission denied error when command : terraform init runs in pipeline

Please find below pipeline script and jenkins console output which is framed after running the jenkins job.Below shared is pipeline code jenkins and jenkins console output:
pipeline {
agent any
parameters {
string(name: 'myvmname', defaultValue: 'az-k8s', description: 'VM name')
string(name: 'mymodulename', defaultValue: 'aks', description: 'Module name')
string(name: 'operation', defaultValue: 'apply', description: 'terraform operation apply/destroy')
booleanParam(name: 'AKS', defaultValue: true, description: 'AKS Cluster Creation')
}
stages {
stage('IAC AKS -Git Clone') {
when
{
expression {params.AKS==true}
}
steps {
git branch: 'main', credentialsId: 'xxxx-xxxx-xxxx-xxx', url: 'http://xxx.xx.xx.x/root/rapidopsiacaks.git'
}
}
stage('AKS Cluster Creation') {
when
{
expression {params.AKS==true}
}
steps {
sh '''echo "AKS cluster creation"
mkdir -p /var/lib/jenkins/workspace/VM_Prov_Terraform/${myvmname}
cp -r /var/lib/jenkins/workspace/TerraformAKS/* /var/lib/jenkins/workspace/VM_Prov_Terraform/${myvmname}
cd /var/lib/jenkins/workspace/VM_Prov_Terraform/${myvmname}/${mymodulename}
terraform init
terraform plan
terraform ${operation} -auto-approve
'''
}
}
}
}
WHen i am running the job / triggering the job via jenkins i am getting below error (sharing complete output of console):
+ cd /var/lib/jenkins/workspace/VM_Prov_Terraform/az-k8s/aks
+ terraform init
[31mThere are some problems with the configuration, described below.
The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.[0m[0m
[31m
[1m[31mError: [0m[0m[1mReserved argument name in module block[0m
[0m on main.tf line 24, in module "aks":
24: [4mdepends_on[0m = [azurerm_resource_group.rg]
[0m
The name "depends_on" is reserved for use in a future version of Terraform.
[0m[0m
[31m
[1m[31mError: [0m[0m[1mInvalid version constraint[0m
[0m on provider.tf line 3, in terraform:
3: azurerm = [4m{[0m
[0m 4: [4m source = "hashicorp/azurerm"[0m
[0m 5: [4m version = "~> 2.65"[0m
[0m 6: [4m }[0m
[0m
A string value is required for azurerm.
[0m[0m
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
-------------------------------xoxoxo -------------------------------
Also , when i run ( terraform init command ) in terminal getting below error :
[devopsadmin#jenkins1 aks]$ terraform init
Initializing modules...
╷
│ Error: Failed to update module manifest
│
│ Unable to write the module manifest file: open .terraform/modules/modules.json: permission denied
Please help !
Thanks in advance !

arangoimport: edge attribute missing or invalid

ArangoDB Version: 3.8
Storage Engine:
Deployment Mode: Single Server
Deployment Strategy: Manual Start
Operating System: Ubuntu 20.04
Total RAM in your machine: 32Gb
Disks in use: < SSD
Used Package: < Ubuntu .deb
Affected feature: arangoimport
(base) raphy#pc:~$ arangodb
2021-11-04T09:34:45+01:00 |INFO| Starting arangodb version 0.15.3, build 814f8be component=arangodb
2021-11-04T09:34:45+01:00 |INFO| Using storage engine 'rocksdb' component=arangodb
2021-11-04T09:34:45+01:00 |INFO| Serving as master with ID 'ef664d42' on :8528... component=arangodb
2021-11-04T09:34:45+01:00 |INFO| Waiting for 3 servers to show up.
component=arangodb
2021-11-04T09:34:45+01:00 |INFO| Use the following commands to start other servers: component=arangodb
arangodb --starter.data-dir=./db2 --starter.join 127.0.0.1
arangodb --starter.data-dir=./db3 --starter.join 127.0.0.1
2021-11-04T09:34:45+01:00 |INFO| ArangoDB Starter listening on 0.0.0.0:8528 (:8528) component=arangodb
I'm trying to import data in this way:
(base) raphy#pc:~$ arangoimport --server.database "ConceptNet" --collection "rel_type" "./ConceptNet/conceptnet.jsonl"
But I get these errors:
Connected to ArangoDB 'http+tcp://127.0.0.1:8529, version: 3.8.2, database: 'ConceptNet', username: 'root'
----------------------------------------
database: ConceptNet
collection: rel_type
create: no
create database: no
source filename: ./ConceptNet/conceptnet.jsonl
file type: json
threads: 2
on duplicate: error
connect timeout: 5
request timeout: 1200
----------------------------------------
Starting JSON import...
2021-11-04T14:49:48Z [165643] INFO [9ddf3] {general} processed 1945 bytes (3%) of input file
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 0: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"pm","_to":"am","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/fr","process":"/s/process/wikiparsec/2"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 1: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"red","_to":"amber","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 2: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"proprium","_to":"apelativum","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 3: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"s","_to":"beze\t","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 4: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"euphoria","_to":"bad_trip","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 5: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"gooder","_to":"badder","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 6: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"goodest","_to":"baddest","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 7: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"goodie","_to":"baddie","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2","contributor":"/s/resource/wiktionary/fr"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 8: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"windy","_to":"calm","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 9: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"anger","_to":"calm_down","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/fr","process":"/s/process/wikiparsec/2"}}
2021-11-04T14:49:48Z [165643] WARNING [e5a29] {general} at position 10: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"get_angry","_to":"calm_down","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/fr","process":"/s/process/wikiparsec/2"}}
created: 0
warnings/errors: 11
updated/replaced: 0
ignored: 0
This is the jsonl file I'm trying to import :
conceptnet.jsonl :
{"_from":"pm","_to":"am","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/fr","process":"/s/process/wikiparsec/2"}}
{"_from":"red","_to":"amber","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
{"_from":"proprium","_to":"apelativum","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
{"_from":"s","_to":"beze\t","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
{"_from":"euphoria","_to":"bad_trip","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
{"_from":"gooder","_to":"badder","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
{"_from":"goodest","_to":"baddest","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
{"_from":"goodie","_to":"baddie","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2","contributor":"/s/resource>
{"_from":"windy","_to":"calm","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
{"_from":"anger","_to":"calm_down","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/fr","process":"/s/process/wikiparsec/2"}}
{"_from":"get_angry","_to":"calm_down","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/fr","process":"/s/process/wikiparsec/2"}}
I tried to modify the line in the jsonl file as follows:
{"_from":"pm","_to":"am","rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/fr","process":"/s/process/wikiparsec/2"}
But still get this error:
(base) raphy#pc:~$ arangoimport --server.database "ConceptNet" --collection "rel_type" "./ConceptNet/conceptnet.jsonl"
Please specify a password:
Connected to ArangoDB 'http+tcp://127.0.0.1:8529, version: 3.8.2, database: 'ConceptNet', username: 'root'
----------------------------------------
database: ConceptNet
collection: rel_type
create: no
create database: no
source filename: ./ConceptNet/conceptnet.jsonl
file type: json
threads: 2
on duplicate: error
connect timeout: 5
request timeout: 1200
----------------------------------------
Starting JSON import...
2021-11-04T18:48:55Z [37684] WARNING [e5a29] {general} at position 0: creating document failed with error 'edge attribute missing or invalid', offending document: {"_from":"pm","_to":"am","rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/fr","process":"/s/process/wikiparsec/2"}
What am I doing wrongly or missing? How to solve the problem?
I found that saving the documents into the jsonl file as following, solves the problem:
conceptnet.jsonl :
{"_from":"conceptnet/pm","_to":"conceptnet/am","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/fr","process":"/s/process/wikiparsec/2"}}
{"_from":"conceptnet/red","_to":"conceptnet/amber","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}
{"_from":"conceptnet/proprium","_to":"conceptnet/apelativum","rel":{"rel_type":"Antonym","language":"en","license":"-sa/4.0","sources":"/s/resource/wiktionary/en","process":"/s/process/wikiparsec/2"}}

Internal Server Error 500 while accessing $GITLAB/admin/runners

I have restored Gitlab from a backup, now every time I try to access the runners webpage, I get a Internal Server error.
I have tried uninstalling all configured gitlab-runners and accessing it, the problem persists. Here is a trace from the gitlab-rails/production.log:
Started GET "/admin/runners" for 127.0.0.1 at 2019-01-16 07:17:12 -0500
Processing by Admin::RunnersController#index as HTML
Completed 500 Internal Server Error in 66ms (ActiveRecord: 5.7ms)
ActionView::Template::Error ():
37:
38: .col-sm-6
39: .bs-callout
40: = render partial: 'ci/runner/how_to_setup_runner',
41: locals: { registration_token: Gitlab::CurrentSettings.runners_registration_token,
42: type: 'shared',
43: reset_token_url: reset_registration_token_admin_application_settings_path }
lib/gitlab/crypto_helper.rb:27:in `aes256_gcm_decrypt'
app/models/concerns/token_authenticatable_strategies/encrypted.rb:55:in `get_token'
app/models/concerns/token_authenticatable_strategies/base.rb:33:in `ensure_token!'
app/models/concerns/token_authenticatable.rb:43:in `block in add_authentication_token_field'
app/models/application_setting.rb:409:in `runners_registration_token'
lib/gitlab/current_settings.rb:19:in `method_missing'
app/views/admin/runners/index.html.haml:40:in `_app_views_admin_runners_index_html_haml___977288809002382166_69944849285200'
app/controllers/application_controller.rb:116:in `render'
lib/gitlab/i18n.rb:55:in `with_locale'
lib/gitlab/i18n.rb:61:in `with_user_locale'
app/controllers/application_controller.rb:420:in `set_locale'
lib/gitlab/middleware/multipart.rb:103:in `call'
lib/gitlab/request_profiler/middleware.rb:16:in `call'
lib/gitlab/middleware/go.rb:19:in `call'
lib/gitlab/etag_caching/middleware.rb:13:in `call'
lib/gitlab/middleware/correlation_id.rb:16:in `block in call'
lib/gitlab/correlation_id.rb:15:in `use_id'
lib/gitlab/middleware/correlation_id.rb:15:in `call'
lib/gitlab/middleware/read_only/controller.rb:40:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/request_context.rb:20:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:29:in `call'
lib/gitlab/middleware/release_env.rb:13:in `call'
Started GET "/-/metrics" for 127.0.0.1 at 2019-01-16 07:17:24 -0500
Processing by MetricsController#index as HTML
Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.0ms)
I could resolve the issue by following official gitlab documentation:
gitlab-rails dbconsole
-- Clear project tokens
UPDATE projects SET runners_token = null, runners_token_encrypted = null;
-- Clear group tokens
UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;
-- Clear instance tokens
UPDATE application_settings SET runners_registration_token_encrypted = null;
-- Clear runner tokens
UPDATE ci_runners SET token = null, token_encrypted = null;
https://docs.gitlab.com/ee/raketasks/backup_restore.html#reset-runner-registration-tokens
Edit:
After clearing existing pipeline jobs (see above), I could still not open the ci settings page for some migrated projects where I had set environment variables. In this case try to remove them:
gitlab-rails dbconsole
SELECT * FROM ci_variables;
DELETE FROM ci_variables WHERE project_id='XX';
Hope that helps.
You did a restore, but probably did not restore the secrets store (/etc/gitlab/gitlab-secrets.json or $GITLAB_HOME/config/secrets.json). Same issue as gitlab 500 errors in the admin area
You can reset it like this from the rails console on the gitlab server:
root#gitlab:/# gitlab-rails console
-------------------------------------------------------------------------------------
GitLab: 11.5.1 (c90ae59)
GitLab Shell: 8.4.1
postgresql: 9.6.8
-------------------------------------------------------------------------------------
Both Deployment and its :status machine have defined a different default for "status". Use only one or the other for defining defaults to avoid unexpected behaviors.
Loading production environment (Rails 4.2.10)
irb(main):001:0> ApplicationSetting.current.reset_runners_registration_token!
=> true
i've solved this,maybe it is usefal
a. Reset CI/CD variables
gitlab-rails dbconsole
SELECT * FROM public."ci_group_variables";
SELECT * FROM public."ci_variables";
DELETE FROM ci_group_variables;
DELETE FROM ci_variables;
b. Reset runner registration tokens
gitlab-rails dbconsole
-- Clear project tokens
UPDATE projects SET runners_token = null, runners_token_encrypted = null;
-- Clear group tokens
UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;
-- Clear instance tokens
UPDATE application_settings SET runners_registration_token_encrypted = null;
UPDATE application_settings SET encrypted_ci_jwt_signing_key = null;
-- Clear runner tokens
UPDATE ci_runners SET token = null, token_encrypted = null;
c. Reset pending pipeline jobs
sudo gitlab-rails dbconsole
-- Clear build tokens
UPDATE ci_builds SET token = null, token_encrypted = null;
d. Fix project integrations
gitlab-rails dbconsole
-- truncate web_hooks table
TRUNCATE web_hooks CASCADE;
from gitlab

GitLab (6.6.5) Omnibus configuration for PostgreSQL

ls -la /etc/gitlab/
-rwxrwxrwx 1 git git gitlab.rb
cat /etc/gitlab/gitlab.rb
# Disable the built-in Postgres
postgresql['enable'] = false
# Fill in the values for database.yml
gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'unicode'
gitlab_rails['db_database'] = 'gitlab_db_name'
gitlab_rails['db_pool'] = 10
gitlab_rails['db_host'] = 'aws-RDShost.us-east-1.rds.amazonaws.com'
gitlab_rails['db_port'] = 5432
gitlab_rails['db_username'] = "gitlab_user"
gitlab_rails['db_password'] = 'mypass'
Now the problem is that 6.6.5 gitlab is not overwriting the database.rb file sitting at /var/opt/gitlab/gitlab-rails/etc/database.rb
Full error stack:
Compiling Cookbooks...
Recipe: gitlab::default
* directory[/etc/gitlab] action create
- change owner from 'git' to 'root'
- change group from 'git' to 'root'
[2015-04-16T20:32:04+00:00] WARN: Cloning resource attributes for directory[/var/opt/gitlab/gitlab-rails/etc] from prior resource (CHEF-3694)
[2015-04-16T20:32:04+00:00] WARN: Previous directory[/var/opt/gitlab/gitlab-rails/etc]: /opt/gitlab/embedded/cookbooks/gitlab/recipes/gitlab-rails.rb:36:in `block in from_file'
[2015-04-16T20:32:04+00:00] WARN: Current directory[/var/opt/gitlab/gitlab-rails/etc]: /opt/gitlab/embedded/cookbooks/gitlab/definitions/unicorn_config.rb:21:in `block in from_file'
[2015-04-16T20:32:04+00:00] WARN: Cloning resource attributes for service[unicorn] from prior resource (CHEF-3694)
[2015-04-16T20:32:04+00:00] WARN: Previous service[unicorn]: /opt/gitlab/embedded/cookbooks/gitlab/recipes/default.rb:58:in `block in from_file'
[2015-04-16T20:32:04+00:00] WARN: Current service[unicorn]: /opt/gitlab/embedded/cookbooks/runit/definitions/runit_service.rb:173:in `block in from_file'
[2015-04-16T20:32:04+00:00] WARN: Cloning resource attributes for service[sidekiq] from prior resource (CHEF-3694)
[2015-04-16T20:32:04+00:00] WARN: Previous service[sidekiq]: /opt/gitlab/embedded/cookbooks/gitlab/recipes/default.rb:58:in `block in from_file'
[2015-04-16T20:32:04+00:00] WARN: Current service[sidekiq]: /opt/gitlab/embedded/cookbooks/runit/definitions/runit_service.rb:173:in `block in from_file'
Recipe: gitlab::postgresql_disable
* link[/opt/gitlab/service/postgresql] action delete (up to date)
Recipe: gitlab::database_migrations
* execute[initialize database] action nothing (skipped due to action :nothing)
* execute[chown git:git /opt/gitlab/embedded/service/gitlab-rails/db/schema.rb] action run
- execute chown git:git /opt/gitlab/embedded/service/gitlab-rails/db/schema.rb
* execute[migrate database] action run
Error executing action `run` on resource 'execute[migrate database]'
Mixlib::ShellOut::ShellCommandFailed
Expected process to exit with [0], but received '1'
Begin output of /opt/gitlab/bin/gitlab-rake db:migrate
STDOUT:
STDERR: rake aborted!
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?

Resources