A lot of problem when setting up hawkbit update server - eclipse-hawkbit

I'm trying to set up hawkBit update server as per https://www.eclipse.org/hawkbit/gettingstarted/ but a lot of issues that I do not know how to resolve.
The following is summary setup steps:
Prerequisites
sudo apt install -y default-jre default-jdk
java -version
*openjdk version "11.0.9.1" 2020-11-04*
*OpenJDK Runtime Environment (build 11.0.9.1+1-Ubuntu-0ubuntu1.18.04)*
*OpenJDK 64-Bit Server VM (build 11.0.9.1+1-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)*
sudo apt install maven
sudo apt-get -y install mariadb-server
Download build source code
git clone https://github.com/eclipse/hawkbit.git
cd hawkbit
mvn clean install
*Start hawkBit update server*
*Access http://localhost:8080/UI/login to check*
java -jar ./hawkbit-runtime/hawkbit-update-server/target/hawkbit-update-server-0.3.0-SNAPSHOT.jar
Customize hawkBit
3.1. Set up data base
sudo mysql -u root
*Welcome to the MariaDB monitor. Commands end with ; or \g.*
*Your MariaDB connection id is 31*
*Server version: 10.1.47-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04*
*Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.*
*Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.*
MariaDB [(none)]> USE mysql;
*Reading table information for completion of table and column names*
*You can turn off this feature to get a quicker startup with -A*
*Database changed*
MariaDB [mysql]> CREATE USER 'test'#'localhost' IDENTIFIED BY '123456a#';
*Query OK, 0 rows affected (0.00 sec)*
*MariaDB [mysql]> UPDATE user SET authentication_string=password('123456a#') where user='test';*
*Query OK, 0 rows affected (0.00 sec)*
*Rows matched: 1 Changed: 0 Warnings: 0*
MariaDB [mysql]> GRANT ALL PRIVILEGES ON *.* TO 'test'#'localhost';
*Query OK, 0 rows affected (0.00 sec)*
MariaDB [mysql]> UPDATE user SET plugin='mysql_native_password' WHERE User='test';
*Query OK, 1 row affected (0.01 sec)*
*Rows matched: 1 Changed: 1 Warnings: 0*
MariaDB [mysql]> FLUSH PRIVILEGES;
*Query OK, 0 rows affected (0.00 sec)*
MariaDB [mysql]> CREATE DATABASE hawkbit;
*Query OK, 1 row affected (0.00 sec)*
MariaDB [mysql]> ALTER DATABASE hawkbit COLLATE latin1_bin;
Query OK, 1 row affected (0.00 sec)
MariaDB [mysql]> grant all privileges on hawkbit.* TO 'test'#'localhost' identified by '123456a#';
*Query OK, 0 rows affected (0.01 sec)*
MariaDB [(none)]> exit
*Bye*
sudo service mysql restart
Append the following code to hawkbit-runtime/hawkbit-update-server/pom.xml
<dependency>
<groupId>org.mariadb.jdbc</groupId>
<artifactId>mariadb-java-client</artifactId>
<scope>compile</scope>
</dependency>
Override the following code to hawkbit-runtime/hawkbit-update-server/src/main/resources/application-mysql.properties:
spring.jpa.database=mysql
spring.datasource.url=jdbc:mysql://localhost:3306/hawkbit
spring.datasource.username=test
spring.datasource.password=123456a#
spring.datasource.driverClassName=org.mariadb.jdbc.Driver
Application.properties is appended with new URL:
server.hostname=http://<my_ip_addr>
server.port=<my_port>
Rebuild and run:
cd hawkbit
mvn clean install
java -jar ./hawkbit-runtime/hawkbit-update-server/target/hawkbit-update-server-0.3.0-SNAPSHOT.jar --spring.profiles.active=mysql
Open http://<my_ip_addr>:<my_port>, I cannot log in by "admin" username and "admin" password.
Run hawkBit without --spring.profiles.active=mysql, I succeed to log in by admin, admin.
Even the application.properties is kept unchanged, the results are the same.
I try to comment out all User Security:
# spring.security.user.name=admin
# spring.security.user.password={noop}admin-pwd
# spring.main.allow-bean-definition-overriding=true
and uncomment Define own users instead of default "admin" user:
hawkbit.server.im.users[0].username=test
hawkbit.server.im.users[0].password={noop}123456a#
hawkbit.server.im.users[0].firstname=Huong
hawkbit.server.im.users[0].lastname=Ha
hawkbit.server.im.users[0].permissions=ALL
the build failed with the following log:
[INFO] hawkBit :: Runtime :: Update Server ................ FAILURE [ 7.579 s]
[INFO] hawkBit :: Test Report ............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14:59 min
[INFO] Finished at: 2020-12-26T16:47:05+07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.1:test (default-test) on project hawkbit-update-server: There are test failures.
[ERROR]
[ERROR] Please refer to /home/huong/software-update-server/hawkbit/hawkbit-runtime/hawkbit-update-server/target/surefire-reports for the individual test results.
So I have some questions and hope someone could help:
The exact steps to setup hawkBit update server. Any experiences would be helpful
The hawkbit database (is created as above) is an empty database, do you have an database which is compatible with hawkBit?
Thanks,
Huong Ha

The procedure you are following is quite correct. However, there are couple of remarks regarding your approach:
You can omit running all the tests while building Hawkbit by specifying skipTests maven argument: mvn clean install -DskipTests. It will drastically decrease the build time.
You don't need to modify the application.properties in order to use MySQL database. Instead just specify the --spring.profiles.active=mysql program argument for activating the "mysql" Spring profile and adapt the application-mysql.properties as needed (in your case changing the username and password datasource config).
You need to either use the spring.security.user.name and spring.security.user.password or hawkbit.server.im.users, not both. Spring security user should work out of the box. Additionally, you should NOT comment out spring.main.allow-bean-definition-overriding=true, it is responsible for the bean overriding mechanism of Spring and has nothing to do with Spring Security.
Hawkbit is using Flyway DB version control (https://flywaydb.org) meaning that it will execute all of the required DB migration scripts during the application start.
And yes, you can run Hawkbit with java11 (see https://github.com/eclipse/hawkbit/commit/ed9a4b1bb31f4cd996e8af8867d67cdcda682d00)
Hope that brings you further ;)

Related

arangodb starter mode does not start

I have d/l'd arangodb3-linux-3.9.2 from GIT on Centos 7. I created a database dir and ran the README instructions for a standalone start. The first time it runs, I get 100 failures, the key INFO log lines seem to be
... [INFO] server started component=arangodb pid=49827 type=single
... [INFO] Wait on 49827 returned component=arangodb exit-status=1 trap-cause=-1
It creates the log file, setup.json and a single8529 dir in the database dir I sped'd. Is it just taking too long to start? The whole 100 fails take about 1 or 2 seconds.
If I try to run it again with the same README instructions, the next time I get this error:
... [FATAL] Failed to run service error="open /.../single8529/data/ENGINE: no such file"
I have also tried with --starter.host 127.0.0.1 -- to simplify
Also I and can confirm that port 8529 is open
I couldn't get arangodb 'starter' according to their README to work, but this does start the server:
arangod --database.directory MYDIR --rocksdb.max-background-jobs 4

cassandra service (3.11.5) stops automaticall after it starts/restart on AWS linux

cassandra service (3.11.5) stops automatically after it starts/restart on AWS linux.
I have fresh installation of cassandra on new instance of AWS linux (t3.xlarge) and
sudo service cassandra start
or
sudo service cassandra restart
after 1 or 2 seconds, the service stop automatically. I looked into logs and I found these.
I am not sure, I havent change configs related to snitch and its always SimpleSnitch. I dont have any multiple cassandras. Just only on single EC2.
Logs
INFO [main] 2020-02-12 17:40:50,833 ColumnFamilyStore.java:426 - Initializing system.schema_aggregates
INFO [main] 2020-02-12 17:40:50,836 ViewManager.java:137 - Not submitting build tasks for views in keyspace system as storage service is not initialized
INFO [main] 2020-02-12 17:40:51,094 ApproximateTime.java:44 - Scheduling approximate time-check task with a precision of 10 milliseconds
ERROR [main] 2020-02-12 17:40:51,137 CassandraDaemon.java:759 - Cannot start node if snitch's data center (datacenter1) differs from previous data center (dc1). Please fix the snitch configuration, decommission and rebootstrap this node or use the flag -Dcassandra.ignore_dc=true.
Installation steps
sudo curl -OL https://www.apache.org/dist/cassandra/redhat/311x/cassandra-3.11.5-1.noarch.rpm
sudo rpm -i cassandra-3.11.5-1.noarch.rpm
sudo pip install cassandra-driver
export CQLSH_NO_BUNDLED=true
sudo chkconfig --levels 3 cassandra on
The issue is in your log file:
ERROR [main] 2020-02-12 17:40:51,137 CassandraDaemon.java:759 - Cannot start node if snitch's data center (datacenter1) differs from previous data center (dc1). Please fix the snitch configuration, decommission and rebootstrap this node or use the flag -Dcassandra.ignore_dc=true.
It seems that you started the cluster, stopped it and renamed the datacenter from dc1 to datacenter1.
In order to fix:
If no data is stored, delete the data directories
If data is stored, rename the datacenter back to dc1 in the config
I had the same problem , where cassandra service immediately stops after it was started.
in the cassandra configuration file located at /etc/cassandra/cassandra.yaml change the cluster_name to the previous one, like this:
...
# The name of the cluster. This is mainly used to prevent machines in
# one logical cluster from joining another.
cluster_name: 'dc1'
# This defines the number of tokens randomly assigned to this node on the ring
# The more tokens, relative to other nodes, the larger the proportion of data
...

Failed to bring up Cloud SQL Metastore When create dataproc cluster using preview image

I am using Spark to do some computation over some data and then push to Hive. The Cloud Dataproc versions is 1.2 with Hive 2.1 included. The Merge command in Hive is only support by version 2.2 onwards. So I have to use preview version for dataproc cluster. When I use version 1.2 for dataproc cluster, I can create the cluster without any issue. I got this error "Failed to bring up Cloud SQL Metastore" when using preview version.
The initialisation script is here. Has anyone every met this problem before?
hive-metastore.service is not a native service, redirecting to systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install is-enabled hive-metastore
mysql.service is not a native service, redirecting to systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable mysql
insserv: warning: current start runlevel(s) (empty) of script `mysql` overrides LSB defaults (2 3 4 5).
insserv: warning: current stop runlevel(s) (0 1 2 3 4 5 6) of script `mysql' overrides LSB defaults (0 1 6).
Created symlink /etc/systemd/system/multi-user.target.wants/cloud-sql-proxy.service → /usr/lib/systemd/system/cloud-sql-proxy.service.
Cloud SQL Proxy installation succeeded
hive-metastore.service is not a native service, redirecting to systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install is-enabled hive-metastore
[2018-06-06T12:43:55+0000]: Failed to bring up Cloud SQL Metastore
I believe the issue may be that your metastore was initialized from an older version of Dataproc and thus has outdated schema.
If you have the failed cluster (if not, please create a new one as before, you can use --single-node option to reduce cost), then SSH to master node and upgrade schema:
$ gcloud compute ssh my-cluster-m
$ /usr/lib/hive/bin/schematool -dbType mysql -info
Hive distribution version: 2.3.0
Metastore schema version: 2.1.0 <-- you will need this
org.apache.hadoop.hive.metastore.HiveMetaException: Metastore schema version is
not compatible. Hive Version: 2.3.0, Database Schema Version: 2.1.0
*** schemaTool failed ***
$ /usr/lib/hive/bin/schematool -dbType mysql -upgradeSchemaFrom 2.1.0
Unfortunately this cluster cannot be returned to running state, so please delete and recreate it.
I have created this PR to make issue more discoverable:
https://github.com/GoogleCloudPlatform/dataproc-initialization-actions/pull/278

Submitting first job to pacemaker

I followed this guide:
https://clusterlabs.org/doc/en-US/Pacemaker/1.1-pcs/html/Clusters_from_Scratch/
I stayed with the Active/Passive DRBD file system sharing. I had to reboot my cluster and now I am getting the following error:
Current DC: rbx-1 (version 1.1.16-12.el7_4.4-94ff4df) - partition with quorum
Last updated: Tue Nov 28 17:01:14 2017
Last change: Tue Nov 28 16:40:09 2017 by root via cibadmin on rbx-1
2 nodes configured
5 resources configured
Node rbx-2: UNCLEAN (offline)
Online: [ rbx-1 ]
Full list of resources:
ClusterIP (ocf::heartbeat:IPaddr2): Started rbx-1
WebSite (ocf::heartbeat:apache): Stopped
Master/Slave Set: WebDataClone [WebData]
WebData (ocf::linbit:drbd): FAILED rbx-1 (blocked)
Stopped: [ rbx-2 ]
WebFS (ocf::heartbeat:Filesystem): Stopped
Failed Actions:
* WebData_stop_0 on rbx-1 'invalid parameter' (2): call=20, status=complete, exitreason='none',
last-rc-change='Tue Nov 28 16:27:58 2017', queued=0ms, exec=3ms
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
Any ideas?
Also does anyone have any recommended guides for submitting jobs?
This post is relatively old at this point but I'll leave this here for others to find if they stumble upon the same issue.
This problem has to do with an issue with the DRBD integration script that pacemaker uses. If it's broken, missing, has incorrect permissions, etc. you can get an error like this. In CentOS 7 that script is located at /usr/lib/ocf/resource.d/drbd
Note: This is specifically for the guide mentioned by OP but may help you:
Section 7.1 has a big "IMPORTANT" block that talks about replacing the Pacemaker integration script due to a bug. If you use the command it tells you to there, you actually replace the script with a 404 Error page which obviously doesn't work, causing the error. You can fix this issue by replacing the script with the original, either by reinstalling DRBD...
yum remove -y kmod-drbd84 drbd84-utils
yum install -y kmod-drbd84 drbd84-utils
...or finding just the drbd script elsewhere and adding/replacing it to /usr/lib/ocf/resource.d/drbd. Make sure its permissions are correct and that it is set as executable.
Hope that helps!

Cannot install Cassandra 2.X in travis-ci

I am having issues create a build with cassandra as a service with a version that is higher than 2.X (let x be higher than 1).
I have verified that
services:
- cassandra
produce a cassandra 2.0.9 as i put in my .travis.yml
$ cqlsh --execute="show version" 127.0.0.1
[cqlsh 4.1.1 | Cassandra 2.0.9 | CQL spec 3.1.1 | Thrift protocol 19.39.0]
However my project requires 2.2.4 as a minimum.
when i tried doing as per travis suggested:
before_install:
- sudo rm -rf /var/lib/cassandra/*
- wget https://archive.apache.org/dist/cassandra/2.2.4/apache-cassandra-2.2.4-bin.tar.gz && tar -xvzf apache-cassandra-2.2.4-bin.tar.gz && sudo sh apache-cassandra-2.2.4/bin/cassandra
- sleep 30
It fails to boot the cassandra as it waits at the following line:
Connection error: Could not connect to 127.0.0.1:9160
While dumping the raw_log it gets stuck here:
INFO 16:01:31 Loading org.apache.cassandra.config.CFMetaData#2716f853[cfId=5f2fbdad-91f1-3946-bd25-d5da3a5c35ec,ksName=system_auth,cfName=resource_role_permissons_index,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),comment=index of db roles with permissions granted on a resource,readRepairChance=0.0,dcLocalReadRepairChance=0.0,gcGraceSeconds=7776000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=role, type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING_COLUMN, componentIndex=0, indexName=null, indexType=null}, ColumnDefinition{name=resource, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null}],compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=3600000,caching={"keys":"ALL", "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=false]
INFO 16:01:31 Initializing system_auth.resource_role_permissons_index
Does anyone have any idea how to get travis to successful build a cassandra 2.X higher than its default?
NOTES:
My project is PHP base.
I have tried this in Container mode and as
sudo (as per travis instructions also

Resources