Jenkins ImportError module not found - python-3.x

I have a file structure like this:
Main/
|----unit_tests
| |---test_main.py
| |---__init__.py
|--main.py
|--__init__.py
|--requirements.txt
test_main.py:
import unittest.mock
from main import MainFunction
from nose.tools import assert_is_not_none, assert_is_none, assert_equal
class TestMainLogic(unittest.case):
~~~ code here ~~~~
When running my unit tests in test_main.py from Pycharm, all is well. However when I try to run it in a Jenkins setup, I get:
+ python3 unit_tests/test_main.py
Traceback (most recent call last):
File "unit_tests/test_main.py", line 4, in <module>
from main import MainFunction
ModuleNotFoundError: No module named 'main'
Here's my Jenkinsfile:
pipeline
{
environment
{
PATH = "/home/jenkins/.local/bin:$PATH"
}
stages
{
stage('build')
{
steps
{
sh 'python3 --version'
sh 'wget https://bootstrap.pypa.io/get-pip.py'
sh 'python3 get-pip.py'
sh 'pip3 install -r requirements.txt'
echo 'build phase has finished'
}
}
stage('Lint')
{
steps
{
sh 'pylint unit_tests/test_main.py'
}
}
stage('test')
{
steps
{
sh 'python3 unit_tests/test_main.py'
}
post
{
always
{
cleanWs()
}
}
}
}
}
I've followed several posts and I'm just not getting an answer that works. I tried:Python module import failure in Jenkins and its linked answers. I do not have access to ssh into the Jenkins server and find the path so I need a way to set it at execution.

you need to use absolute import instead of relative in your test_main.py file.
in test_main.py
replace:
from main import MainFunction
# SOME CODE
with:
from Main.main import MainFunction
# SOME CODE

Related

Vscode python file runs fine but ModuleNotFoundError is thrown when code-runner is used

Hello I have the following directory structure for my project in vscode:
--- my_programs/----|
|--my_code/----|
|__tests/
|
|__libs/my_package/classes.py
|
|__main/main.py
When I run main.py in the vscode terminal it runs fine. As soon as I run it using "Run Code" (which is an extension) it complains about the following:
`
Running] /Users/username/my_programs/venv/bin/python -u "/Users/username/my_programs/my_code/main/main.py"
Traceback (most recent call last):
File "/Users/username/my_programs/my_code/main/main.py", line 2, in <module>
from libs.my_package.classes import MyClass
ModuleNotFoundError: No module named 'libs'
`
main.py looks like this:
import os
from libs.my_package.classes import MyClass
dir_path = os.path.dirname(os.path.realpath(__file__))
print (dir_path)
cwd = os.getcwd()
print(cwd)
os.chdir(dir_path)
cwd = os.getcwd()
print (cwd)
m = MyClass("Hello Hello")
print(m)
My settings.json looks like the following:
{
"terminal.integrated.env.osx": {"PYTHONPATH": "${workspaceFolder}/my_code"},
"python.envFile": "${workspaceFolder}/.vscode/env",
"python.analysis.extraPaths": [
"my_programs",
],
"python.testing.pytestArgs": [
"my_code"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true,
"python.testing.cwd": "${workspaceFolder}",
"python.terminal.activateEnvironment": true,
"python.terminal.activateEnvInCurrentTerminal": true,
"code-runner.fileDirectoryAsCwd": false,
"code-runner.executorMap":
{
"python": "$pythonPath -u $fullFileName"
},
}
Why does it work when I click "Run Python File" and not when I do "Run Code"? FYI, it also works manually from the terminal.

How to use global credentials in python script invoked by jenkins pipeline

I'm very new to working with jenkins, so far I was able to run simple pipeline with simple pip install, but I need to pass global credentials from Jenkins into python script test.py invoked by jenkinsfile.
pipeline {
options { timeout(time: 30, unit: 'MINUTES')
buildDiscarder(logRotator(numToKeepStr: '30', artifactNumToKeepStr: '30')) }
agent { label 'ops_slave' }
stages {
stage('Environment Build') {
steps {
echo "Hello World!"
sh "echo Hello from the shell"
sh "hostname"
sh "uptime"
sh "python3 -m venv test_env"
sh "source ./test_env/bin/activate"
sh "pip3 install pandas psycopg2"
sh """echo the script is working"""
withCredentials([[
$class: 'UsernamePasswordMultiBinding',
credentialsId: 98,
usernameVariable: 'user',
passwordVariable: 'pw',
]])
sh """python3 bartek-jenkins-testing/python/test.py"""
}
}
}
}
I've seen implementation where you use argparse, but it's way above my level at this point, and I believe there is a way to reference it from python script or jenkins directly to pass to python script. I was googling for some time now, but I'm not sure questions I'm asking are correct.
My python script should be able to get username and password from Jenkins global credentials ID98:
print('Hello World this is python')
import pandas as pd
print(pd.__version__)
import pyodbc
import psycopg2
# can pass environemtn variables
connection = psycopg2.connect(
host="saturn-dv",
database="saturn_dv",
port='8080',
user='saturn_user_bartek_malysz',
password='')
connection.set_session(readonly=True)
query = """
SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'
ORDER BY table_schema,table_name;"""
data = pd.read_sql(query, connection)
print(data)
A straightforward way is to leverage environment variable as following
// Jenkinsfile
withCredentials([[
$class: 'UsernamePasswordMultiBinding',
credentialsId: 98,
usernameVariable: 'user',
passwordVariable: 'pw',
]]) {
sh """
export DB_USERNAME="${user}"
export DB_PASSWORD="${pw}"
python3 bartek-jenkins-testing/python/test.py
"""
}
// test.py
connection = psycopg2.connect(
host="saturn-dv",
database="saturn_dv",
port='8080',
user=os.getenv('DB_USERNAME'),
password=os.getenv('DB_PASSWORD'))

Importing class from the root folder from a class in a package: unable to resolve class

I have the following directory/package structure in my groovy project:
src
+ progressManagers (package)
| |
| + LoadSensitiveProgressManager.groovy
+ TemporaryRegistry.groovy
Contents of TemporaryRegistry.groovy
// here is no package definition set
public class TemporaryRegistry {}
Contents of LoadSensitiveProgressManager.groovy
package progressManagers
// this gets marked red in my IDE (Intellij Idea)
import TemporaryRegistry
class LoadSensitiveProgressManager implements GenericProgressManager {
}
When i run my code in Scriptrunner for Jira Server the class is not found:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
file:/data/atlassian/application-data/jira/scripts/progressManagers/LoadSensitiveProgressManager.groovy: 7: unable to resolve class TemporaryRegistry
# line 7, column 1.
import TemporaryRegistry
^
1 error
Why is that?
I can "import TemporaryRegistry" from every groovy file in the src folder.
Thanks in advance
Jens

Jenkins. Invalid agent type "docker" specified. Must be one of [any, label, none]

My JenkinsFile looks like:
pipeline {
agent {
docker {
image 'node:12.16.2'
args '-p 3000:3000'
}
}
stages {
stage('Build') {
steps {
sh 'node --version'
sh 'npm install'
sh 'npm run build'
}
}
stage ('Deliver') {
steps {
sh 'readlink -f ./package.json'
}
}
}
}
I used to have Jenkins locally and this configuration worked, but I deployed it to a remote server and get the following error:
WorkflowScript: 3: Invalid agent type "docker" specified. Must be one of [any, label, none] # line 3, column 9.
docker {
I could not find a solution to this problem on the Internet, please help me
You have to install 2 plugins: Docker plugin and Docker Pipeline.
Go to Jenkins root page > Manage Jenkins > Manage Plugins > Available and search for the plugins. (Learnt from here).
instead of
agent {
docker {
image 'node:12.16.2'
args '-p 3000:3000'
}
}
try
agent {
any {
image 'node:12.16.2'
args '-p 3000:3000'
}
}
that worked for me.
For those that are using CasC you might want to include in plugin declaration
docker:latest
docker-commons:latest
docker-workflow:latest

How using Vim alphabetically sort JS es6 imports

I use vim as IDE for typescript project.
import { FlightInfo } from './FlightInfo'
import { InfoBlockProps, InfoRowProps, INavigationFlightOfferDataProps } from './interfaces'
import { getDiscountData, formatDataByServicesType, selectAdministrationFee } from './functions'
Also, I use ts-lint rule for check sorting:
...
ordered-imports": [
true,
{
"import-sources-order": "lowercase-first",
"named-imports-order": "lowercase-first"
}
],
...
And get errors:
ERROR: 11:1 ordered-imports Import sources within a group must be alphabetized.
ERROR: 11:10 ordered-imports Named imports must be alphabetized.
ERROR: 12:1 ordered-imports Import sources within a group must be alphabetized.
ERROR: 12:10 ordered-imports Named imports must be alphabetized.
I am searching for a solution or plugin for fix this sorting errors.
In this situation, for me very well works ts-lint --fix -c ./ts-congig.json my-file.ts command.

Resources