Import all libraries from Python package in pantsbuild - python-3.x

I want to import all the resources in my models/sources directory, into another target in pants build my contents of root/models/sources:
├── base.py
├── BUILD
├── design.py
├── email.py
├── __init__.py
└── project_management.py
The contents of my build file are as follows:
python_library(
name="base",
sources=["base.py"],
dependencies=["root/models/tokens/__init__.py:__init__"]
)
python_library(
name="design",
sources=["design.py"],
dependencies=[":base"]
)
python_library(
name="email",
sources=["email.py"],
dependencies=[":base"]
)
python_library(
name="project_management",
sources=["project_management.py"],
dependencies=[
":base",
"root/models/tokens/__init__.py:__init__"
]
)
python_library(
name="__init__",
sources=["__init__.py"],
dependencies=[
":base",
":design",
":email",
":project_management"
]
)
Now in another target, I have an
from root.models.sources import *
How would I set up the dependencies of this other target to import all the libraries in models/sources ?
p.s I know using * is not best, but that's the simplest form. Also, init is a Factory.

import the init as long as it's a Factory that references all the other files in the package.
eg:
from root.models.sources.base import Source
from root.models.sources.design import DesignSource
from root.models.sources.email import EmailSource
from root.models.sources.project_management import \
ProjectManagementSource
def SourceFactory(source: str = "default"):
sources = {
"default": Source,
"foo1": ProjectManagementSource,
"foo2": ProjectManagementSource,
"foo3": ProjectManagementSource,
"foo4": ProjectManagementSource,
"foo5": ProjectManagementSource,
"foo6": ProjectManagementSource,
"foo7": ProjectManagementSource,
"foo8": EmailSource,
"foo9": DesignSource,
}
return sources[source] if source in sources.keys() else Source

Related

azure function app python module import failure

I'm not able to execute my cloud function due to failures to import modules, module azure-identity for example, it's telling me module not found, I had my requirements.txt file:
msrestazure
azure-identity
azure.keyvault.secrets
azure-cli
and my code imports:
import logging
import json
import base64
import azure.functions as func
import requests
from azure.identity import AzureCliCredential
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.network import NetworkManagementClient
from azure.mgmt.compute import ComputeManagementClient
import os
Result: Failure Exception: ModuleNotFoundError: No module named 'azure.identity'. Troubleshooting Guide: https://aka.ms/functions-modulenotfound Stack: File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/dispatcher.py", line 318, in _handle__function_load_request func_request.metadata.entry_point) File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 42, in call raise extend_exception_message(e, message) File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 40, in call return func( * args, ** kwargs) File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/loader.py", line 85, in load_function mod = importlib.import_module(fullmodname) File "/usr/local/lib/python3.6/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/site/wwwroot/dworkerfn/init.py", line 6, in from azure.identity import AzureCliCredential**
My files hierarchy:
├── dworkerfn
│   ├── function.json
│   ├── __init__.py
│   ├── launch-worker.sh
│   └── sample.dat
├── host.json
└── requirements.txt
This issue is not related to the module itself, I tried with different modules, requests for example and always the same issue.

Application Factory Pattern: AttributeError: 'tuple' object has no attribute 'shell_context_processor' [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 2 years ago.
Improve this question
I am trying to cast my Flask app into a App Factory Pattern, but I got stuck on the following error:
Traceback (most recent call last):
File "/usr/local/bin/flask", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 967, in main
cli.main(args=sys.argv[1:], prog_name="python -m flask" if as_module else None)
File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/decorators.py", line 73, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 848, in run_command
app = DispatchingApp(info.load_app, use_eager_loading=eager_loading)
File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 305, in __init__
self._load_unlocked()
File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 330, in _load_unlocked
self._app = rv = self.loader()
File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 388, in load_app
app = locate_app(self, import_name, name)
File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 240, in locate_app
__import__(module_name)
File "/home/chrdina/python/werda/werda.py", line 7, in <module>
#app.shell_context_processor
AttributeError: 'tuple' object has no attribute 'shell_context_processor'
I tried to find out where the AttributeError happens, but I'm totally stuck. I don't unterstand where the mentioned tuple is located in my code. I tried deleting the shell_context_processor part, but then the app seems to not know it's an app at all and throws this "Error: Failed to find Flask application or factory in module "werda". Use "FLASK_APP=werda:name to specify one.").
If I then try to export FLASK_APP again (it's already set in my .flaskenv) nothing changes.
I define the shell_context_processor in my werda.py (which is my app) in the top level of my directory.
werda.py:
from app import create_app, db
from app.models import User, Role, Employee, Language, Employee_Languages, Attendance, AttendanceArchive
app = create_app()
#app.shell_context_processor
def make_shell_context():
return {'db': db, 'User': User, 'Role': Role, 'Employee':Employee, 'Language':Language, 'Employee_Languages':Employee_Languages, 'Attendance':Attendance, 'AttendanceArchive':AttendanceArchive}
init.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from flask_admin import Admin
from flask_bootstrap import Bootstrap
from flask_security import Security, SQLAlchemyUserDatastore
from flask_mail import Mail
import logging
from logging.handlers import SMTPHandler, RotatingFileHandler
import os
from config import Config
db = SQLAlchemy()
migrate = Migrate()
bootstrap = Bootstrap()
from app.models import User, Role
user_datastore = SQLAlchemyUserDatastore(db, User, Role)
security = Security()
mail = Mail()
admin = Admin()
def create_app(config_class=Config):
app=Flask(__name__)
app.config.from_object(Config)
db.init_app(app)
migrate.init_app(app, db)
from app.models import User, Role
user_datastore = SQLAlchemyUserDatastore(db, User, Role)
security.init_app(app, user_datastore)
bootstrap.init_app(app)
mail.init_app(app)
app.config['FLASK_ADMIN_SWATCH'] = 'cerulean'
from app.errors import bp as errors_bp
app.register_blueprint(errors_bp)
from app.main import bp as main_bp
app.register_blueprint(main_bp)
admin = Admin(app, name='werda', template_mode='bootstrap3')
#send errors via mail:
if not app.debug:
if app.config['MAIL_SERVER']:
auth = None
if app.config['MAIL_USERNAME'] or app.config['MAIL_PASSWORD']:
auth = (app.config['MAIL_USERNAME'], app.config['MAIL_PASSWORD'])
secure = None
if app.config['MAIL_USE_TLS']:
secure = ()
mail_handler = SMTPHandler(
mailhost=(app.config['MAIL_SERVER'], app.config['MAIL_PORT']),
fromaddr='no-reply#' + app.config['MAIL_SERVER'],
toaddrs=app.config['ADMINS'], subject='werda Failure',
credentials=auth, secure=secure)
mail_handler.setLevel(logging.ERROR)
app.logger.addHandler(mail_handler)
#save errors to log file:
if not os.path.exists('logs'):
os.mkdir('logs')
file_handler = RotatingFileHandler('logs/werda.log', maxBytes=10240,
backupCount=10)
file_handler.setFormatter(logging.Formatter(
'%(asctime)s %(levelname)s: %(message)s [in %(pathname)s:%(lineno)d]'))
file_handler.setLevel(logging.INFO)
app.logger.addHandler(file_handler)
app.logger.setLevel(logging.INFO)
app.logger.info('werda startup')
return app, admin
from app import models
Directory Structure
.
├── app
│   ├── cli.py
│   ├── errors
│   │   ├── handlers.py
│   │   ├── __init__.py
│   │   └── __pycache__
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── main
│   │   ├── forms.py
│   │   ├── __init__.py
│   │   ├── __pycache__
│   │   └── routes.py
│   ├── models.py
│   └── templates
│   ├── addnextweek.html
│   ├── base.html
│   ├── errors
│   │   ├── 403.html
│   │   ├── 404.html
│   │   └── 500.html
│   ├── index.html
│   ├── movetoarchive.html
│   ├── nextweek.html
│   ├── now.html
│   ├── russ.html
│   ├── security
│   │   ├── login_user.html
│   │   └── register_user.html
│   ├── thisweek.html
│   ├── thisweek_working.html
│   └── today.html
├── app.db
├── config.py
├── libc6_2.31-0ubuntu8+lp1871129~1_amd64.deb
├── README.md
├── requirements.txt
├── werda.py
Consider to use the init_app() function in your factory function. You also should modify your app.config.from_object(Config) to app.config.from_object(class_config)
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from flask_admin import Admin
from flask_bootstrap import Bootstrap
from flask_security import Security, SQLAlchemyUserDatastore
from flask_mail import Mail
import logging
from logging.handlers import SMTPHandler, RotatingFileHandler
import os
from config import Config
from app.models import User, Role #<-----------repositioned
db = SQLAlchemy()
migrate = Migrate()
bootstrap = Bootstrap()
user_datastore = SQLAlchemyUserDatastore() #<-------------modified
security = Security()
mail = Mail()
admin = Admin()
def create_app(config_class=Config):
app=Flask(__name__)
app.config.from_object(class_config) #<------modified
db.init_app(app)
migrate.init_app(app, db)
user_datastore.init_app = SQLAlchemyUserDatastore(app, db, User, Role) #<-------modified
security.init_app(app, user_datastore)
bootstrap.init_app(app)
mail.init_app(app)
app.config['FLASK_ADMIN_SWATCH'] = 'cerulean'
from app.errors import bp as errors_bp
app.register_blueprint(errors_bp)
from app.main import bp as main_bp
app.register_blueprint(main_bp)
admin = Admin(app, name='werda', template_mode='bootstrap3')
#send errors via mail:
if not app.debug:
if app.config['MAIL_SERVER']:
auth = None
if app.config['MAIL_USERNAME'] or app.config['MAIL_PASSWORD']:
auth = (app.config['MAIL_USERNAME'], app.config['MAIL_PASSWORD'])
secure = None
if app.config['MAIL_USE_TLS']:
secure = ()
mail_handler = SMTPHandler(
mailhost=(app.config['MAIL_SERVER'], app.config['MAIL_PORT']),
fromaddr='no-reply#' + app.config['MAIL_SERVER'],
toaddrs=app.config['ADMINS'], subject='werda Failure',
credentials=auth, secure=secure)
mail_handler.setLevel(logging.ERROR)
app.logger.addHandler(mail_handler)
#save errors to log file:
if not os.path.exists('logs'):
os.mkdir('logs')
file_handler = RotatingFileHandler('logs/werda.log', maxBytes=10240,
backupCount=10)
file_handler.setFormatter(logging.Formatter(
'%(asctime)s %(levelname)s: %(message)s [in %(pathname)s:%(lineno)d]'))
file_handler.setLevel(logging.INFO)
app.logger.addHandler(file_handler)
app.logger.setLevel(logging.INFO)
app.logger.info('werda startup')
return app, admin
from app import models
I got it!
I return'ed app and admin in create_app(), which caused the problem. And it seems that's also where the mysterious Tuple was situated.
If create_app() only return's app my app starts as it should. Now to work out how to run flask-admin correctly.
Corrected code for now:
init.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from flask_admin import Admin
from flask_bootstrap import Bootstrap
from flask_security import Security, SQLAlchemyUserDatastore
from flask_mail import Mail
import logging
from logging.handlers import SMTPHandler, RotatingFileHandler
import os
from config import Config
db = SQLAlchemy()
migrate = Migrate()
bootstrap = Bootstrap()
from app.models import User, Role
user_datastore = SQLAlchemyUserDatastore(db, User, Role)
security = Security()
mail = Mail()
admin = Admin()
def create_app(config_class=Config):
app=Flask(__name__)
app.config.from_object(Config)
db.init_app(app)
migrate.init_app(app, db)
from app.models import User, Role
user_datastore = SQLAlchemyUserDatastore(db, User, Role)
security.init_app(app, user_datastore)
bootstrap.init_app(app)
mail.init_app(app)
app.config['FLASK_ADMIN_SWATCH'] = 'cerulean'
from app.errors import bp as errors_bp
app.register_blueprint(errors_bp)
from app.main import bp as main_bp
app.register_blueprint(main_bp)
admin = Admin(app, name='werda', template_mode='bootstrap3')
#send errors via mail:
if not app.debug:
if app.config['MAIL_SERVER']:
auth = None
if app.config['MAIL_USERNAME'] or app.config['MAIL_PASSWORD']:
auth = (app.config['MAIL_USERNAME'], app.config['MAIL_PASSWORD'])
secure = None
if app.config['MAIL_USE_TLS']:
secure = ()
mail_handler = SMTPHandler(
mailhost=(app.config['MAIL_SERVER'], app.config['MAIL_PORT']),
fromaddr='no-reply#' + app.config['MAIL_SERVER'],
toaddrs=app.config['ADMINS'], subject='werda Failure',
credentials=auth, secure=secure)
mail_handler.setLevel(logging.ERROR)
app.logger.addHandler(mail_handler)
#save errors to log file:
if not os.path.exists('logs'):
os.mkdir('logs')
file_handler = RotatingFileHandler('logs/werda.log', maxBytes=10240,
backupCount=10)
file_handler.setFormatter(logging.Formatter(
'%(asctime)s %(levelname)s: %(message)s [in %(pathname)s:%(lineno)d]'))
file_handler.setLevel(logging.INFO)
app.logger.addHandler(file_handler)
app.logger.setLevel(logging.INFO)
app.logger.info('werda startup')
return app #deleted admin here
from app import models

AWS SAM Nested Application in Python with Dynamorm

I am using AWS SAM to build a Serverless application. I followed the instruction to build a nested application.
My application structure is basically the following:
.
├── MAKEFILE
├── README.md
├── __init__.py
├── apps
│ ├── __init__.py
│ ├── account
│ │ ├── __init__.py
│ │ ├── endpoints.py
│ │ ├── models.py
│ │ ├── requirements.txt
│ │ └── template.yaml
├── samconfig.toml
└── template.yaml
The requirements.txt in the folder apps/account/ has the following python packages: boto3 marshmallow and dynamorm.
The sam build and sam deploy works fine and the lambda functions are deployed correctly. However, I receive an error when calling the lambda function. The logs show the following error Unable to import module 'endpoints': No module named 'dynamorm'.
Here are excerpts from my code:
endpoints.py
import json
import boto3
from models import Account
print('Loading function')
def account_info(event, context):
apiKey = event["requestContext"]["identity"]["apiKeyId"]
account_info = Account.get(id= apiKey)
return {
"statusCode": 200,
"body": json.dumps(account_info)
}
models.py
import datetime
from dynamorm import DynaModel, GlobalIndex, ProjectAll
from marshmallow import Schema, fields, validate, validates, ValidationError
class Account(DynaModel):
# Define our DynamoDB properties
class Table:
name = 'XXXXXXXXXX'
hash_key = 'id'
read = 10
write = 5
class Schema:
id = fields.String(required=True)
name = fields.String()
email = fields.String()
phonenumber = fields.String()
status = fields.String()
I am not sure what am I missing? Are there additional instructions to build a nested app in SAM?
Thank you so much for the help!
According to https://github.com/awslabs/aws-sam-cli/issues/1213, this feature is not supported yet.
In my case, I did 'sam build' on every nested stacks and fix parent yaml template as following (use template.yaml generated by sam build command), then works. But just workaround and not nice way.
XXX_APP:
Type: AWS::Serverless::Application
Properties:
Location: nest_application/.aws-sam/build/template.yaml

Aliasing modules using NodeJS

Some context here: It's not that I cannot use Webpack, it's that I do not want to use Webpack. I would like to keep everything as "vanilla" as possible.
Currently when creating modules in a project you have to require them using either a relative or absolute path, for example in the following directory..
project/
├── index.js
├── lib/
│ ├── network/
│ │ request.js
│ │ response.js
├── pages/
│ ├── foo.js
Considering we're in index.js we would import request via
var networkRequest = require('./lib/network/request.js')
and if we're in foo.js we would import request via
var networkRequest = require('../lib/network/request.js')
What I'm wondering is that if there's any way to perhaps, set a local alias in Package.json or anywhere else like so:
localPackages = [
{ name: 'network-request', path: './lib/network/request.js' }
];
In which you could just do
var networkRequest = require('network-request')
From any file and it will provide the correct path.
Yep, that's what npm link is for. Native and out of the box.
You can also set local paths in package.json
{
"name": "baz",
"dependencies": {
"bar": "file:../foo/bar"
}
}

Hiera 3 + Puppet 4.2 can't manage empty value in yaml datasource

I'm facing some issues to use Hiera 3 with Puppet 4.2.
When applying my puppet manifest using this command:
puppet apply environments/production/manifests/init.pp --hiera_config=hiera.yaml
I get the following error:
Error: Evaluation Error: Error while evaluating a Function Call, Could not find data item myclass::ensure in any Hiera data file and no default supplied at /home/vagrant/temp/environments/production/manifests/init.pp:13:39 on node a
Here is the directory structure :
$ tree
.
├── environments
│   └── production
│   ├── config.yaml
│   └── manifests
│   └── init.pp
└── hiera.yaml
content of config.yaml:
$ cat hiera.yaml
---
:backends:
- yaml
:hierarchy:
- config
:yaml:
:datadir: environments/production
content of init.pp:
class myclass(
$version = $myclass::params::version,
$ensure = $myclass::params::ensure,
) inherits myclass::params {
notify {"$version": }
notify {"$ensure": }
}
class myclass::params {
$version = hiera('myclass::version', '1.0.0')
$ensure = hiera('myclass::ensure', 'running')
}
class {'myclass': }
content of hiera data source:
$ cat config.yaml
---
myclass::version: '1.0.0'
myclass::ensure:
Looks like Hiera can't handle empty values from a yaml data source and/or replace them by default value?

Resources