Pynamodb Transaction not using local endpoint - python-3.x

Regardless of whatever I tried, pynamodb is not connecting to my local dynamodb.
Below is my model,
class OrderMarker(Model):
class Meta:
table_name = 'ddb_mrker_table'
connection = TableConnection(table_name='ddb_mrker_table', host='http://localhost:8000')
OrderUniqueueId = UnicodeAttribute(hash_key=True)
Below is the exception,
Traceback (most recent call last):
File "C:\backend\orders\snapshot\app.py", line 451, in <module>
main()
File "C:\backend\orders\snapshot\app.py", line 445, in main
OrderMarker.create_table(wait=True)
File "C:\Users\user1\AppData\Local\Programs\Python\Python39\lib\site-packages\pynamodb\models.py", line 798, in create_table
if not cls.exists():
File "C:\Users\user1\AppData\Local\Programs\Python\Python39\lib\site-packages\pynamodb\models.py", line 762, in exists
cls._get_connection().describe_table()
File "C:\Users\user1\AppData\Local\Programs\Python\Python39\lib\site-packages\pynamodb\connection\table.py", line 294, in describe_table
return self.connection.describe_table(self.table_name)
File "C:\Users\user1\AppData\Local\Programs\Python\Python39\lib\site-packages\pynamodb\connection\base.py", line 787, in describe_table
raise TableError("Unable to describe table: {}".format(e), e)
pynamodb.exceptions.TableError: Unable to describe table: Could not connect to the endpoint URL: "https://dynamodb.us-east-1.amazonaws.com/"
As you can see , pynamodb is trying to access https://dynamodb.us-east-1.amazonaws.com/ eventhough I have specified the local endpoint url. This is the case always, even when I try transactions too.
Local ddb server is run as shown below,
java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -inMemory -delayTransientStatuses
A quick check using aws dynamodb list-tables --endpoint-url http://localhost:8000 shows local server is running fine!
Any idea?

Related

I got a error while running server in django i was using git before

bibek#sweety-babe:~/Desktop/Friends$ python3 manage.py runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
Exception in thread django-main-thread:
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 946, in run
self._target(*self._args, **self._kwargs)
File "/home/bibek/.local/lib/python3.10/site-packages/django/utils/autoreload.py", line 53, in wrapper
fn(*args, **kwargs)
File "/home/bibek/.local/lib/python3.10/site-packages/django/core/management/commands/runserver.py", line 120, in inner_run
self.check_migrations()
File "/home/bibek/.local/lib/python3.10/site-packages/django/core/management/base.py", line 458, in check_migrations
executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
File "/home/bibek/.local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 18, in __init__
self.loader = MigrationLoader(self.connection)
File "/home/bibek/.local/lib/python3.10/site-packages/django/db/migrations/loader.py", line 49, in __init__
self.build_graph()
File "/home/bibek/.local/lib/python3.10/site-packages/django/db/migrations/loader.py", line 274, in build_graph
raise exc
File "/home/bibek/.local/lib/python3.10/site-packages/django/db/migrations/loader.py", line 248, in build_graph
self.graph.validate_consistency()
File "/home/bibek/.local/lib/python3.10/site-packages/django/db/migrations/graph.py", line 195, in validate_consistency
[n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
File "/home/bibek/.local/lib/python3.10/site-packages/django/db/migrations/graph.py", line 195, in <listcomp>
[n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
File "/home/bibek/.local/lib/python3.10/site-packages/django/db/migrations/graph.py", line 58, in raise_error
raise NodeNotFoundError(self.error_message, self.key, origin=self.origin)
django.db.migrations.exceptions.NodeNotFoundError: Migration base.0001_initial dependencies reference nonexistent parent node ('auth', '0012_alter_user_first_name_max_length')
i am getting this error suddenly after git push. I dont know what is the reason behind this please help me.
this is just dummy (The POST method is used to send data to the server to create/update a resource on the server. In this HTTP POST request example, the Content-Type request header indicates the data type in the body of the POST message, and the Content-Length request header indicates the size of the data in the body of the POST request. The sample data is listed in the Content tab, and additional headers are listed in the Headers tab. The data is sent to the server in the body of the POST message.)
I'm thinking you probably didn't track the migration file when pushing, try pulling and see. if it doesn't work then I suggest you delete the files in the migration folder apart from the init file then run make migrations again

Unable to run index migration in OpenSearch

I have a docker compose running where a django backend, opensearch & opensearch dashboard are running. I have connected the backend to talk to opensearch and I'm able to query it successfully. I'm trying to create indexes using this command inside the docker container.
./manage.py opensearch --rebuild
Reference: https://django-opensearch-dsl.readthedocs.io/en/latest/getting_started/#create-and-populate-opensearchs-indices
I get the following error when I run the above command
root#ed186e462ca3:/app# ./manage.py opensearch --rebuild
/usr/local/lib/python3.6/site-packages/OpenSSL/crypto.py:8: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
from cryptography import utils, x509
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 224, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 37, in load_command_class
return module.Command()
File "/usr/local/lib/python3.6/site-packages/django_opensearch_dsl/management/commands/opensearch.py", line 32, in __init__
if settings.TESTING: # pragma: no cover
File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 80, in __getattr__
val = getattr(self._wrapped, name)
AttributeError: 'Settings' object has no attribute 'TESTING'
Sentry is attempting to send 1 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit
I'm not sure where I'm going wrong. Any help would be greatful.
TIA
For future reference, this was indeed a bug in django-opensearch-dsl which was fix in the 0.3.0 release.

Getting psycopg.ProgrammingError: invalid connection option "database" when pasing database parameter

Hey I am new to Databases and I decided to use Postgresql for convinience. And I am using an adapter for the Python programming language of the database named Psycopg I followed the installation tutorial of Psycopg2 but I was getting an error so I decided to install psycopg3 and it installed successfully! but when I pass the database parametere I get the following error:
Traceback (most recent call last):
File "C:\Users\Aditya\AppData\Local\Programs\Python\Python310\lib\site-packages\psycopg\conninfo.py", line 97, in _parse_conninfo
return pq.Conninfo.parse(conninfo.encode())
File "psycopg_binary\\pq/conninfo.pyx", line 30, in psycopg_binary.pq.Conninfo.parse
psycopg.OperationalError: invalid connection option "database"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\Users\Aditya\Desktop\Aditya\TGbot\dbhelper.py", line 3, in <module>
conn = psycopg.connect(
File "C:\Users\Aditya\AppData\Local\Programs\Python\Python310\lib\site-packages\psycopg\connection.py", line 561, in connect
conninfo = make_conninfo(**params)
File "C:\Users\Aditya\AppData\Local\Programs\Python\Python310\lib\site-packages\psycopg\conninfo.py", line 56, in make_conninfo
_parse_conninfo(conninfo)
File "C:\Users\Aditya\AppData\Local\Programs\Python\Python310\lib\site-packages\psycopg\conninfo.py", line 99, in _parse_conninfo
raise e.ProgrammingError(str(ex))
psycopg.ProgrammingError: invalid connection option "database"
But when i didnt pass the database argument i didnt get any kind of error... here's my code:
import psycopg
conn = psycopg.connect(
host="localhost",
database="suppliers",
user="postgres",
password="pas")
What i am doing wrong here? I am sure that the i have created the database with that name and the password is also correct.
You look to be using psycopg3. database was a deprecated parameter in psycopg2 and not allowed in psycopg3. You will need to use dbname per the list here as found in page in the psycopg3 page for connect.

Python Firestore insert return error 503 DNS resolution failed

I have a problem during the execution of my python script from crontab, which consists of an insert operation in the firestore database.
db.collection(u'ab').document(str(row["Name"])).collection(str(row["id"])).document(str(row2["id"])).set(self.packStructure(row2))
When I execute normally with python3 script.py command it works, but when I execute it from crontab it return the following error:
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/axatel/angel_bridge/esportazione_firebase/main.py", line 23, in <module>
dato.getDati(dato, db, cursor, cursor2, fdb, select, anagrafica)
File "/home/axatel/angel_bridge/esportazione_firebase/dati.py", line 19, in getDati
db.collection(u'ab').document(str(row["Name"])).collection(str(row["id"])).document(str(row2["id"])).set(self.packStructure(row2))
File "/home/axatel/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/document.py", line 234, in set
write_results = batch.commit()
File "/home/axatel/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/batch.py", line 147, in commit
metadata=self._client._rpc_metadata,
File "/home/axatel/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/gapic/firestore_client.py", line 1121, in commit
request, retry=retry, timeout=timeout, metadata=metadata
File "/home/axatel/.local/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py", line 145, in __call__
return wrapped_func(*args, **kwargs)
File "/home/axatel/.local/lib/python3.7/site-packages/google/api_core/retry.py", line 286, in retry_wrapped_func
on_error=on_error,
File "/home/axatel/.local/lib/python3.7/site-packages/google/api_core/retry.py", line 184, in retry_target
return target()
File "/home/axatel/.local/lib/python3.7/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout
return func(*args, **kwargs)
File "/home/axatel/.local/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.ServiceUnavailable: 503 DNS resolution failed for service: firestore.googleapis.com:443
I really don't understand what's the problem, because the connection at the database works every time the script is started in both ways.
Is there a fix for this kind of issue?
I found something that might be helpful. There is nice troubleshooting guide and there is a part there, which seems to be related:
If your command works by invoking a runtime like python some-command.py perform a few checks to determine that the runtime
version and environment is correct. Each language runtime has quirks
that can cause unexpected behavior under crontab.
For python you might find that your web app is using a virtual
environment you need to invoke in your crontab.
I haven't seen such error running Firestore API, but this seems to match to your issue.
I found the solution.
The problem occured because the timeout sleep() value was lower than expected, so the database connection function starts too early during boot phase of machine. Increasing this value to 45 or 60 seconds fixed the problem.
#time.sleep(10) # old version
time.sleep(60) # working version
fdb = firebaseConnection()
def firebaseConnection():
# firebase connection
cred = credentials.Certificate('/database/axatel.json')
firebase_admin.initialize_app(cred)
fdb = firestore.client()
if fdb:
return fdb
else:
print("Error")
sys.exit()

Cqlsh error on cassandra 2.0.1

We recently upgraded to cassandra 2.0.1 with cqlsh 4.0.1. I am seeing timeout errors/ broken pipe while using the cqlsh client. Please see error trace below. I have verified that the cluster is Up using nodetool and I am able to read/write using mapreduce. Please advice.
Thanks,
Prateek
Traceback (most recent call last):
File "./bin/cqlsh", line 897, in perform_statement_untraced
self.cursor.execute(statement, decoder=decoder)
File "./bin/../lib/cql-internal-only-1.4.0.zip/cql-1.4.0/cql/cursor.py", line 80, in execute
response = self.get_response(prepared_q, cl)
File "./bin/../lib/cql-internal-only-1.4.0.zip/cql-1.4.0/cql/thrifteries.py", line 77, in get_response
return self.handle_cql_execution_errors(doquery, compressed_q, compress, cl)
File "./bin/../lib/cql-internal-only-1.4.0.zip/cql-1.4.0/cql/thrifteries.py", line 96, in handle_cql_execution_errors
return executor(*args, **kwargs)
File "./bin/../lib/cql-internal-only-1.4.0.zip/cql-1.4.0/cql/cassandra/Cassandra.py", line 1782, in execute_cql3_query
self.send_execute_cql3_query(query, compression, consistency)
File "./bin/../lib/cql-internal-only-1.4.0.zip/cql-1.4.0/cql/cassandra/Cassandra.py", line 1793, in send_execute_cql3_query
self._oprot.trans.flush()
File "./bin/../lib/thrift-python-internal-only-0.9.1.zip/thrift/transport/TTransport.py", line 292, in flush
self.__trans.write(buf)
File "./bin/../lib/thrift-python-internal-only-0.9.1.zip/thrift/transport/TSocket.py", line 128, in write
plus = self.handle.send(buff)
error: [Errno 32] Broken pipe
If you have an open cqlsh session, it will always give you Errno 32 if the Cassandra instance that it connected to was stopped or even just restarted. You will have to restart cqlsh in order to re-establish a connection to the server.
If you see this problem without having stopped or restarted a Cassandra server, then please supply and additional details about conditions that lead up to this error.

Resources