I have been using Kusto Query Language (KQL) and running queries from Jupyter Lab through the help of magic-command %%kql as instructed in the link below:
https://github.com/microsoft/jupyter-Kqlmagic
Lately, I started getting the following error for the same queries that I used to run without any issues. Sometimes when I refresh my jupyter lab and connection to kql, the problem disappears but there doesn't seem to be any consistency and the problem seems to get worse more recently.
The error:
ChunkedEncodingError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))
I would appreciate any guidance on how to solve this problem.
p.s. I tried to create the Kqlmagic tag under the tags as recommended in (https://github.com/microsoft/jupyter-Kqlmagic) however the system doesn't allow me to create this new tag as I don't have the required 'reputation score'...
Thank you
Baran
Hi (I am the author of Kqlmagic),
pls install Kqlmagic latest dev version (I think it is dev21), try to replicate the error, once it happens, execute "%kql --bugreport" and submit it to Kqlmagic Github. The bugreport, will help to toubleshoot it.
Related
I recently created a solution to automate some copy-pasting of Excel files between SharePoint folders.
To establish the connection to SharePoint, I use the Office 365 REST Python client.
As an authentication method, I use clientcredentials (this allowed me to bypass TFA on the SharePoint).
I have verified that the ID and secret that I am using are still valid.
Until recently, the code was working like a charm.
Yesterday, however, I got the following error (and have been getting it ever since):
Requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))
Weidly, sometimes I get the following error instead:
File "...\acs_token_provider.py", line
45, in get_app_only_access_token
self.error = e.response.text
AttributeError: 'NoneType' object has no attribute 'text'
I have encountered several posts mentioning the first error.
However, none of them were related to the package that I am using.
Below is a simplified example of my code which produces the same error:
from office365.runtime.auth.client_credential import ClientCredential
from office365.sharepoint.client_context import ClientContext
CLIENT_ID = "thecorrectclientid"
CLIENT_SECRET = "thecorrectclientsecret"
site_url = "https://{xxxx}.sharepoint.com/sites/{nameofthesite}"
ctx = ClientContext(site_url)
creds = ClientCredential(CLIENT_ID, CLIENT_SECRET)
ctx.with_credentials(creds)
web = ctx.web
ctx.load(web)
ctx.execute_query()
print("Web title: {0}".format(web.properties['Title']))
Does anyone have an idea what may be causing this and how I could possibly fix this?
Any help would be greatly appreciated!
UPDATE: It seems to be a problem related to my local machine.
I just cloned the repo on my other laptop and it runs smoothly.
I have also noticed the following error when opening files from SharePoint (which might be related):
the server you are trying to access is using an authentication protocol not supported by office sharepoint excel
I tried refreshing the Client Id and Client secret (to make sure I wasn't using expired credentials).
I have browsed the web for similar errors, but none of them gave me the answer for my specific situation.
I have double checked if anything changed in the URL of the site that I am trying to access.
I tried running it on another laptop and that worked for me. But the root cause is still a mystery.
No results...
UPDATE: I tried running it on another laptop and that worked for me. But the root cause is still a mystery.
After creating a compute instance under Microsoft Azure Machine Learning and select Jupyter to open Jupyter Notebook, I am selecting a New terminal from the menu. However, I am getting the following errors:
terminado.js:4 WebSocket connection to 'wss://xalapa.eastus2.instances.azureml.ms/terminals/websocket/2' failed: Error during WebSocket handshake: Unexpected response code: 426
make_terminal # terminado.js:4
index.js:5 Uncaught TypeError: Cannot read property 'parentElement' of undefined
at proposeGeometry (index.js:5)
at fit (index.js:30)
at Terminal.terminalConstructor.fit (index.js:44)
at window.onresize (main.js:54)
I have recreated the compute instance with no luck.
Do you know what is the issue related?
Thanks
Looks like you might have your websockets blocked. Here is the troubleshooting doc: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-run-jupyter-notebooks#troubleshooting
If you can't connect to a notebook, ensure that web socket
communication is not disabled. For compute instance Jupyter
functionality to work, web socket communication must be enabled.
Please ensure your network allows websocket connections to
*.instances.azureml.net and *.instances.azureml.ms.
Looks like you cannot connect to your compute instance, and 426 means the server refuses to connect.
Could you try to open Jupyter / JupyterLab from the same compute instance?
When I am trying to load a query into a tabular dateset (from a devops docker image) I will get the following error:
raise DatasetValidationError(error_message + '\n' + str(e), e)
azureml.data.dataset_error_handling.DatasetValidationError: Cannot load any data from the datastore using the SQL query "<azureml.data.datapath.DataPath object at 0x>". Please make sure the datastore and query is correct.
Error Code: ScriptExecution.DatabaseConnection.Unexpected
Failed Step: 9ad57100-4870-49d2-a32f-1c9c15c244e0
Error Message: ScriptExecutionException was caused by DatabaseConnectionException.
DatabaseConnectionException was caused by UnexpectedException.
'MSSQL' encountered unexpected exception of type 'InvalidOperationException' with HResult 'x80131509' while opening connection.
Internal connection fatal error.
I believe that I have allowed the connection in firewall (I might not have done it quite right).
I don't get the error when I am running it from the notebook (on the compute instance).
Is the code that is running on the compute instance the exact same as the docker image? If so, it's likely that the firewall is causing connectivity issues. One way of checking this is to (be cautious) disable the firewall and see if the connectivity issue is gone.
If you're comfortable sharing the firewall/network configurations, that will also help troubleshoot this.
I can confirm the 3-replica cluster of h2o inside K3s is correctly deployed, as executing in the Python3 interpreter h2o.init(ip="x.x.x.x") works as expected. I followed the instructions noted here: https://www.h2o.ai/blog/running-h2o-cluster-on-a-kubernetes-cluster/
Nevertheless, I had to modify the service.yaml and comment out the line which says clusterIP: None, as K3s was complaining about something related to its inability to set the clusterIP to None. But even though, I can certify it is working correctly, and I am able to use an external IP to connect to the cluster.
If I try to load the dataset using the h2o cluster inside the K3s cluster using the exact same steps as described here http://docs.h2o.ai/h2o/latest-stable/h2o-docs/automl.html, this is the output that I get:
>>> train = h2o.import_file("https://s3.amazonaws.com/erin-data/higgs/higgs_train_10k.csv")
...
h2o.exceptions.H2OResponseError: Server error java.lang.IllegalArgumentException:
Error: Key not loaded: Key<Frame> https://s3.amazonaws.com/erin-data/higgs/higgs_train_10k.csv
Request: POST /3/ParseSetup
data: {'check_header': '0', 'source_frames': '["https://s3.amazonaws.com/erin-data/higgs/higgs_train_10k.csv"]'}
The same error occurs if I use the h2o.upoad_file("x.csv") method.
There is a clue about what may be happening here: Key not loaded: Key<Frame> while POSTing source frame through ParseSetup in H2O API call but I am not using curl, and I can not find any parameter that could help me overcome this issue: http://docs.h2o.ai/h2o/latest-stable/h2o-py/docs/h2o.html?highlight=import_file#h2o.import_file
I need to use the Python client inside the same K3s cluster due to different technical reasons, so I am not able to kick off nor Flow nor Firebug to know what may be happening.
I can confirm it is working correctly when I simply issue a h2o.init(), using the local Java instance.
UPDATE 1:
I have tried in different K3s clusters without success. I changed the service.yaml to a NodePort, and now this is the error traceback:
>>> train = h2o.import_file("https://s3.amazonaws.com/erin-data/higgs/higgs_train_10k.csv")
...
h2o.exceptions.H2OResponseError: Server error java.lang.IllegalArgumentException:
Error: Job is missing
Request: GET /3/Jobs/$03010a2a016132d4ffffffff$_a2366be93ec99a78d7bc161de8c54d67
UPDATE 2:
I have tried using different services (NodePort, LoadBalancer, ClusterIP) and none of them work. I also have tried using Minikube with the official image, and with a custom image made by me, without success. I suspect this is something related to either h2o itself, or the clustering between pods. I will keep digging and let's think there will be some gold in it.
UPDATE 3:
I also found out that the post about running H2O in Docker is really outdated https://www.h2o.ai/blog/h2o-docker/ nor is working the Dockerfile present at GitHub (I changed it to uncomment the ENTRYPOINT section without success): https://github.com/h2oai/h2o-3/blob/master/Dockerfile
Even though, I tried with the custom image I built for h2o-k8s and it is working seamlessly in pure Docker. I am wondering why it is still not working in K8s...
UPDATE 4:
I have tried modifying the environment variable called H2O_KUBERNETES_SERVICE_DNS without success.
In the meantime, the cluster started to be unavailable, that is, the readinessProbe's would not successfully complete. No matter what I change now, it does not work.
I spinned up a K3d cluster in local to see what happened, and surprisingly, the readinessProbe's were not failing, using v3.30.0.6. But now I started testing it with R instead of Python. I am glad I tried, because I may have pinpointed what was wrong. There is a version mismatch between the client and the server. So I updated accordingly the image to v3.30.0.1.
But now again, the readinessProbe is not working in my k3d cluster, so I am unable to test it.
It seems it is working now. R client version 3.30.0.1 with server version 3.30.0.1. Also tried with Python version 3.30.0.7 and server version 3.30.0.7 and it started working. Marvelous. The problem was caused by a version mismatch between the client and the server, as the python client was updated to 3.30.0.7 while the latest server for docker was 3.30.0.6.
My script is listed below:
import pyodbc
cnxn = pyodbc.connect(driver=driver,
server=server,
user=username,
database=database,
Trusted_Connection='yes',
autocommit = True)
However, I got error message:
('HY000', '[HY000] [Microsoft][ODBC Driver 13 for SQL Server]SQL Server Network Interfaces: The Microsoft Online Services Sign-In Assistant encountered an error that might not occur if retried. Possible reasons include a transient network error. [x80048849]. (-2147186615) (SQLDriverConnect)')
Anyone has idea? My error came out when I tried to connect to azure. In the error message, it said retried may work, but I have tried several times, and I still saw this error. I see most of people have this issue when they run the query. Please feel free to leave your comment.
I have figured out the issue. Because it is connected to Azure, i dont need to include Trusted_Connection='yes', which is for windows authentication.