i want to download artifact from remote server i used for mlflow tracking with below code :
from mlflow.tracking import MlflowClient
import pandas as pd
import mlflow
uri="http://some_url"
ex_id = '1'
mlflow.set_tracking_uri(uri)
experiments = mlflow.list_experiments()
runs = mlflow.search_runs(ex_id)
runs=runs[runs['tags.dataset'] == 'my_dataset']
client = MlflowClient()
client.download_artifacts("c21d2c4e75c047608003235f213e5bb5", "myarts/arts.csv", dst_path="./local_arts/")
but after a minute it give me below error :
The following failures occurred while downloading one or more artifacts from http://some_url/api/2.0/mlflow-artifacts/artifacts/1/c21d2c4e75c047608003235f213e5bb5/artifacts: {'myarts/': 'MlflowException("API request to http://some_url/api/2.0/mlflow-artifacts/artifacts/1/c21d2c4e75c047608003235f213e5bb5/artifacts/myarts/ failed with exception HTTPConnectionPool(host='some_url', port=80): Max retries exceeded with url: /api/2.0/mlflow-artifacts/artifacts/1/c21d2c4e75c047608003235f213e5bb5/artifacts/myarts/ (Caused by ResponseError('too many 500 error responses'))")'}
what is wrong with it ?
i upgraded mlflow but nothing changed
Related
I am trying to debug random ConnectTimeout happening in our infrastructure.
The symptom: Every now and then we receive a ConnectTimeout.
What I have tested:
Change host file: We changed the host file to route to the public IP, to avoid DNS lookups which may cause similar behaviour. This did not provide any significant benefit.
Change client:
Using aiohttp.ClientSession() - We get that occasionally, instead of a ConnectTimeout that the connection never terminates and remains open forever.
async with aiohttp.ClientSession() as session:
async with session.request(method, url, json=data, headers=headers, timeout=timeout, raise_for_status=False) as resp:
if resp.content_type == "application/json":
data = await resp.json()
else:
data = await resp.text()
if resp.ok:
return data
if isinstance(data, dict) and "detail" in data:
raise RawTextError(data["detail"])
resp.raise_for_status()
Replacing the client with httpx.AsyncClient, I get a ConnectTimeout sometimes and increasing connect timeout removes the connectTimeouts received. Sample Code
async with httpx.AsyncClient(http2=True, timeout=30.0) as client:
result = await client.post(cmdurl, data=json.dumps(wf_event["stages"], cls=JSONEncoder))
res = result.read()
Running the following script without timeouts (Default Httpx timeout is 5 s) I can reproduce the behaviour from multiple servers and locations
import httpx
import asyncio
import datetime
import sys, traceback
async def testurl():
cmdurl = "url
headers = {"authorization" : "token"}
while True:
try:
t0 = datetime.datetime.now()
async with httpx.AsyncClient(http2=True) as client:
#print("Fetching from URL: {}".format(cmdurl))
result = await client.get(cmdurl,headers=headers)
res = result.json()
#print(res)
except Exception as e:
print("Start : ", t0)
print("End : ", datetime.datetime.now())
print(e)
traceback.print_exc(file=sys.stdout)
loop = asyncio.get_event_loop()
loop.run_until_complete(testurl())
Upgrade NGINX: The original Nginx is version 1.18. We created a separate Nginx on 1.21, we created a new URL for the new Nginx and could replicate the ConnectTimeouts with the above httpx script on the new endpoint, indicating that NGINX version is fine and that the load on Nginx is probably not the issue.
Regarding aiohttp.web.Application: Here we have 14 load-balancing dockers in play. I couldn't find anything suggesting a max connection count could be the issue here. I am also not sure if the upstream aiohttp is the issue as I don't see any related issues in the nginx error logs for the corresponding ConnectTimeout.
So while using the Httpx with timeouts almost completely resolved the symptom I still have no Idee why a ConnectTimeout would occur and why it sometimes takes longer than 5 seconds to connect. I haven't been able to reproduce this locally, but then our live service does handle 5000 concurrent connections at any given time.
Hope someone can point me in the direction of where to look.
Thanks for all the help in advance.
I am currently developing a function using Flask-restful to call a GET function of get a QR code from another site. But got the following error :
requests.exceptions.SSLError: HTTPSConnectionPool(host='testing-site', port=443): Max retries exceeded with url: /testing/getQR?clientID={clientID}&responseType={response-type}&source=PC_Browser&redirectURI={redirect-url} (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1122)')))
My code are as follow:
qr_request_body_web = {'clientID' : clientID, 'responseType' : 'code', 'source' : 'PC_Browser', 'redirectURI' : server_domain + authcallback_path}
def getQR():
response = requests.get('testing-site' + api_auth_getqr_path, params = qr_request_body_web, allow_redirects=True)
return redirect(response.url)
if __name__ == '__main__':
app.run(port=80, debug=True)
I can call the GET function on Postman & Chrome, is there any configuration I need to set for python to get it work? Thank you very much.
Make sure that the "http://" is the same in the browser and the requests URL parameter. If not make it the one that you see in the browser. Let me know if that helps you or not.
I want to perform load test on my mongodb database.So I added mongodb driver to bin/ext folder and restarted jmeter. I selected JSR223 Sampler and selected groovy as my language and added connection code as below
import com.mongodb.*
import com.mongodb.client.MongoClients;
import com.mongodb.client.MongoClient;
import com.mongodb.MongoClientSettings;
import com.mongodb.ServerAddress;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;
import java.util.Arrays;
try {
MongoClientSettings settings = MongoClientSettings.builder()
.applyToClusterSettings {builder ->
builder.hosts(Arrays.asList(new ServerAddress(vars.get("mongoHost"),vars.get("mongoPort").toInteger())))}
.build();
MongoClient mongoClient = MongoClients.create(settings);
MongoDatabase database = mongoClient.getDatabase(vars.get("databaseName"));
MongoCollection<Document> collection = database.getCollection(vars.get("collectionName"));
vars.putObject("collection", collection);
return "Connected to " + vars.get("collectionName");
}
catch (Exception e) {
SampleResult.setSuccessful(false);
SampleResult.setResponseCode("500");
SampleResult.setResponseMessage("Exception: " + e);
}
mongoHost,mongoPort,databaseName,collectionName are configured in user defined variables(Test Plan).
When I run sampler getting error
Thread Name:myThread Group 1-1
Sample Start:2020-01-27 15:57:35 IST
Load time:39
Connect Time:0
Latency:0
Size in bytes:0
Sent bytes:0
Headers size in bytes:0
Body size in bytes:0
Sample Count:1
Error Count:1
Data type ("text"|"bin"|""):text
Response code:500
Response message:javax.script.ScriptException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script3.groovy: 6: unable to resolve class com.mongodb.ServerAddress
# line 6, column 1.
import com.mongodb.ServerAddress;
^
Script3.groovy: 5: unable to resolve class com.mongodb.MongoClientSettings
# line 5, column 1.
import com.mongodb.MongoClientSettings;
^
Script3.groovy: 10: unable to resolve class org.bson.Document
# line 10, column 1.
import org.bson.Document;
^
3 errors
SampleResult fields:
ContentType:
DataEncoding: null
You need to add the following libraries to JMeter Classpath:
mongo-java-driver
bson
The libraries version must match (or at least be compatible) with your MongoDB version, it can be obtained by running db.version() query in the MongoDB shell
JMeter restart will be required to pick up the libraries
More information: MongoDB Performance Testing with JMeter
I'm using tensorflow and ImageAI in my python code, but when trying to upload it on Lambda getting following warning:
Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.
It's successfull and working when I removed this line from my code:
from imageai.Prediction.Custom import CustomImagePrediction
I have created virtual environment and uploaded package on lambda using zappa, but no success
from flask import Flask, jsonify
app = Flask(__name__)
#app.route('/')
def hello_world():
from imageai.Prediction.Custom import CustomImagePrediction
import os
filename = 'Image path'
prediction = CustomImagePrediction()
prediction.setModelTypeAsResNet()
prediction.setModelPath("Model path")
prediction.setJsonPath("Model json path")
prediction.loadModel(num_objects=7)
predictions, probabilities = prediction.predictImage(filename, result_count=3)
data = {}
for eachPrediction, eachProbability in zip(predictions, probabilities):
data[eachPrediction] = eachProbability
return jsonify({"status":"200","message": "Ok"})
if __name__ == '__main__':
app.run(threaded=False)
Expected Result: Your updated Zappa deployment is live!:
https://i8ppr4snid.execute-api.us-east-1.amazonaws.com/
Actual Result: Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.
Hy. I try to write test for webHander:
import pytest
import tornado
from tornado.testing import AsyncTestCase
from tornado.httpclient import AsyncHTTPClient
from tornado.web import Application, RequestHandler
import urllib.parse
class TestRESTAuthHandler(AsyncTestCase):
#tornado.testing.gen_test
def test_http_fetch_login(self):
data = urllib.parse.urlencode(dict(username='admin', password='123456'))
client = AsyncHTTPClient(self.io_loop)
response = yield client.fetch("http://localhost:8080//#/login", method="POST", body=data)
# Test contents of response
self.assertIn("Automation web console", response.body)
Received error when running test:
raise TimeoutError('Operation timed out after %s seconds' % timeout)
tornado.ioloop.TimeoutError: Operation timed out after 5 seconds
Set ASYNC_TEST_TIMEOUT environment variable.
Runs the IOLoop until stop is called or timeout has passed.
In the event of a timeout, an exception will be thrown. The default timeout is 5 seconds; it may be overridden with a timeout keyword argument or globally with the ASYNC_TEST_TIMEOUT environment variable. -- from http://www.tornadoweb.org/en/stable/testing.html#tornado.testing.AsyncTestCase.wait
You need to use AsyncHTTPTestCase, not just AsyncTestCase. A nice example is in Tornado's self-tests:
https://github.com/tornadoweb/tornado/blob/d7d9c467cda38f4c9352172ba7411edc29a85196/tornado/test/httpclient_test.py#L130-L130
You need to implement get_app to return an application with the RequestHandler you've written. Then, do something like:
class TestRESTAuthHandler(AsyncHTTPTestCase):
def get_app(self):
# implement this
pass
def test_http_fetch_login(self):
data = urllib.parse.urlencode(dict(username='admin', password='123456'))
response = self.fetch("http://localhost:8080//#/login", method="POST", body=data)
# Test contents of response
self.assertIn("Automation web console", response.body)
AsyncHTTPTestCase provides convenient features so you don't need to write coroutines with "gen.coroutine" and "yield".
Also, I notice you're fetching a url with a fragment after "#", please note that in real life web browsers do not include the fragment when they send the URL to the server. So your server would see the URL only as "//", not "//#/login".