How to handle multipart/form-data upload in rocket (rust)? - rust

I'm trying to handle POST requests on my webserver for file uploads, but i haven't found good documentation or examples on how to do it.
Here is the POST function i came up with after a long time:
#[post("/upload",data="<file>")]
fn upload_handler(file: Data) -> Redirect {
let mut buffer = Vec::new();
file.stream_to(&mut buffer).unwrap();
let split_pos = buffer.windows(4).position(|pos|pos == b"\r\n\r\n").unwrap();
let split_end = buffer.windows(8).position(|pos|pos == b"\r\n------").unwrap();
let headers = String::from_utf8_lossy(&buffer[0..split_pos]);
let content = &buffer[split_pos+4..split_end];
println!("{}",&headers);
let re = Regex::new("filename=\"(?P<filename>.*)\"").unwrap();
let captures = re.captures(&headers).unwrap();
let filename = &captures["filename"].to_owned();
let mut local_file = File::create(filename).unwrap();
local_file.write(content).unwrap();
Redirect::to("/")
}
It kinda works but has a lot of flaws like:
I can't limit the file upload size.
I don't know the size of the file being uploaded.
I had to manually regex the filename (which i don't think is a thing you have to do when using a framework).
I can't see the entire request header.
When spliting the headers from the content this is all i got:
Content-Disposition: form-data; name="file"; filename="image.jpg"
Content-Type: image/jpeg
Here's a simple working code i had on a python flask server that did the job:
#app.route("/upload",methods=["POST"])
def upload():
if int(request.headers["Content-Length"]) > app.config["MAX_CONTENT_LENGTH"]:
abort(413)
f = request.files["file"]
filename = secure_filename(f.filename)
if filename.endswith(ALLOWED_EXTENSIONS):
f.save("src/"+filename)
return render_template("upload.html",file=filename),{"Refresh": "2; url=/"}
else:
return render_template("denied.html")
If anyone knows a better/right way of doing it please tell me.

Related

How to return contents as a file download in Axum?

I have a Rust application that is acting as a proxy. From the user perspective there is a web UI front end. This contains a button that when invoked will trigger a GET request to the Rust application. This in turn calls an external endpoint that returns the CSV file.
What I want is have the file download to the browser when the user clicks the button. Right now, the contents of the CSV file are returned to the browser rather than the file itself.
use std::net::SocketAddr;
use axum::{Router, Server};
use axum::extract::Json;
use axum::routing::get;
pub async fn downloadfile() -> Result<Json<String>, ApiError> {
let filename = ...;
let endpoint = "http://127.0.0.1:6101/api/CSV/DownloadFile";
let path = format!("{}?filename={}", endpoint, filename);
let response = reqwest::get(path).await?;
let content = response.text().await?;
Ok(Json(content))
}
pub async fn serve(listen_addr: SocketAddr) {
let app = Router::new()
.route("/downloadfile", get(downloadfile));
Server::bind(&listen_addr)
.serve(app.into_make_service())
.await
.unwrap();
}
I understand the reason I'm getting the contents is because I'm returning the content string as JSON. This makes sense. However, what would I need to change to return the file itself so the browser downloads it directly for the user?
I've managed to resolve it and now returns the CSV as a file. Here's the function that works:
use axum::response::Headers;
use http::header::{self, HeaderName};
pub async fn downloadfile() -> Result<(Headers<[(HeaderName, &'static str); 2]>, String), ApiError> {
let filename = ...;
let endpoint = "http://127.0.0.1:6101/api/CSV/DownloadFile";
let path = format!("{}?filename={}", endpoint, filename);
let response = reqwest::get(path).await?;
let content = response.text().await?;
let headers = Headers([
(header::CONTENT_TYPE, "text/csv; charset=utf-8"),
(header::CONTENT_DISPOSITION, "attachment; filename=\"data.csv\""),
]);
Ok((headers, content))
}
I'm still struggling with being able to set a dynamic file name, but that for another question.

Expiration of CachedURLResponse not working

I was trying to set custom headers for 'Cache-Control' to achieve a cache at client side(server side has 'Cache-Control: no-cache'). Trying to achieve following two major things.
Response of some of the end points should be cached in memory and
should have an expiration(user defined)
Once expiration time is over, then app should ignore the cache and should fetch data from server.
I followed this link and was able to achieve first target but somehow even after expiry app is still using cache and not triggering any API calls. Not sure if 'max-age' set in header is ignored by the app. Please guide me if I am missing something here.
Here are the code snippets.
Session Configuration:
let sessionConfiguration: URLSessionConfiguration = URLSessionConfiguration.ephemeral
sessionConfiguration.requestCachePolicy = .returnCacheDataElseLoad
sessionConfiguration.urlCache = .shared
self.currentURLSession = URLSession(configuration: sessionConfiguration, delegate: self, delegateQueue: nil)
Request:
if let urlPath = URL(string: <WEB_API_END_POINT>){
var aRequest = URLRequest(url: urlPath, cachePolicy: .useProtocolCachePolicy, timeoutInterval: 60)
aRequest.addValue("private", forHTTPHeaderField: "Cache-Control")
let aTask = self.currentURLSession.dataTask(with: aRequest)
aTask.resume()
}
Caching logic:
func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, willCacheResponse proposedResponse: CachedURLResponse, completionHandler: #escaping (CachedURLResponse?) -> Void) {
if proposedResponse.response.url?.path.contains("/employees") == true {
let updatedResponse = proposedResponse.response(withExpirationDuration: 60)
completionHandler(updatedResponse)
} else {
completionHandler(proposedResponse)
}
}
CachedURLResponse Extension:
extension CachedURLResponse {
func response(withExpirationDuration duration: Int) -> CachedURLResponse {
var cachedResponse = self
if let httpResponse = cachedResponse.response as? HTTPURLResponse, var headers = httpResponse.allHeaderFields as? [String : String], let url = httpResponse.url{
headers["Cache-Control"] = "max-age=\(duration)"
headers.removeValue(forKey: "Expires")
headers.removeValue(forKey: "s-maxage")
if let newResponse = HTTPURLResponse(url: url, statusCode: httpResponse.statusCode, httpVersion: "HTTP/1.1", headerFields: headers) {
cachedResponse = CachedURLResponse(response: newResponse, data: cachedResponse.data, userInfo: headers, storagePolicy: .allowedInMemoryOnly)
}
}
return cachedResponse
}
}
Was able to fix it at my own. Still sharing the answer in case if helps someone else in need.
Added 'Cache-Control' response header in server response and there we have 'max-age: 60', which indicates that response can be valid till 60 seconds only. So till 60 seconds, app will cache that data and after 60 seconds if making another request, this will fetch fresh data from server.
With that, At client side other than defining your cache policy, nothing else is required.
You can do this either for entire URL Session:
let sessionConfiguration: URLSessionConfiguration = URLSessionConfiguration.ephemeral
sessionConfiguration.requestCachePolicy = .useProtocolCachePolicy
sessionConfiguration.urlCache = .shared
self.currentURLSession = URLSession(configuration: sessionConfiguration, delegate: self, delegateQueue: nil)
Or can do it for specific request.
if let urlPath = URL(string: <WEB_API_END_POINT>) {
var aRequest = URLRequest(url: urlPath, cachePolicy: .useProtocolCachePolicy, timeoutInterval: 60)
let aTask = self.currentURLSession.dataTask(with: aRequest)
aTask.resume()
}

How can I download a website's content into a string?

I would simply like to download a website and put its contents into a String.
Similar to how this is done in C#:
WebClient c = new WebClient();
string ex = c.DownloadString("http://url.com");
Rust has no HTTP functionality in the standard library, so you probably want to use another crate (library) to handle HTTP stuff for you. There are several different crates for this purpose.
reqwest: "higher level HTTP client library"
let body = reqwest::get("http://url.com")?.text()?;
ureq: "Minimal HTTP request library"
let body = ureq::get("http://url.com").call().into_string()?
isahc: "The practical HTTP client that is fun to use."
let mut response = isahc::get("https://example.org")?;
let body = response.text()?;
curl: "Rust bindings to libcurl for making HTTP requests"
use curl::easy::Easy;
// First write everything into a `Vec<u8>`
let mut data = Vec::new();
let mut handle = Easy::new();
handle.url("http://url.com").unwrap();
{
let mut transfer = handle.transfer();
transfer.write_function(|new_data| {
data.extend_from_slice(new_data);
Ok(new_data.len())
}).unwrap();
transfer.perform().unwrap();
}
// Convert it to `String`
let body = String::from_utf8(data).expect("body is not valid UTF8!");
hyper?
Hyper is a very popular HTTP library, but it's rather low-level. This makes it usually too hard/verbose to use for small scripts. However, if you want to write a HTTP server, Hyper sure is the way to go (that's why Hyper is used by most Rust Web Frameworks).
Many others!
I couldn't list all available libraries in this answer. So feel free to search crates.io for more crates that could help you.

In Swift, URLSession.shared.dataTask sometimes work and sometimes not

client: swift, xcode 8.3
server: nodejs, express, mongodb
In client side, it first send login request (username & password) to server, upon receive a valid token, it proceeds to the next screen to send another request to get a list of books.
The strange point is that, for the second request for books, sometimes it works - if with debug mode and a number of breakpoints proceeding "slowly", while sometimes it does not work - if proceed the breakpoints not such slow or without them. I repeated a number of times and figured out this pattern.
Here is the piece of code which send request - millions of thanks!!!
let url = URL(string: "http://" + SERVER_IP + ":" + PORT + "/books")
print("Query:>> url: ")
print(url)
var request = URLRequest(url: url!)
request.httpMethod = "GET"
let task = URLSession.shared.dataTask(with: request) {(data, response, error) in
print("Query>> response from loadBooks")
if let data = data,
let json = try? JSONSerialization.jsonObject(with: data, options: []) {
var books = [Book]()
if let array = json as? [Any] {
for i in 0...array.count-1 {
var bookJson = array[i] as? [String: Any]
let b = Book(json: bookJson!)
books.append(b!)
}
}
print ("Query>> \(books.count)" + " books loaded, callback completion")
completion(books)
}
}
task.resume()
Console output when the request is not successful:
Query:>> url: Optional(localhost:3001/books)
2017-09-07 15:31:45.173596+0800 SimplyRead[76273:2379182] [] nw_endpoint_flow_service_writes [1.1 ::1.3001 ready socket-flow (satisfied)] Write request has 4294967295 frame count, 0 byte count Query>> response from loadBooks
2017-09-07 15:51:41.210732+0800 SimplyRead[76273:2390856] [] tcp_connection_write_eof_block_invoke Write close callback received error: [89] Operation canceled

SoapUI. Add multipart body part using groovy

I need to add json as multipart body part to "multipart/form-data" request using groovy.
I can do this using file attachments:
testRunner.testCase
.testSteps["/job/result"]
.getHttpRequest()
.attachBinaryData(json.toString().getBytes(), "application/json").contentID = "info"
My problem is "attachBinaryData" creates a temp file per request. It is not good for load tests :)
Is there other possibility to add body parts, without file attachments?
something like :
testRunner.testCase
.testSteps["/job/result"]
.getHttpRequest()
.addBodyPart("application/json", json.toString())
P.S. it must be the "add", because request has also one static attachment.
If you want to add a json to your request as a multipart/form-data using groovy script testStep you can use the follow code:
def jsonStr = "{'id':'test','someValue':'3'}"
def testStep = context.testCase.testSteps["/job/result"]
// set the content for the request
testStep.getHttpRequest().setRequestContent(jsonStr)
// and set the media type
testStep.testRequest.setMediaType('application/json')
// if you want to send as a multipart/form-data then use follow line instead
// testStep.testRequest.setMediaType('multipart/form-data')
This results in a request configured as follows:
Hope it helps,
Have found a solution using https://github.com/jgritman/httpbuilder
def http = new HTTPBuilder(serviceEndPoint)
def scanResultFile = new File(testRunner.testCase.getPropertyValue("ScanResultFile"))
http.request( POST ){ req ->
headers.'Connection' = 'Keep-Alive'
headers.'User-Agent' = 'SoapUI 4.5.1'
requestContentType = 'multipart/form-data'
ByteArrayBody bin = new ByteArrayBody(scanResultFile.readBytes(), "application/octet-stream", "jobResult");
StringBody info = new StringBody(testRunner.testCase.getPropertyValue("JsonScanResult"), "application/json", java.nio.charset.StandardCharsets.UTF_8);
MultipartEntity entity = new MultipartEntity()
entity.addPart("info", info);
entity.addPart("jobResult", bin)
req.entity = entity
}

Resources