How to get path from URL in warp? - rust

I am using warp to build a Proxy. The proxy doesn't care about parameters or path, it just delegate requests from clients. So the client may query like:
https://proxy.com/p1?a=1&b=2 or
https://proxy.com/p2?c=1 or many other different paths
I want to write something like this:
/*
client query: https://proxy.com/test_path1?a=1
client query: https://proxy.com/test_path2?b=2
*/
let filter = warp::any()
.and(warp::query::<HashMap<String, String>>())
.map(|p: HashMap<String, String>| {
let path = xxx.get_path(); // path = "/test_path1" or "/test_path2"
println!("{:?}",p); // p = {"a":1} or {"b":2} respectively
});
instead of this:
let path1 = warp::path("test_path1").and(warp::query::<HashMap<String, String>>()).map(|p: HashMap<String,String>|{
let path = "test_path1".to_string();
println!("{:?}",p); // p = {"a":1} or {"b":2} respectively
});
let path2 = warp::path("test_path2").and(warp::query::<HashMap<String, String>>()).map(|p: HashMap<String,String>|{
let path = "test_path2".to_string();
println!("{:?}",p); // p = {"a":1} or {"b":2} respectively
});
How can I do it?

Without any further detail, this does what you want by utilizing the full filter:
use warp::{self, path::FullPath, Filter};
let filter = warp::any()
.and(warp::query::<HashMap<String, String>>())
.and(warp::path::full())
.map(|q: HashMap<_, _>, p: FullPath| {
let path = p.as_str();
format!("path: {}\nquery: {:?}", path, q)
});

https://docs.rs/warp/latest/warp/filters/path/fn.tail.html
But are you even using warp::any? That's useful as a base to BYO (e.g. to add external data to the request) but warp::query is a filter, if you're using only that it doesn't need to be chained to anything.
And HTTP allows repeating query parameters so technically if you reify to a hashmap there is no guarantee the "proxy" will fully replicate the original request.

Related

Trying to get rid of unwraps

I have this line in my program:
let date = file.metadata().unwrap().modified().unwrap();
Can it be changed into form of if let Ok(date) = file.metadata().something.... and still be one liner?
Forgot to add: can't use ? operator, bc this is in a closure in for_each().
Using Result::and_then:
if let Ok(date) = file.metadata().and_then(|md| md.modified()) {
// stuff
}
Using the "try" operator (?):
// containing function returns `Result<T, E>` where `E: From<io::Error>`
let date = file.metadata()?.modified()?;
If you're inside a closure which must return (), and you want to ignore the error, I'd actually recommend using let else as such:
let Ok(metadata) = file.metadata() else { return };
let Ok(date) = metadata.modified() else { return };
// ...
This has the advantage that it doesn't increase the indentation level.

Change relative location for PathBuf

I have a few PathBufs in my Rust application:
let mut dog_path = PathBuf::from("./animals/dog.png");
let mut cow_path = PathBuf::from("./animals/bovine/cow.jpg");
How could I change these PathBufs so that they're being referred to from the ./animals directory?
// an operation on dog_path
// same operation on cow_path
assert_eq!(PathBuf::from("./dog.png"), dog_path);
assert_eq!(PathBuf::from("./bovine/cow.jpg"), cow_path);
I think you want Path::strip_prefix:
let dog_path = PathBuf::from("./animals/dog.png");
let cow_path = PathBuf::from("./animals/bovine/cow.jpg");
let dog_path_rel = dog_path.strip_prefix("./animals").unwrap();
let cow_path_rel = cow_path.strip_prefix("./animals").unwrap();
assert_eq!(Path::new("dog.png"), dog_path_rel);
assert_eq!(Path::new("bovine/cow.jpg"), cow_path_rel);
But that won't include the leading ./. If that's important to you, you can add it manually:
let dog_path_prefixed = Path::new("./").join(dog_path_rel);
let cow_path_prefixed = Path::new("./").join(cow_path_rel);
assert_eq!(PathBuf::from("./dog.png"), dog_path_prefixed);
assert_eq!(PathBuf::from("./bovine/cow.jpg"), cow_path_prefixed);
playground
Note that strip_prefix returns a Result, meaning it could fail if the path doesn't begin with the given prefix. You may want to handle this case instead of unwraping the result (causing your program to exit with a panic), or you may want to use .expect("your message here") instead to provide a meaningful error message.
If you want a general solution you could look at relative-path crate. It looks like it provide the functionality you want.
use std::path::PathBuf;
use relative_path::RelativePath;
fn main() {
let dog_path = PathBuf::from("./animals/dog.png");
let cow_path = PathBuf::from("./animals/bovine/cow.jpg");
let dog_path = RelativePath::from_path(&dog_path).unwrap();
let cow_path = RelativePath::from_path(&cow_path).unwrap();
let animals_dir = RelativePath::new("./animals");
let dog_path = animals_dir.relative(&dog_path).to_path(".");
let cow_path = animals_dir.relative(&cow_path).to_path(".");
assert_eq!(PathBuf::from("./dog.png"), dog_path);
assert_eq!(PathBuf::from("./bovine/cow.jpg"), cow_path);
}
This is a quick draft, but it shows how to do in a generic way what you are trying to accomplish. I think it could be further optimized, but I literally found this crate 10 minutes ago.

How can I pull data out of an Option for independent use?

Is there a way to 'pull' data out of an Option? I have an API call that returns Some(HashMap). I want to use the HashMap as if it weren't inside Some and play with the data.
Based on what I've read, it looks like Some(...) is only good for match comparisons and some built-in functions.
Simple API call pulled from crate docs:
use std::collections::HashMap;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let resp = reqwest::blocking::get("https://httpbin.org/ip")?
.json::<HashMap<String, String>>()?;
println!("{:#?}", resp.get("origin"));
Ok(())
}
Result:
Some("75.69.138.107")
if let Some(origin) = resp.get("origin") {
// use origin
}
If you can guarantee that it's impossible for the value to be None, then you can use:
let origin = resp.get("origin").unwrap();
Or:
let origin = resp.get("origin").expect("This shouldn't be possible!");
And, since your function returns a Result:
let origin = resp.get("origin").ok_or("This shouldn't be possible!")?;
Or with a custom error type:
let origin = resp.get("origin").ok_or(MyError::DoesntExist)?;
The most common way is with if let:
if let Some(origin) = resp.get("origin") {
origin.do_stuff()
}
For more fine grained control, you can use pattern matching:
match resp.get("origin") {
Some(origin) => origin.do_stuff(),
None => panic!("origin not found!")
}
You could also use unwrap, which will give you the underlying value of the option, or panic if it is None:
let origin = resp.get("origin").unwrap();
You can customize the panic message with expect:
let origin = resp.get("origin").expect("Oops!");
Or compute a default value with unwrap_or:
let origin = resp.get("origin").unwrap_or(&String::from("192.168.0.1"));
You can also return an error instead of panicking:
let origin = resp.get("origin").ok_or(Error::UnknownOrigin)?;
Your options are a plenty.
if let Some(origin) = resp.get("origin") {
// do stuff using origin
}
origin = resp.get("origin").unwrap()
// will panic if None
resp.get("origin").map(|origin| {
// do stuff using inner value, returning another option
})
resp.get("origin").and_then(|origin| {
// same as map but short-circuits if there is no inner value
})

How do I parse a page with html5ever, modify the DOM, and serialize it?

I would like to parse a web page, insert anchors at certain positions and render the modified DOM out again in order to generate docsets for Dash. Is this possible?
From the examples included in html5ever, I can see how to read an HTML file and do a poor man's HTML output, but I don't understand how I can modify the RcDom object I retrieved.
I would like to see a snippet inserting an anchor element (<a name="foo"></a>) to an RcDom.
Note: this is a question regarding Rust and html5ever specifically ... I know how to do it in other languages or simpler HTML parsers.
Here is some code that parses a document, adds an achor to the link and prints the new document:
extern crate html5ever;
use html5ever::{ParseOpts, parse_document};
use html5ever::tree_builder::TreeBuilderOpts;
use html5ever::rcdom::RcDom;
use html5ever::rcdom::NodeEnum::Element;
use html5ever::serialize::{SerializeOpts, serialize};
use html5ever::tendril::TendrilSink;
fn main() {
let opts = ParseOpts {
tree_builder: TreeBuilderOpts {
drop_doctype: true,
..Default::default()
},
..Default::default()
};
let data = "<!DOCTYPE html><html><body></body></html>".to_string();
let dom = parse_document(RcDom::default(), opts)
.from_utf8()
.read_from(&mut data.as_bytes())
.unwrap();
let document = dom.document.borrow();
let html = document.children[0].borrow();
let body = html.children[1].borrow(); // Implicit head element at children[0].
{
let mut a = body.children[0].borrow_mut();
if let Element(_, _, ref mut attributes) = a.node {
attributes[0].value.push_tendril(&From::from("#anchor"));
}
}
let mut bytes = vec![];
serialize(&mut bytes, &dom.document, SerializeOpts::default()).unwrap();
let result = String::from_utf8(bytes).unwrap();
println!("{}", result);
}
This prints the following:
<html><head></head><body></body></html>
As you can see, we can navigate through the child nodes via the children attribute.
And we can change an attribute present in the vector of attributes of an Element.

How to use stringByAddingPercentEncodingWithAllowedCharacters() for a URL in Swift 2.0

I was using this, in Swift 1.2
let urlwithPercentEscapes = myurlstring.stringByAddingPercentEscapesUsingEncoding(NSUTF8StringEncoding)
This now gives me a warning asking me to use
stringByAddingPercentEncodingWithAllowedCharacters
I need to use a NSCharacterSet as an argument, but there are so many and I cannot determine what one will give me the same outcome as the previously used method.
An example URL I want to use will be like this
http://www.mapquestapi.com/geocoding/v1/batch?key=YOUR_KEY_HERE&callback=renderBatch&location=Pottsville,PA&location=Red Lion&location=19036&location=1090 N Charlotte St, Lancaster, PA
The URL Character Set for encoding seems to contain sets the trim my
URL. i.e,
The path component of a URL is the component immediately following the
host component (if present). It ends wherever the query or fragment
component begins. For example, in the URL
http://www.example.com/index.php?key1=value1, the path component is
/index.php.
However I don't want to trim any aspect of it.
When I used my String, for example myurlstring it would fail.
But when used the following, then there were no issues. It encoded the string with some magic and I could get my URL data.
let urlwithPercentEscapes = myurlstring.stringByAddingPercentEscapesUsingEncoding(NSUTF8StringEncoding)
As it
Returns a representation of the String using a given encoding to
determine the percent escapes necessary to convert the String into a
legal URL string
Thanks
For the given URL string the equivalent to
let urlwithPercentEscapes = myurlstring.stringByAddingPercentEscapesUsingEncoding(NSUTF8StringEncoding)
is the character set URLQueryAllowedCharacterSet
let urlwithPercentEscapes = myurlstring.stringByAddingPercentEncodingWithAllowedCharacters( NSCharacterSet.URLQueryAllowedCharacterSet())
Swift 3:
let urlwithPercentEscapes = myurlstring.addingPercentEncoding( withAllowedCharacters: .urlQueryAllowed)
It encodes everything after the question mark in the URL string.
Since the method stringByAddingPercentEncodingWithAllowedCharacters can return nil, use optional bindings as suggested in the answer of Leo Dabus.
It will depend on your url. If your url is a path you can use the character set
urlPathAllowed
let myFileString = "My File.txt"
if let urlwithPercentEscapes = myFileString.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed) {
print(urlwithPercentEscapes) // "My%20File.txt"
}
Creating a Character Set for URL Encoding
urlFragmentAllowed
urlHostAllowed
urlPasswordAllowed
urlQueryAllowed
urlUserAllowed
You can create also your own url character set:
let myUrlString = "http://www.mapquestapi.com/geocoding/v1/batch?key=YOUR_KEY_HERE&callback=renderBatch&location=Pottsville,PA&location=Red Lion&location=19036&location=1090 N Charlotte St, Lancaster, PA"
let urlSet = CharacterSet.urlFragmentAllowed
.union(.urlHostAllowed)
.union(.urlPasswordAllowed)
.union(.urlQueryAllowed)
.union(.urlUserAllowed)
extension CharacterSet {
static let urlAllowed = CharacterSet.urlFragmentAllowed
.union(.urlHostAllowed)
.union(.urlPasswordAllowed)
.union(.urlQueryAllowed)
.union(.urlUserAllowed)
}
if let urlwithPercentEscapes = myUrlString.addingPercentEncoding(withAllowedCharacters: .urlAllowed) {
print(urlwithPercentEscapes) // "http://www.mapquestapi.com/geocoding/v1/batch?key=YOUR_KEY_HERE&callback=renderBatch&location=Pottsville,PA&location=Red%20Lion&location=19036&location=1090%20N%20Charlotte%20St,%20Lancaster,%20PA"
}
Another option is to use URLComponents to properly create your url
Swift 3.0 (From grokswift)
Creating URLs from strings is a minefield for bugs. Just miss a single / or accidentally URL encode the ? in a query and your API call will fail and your app won’t have any data to display (or even crash if you didn’t anticipate that possibility). Since iOS 8 there’s a better way to build URLs using NSURLComponents and NSURLQueryItems.
func createURLWithComponents() -> URL? {
var urlComponents = URLComponents()
urlComponents.scheme = "http"
urlComponents.host = "www.mapquestapi.com"
urlComponents.path = "/geocoding/v1/batch"
let key = URLQueryItem(name: "key", value: "YOUR_KEY_HERE")
let callback = URLQueryItem(name: "callback", value: "renderBatch")
let locationA = URLQueryItem(name: "location", value: "Pottsville,PA")
let locationB = URLQueryItem(name: "location", value: "Red Lion")
let locationC = URLQueryItem(name: "location", value: "19036")
let locationD = URLQueryItem(name: "location", value: "1090 N Charlotte St, Lancaster, PA")
urlComponents.queryItems = [key, callback, locationA, locationB, locationC, locationD]
return urlComponents.url
}
Below is the code to access url using guard statement.
guard let url = createURLWithComponents() else {
print("invalid URL")
return nil
}
print(url)
Output:
http://www.mapquestapi.com/geocoding/v1/batch?key=YOUR_KEY_HERE&callback=renderBatch&location=Pottsville,PA&location=Red%20Lion&location=19036&location=1090%20N%20Charlotte%20St,%20Lancaster,%20PA
In Swift 3.1, I am using something like the following:
let query = "param1=value1&param2=" + valueToEncode.addingPercentEncoding(withAllowedCharacters: .alphanumeric)
It's safer than .urlQueryAllowed and the others, because it this will encode every characters other than A-Z, a-z and 0-9. This works better when the value you are encoding may use special characters like ?, &, =, + and spaces.
In my case where the last component was non latin characters I did the following in Swift 2.2:
extension String {
func encodeUTF8() -> String? {
//If I can create an NSURL out of the string nothing is wrong with it
if let _ = NSURL(string: self) {
return self
}
//Get the last component from the string this will return subSequence
let optionalLastComponent = self.characters.split { $0 == "/" }.last
if let lastComponent = optionalLastComponent {
//Get the string from the sub sequence by mapping the characters to [String] then reduce the array to String
let lastComponentAsString = lastComponent.map { String($0) }.reduce("", combine: +)
//Get the range of the last component
if let rangeOfLastComponent = self.rangeOfString(lastComponentAsString) {
//Get the string without its last component
let stringWithoutLastComponent = self.substringToIndex(rangeOfLastComponent.startIndex)
//Encode the last component
if let lastComponentEncoded = lastComponentAsString.stringByAddingPercentEncodingWithAllowedCharacters(NSCharacterSet.alphanumericCharacterSet()) {
//Finally append the original string (without its last component) to the encoded part (encoded last component)
let encodedString = stringWithoutLastComponent + lastComponentEncoded
//Return the string (original string/encoded string)
return encodedString
}
}
}
return nil;
}
}
Swift 4.0
let encodedData = myUrlString.addingPercentEncoding(withAllowedCharacters: CharacterSet.urlHostAllowed)

Resources