The following is what I desire to format to:
async fn main() {
let req = reqwest::get("http://wttr.in/brisbane")
.await
.expect("Failed to get weather data")
.text()
.await;
}
This is what I get (unless it does not fit on the one line, then it will format it like above):
let req = reqwest::get("http://wttr.in/brisbane").await.expect("Failed to get weather data").text().await;
I want it to always format vertically, regardless of width. How can I do this?
Related
I would like to read a CSV into a Polars DataFrame and get the mean of a column as println but I struggle to make it work.
use polars::prelude::*;
use std::fs::File;
fn example() -> Result<DataFrame> {
let file = File::open("data.csv").expect("could not open file");
return CsvReader::new(file)
.infer_schema(None)
.has_header(true)
.finish()
}
fn main() {
let _s = example();
println!("{}", _s.select([col("A").sum()]))
}
Could someone help me understand how to do that? it looks like what I return with the example function is a Result<DataFrame> but then I don't know how to work with that.
How can I obtain the values in the message headers of an AMQP message via crate lapin (RabbitMQ client)?
I am trying to obtain the values of message headers from the lapin::message::Delivery struct.
I am using Delivery.properties.headers() which returns Option<amq_protocol_types::FieldTable>
How do I read the values in the FieldTable?
Are there any examples that show how to do so?
let mut consumer = channel
.basic_consume(
"hello",
"my_consumer",
BasicConsumeOptions::default(),
FieldTable::default(),
)
.await?;
while let Some(delivery) = consumer.next().await {
let (_, delivery2) = delivery.expect("error in consumer");
message_cnt+=1;
let payload_str:String = match String::from_utf8(delivery2.data.to_owned()) {//delivery.data is of type Vec<u8>
Ok(v) => v,
Err(e) => panic!("Invalid UTF-8 sequence: {}", e),
};
let log_message:String=format!("message_cnt is:{}, delivery_tag is:{}, exchange is:{}, routing_key is:{}, redelivered is:{}, properties is:'{:?}', received data is:'{:?}'"
,&message_cnt
,&delivery2.delivery_tag
,&delivery2.exchange
,&delivery2.routing_key
,&delivery2.redelivered
,&delivery2.properties//lapin::BasicProperties Contains the properties and the headers of the message.
,&payload_str
);
let amqp_msg_headers_option:&Option<amq_protocol_types::FieldTable>=delivery2.properties.headers();
let amqp_msg_headers:&amq_protocol_types::FieldTable=match amqp_msg_headers_option{
None=>{
let bt=backtrace::Backtrace::new();
let log_message=format!(">>>>>At receive_message_from_amqp(), message received has no headers, backtrace is '{:?}'",&bt);
slog::error!(my_slog_logger,"{}",log_message);
let custom_error=std::io::Error::new(std::io::ErrorKind::Other, &log_message.to_string()[..]);
return std::result::Result::Err(Box::new(custom_error));
}
,Some(amqp_msg_headers)=>{amqp_msg_headers}
};
if amqp_msg_headers.contains_key("worker_id"){
//let worker_id2:String=amqp_msg_headers.get("worker_id").into();
let amqp_msg_headers_btm:&std::collections::BTreeMap<amq_protocol_types::ShortString, lapin::types::AMQPValue>=amqp_msg_headers.inner();
let worker_id2_option=amqp_msg_headers_btm.get(lapin::types::AMQPValue::ShortString("worker_id".into()));
}
delivery2
.ack(BasicAckOptions::default())
.await?;
}
Well I found a solution of sorts. This entails the use of serde_json. I made use of serde_json to obtain serde_json::Value object from the object of amq_protocol_types::FieldTable obtained from the properties field of lapin::message::Delivery.
I then processed the serde_json::Value to obtain the header key and value.
Below is the function I have written for the above logic.
fn extract_amqp_msg_headers_values(
my_slog_logger:&slog::Logger
,amqp_msg_headers_basic_properties:&lapin::BasicProperties
)->std::result::Result<std::collections::HashMap<String,String>, Box<std::io::Error>> {
let mut amqp_msg_headers_hm:std::collections::HashMap<String,String>=std::collections::HashMap::new();
let amqp_msg_headers_option=amqp_msg_headers_basic_properties.headers();
let amqp_msg_headers=match amqp_msg_headers_option{
None=>{
let bt=backtrace::Backtrace::new();
let log_message=format!(">>>>>At extract_amqp_msg_headers_values(), message received has no headers, backtrace is '{:?}'",&bt);
slog::error!(my_slog_logger,"{}",log_message);
let custom_error=std::io::Error::new(std::io::ErrorKind::Other, &log_message.to_string()[..]);
return std::result::Result::Err(Box::new(custom_error));
}
,Some(amqp_msg_headers)=>{amqp_msg_headers}
};
//let mut worker_id2:String="".to_owned();
let amqp_msg_headers_btm:&std::collections::BTreeMap<amq_protocol_types::ShortString, lapin::types::AMQPValue>=amqp_msg_headers.inner();
let amqp_msg_headers_serde_value_option=serde_json::to_value(&amqp_msg_headers_btm);
let amqp_msg_headers_serde_value:serde_json::value::Value=match amqp_msg_headers_serde_value_option{
Err(err)=>{
let bt=backtrace::Backtrace::new();
let log_message=format!(">>>>>At extract_amqp_msg_headers_values(), pos 2b, some error has been encountered transforming amqp_msg_headers to json, amqp_msg_headers is:'{:?}', error is:'{:?}', backtrace is '{:?}'",&amqp_msg_headers,&err,&bt);
slog::error!(my_slog_logger,"{}",log_message);
let custom_error=std::io::Error::new(std::io::ErrorKind::Other, &log_message.to_string()[..]);
return std::result::Result::Err(Box::new(custom_error));
}
,Ok(serde_json_value)=>{serde_json_value}
};
let serde_json_map:&serde_json::Map<String,serde_json::Value>=amqp_msg_headers_serde_value.as_object().unwrap();
let amqp_msg_headers_serde_value_key_vec2:Vec<String>=serde_json_map.keys().cloned().collect();
for amqp_msg_headers_serde_value_key2 in amqp_msg_headers_serde_value_key_vec2{
let amqp_msg_headers_serde_value2:&serde_json::value::Value=&serde_json_map[&amqp_msg_headers_serde_value_key2];
let serde_json_map3:&serde_json::Map<String,serde_json::Value>=amqp_msg_headers_serde_value2.as_object().unwrap();
let some_header_key:String=remove_quotes(&amqp_msg_headers_serde_value_key2).to_owned();
let amqp_msg_headers_serde_value_key_vec3:Vec<String>=serde_json_map3.keys().cloned().collect();
for amqp_msg_headers_serde_value_key3 in amqp_msg_headers_serde_value_key_vec3{
let some_header_value:String=remove_quotes(&serde_json_map3[&amqp_msg_headers_serde_value_key3].to_string()).to_owned();
amqp_msg_headers_hm.insert(some_header_key.to_owned(),some_header_value.to_owned());
}
}
//slog::info!(my_slog_logger," amqp_msg_headers_hm is:'{:?}'",&amqp_msg_headers_hm);
Ok(amqp_msg_headers_hm)
}
/// This is a general purpose function to strip double quotes surrounding the supplied &str value.
pub fn remove_quotes(some_str:&str)->&str
{
// The trimmed string is a slice to the original string, hence no new
// allocation is performed
let chars_to_trim: &[char] = &['"',' ', ','];
let some_str: &str = some_str.trim_matches(chars_to_trim);
//println!("some_str is:'{}'", some_str);
return some_str;
}
I built a microservice in Rust. I receive messages, request a document based on the message, and call a REST api with the results. I built the REST api with warp and send out the result with reqwest. We use jaeger for tracing and the "b3" format. I have no experience with tracing and am a Rust beginner.
Question: What do I need to add the the warp / reqwest source below to propagate the tracing information and add my own span?
My version endpoint (for simplicity) looks like:
pub async fn version() -> Result<impl warp::Reply, Infallible> {
Ok(warp::reply::with_status(VERSION, http::StatusCode::OK))
}
I assume I have to extract e.g. the traceid / trace information here.
A reqwest call I do looks like this:
pub async fn get_document_content_as_text(
account_id: &str,
hash: &str,
) -> Result<String, Box<dyn std::error::Error>> {
let client = reqwest::Client::builder().build()?;
let res = client
.get(url)
.bearer_auth(TOKEN)
.send()
.await?;
if res.status().is_success() {}
let text = res.text().await?;
Ok(text)
}
I assume I have to add the traceid / trace information here.
You need to add a tracing filter into your warp filter pipeline.
From the documentation example:
use warp::Filter;
let route = warp::any()
.map(warp::reply)
.with(warp::trace(|info| {
// Create a span using tracing macros
tracing::info_span!(
"request",
method = %info.method(),
path = %info.path(),
)
}));
I'll assume that you're using tracing within your application and using opentelemetry and opentelemetry-jaeger to wire it up to an external service. The specific provider you're using doesn't matter. Here's a super simple setup to get that all working that I'll assume you're using on both applications:
# Cargo.toml
[dependencies]
opentelemetry = "0.17.0"
opentelemetry-jaeger = "0.16.0"
tracing = "0.1.33"
tracing-subscriber = { version = "0.3.11", features = ["env-filter"] }
tracing-opentelemetry = "0.17.2"
reqwest = "0.11.11"
tokio = { version = "1.21.1", features = ["macros", "rt", "rt-multi-thread"] }
warp = "0.3.2"
opentelemetry::global::set_text_map_propagator(opentelemetry_jaeger::Propagator::new());
tracing_subscriber::registry()
.with(tracing_opentelemetry::layer().with_tracer(
opentelemetry_jaeger::new_pipeline()
.with_service_name("client") // or "server"
.install_simple()
.unwrap())
).init();
Let's say the "client" application is set up like so:
#[tracing::instrument]
async fn call_hello() {
let client = reqwest::Client::default();
let _resp = client
.get("http://127.0.0.1:3030/hello")
.send()
.await
.unwrap()
.text()
.await
.unwrap();
}
#[tokio::main]
async fn main() {
// ... initialization above ...
call_hello().await;
}
The traces produced by the client are a bit chatty because of other crates but fairly simple, and does not include the server-side:
Let's say the "server" application is set up like so:
#[tracing::instrument]
fn hello_handler() -> &'static str {
tracing::info!("got hello message");
"hello world"
}
#[tokio::main]
async fn main() {
// ... initialization above ...
let routes = warp::path("hello")
.map(hello_handler);
warp::serve(routes).run(([127, 0, 0, 1], 3030)).await;
}
Likewise, the traces produced by the server are pretty bare-bones:
The key part to marrying these two traces is to declare the client-side trace as the parent of the server-side trace. This can be done over HTTP requests with the traceparent and tracestate headers as designed by the W3C Trace Context Standard. There is a TraceContextPropagator available from the opentelemetry crate that can be used to "extract" and "inject" these values (though as you'll see, its not very easy to work with since it only works on HashMap<String, String>s).
For the "client" to send these headers, you'll need to:
get the current tracing Span
get the opentelemetry Context from the Span (if you're not using tracing at all, you can skip the first step and use Context::current() directly)
create the propagator and fields to propagate into and "inject" then from the Context
use those fields as headers for reqwest
#[tracing::instrument]
async fn call_hello() {
let span = tracing::Span::current();
let context = span.context();
let propagator = TraceContextPropagator::new();
let mut fields = HashMap::new();
propagator.inject_context(&context, &mut fields);
let headers = fields
.into_iter()
.map(|(k, v)| {(
HeaderName::try_from(k).unwrap(),
HeaderValue::try_from(v).unwrap(),
)})
.collect();
let client = reqwest::Client::default();
let _resp = client
.get("http://127.0.0.1:3030/hello")
.headers(headers)
.send()
.await
.unwrap()
.text()
.await
.unwrap();
}
For the "server" to make use of those headers, you'll need to:
pull them out from the request and store them in a HashMap
use the propagator to "extract" the values into a Context
set that Context as the parent of the current tracing Span (if you didn't use tracing, you could .attach() it instead)
#[tracing::instrument]
fn hello_handler(traceparent: Option<String>, tracestate: Option<String>) -> &'static str {
let fields: HashMap<_, _> = [
dbg!(traceparent).map(|value| ("traceparent".to_owned(), value)),
dbg!(tracestate).map(|value| ("tracestate".to_owned(), value)),
]
.into_iter()
.flatten()
.collect();
let propagator = TraceContextPropagator::new();
let context = propagator.extract(&fields);
let span = tracing::Span::current();
span.set_parent(context);
tracing::info!("got hello message");
"hello world"
}
#[tokio::main]
async fn main() {
// ... initialization above ...
let routes = warp::path("hello")
.and(warp::header::optional("traceparent"))
.and(warp::header::optional("tracestate"))
.map(hello_handler);
warp::serve(routes).run(([127, 0, 0, 1], 3030)).await;
}
With all that, hopefully your traces have now been associated with one another!
Full code is available here and here.
Please, someone let me know if there is a better way! It seems ridiculous to me that there isn't better integration available. Sure some of this could maybe be a bit simpler and/or wrapped up in some nice middleware for your favorite client and server of choice... But I haven't found a crate or snippet of that anywhere!
i have made this code to check for alive urls in a text file it was first to check for a single url the script worked but then i wanted to make it multithreaded i got this error
error
here is the original code :
use hyper_tls::HttpsConnector;
use hyper::Client;
use tokio::io::BufReader;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let https = HttpsConnector::new();
let url = std::env::args().nth(1).expect("no list given");
let client = Client::builder().build::<_, hyper::Body>(https);
let reader = BufReader::new(url);
let lines = reader.lines();
for l in lines {
let sep = l.parse()?;
// Await the response...
let resp = client.get(sep).await?;
if resp.status() == 200 {
println!("{}", l);}
if resp.status() == 301 {
println!("{}", l); }
}
Ok(())
}
the issue seems to be that you are passing in the file's name as opposed to its content to the BufReader.
In order to read the contents instead, you can use a tokio::fs:File.
Here's an example of reading a file and printing its lines to stdout using tokio and a BufReader:
use tokio::{
fs::File,
io::{
// This trait needs to be imported, as the lines function being
// used on reader is defined there
AsyncBufReadExt,
BufReader
}
};
#[tokio::main]
async fn main() {
// get file command line argument
let file_argument = std::env::args().nth(1).expect("Please provide a file as command line argument.");
// open file
let file = File::open(file_argument).await.expect("Failed to open file");
// create reader using file
let reader = BufReader::new(file);
// get iterator over lines
let mut lines = reader.lines();
// this has to be used instead of a for loop, since lines isn't a
// normal iterator, but a Lines struct, the next element of which
// can be obtained using the next_line function.
while let Some(line) = lines.next_line().await.expect("Failed to read file") {
// print current line
println!("{}", line);
}
}
So I have gone through 90% of the tutorial on Rust and I think I mostly have a grasp on the syntax. I'm attempting to start writing code with it I'm currently using the rustc_serialize library to parse JSON from stdin and I'm not getting the results I expect. I have the following JSON file called message.txt the following content:
{"text": "hello world"}
Here is the Rust code to accept stdin and parse out the text field:
extern crate rustc_serialize;
use std::io::{self, Read};
use rustc_serialize::json::Json;
fn main() {
// provide a buffer for stdin
let mut buffer = String::new();
let _ = io::stdin().read_to_string(&mut buffer);
// parse the json
let message = match Json::from_str(&mut buffer) {
Ok(m) => m,
Err(_) => panic!("Stdin provided invalid JSON")
};
// get the message object and "text" field string
let message_object = message.as_object().unwrap();
let message_string = message_object.get("text").unwrap();
println!("{}", message_string);
println!("{}", &message_string.to_string()[0..4]);
}
The following code outputs:
"Hello World"
"Hel
I'm currently outputting the byte slice to make sure the quote wasn't something that was added by print. According to the docs message_string shouldn't have quotes around it.
If I print out the data using the example from the documentation then it prints the value of "text" without quotes:
for (key, value) in message_object.iter() {
println!("{}: {}", key, match *value {
Json::U64(v) => format!("{} (u64)", v),
Json::String(ref v) => format!("{} (string)", v),
_ => format!("other")
});
}
Output:
text: hello world (string)
I'm a newbie to Rust so I probably just don't understand the string manipulation parts of Rust all that well.
The problem is that message_string isn't what you think it is. I discovered that when I tried to use len on the "string", which didn't work (I assume that's why you have a to_string when you are slicing). Let's make the compiler tell us what it is:
let () = message_string;
Has the error:
error: mismatched types:
expected `&rustc_serialize::json::Json`,
found `()`
It's a Json! We need to convert that enumerated type into a string-like thing:
let message_object = message.as_object().unwrap();
let message_json = message_object.get("text").unwrap();
let message_string = message_json.as_string().unwrap();
Ultimately, I'd argue that Display (which allows the {} format string) should not have been implemented for this type, as Display means format in an end-user-focused manner. It's probably too late to change that decision now though.
I know that unwrap is great for quick prototyping, but I'd be remiss in not showing a slightly more idiomatic way of doing this:
fn main() {
let mut buffer = String::new();
io::stdin().read_to_string(&mut buffer).expect("Could not read from stdin");
let message = Json::from_str(&mut buffer).expect("Stdin provided invalid JSON");
let message_string = message.as_object().and_then(|obj| {
obj.get("text").and_then(|json| {
json.as_string()
})
}).expect("The `text` key was missing or not a string");
println!("{}", message_string);
}
Ignoring the Result from read_to_string is worse than panicking. ^_^