I'm learning tracing and open-telemetry in Rust. I feel there are too many concepts and too many crates (at least in Rust) to see traces.
I wrote a simple lib app that adds two u32s:
use std::ops::Add;
pub fn add(f: u32, s: u32) -> u32 {
let span = tracing::info_span!("Add function", ?f, ?s);
let _guard = span.enter();
tracing::info!("Info event");
f.add(s)
}
And then I'm using the lib in my binary app:
use TracedLibrary::add;
use tracing_opentelemetry::OpenTelemetryLayer;
use tracing_subscriber::util::SubscriberInitExt;
use opentelemetry::{global, sdk::propagation::TraceContextPropagator};
use tracing_subscriber::layer::SubscriberExt;
use tracing_subscriber::Registry;
fn main() {
setup_global_subscriber();
let sum = add::add(1, 2);
}
fn setup_global_subscriber() {
global::set_text_map_propagator(TraceContextPropagator::new());
let (tracer, _uninstall) = opentelemetry_jaeger::new_pipeline()
.with_service_name("trace_demo_2")
.install().expect("Error initializing Jaeger exporter");
let telemetry = tracing_opentelemetry::layer().with_tracer(tracer);
Registry::default()
.with(telemetry).init();
}
The most confusing part is my apps Cargo.toml which looks like
tracing-subscriber = { version = "0.2.15" }
tracing-opentelemetry = { version= "0.11.0"}
opentelemetry = "0.12.0"
opentelemetry-jaeger = {version = "0.11.0" }
What on earth are those different crates are for? The only crate that makes sense is opentelemetry-jaeger. Are others even required?
And to my main question: I'm running Jaeger's all-in-one docker container. But when I visit http://localhost:16686, I see no traces.
Does anyone know what's happening?
Turns out when I create the Jaeger pipeline in setup_global_subscriber(), the _uninstall being returned gets dropped at the end of the function.
And when it gets dropped, collector shuts down.
To get traces I had to move contents of setup_global_subscriber() in main().
I think this is what you need.
tracing::subscriber::set_global_default(subscriber);
This sets a global Dispatcher, which Dispatcher.inner is your subscriber.
Related
I'm new to rust and encountered an issue while building an API with warp. I'm trying to pass some requests to another thread with a channel(trying to avoid using arc/mutex). Still, I noticed that when I pass an mpsc::sync::Sender to a warp handler, I get this error.
"std::sync::mpsc::Sender cannot be shared between threads safely"
and
"the trait Sync is not implemented for `std::sync::mpsc::Sender"
Can someone lead me in the right direction?
use std::sync::mpsc::Sender;
pub async fn init_server(run_tx: Sender<Packet>) {
let store = Store::new();
let store_filter = warp::any().map(move || store.clone());
let run_tx_filter = warp::any().map(move || run_tx.clone());
let update_item = warp::get()
.and(warp::path("v1"))
.and(warp::path("auth"))
.and(warp::path::end())
.and(post_json())
.and(store_filter.clone())
.and(run_tx_filter.clone()) //where I'm trying to send "Sender"
.and_then(request_token);
let routes = update_item;
println!("HTTP server started on port 8080");
warp::serve(routes).run(([127, 0, 0, 1], 3030)).await;
}
pub async fn request_token(
req: TokenRequest,
store: Store,
run_tx: Sender<Packet>,
) -> Result<impl warp::Reply, warp::Rejection> {
let (tmp_tx, tmp_rx) = std::sync::mpsc::channel();
run_tx
.send(Packet::IsPlayerLoggedIn(req.address, tmp_tx))
.unwrap();
let logged_in = tmp_rx.recv().unwrap();
if logged_in {
return Ok(warp::reply::with_status(
"Already logged in",
http::StatusCode::BAD_REQUEST,
));
}
Ok(warp::reply::with_status("some token", http::StatusCode::OK))
}
I've looked through some of the examples for warp, and was also wondering what are some good resources to get knowledgable of the crate. Thank you!
This is because you're using std::sync::mpsc::Sender. std::sync implements !Sync, so you won't be able to use that. You don't want to use that anyway since it's blocking.
When you use async functionality in rust, you need to provide a runtime for it. The good news is that warp runs on the tokio runtime, so you should have access to tokio::sync::mpsc If you take a look at that link, the Sender for that mpsc implementation implements Sync and Send so it's safe to share across threads.
TLDR:
Use tokio::sync::mpsc instead of std::sync::mpsc and this won't be an issue.
I am developing a CLI program for rendering template files using the new MiniJinja library by mitsuhiko.
The program is here: https://github.com/benwilber/temple.
I would like to be able to extend the program by allowing the user to load custom Lua scripts for things like custom filters, functions, and tests. However, I am running into Rust lifetime errors that I've not been able to solve.
Basically, I would like to be able to register a Lua function as a custom filter function. But it's showing an error when compiling. Here is the code:
https://github.com/benwilber/temple/compare/0.3.1..lua
Error:
https://gist.github.com/c649a0b240cf299d3dbbe018c24cbcdc
How can I call a Lua function from the MiniJinja add_filter function? I would prefer to try to do this in the regular/safe way. But I'm open to unsafe alternatives if required.
Thanks!
Edit: Posted the same on Reddit and users.rust-lang.org
Lua uses state that is not safe to use from more than one thread.
A consequence of this is that LuaFunction is neither Sync or Send.
This is being enforced by this part of the error message:
help: within `LuaFunction<'_>`, the trait `Sync` is not implemented for `*mut rlua::ffi::lua_State`
In contrast a minijinja::Filter must implement Send + Sync + 'static.
(See https://docs.rs/minijinja/0.5.0/minijinja/filters/trait.Filter.html)
This means we can't share LuaFunctions (or even LuaContext) between calls to the Filters.
One option is to not pass your lua state into the closures, and instead create a new lua state every call, something like this.
env.add_filter(
"concat2",
|_env: &Environment, s1: String, s2: String|
-> anyhow::Result<String, minijinja::Error> {
lua.context(|lua_ctx| {
lua_ctx.load(include_str!("temple.lua")).exec().unwrap();
let globals = lua_ctx.globals();
let temple: rlua::Table = globals.get("temple").unwrap();
let filters: rlua::Table = temple.get("_filters").unwrap();
let concat2: rlua::Function = filters.get("concat2").unwrap();
let res: String = concat2.call::<_, String>((s1, s2)).unwrap();
Ok(res)
}
}
);
This is likely to have relatively high overhead.
Another option is to create your rlua state in one thread and communicate with it via pipes. This would look more like this:
pub fn test() {
let mut env = minijinja::Environment::new();
let (to_lua_tx, to_lua_rx) = channel::<(String,String,SyncSender<String>)>();
thread::spawn(move|| {
let lua = rlua::Lua::new();
lua.context(move |lua_ctx| {
lua_ctx.load("some_code").exec().unwrap();
let globals = lua_ctx.globals();
let temple: rlua::Table = globals.get("temple").unwrap();
let filters: rlua::Table = temple.get("_filters").unwrap();
let concat2: rlua::Function = filters.get("concat2").unwrap();
while let Ok((s1,s2, channel)) = to_lua_rx.recv() {
let res: String = concat2.call::<_, String>((s1, s2)).unwrap();
channel.send(res).unwrap()
}
})
});
let to_lua_tx = Mutex::new(to_lua_tx);
env.add_filter(
"concat2",
move |_env: &minijinja::Environment,
s1: String,
s2: String|
-> anyhow::Result<String, minijinja::Error> {
let (tx,rx) = sync_channel::<String>(0);
to_lua_tx.lock().unwrap().send((s1,s2,tx)).unwrap();
let res = rx.recv().unwrap();
Ok(res)
}
);
}
It would even be possible to start multiple lua states this way, but would require a bit more plumbing.
DISCLAIMER: This code is all untested - however, it builds with a stubbed version of minijinja and rlua in the playground. You probably want better error handling and might need some additional code to handle cleanly shutting down all the threads.
mod loginfo{
use std::io::Error;
use chrono::prelude::*;
use std::io::prelude::*;
use std::fs::OpenOptions;
const LOG_SYS :&'static str = "log.txt";
const LOG_ERR :&'static str = "log_error.txt";
pub fn set_log_error(info: String)->Result<(), String>{
let mut handler = OpenOptions::new().append(true)
.open(LOG_ERR);
if handler.is_err(){
create_file(LOG_ERR.to_owned()).unwrap();
set_log_error(info).unwrap();
}
if let Err(_errno) = handler.write_fmt(
format_args!("{:?}\t{:?} ->[Last OS error({:?})]\n",
Utc::now().to_rfc2822().to_string(), info,
Error::last_os_error()) ){
panic!(
"\nCannot write info log error\t Info\t:{:?}\n",
Error::last_os_error());
}
Ok(())
}
pub fn set_log(info: String)->Result<(), String>{
let mut handler = OpenOptions::new().append(true)
.open(LOG_SYS);
if handler.is_err(){
set_log_error("Cannot write info log".to_owned())
.unwrap();
}
if let Err(_errno) = handler.write_fmt(
format_args!("{:?}\t{:?}\n",
Utc::now().to_rfc2822().to_string(), info)){
set_log_error("Cannot write data log file".to_owned())
.unwrap();
}
Ok(())
}
pub fn create_file(filename : String)->Result<(), String>{
let handler = OpenOptions::new().write(true)
.create(true).open(filename);
if handler.is_err(){
panic!(
"\nCannot create log file\t Info\t:{:?}\n",
Error::last_os_error());
}
Ok(())
}
}
When compiling, I get the following errors, "error[E0599]: no method named write_fmt found for enum std::result::Result<std::fs::File, std::io::Error> in the current scope --> src/loginfo.rs:19:38`"
but despite using the right imports, I still get the same errors. Is this due to a bad implementation of the module?
Thank you in advance for your answers and remarks?
+1 #Masklinn Ok I think I understand it would be easier to just write
pub fn foo_write_log( info: String){
let mut handler = OpenOptions::new().append(true)
.create(true).open(LOG_SYS).expect("Cannot create log");
handler.write_fmt(
format_args!("{:?}\t{:?} ->[Last OS error({:?})]\n",
Utc::now().to_rfc2822().to_string(), info,
Error::last_os_error())).unwrap();
}
but despite using the right imports, I still get the same errors. Is this due to a bad implementation of the module?
Kind-of? If you look at the type specified in the error, handler is a Result<File, Error>. And while io::Write is implemented on File, it's not implemented on Result.
The problem is that while you're checking whether handler.is_err() you never get the file out of it, nor do you ever return in the error case. Normally you'd use something like match or if let or one of the higher-order methods (e.g. Result::map, Result::and_then) in order to handle or propagate the various cases.
And to be honest the entire thing is rather odd and awkward e.g. your functions can fail but they panic instead (you never actually return an Err); if you're going to try and create a file when opening it for writing fails, why not just do that directly[0]; you are manually calling write_fmt and format_args why not just write!; write_fmt already returns an io::Error why do you discard it then ask for it again via Error::last_os_error; etc...
It's also a bit strange to hand-roll your own logger thing when the rust ecosystem already has a bunch of them though you do you; and the naming is also somewhat awkward e.g. I'd expect something called set_X to actually set the X, so to me set_log would be a way to set the file being logged to.
[0] .create(true).append(true) should open the file in append mode if it exists and create it otherwise; not to mention your version has a concurrency issue: if the open-for-append fails you create the file in write mode, but someone else could have created the file -- with content -- between the two calls, in which case you're going to partially overwrite the file
Does rust currently have a library implement function similar to JavaScript's setTimeout and setInverval?, that is, a library that can call multiple setTimeout and setInterval to implement management of multiple tasks at the same time.
I feel that tokio is not particularly convenient to use. I imagine it to be used like this:
fn callback1() {
println!("callback1");
}
fn callback2() {
println!("callback2");
}
set_interval(callback1, 10);
set_interval(callback1, 20);
set_timeout(callback1, 30);
Of course, I can simulate a function to make it work:
// just for test, not what I wanted at all
type rust_listener_callback = fn();
fn set_interval(func: rust_listener_callback, duration: i32) {
func()
}
fn set_timeout(func: rust_listener_callback, duration: i32) {
func();
}
If a set_interval is implemented in this way, multiple combinations, dynamic addition and deletion, and cancellation are not particularly convenient：
use tokio::time;
async fn set_interval(func: rust_listener_callback, duration: u64) {
let mut interval = time::interval(Duration::from_millis(duration));
tokio::spawn(async move {
loop {
interval.tick().await;
func()
}
}).await;
}
// emm maybe loop can be removed, just a sample
While, What I want to know is if there is a library to do this, instead of writing it myself.
I have some idea if I would write it myself. Generally, all functions are turned into a task queue or task tree, and then tokio::time::delay_for can be used to execute them one by one, but the details are actually more complicated.
However, I think that this general capability may have already been implemented but I has not found for the time being, so I want to ask here, Thank you very much.
And importantly, I hope it can support single thread
setTimeout can be done like this without the need for a crate:
tokio::spawn(async move {
tokio::time::sleep(Duration::from_secs(5)).await;
// code goes here
});
I asked myself the same question a few days ago, created a solution for this (for tokio runtimes), and found your stackoverflow post just now.
https://crates.io/crates/tokio-js-set-interval
code.rs
use std::time::Duration;
use tokio_js_set_interval::{set_interval, set_timeout};
#[tokio::main]
async fn main() {
println!("hello1");
set_timeout!(println!("hello2"), 0);
println!("hello3");
set_timeout!(println!("hello4"), 0);
println!("hello5");
// give enough time before tokios runtime exits
tokio::time::sleep(Duration::from_millis(1)).await;
}
But this must be used with caution. There is no guarantee that the futures will be executed (because tokios runtime must run long enough). Use it only:
for educational purposes,
and if you have low priority background tasks that you don't expect to get executed always
I created a library just for this which allows setting many timeouts using only 1 tokio task (instead of spawning a new task for each timeout) which provides better performance and lower memory usage.
The library also supports cancelling timeouts, and provides some ways to optimize the performance of the timeouts.
Check it out:
https://crates.io/crates/set_timeout
Usage example:
#[tokio::main]
async fn main() {
let scheduler = TimeoutScheduler::new(None);
// schedule a future which will run after 1.234 seconds from now.
scheduler.set_timeout(Duration::from_secs_f32(1.234), async move {
println!("It works!");
});
// make sure that the main task doesn't end before the timeout is executed, because if the main
// task returns the runtime stops running.
tokio::time::sleep(Duration::from_secs(2)).await;
}
I'm trying to implement a project where I can tail the logs of multiple Kubernetes container logs simultaneously. Think tmux split pane with two tails in each pane. Anyway, I'm far far away from my actual project because I'm stuck right at the beginning. If you look at the following code then the commented out line for lp.follow = true will keep the log stream open and stream logs forever. I'm not sure how to actually use this. I found a function called .into_stream() that I can tack onto the pods.log function, but then I'm not sure how to actually use the stream. I'm not experienced enough to know if this is a limitation of the kube library, or if I'm just doing something wrong. Anyway, here is the repo if you want to look at anything else. https://github.com/bloveless/kube-logger
I'd be forever grateful for any advice or resources I can look at. Thanks!
use kube::{
api::Api,
client::APIClient,
config,
};
use kube::api::{LogParams, RawApi};
use futures::{FutureExt, Stream, future::IntoStream, StreamExt};
#[tokio::main]
async fn main() -> anyhow::Result<()> {
std::env::set_var("RUST_LOG", "info,kube=trace");
let config = config::load_kube_config().await?;
let client = APIClient::new(config);
// Manage pods
let pods = Api::v1Pod(client).within("fritzandandre");
let mut lp = LogParams::default();
lp.container = Some("php".to_string());
// lp.follow = true;
lp.tail_lines = Some(100);
let log_string = pods.log("fritzandandre-php-0", &lp).await?;
println!("FnA Log: {}", log_string);
Ok(())
}
Originally posted here https://www.reddit.com/r/learnrust/comments/eg49tx/help_with_futuresstreams_and_the_kubers_library/