I'm trying to write a Rocket / Juniper / Rust based GraphQL Server using PickleDB - an in-memory key/value store.
The pickle db is created / loaded at the start and given to rocket to manage:
fn rocket() -> Rocket {
let pickle_path = var_os(String::from("PICKLE_PATH")).unwrap_or(OsString::from("pickle.db"));
let pickle_db_dump_policy = PickleDbDumpPolicy::PeriodicDump(Duration::from_secs(120));
let pickle_serialization_method = SerializationMethod::Bin;
let pickle_db: PickleDb = match Path::new(&pickle_path).exists() {
false => PickleDb::new(pickle_path, pickle_db_dump_policy, pickle_serialization_method),
true => PickleDb::load(pickle_path, pickle_db_dump_policy, pickle_serialization_method).unwrap(),
};
rocket::ignite()
.manage(Schema::new(Query, Mutation))
.manage(pickle_db)
.mount(
"/",
routes![graphiql, get_graphql_handler, post_graphql_handler],
)
}
And I want to retrieve the PickleDb instance from the Rocket State in my Guard:
pub struct Context {
pickle_db: PickleDb,
}
impl juniper::Context for Context {}
impl<'a, 'r> FromRequest<'a, 'r> for Context {
type Error = ();
fn from_request(_request: &'a Request<'r>) -> request::Outcome<Context, ()> {
let pickle_db = _request.guard::<State<PickleDb>>()?.inner();
Outcome::Success(Context { pickle_db })
}
}
This does not work because the State only gives me a reference:
26 | Outcome::Success(Context { pickle_db })
| ^^^^^^^^^ expected struct `pickledb::pickledb::PickleDb`, found `&pickledb::pickledb::PickleDb`
When I change my Context struct to contain a reference I get lifetime issues which I'm not yet familiar with:
15 | pickle_db: &PickleDb,
| ^ expected named lifetime parameter
I tried using 'static which does make rust quite unhappy and I tried to use the request lifetime (?) 'r of the FromRequest, but that does not really work either...
How do I get this to work? As I'm quite new in rust, is this the right way to do things?
I finally have a solution, although the need for unsafe indicates it is sub-optimal :)
#![allow(unsafe_code)]
use pickledb::{PickleDb, PickleDbDumpPolicy, SerializationMethod};
use serde::de::DeserializeOwned;
use serde::Serialize;
use std::env;
use std::path::Path;
use std::time::Duration;
pub static mut PICKLE_DB: Option<PickleDb> = None;
pub fn cache_init() {
let pickle_path = env::var(String::from("PICKLE_PATH")).unwrap_or(String::from("pickle.db"));
let pickle_db_dump_policy = PickleDbDumpPolicy::PeriodicDump(Duration::from_secs(120));
let pickle_serialization_method = SerializationMethod::Json;
let pickle_db = match Path::new(&pickle_path).exists() {
false => PickleDb::new(
pickle_path,
pickle_db_dump_policy,
pickle_serialization_method,
),
true => PickleDb::load(
pickle_path,
pickle_db_dump_policy,
pickle_serialization_method,
)
.unwrap(),
};
unsafe {
PICKLE_DB = Some(pickle_db);
}
}
pub fn cache_get<V>(key: &str) -> Option<V>
where
V: DeserializeOwned + std::fmt::Debug,
{
unsafe {
let pickle_db = PICKLE_DB
.as_ref()
.expect("cache uninitialized - call cache_init()");
pickle_db.get::<V>(key)
}
}
pub fn cache_set<V>(key: &str, value: &V) -> Result<(), pickledb::error::Error>
where
V: Serialize,
{
unsafe {
let pickle_db = PICKLE_DB
.as_mut()
.expect("cache uninitialized - call cache_init()");
pickle_db.set::<V>(key, value)?;
Ok(())
}
}
This can be simply imported and used as expected, but I think I'll run into issues when the load gets to high...
Related
I am attempting to use a generic method to retrieve values from database tables that are structurally identical, but have different types for one of their columns. Simplified example below
async fn query<'a, 'r, T: DatabaseType<Item=T> + Decode<'r, Sqlite> + Type<Sqlite>>(&self, name: &'a str) -> Result<Vec<NamedValue<'a, T>>> {
let mut connection = self.pool.acquire().await?;
let mut rows = sqlx::query("Select id, value from table where name = $1")
.bind(name)
.fetch(&mut connection);
let mut results = Vec::new();
while let Some(row) = rows.try_next().await? {
results.push(NamedValue {
name,
value: row.try_get("value")?
})
}
Ok(results)
}
This will not compile, with the error: borrowed value does not live long enough, argument requires that 'row' is borrowed for 'r. The lifetime sqlx::Decode wants ('r), has to be declared as part of the query function's signature, but the resource the lifetime refers to does not exist yet, and only exists when the query executes and the stream is iterated over. I can't omit this bound on the generic, because the type does need to be decodable for try_get to work, so how do I tell the compiler that it is actually completely safe, and that the decoding is happening against a row that will definitely live longe enough for the try_get? Once the value is decoded, it will always have a static lifetime.
Rust playground doesn't include SQLx, an example that can be compiled at home is below:
[package]
name = "sqlx-minimal-example"
version = "0.1.0"
edition = "2021"
[dependencies]
tokio = { version = "1", features = ["full"] }
sqlx = { version = "0.6", features = ["runtime-tokio-rustls", "sqlite"] }
anyhow = "1.0"
futures = "0.3"
And the full application would be:
use anyhow::Result;
use sqlx::{Decode, Row, Sqlite, SqlitePool, Type};
use futures::TryStreamExt;
#[tokio::main]
async fn main() -> Result<()> {
println!("Hello, world!");
Ok(())
}
struct NamedValue<'a ,T> {
name: &'a str,
value: T
}
struct SqliteBackend {
pool: SqlitePool
}
trait DatabaseType {
type Item;
}
impl DatabaseType for f32 {
type Item = f32;
}
impl DatabaseType for i32 {
type Item = i32;
}
impl SqliteBackend {
async fn query<'a, 'r, T: DatabaseType<Item=T> + Decode<'r, Sqlite> + Type<Sqlite>>(&self, name: &'a str) -> Result<Vec<NamedValue<'a, T>>> {
let mut connection = self.pool.acquire().await?;
let mut rows = sqlx::query("Select id, value from table where name = $1")
.bind(name)
.fetch(&mut connection);
let mut results = Vec::new();
while let Some(row) = rows.try_next().await? {
results.push(NamedValue {
name,
value: row.try_get("value")?
})
}
Ok(results)
}
}
Higher-ranked trait bounds were the answer. This tells the compiler the type is decodable for all possible lifetimes.
Working function below:
async fn query<'a, T: DatabaseType<Item=T> + for<'r> Decode<'r, Sqlite> + Type<Sqlite>>(&self, name: &'a str) -> Result<Vec<NamedValue<'a, T>>> {
let mut connection = self.pool.acquire().await?;
let mut rows = sqlx::query("Select id, value from table where name = $1")
.bind(name)
.fetch(&mut connection);
let mut results = Vec::new();
while let Some(row) = rows.try_next().await? {
results.push(NamedValue {
name,
value: row.try_get("value")?
})
}
Ok(results)
}
I know the issue is that I have two Result types from different libraries but can't find how to fix it.
[dependencies]
crossterm = "0.23"
time = "0.3.9"
tokio = { version = "1", features = ["full"] }
reqwest = { version = "0.11", features = ["blocking", "json"] }
use time::Instant;
use std::collections::HashMap;
use crossterm::{
event::{self, Event, KeyCode, KeyEvent},
Result,
};
pub fn read_char() -> Result<char> {
loop {
if let Event::Key(KeyEvent {
code: KeyCode::Char(c),
..
}) = event::read()?
{
return Ok(c);
}
}
}
fn main() -> Result<(), Box<dyn std::error::Error>> {
let instant = Instant::now();
let response = reqwest::blocking::get("https://httpbin.org/ip")?
.json::<HashMap<String, String>>()?;
let duration = instant.elapsed();
println!("ns = {:?}, response: {:#?}, ", duration.whole_nanoseconds(), response);
// Any key to continue
println!("Press any key to continue:");
println!("{:?}", read_char());
Ok(())
}
Gives the error:
error[E0107]: this type alias takes 1 generic argument but 2 generic arguments were supplied
--> src\main.rs:20:14
|
20 | fn main() -> Result<(), Box<dyn std::error::Error>> {
| ^^^^^^ -------------------------- help: remove this generic argument
| |
| expected 1 generic argument
How do I fix this? I have searched but am likely looking for incorrect terms e.g. namespace alias and core::Result error[E0107] is not really helping.
I have tried this without success:
fn main() -> core::Result<(), Box<dyn std::error::Error>> {
You have crossterm ::Result in scope, so you would have to disambiguate the result you want to return, otherwise it just thinks you want to return the crossterm type:
fn main() -> std::result::Result<(), Box<dyn std::error::Error>> {
...
Ok(())
}
I am new to Rust, as will probably be obvious.
Basically I have this scenario you can see below where, I create a new type that has a closure added to it, but this closure needs to access data which has not yet been created. The data will be created by the time the closure gets called, but when the closure is initially created the data is not yet available.
What is the best way to do deal with?
I am also curious if my closure was not a closure, but rather a private function in my implementation, how would I access that data? This closure/function is a callback from WasmTime and requires an explicit method signature which does not allow me to add $self to it. So how could I get at the instance fields of the implementation without a reference to $self in the function parameters?
pub struct EmWasmNode {
wasmStore: Store<WasiCtx>,
wasmTable: Table,
}
impl EmWasmNode {
pub fn new(filePath: &str) -> Result<Self> {
let engine = Engine::default();
// let module = Module::from_file(&engine, "wasm/index.wast")?;
let module = Module::from_file(&engine, filePath)?;
let mut linker = Linker::new(&engine);
wasmtime_wasi::add_to_linker(&mut linker, |s| s)?;
let wasi = WasiCtxBuilder::new()
.inherit_stdio()
.inherit_args()?
.build();
let mut store = Store::new(&engine, wasi);
linker.func_wrap("env", "emscripten_set_main_loop", |p0: i32, p1: i32, p2: i32| {
println!("emscripten_set_main_loop {} {} {}", p0, p1, p2);
/*** How would I access wasmTable and wasmStore from here to execute more methods??? ***/
//let browserIterationFuncOption:Option<wasmtime::Val> = Self::wasmTable.get(&mut Self::wasmStore, p0 as u32);
// browserIterationFuncOption.unwrap().unwrap_funcref().call(&store, ());
})?;
let instance = linker.instantiate(&mut store, &module)?;
let table = instance
.get_export(&mut store, "__indirect_function_table")
.as_ref()
.and_then(extern_table)
.cloned();
let start = instance.get_typed_func::<(), (), _>(&mut store, "_start")?;
start.call(&mut store, ())?;
Ok(EmWasmNode {
wasmStore: store,
wasmTable: table.unwrap(),
})
}
You have to instantiate a struct before. I suggest the more simple code below to see my idea.
struct Atype
{
name: String,
}
impl Atype
{
pub fn new() -> Self
{
Self{ name: String::from("zeppi")}
}
pub fn test(&self) -> ()
{
let func = | x | { println!("{} {}", &self.name, x);};
func(3)
}
}
fn main() {
let o = Atype::new();
o.test();
}
I'm new to rust and still have some problems with ownership/borrowing. In this case, I want to store a FnOnce in an enum and then call it later on from another thread. I tried a lot of different variants but always get stuck somewhere. Here is a reduced variant of what I currently have:
#![feature(fnbox)]
use std::sync::{Arc, Mutex};
use std::boxed::{Box, FnBox};
enum Foo<T> {
DoNothing,
CallFunction(Box<FnBox(&T) + Send>)
}
struct FooInMutex<T> {
foo: Arc<Mutex<Foo<T>>>
}
impl<T> FooInMutex<T> {
fn put_fn(&self, f: Box<FnBox(&T)+Send>) {
let mut foo = self.foo.lock().unwrap();
let mut new_foo : Foo<T>;
match *foo {
Foo::DoNothing =>
new_foo = Foo::CallFunction(f),
_ =>
new_foo = Foo::DoNothing
}
*foo = new_foo;
}
fn do_it(&self, t: T) {
let mut foo = self.foo.lock().unwrap();
let mut new_foo : Foo<T>;
match *foo {
Foo::CallFunction(ref mut f) => {
//f(&t)
f.call_box((&t,));
new_foo = Foo::DoNothing;
}
_ =>
panic!("...")
}
*foo = new_foo;
}
}
#[test]
fn it_works() {
let x = FooInMutex { foo: Arch::new(Mutex::new(Foo::DoNothing)) };
x.put_fn(Box::new(|| panic!("foo")));
x.do_it();
}
I use "rustc 1.4.0-nightly (e35fd7481 2015-08-17)".
The error message:
src/lib.rs:35:17: 35:18 error: cannot move out of borrowed content
src/lib.rs:35 f.call_box((&t,));
^
As I understand it, f is owned by the enum in the mutex and I only borrow it via *foo. But for calling f, I need move it out. But how to do this? Or what else do I have to change to make this example work?
std::mem::replace is what you should use there, like this:
use std::mem;
…
fn do_it(&self, t: T) {
match mem::replace(self.foo.lock().unwrap(), Foo::DoNothing) {
Foo::CallFunction(f) => {
f.call_box((&t,));
}
_ => panic!("...")
}
}
Say we want to have objects implementations switched at runtime, we'd do something like this:
pub trait Methods {
fn func(&self);
}
pub struct Methods_0;
impl Methods for Methods_0 {
fn func(&self) {
println!("foo");
}
}
pub struct Methods_1;
impl Methods for Methods_1 {
fn func(&self) {
println!("bar");
}
}
pub struct Object<'a> { //'
methods: &'a (Methods + 'a),
}
fn main() {
let methods: [&Methods; 2] = [&Methods_0, &Methods_1];
let mut obj = Object { methods: methods[0] };
obj.methods.func();
obj.methods = methods[1];
obj.methods.func();
}
Now, what if there are hundreds of such implementations? E.g. imagine implementations of cards for collectible card game where every card does something completely different and is hard to generalize; or imagine implementations for opcodes for a huge state machine. Sure you can argue that a different design pattern can be used -- but that's not the point of this question...
Wonder if there is any way for these Impl structs to somehow "register" themselves so they can be looked up later by a factory method? I would be happy to end up with a magical macro or even a plugin to accomplish that.
Say, in D you can use templates to register the implementations -- and if you can't for some reason, you can always inspect modules at compile-time and generate new code via mixins; there are also user-defined attributes that can help in this. In Python, you would normally use a metaclass so that every time a new child class is created, a ref to it is stored in the metaclass's registry which allows you to look up implementations by name or parameter; this can also be done via decorators if implementations are simple functions.
Ideally, in the example above you would be able to create Object as
Object::new(0)
where the value 0 is only known at runtime and it would magically return you an Object { methods: &Methods_0 }, and the body of new() would not have the implementations hard-coded like so "methods: [&Methods; 2] = [&Methods_0, &Methods_1]", instead it should be somehow inferred automatically.
So, this is probably extremely buggy, but it works as a proof of concept.
It is possible to use Cargo's code generation support to make the introspection at compile-time, by parsing (not exactly parsing in this case, but you get the idea) the present implementations, and generating the boilerplate necessary to make Object::new() work.
The code is pretty convoluted and has no error handling whatsoever, but works.
Tested on rustc 1.0.0-dev (2c0535421 2015-02-05 15:22:48 +0000)
(See on github)
src/main.rs:
pub mod implementations;
mod generated_glue {
include!(concat!(env!("OUT_DIR"), "/generated_glue.rs"));
}
use generated_glue::Object;
pub trait Methods {
fn func(&self);
}
pub struct Methods_2;
impl Methods for Methods_2 {
fn func(&self) {
println!("baz");
}
}
fn main() {
Object::new(2).func();
}
src/implementations.rs:
use super::Methods;
pub struct Methods_0;
impl Methods for Methods_0 {
fn func(&self) {
println!("foo");
}
}
pub struct Methods_1;
impl Methods for Methods_1 {
fn func(&self) {
println!("bar");
}
}
build.rs:
#![feature(core, unicode, path, io, env)]
use std::env;
use std::old_io::{fs, File, BufferedReader};
use std::collections::HashMap;
fn main() {
let target_dir = Path::new(env::var_string("OUT_DIR").unwrap());
let mut target_file = File::create(&target_dir.join("generated_glue.rs")).unwrap();
let source_code_path = Path::new(file!()).join_many(&["..", "src/"]);
let source_files = fs::readdir(&source_code_path).unwrap().into_iter()
.filter(|path| {
match path.str_components().last() {
Some(Some(filename)) => filename.split('.').last() == Some("rs"),
_ => false
}
});
let mut implementations = HashMap::new();
for source_file_path in source_files {
let relative_path = source_file_path.path_relative_from(&source_code_path).unwrap();
let source_file_name = relative_path.as_str().unwrap();
implementations.insert(source_file_name.to_string(), vec![]);
let mut file_implementations = &mut implementations[*source_file_name];
let mut source_file = BufferedReader::new(File::open(&source_file_path).unwrap());
for line in source_file.lines() {
let line_str = match line {
Ok(line_str) => line_str,
Err(_) => break,
};
if line_str.starts_with("impl Methods for Methods_") {
const PREFIX_LEN: usize = 25;
let number_len = line_str[PREFIX_LEN..].chars().take_while(|chr| {
chr.is_digit(10)
}).count();
let number: i32 = line_str[PREFIX_LEN..(PREFIX_LEN + number_len)].parse().unwrap();
file_implementations.push(number);
}
}
}
writeln!(&mut target_file, "use super::Methods;").unwrap();
for (source_file_name, impls) in &implementations {
let module_name = match source_file_name.split('.').next() {
Some("main") => "super",
Some(name) => name,
None => panic!(),
};
for impl_number in impls {
writeln!(&mut target_file, "use {}::Methods_{};", module_name, impl_number).unwrap();
}
}
let all_impls = implementations.values().flat_map(|impls| impls.iter());
writeln!(&mut target_file, "
pub struct Object;
impl Object {{
pub fn new(impl_number: i32) -> Box<Methods + 'static> {{
match impl_number {{
").unwrap();
for impl_number in all_impls {
writeln!(&mut target_file,
" {} => Box::new(Methods_{}),", impl_number, impl_number).unwrap();
}
writeln!(&mut target_file, "
_ => panic!(\"Unknown impl number: {{}}\", impl_number),
}}
}}
}}").unwrap();
}
The generated code:
use super::Methods;
use super::Methods_2;
use implementations::Methods_0;
use implementations::Methods_1;
pub struct Object;
impl Object {
pub fn new(impl_number: i32) -> Box<Methods + 'static> {
match impl_number {
2 => Box::new(Methods_2),
0 => Box::new(Methods_0),
1 => Box::new(Methods_1),
_ => panic!("Unknown impl number: {}", impl_number),
}
}
}