I'm trying to wrap an API using rust, so long no problems but one endpoint is giving me a big pain in the head.
the endpoint returns a valid JSON as the other endpoints from this API.
The problem is when I try to work with the response I get the following error:
error decoding response body: trailing characters at line 1 column 146
Get function:
pub fn get<T: DeserializeOwned>(&self, endpoint: API, request: Option<String>) -> Result<T> {
let mut url: String = format!("{}{}", self.host, String::from(endpoint));
if let Some(request) = request {
if !request.is_empty() {
url.push_str(format!("?{}", request).as_str());
}
}
let client = &self.inner_client;
let response = client.get(url.as_str()).send()?;
self.handler(response)
}
Handler function:
fn handler<T: DeserializeOwned>(&self, response: Response) -> Result<T> {
match response.status() {
StatusCode::OK => {
let l = response.json::<T>()?;
Ok(l)
},
s => {
bail!(format!("Received response: {:?}", s));
}
}
}
Enpoint funcion:
pub fn get_depth<S>(&self, symbol: S) -> Result<OrderBook>
where
S: Into<String>,
{
let mut parameters: BTreeMap<String, String> = BTreeMap::new();
parameters.insert("symbol".into(), symbol.into());
let request = build_request(parameters);
self.client.get(API::Spot(Spot::Depth), Some(request))
}
Models:
#[derive(Debug, Serialize, Deserialize, Clone)]
#[serde(rename_all = "camelCase")]
pub struct OrderBook {
pub last_update_id: u64,
pub asks: Vec<Asks>,
pub bids: Vec<Bids>,
}
#[derive(PartialEq, Debug, Serialize, Deserialize, Clone)]
pub struct Bids {
#[serde(with = "string_or_float")]
pub price: f64,
#[serde(with = "string_or_float")]
pub qty: f64,
}
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct Asks {
#[serde(with = "string_or_float")]
pub price: f64,
#[serde(with = "string_or_float")]
pub qty: f64,
}
API response:
{
"lastUpdateId": 1595750078,
"bids": [
[
"0.00001661",
"24250.00000000",
[]
],
[
"0.00001660",
"65159.00000000",
[]
]
],
"asks": [
[
"0.00001662",
"26397.00000000",
[]
],
[
"0.00001663",
"54421.00000000",
[]
]
]
}
I was testing this mocking the API response and removing the empty arrays from asks and bids and it worked flawless. I don't know why the [] is a problem and I can figure out how to solve this problem. So really I will appreciate any help.
Your problem boils down to matching your rust struct with the response structure. I'm using the example with String from your playground link.
Response
{
"lastUpdateId":1595750078,
"bids":[
[
"0.00001661",
"24250.00000000",
[
]
],
[
"0.00001660",
"65159.00000000",
[
]
]
],
"asks":[
[
"0.00001662",
"26397.00000000"
],
[
"0.00001663",
"54421.00000000"
]
]
}
Your Structs
#[derive(Debug, Serialize, Deserialize, Clone)]
#[serde(rename_all = "camelCase")]
pub struct OrderBook {
pub last_update_id: u64,
pub asks: Vec<Asks>,
pub bids: Vec<Bids>,
}
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct Bids {
pub price: String,
pub qty: String,
}
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct Asks {
pub price: String,
pub qty: String,
}
Now if you look at the bids json, the empty array is not represented in the Bids structure in your code. Hence calling serde_json::from_str::<OrderBook>(json) fails, as it's encountering a new array where nothing is expected anymore.
Solution
Introduce a new (optional) field in Bids, representing the array. I cannot say which structure this would be, as your example only shows the empty representation. pub x: Vec<()> would also work, though I'm certain the API might sent real data if available.
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct Bids {
pub price: String,
pub qty: String,
pub something_else: Vec<()>, // or Vec<AnotherStruct>
}
Now I want to map object from Favorites to FavMusicResponse, the Favorites object was selected from database and the FavMusicResponse entity was return to the client. I define the map function like this way in FavMusicResponse:
impl From<&Favorites> for FavMusicResponse {
fn from(f: &Favorites, music: Music) -> Self {
Self{
id: f.id,
song_id: None,
created_time: f.created_time,
updated_time: f.updated_time,
user_id: 0,
source_id: "".to_string(),
like_status: 0,
source: 0,
playlist_id: 0,
play_count: 0,
fetched_download_url: None,
downloaded: None,
music: Default::default()
}
}
}
most of the fields from Favorites fields are the same with FavMusicResponse, the only difference is that music was passed from another entity. This is my FavMusicResponse define:
#[derive( Serialize, Queryable, Deserialize,Default, Clone)]
pub struct FavMusicResponse{
pub id: i64,
pub song_id: Option<i64>,
pub created_time: i64,
pub updated_time: i64,
pub user_id: i64,
pub source_id: String,
pub like_status: i32,
pub source: i32,
pub playlist_id: i64,
pub play_count: i32,
pub fetched_download_url: Option<i32>,
pub downloaded: Option<i32>,
pub music: Music
}
the compiler tell me that from expected 1 parameter, found 2, what should I do do pass 2 or more parameter into the composite function in rust? I invoke the function like this way:
let filtered_music:Vec<_> = musics.iter()
.filter(|item| item.source_id == fav.source_id)
.map(|item|FavMusicResponse::from(fav,item.clone()))
.collect();
I want to passed two entity and composite to the finally FavMusicResponse entity. what should I do to make it work like that? This is my minimal reproduce that could run in rust playground to figure out where is going wrong:
fn main() {
let music = Music{
id: 1
};
let favMusic = Favorites{
id: 1
};
let musicRes = FavMusicResponse::from(favMusic,music);
}
pub struct Music {
pub id: i64
}
pub struct Favorites {
pub id: i64
}
pub struct FavMusicResponse {
pub id: i64,
pub music:Music
}
impl From<Favorites> for FavMusicResponse {
fn from(f: Favorites, music: Music) -> Self {
Self{
id: f.id,
music: music
}
}
}
From trait can't take two parameters only one, it's define that way, the only solution if you really want to use From trait is to transform your two parameter into one, a quick solution is to use a tuple:
impl From<(Favorites, Music)> for FavMusicResponse {
fn from((f, music): (Favorites, Music)) -> Self {
Self {
id: f.id,
music,
}
}
}
// let musicRes = FavMusicResponse::from((favMusic, music));
That said you could also just have a new() method on your type:
impl FavMusicResponse {
fn new(f: Favorites, music: Music) -> Self {
Self {
id: f.id,
music,
}
}
}
// let musicRes = FavMusicResponse::new(favMusic, music);
This question already has answers here:
How to deserialize a JSON file which contains null values using Serde?
(3 answers)
Closed 1 year ago.
I am using a structure to receive data from a http request body in rust. This is the struct define:
use rocket::serde::Deserialize;
use rocket::serde::Serialize;
#[derive(Debug, PartialEq, Eq, Deserialize, Serialize)]
#[allow(non_snake_case)]
pub struct LearningWordRequest {
pub wordId: i64,
pub word: String,
pub wordStatus: i32
}
but I have to put all of the fields into the body to parse, if I did not do it like that. The server will show:
2021-11-10T15:02:43 [WARN] - `Json < LearningWordRequest >` data guard failed: Parse("{\"word\":\"hood\",\"wordId\":-1}", Error("missing field `wordStatus`", line: 1, column: 27)).
is it possible to make the parse just ignore the field if I did not put this field? Sometimes it is no need to transfer all fields in the http body, this is my server side receive code:
#[post("/v1/add",data = "<record>")]
pub fn add_learning_word(record: Json<LearningWordRequest>) -> content::Json<String> {
let res = ApiResponse {
result: "ok",
..Default::default()
};
let response_json = serde_json::to_string(&res).unwrap();
return content::Json(response_json);
}
Change the struct member to be optional:
#[derive(Debug, PartialEq, Eq, Deserialize, Serialize)]
#[allow(non_snake_case)]
pub struct LearningWordRequest {
pub wordId: i64,
pub word: String,
pub wordStatus: Option<i32>
}
For example:
fn main() {
let present = r#" {
"wordId": 43,
"word": "foo",
"wordStatus": 1337
}"#;
let absent = r#" {
"wordId": 43,
"word": "foo"
}"#;
println!("{:?}", serde_json::from_str::<LearningWordRequest>(present));
println!("{:?}", serde_json::from_str::<LearningWordRequest>(absent));
}
Outputs:
Ok(LearningWordRequest { wordId: 43, word: "foo", wordStatus: Some(1337) })
Ok(LearningWordRequest { wordId: 43, word: "foo", wordStatus: None })
I'm working on modifying some code that was written by another party. The purpose is to emulate a UniFi Security Gateway to get reporting in the UniFi Controller software. I'm planning to run the emulation on a Zotac Mini PC with 2 NICs running CentOS 8. So that I don't have to worry about additional NAT, I have the 2 NICs setup as a bridge.
The struct for each network device is defined as:
#[derive(PartialEq, Clone, Debug)]
pub(crate) struct UnixNetworkDevice {
name: String,
mac: MacAddr,
interface: NetworkInterface,
statistics: UnixNetworkDeviceStatistics,
}
When the struct is filled in for the eth0 device you get this (wan_device at the beginning is just my tag so I know what I'm looking at):
wan_device Some(UnixNetworkDevice {
name: "eth0",
mac: 00:01:2e:80:3e:1d,
interface: NetworkInterface {
name: "eth0",
index: 2,
mac: Some(00:01:2e:80:3e:1d),
ips: [],
flags: 69699
},
statistics: UnixNetworkDeviceStatistics {
collisions: 0,
multicast: 497,
rx_bytes: 359185,
rx_compressed: 0,
rx_crc_errors: 0,
rx_dropped: 26,
rx_errors: 0,
rx_fifo_errors: 0,
rx_frame_errors: 0,
rx_length_errors: 0,
rx_missed_errors: 0,
rx_nohandler: 0,
rx_over_errors: 0,
rx_packets: 1587,
tx_aborted_errors: 0,
tx_bytes: 74695,
tx_carrier_errors: 0,
tx_compressed: 0,
tx_dropped: 0,
tx_errors: 0,
tx_fifo_errors: 0,
tx_heartbeat_errors: 0,
tx_packets: 474,
tx_window_errors: 0
}
})
I'm also pulling the same information from the Bridge device:
bri_device Some(UnixNetworkDevice {
name: "bri0",
mac: 00:01:2e:80:3e:1d,
interface: NetworkInterface {
name: "bri0",
index: 5,
mac: Some(00:01:2e:80:3e:1d),
ips: [V4(Ipv4Network { addr: 192.168.113.2, prefix: 24 }),
V6(Ipv6Network { addr: fe80::c8ee:a0ff:fe3a:3096, prefix: 64 })],
flags: 69699
},
statistics: UnixNetworkDeviceStatistics {
collisions: 0,
multicast: 0,
rx_bytes: 275467,
rx_compressed: 0,
rx_crc_errors: 0,
rx_dropped: 0,
rx_errors: 0,
rx_fifo_errors: 0,
rx_frame_errors: 0,
rx_length_errors: 0,
rx_missed_errors: 0,
rx_nohandler: 0,
rx_over_errors: 0,
rx_packets: 1371,
tx_aborted_errors: 0,
tx_bytes: 68355,
tx_carrier_errors: 0,
tx_compressed: 0,
tx_dropped: 0,
tx_errors: 0,
tx_fifo_errors: 0,
tx_heartbeat_errors: 0,
tx_packets: 345,
tx_window_errors: 0
}
})
The wan_device doesn't show any IP addresses, since that is assigned to the bridge.
Right now, when it tries to do a clone() of the wan_device, it fails because of the empty ips section in the struct. I would like to copy the IPS portion of the struct from bri_device to wan_device. I hope that is possible.
Here is an MRE that can be made into a project, including the Cargo.toml file after it.
This is the link to the ZIP file of the same:
https://www.dropbox.com/s/7pfxbmv49ra1jxs/Test-OpnFi.zip?dl=0
#[macro_use]
extern crate log;
extern crate lazy_static;
extern crate clap;
extern crate regex;
extern crate simple_logger;
/// Network interface for inform
#[derive(Serialize, Deserialize, PartialEq, Clone, Debug, Default)]
pub struct OpnFiInformNetworkInterface {
pub drops: usize,
pub enabled: bool,
pub full_duplex: bool,
pub gateways: Vec<String>,
pub ip: String,
pub latency: usize,
pub mac: String,
pub name: String,
pub nameservers: Vec<String>,
pub netmask: String,
pub num_port: usize,
pub rx_bytes: usize,
pub rx_dropped: usize,
pub rx_errors: usize,
pub rx_multicast: usize,
pub rx_packets: usize,
pub speed: usize,
pub speedtest_lastrun: usize,
pub speedtest_ping: usize,
pub speedtest_status: String,
pub tx_bytes: usize,
pub tx_dropped: usize,
pub tx_errors: usize,
pub tx_packets: usize,
pub up: bool,
pub uptime: usize,
pub xput_down: usize,
pub xput_up: usize,
}
use serde::{Deserialize, Serialize};
#[derive(Serialize, Deserialize, PartialEq, Clone, Debug)]
pub(crate) struct Config {
pub inform_url: String,
pub capability: Vec<String>,
pub cfgversion: String,
pub selfrun_guest_mode: String,
pub led_enabled: bool,
pub stun_url: String,
pub mgmt_url: String,
pub authkey: String,
pub use_aes_gcm: bool,
pub report_crash: bool,
}
use pnet::{
datalink::{interfaces, NetworkInterface},
util::MacAddr,
};
use std::str::FromStr;
use std::{fs, io, path};
#[derive(PartialEq, Clone, Debug)]
pub(crate) struct UnixNetworkDevice {
name: String,
mac: MacAddr,
interface: NetworkInterface,
statistics: UnixNetworkDeviceStatistics,
}
impl UnixNetworkDevice {
pub fn new(name: &String) -> io::Result<UnixNetworkDevice> {
let device_path = path::Path::new("/sys/class/net").join(name);
if !device_path.as_path().is_dir() {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
format!("Unable to locate UnixNetworkDevice {}", name),
));
}
let mac_string = fs::read_to_string(device_path.join("address"))?;
if mac_string.trim().len() < 15 {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
"Mac address is empty.",
));
}
let mac = MacAddr::from_str(mac_string.trim())
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
let interface = interfaces().into_iter().filter(|i| i.name == *name).next();
if interface.is_none() {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
format!("Unable to locate NetworkInterface {}", name),
));
}
let interface = interface.unwrap();
let statistics = UnixNetworkDeviceStatistics::new(name);
Ok(UnixNetworkDevice {
name: name.clone(),
mac,
interface,
statistics,
})
}
pub fn name(&self) -> String {
self.name.clone()
}
pub fn mac(&self) -> MacAddr {
self.mac
}
pub fn interface(&self) -> NetworkInterface {
self.interface.clone()
}
pub fn statistics(&self) -> UnixNetworkDeviceStatistics {
self.statistics.clone()
}
}
// ===== Statistics =====
#[derive(PartialOrd, PartialEq, Clone, Debug)]
pub(crate) struct UnixNetworkDeviceStatistics {
pub collisions: usize,
pub multicast: usize,
pub rx_bytes: usize,
pub rx_compressed: usize,
pub rx_crc_errors: usize,
pub rx_dropped: usize,
pub rx_errors: usize,
pub rx_fifo_errors: usize,
pub rx_frame_errors: usize,
pub rx_length_errors: usize,
pub rx_missed_errors: usize,
pub rx_nohandler: usize,
pub rx_over_errors: usize,
pub rx_packets: usize,
pub tx_aborted_errors: usize,
pub tx_bytes: usize,
pub tx_carrier_errors: usize,
pub tx_compressed: usize,
pub tx_dropped: usize,
pub tx_errors: usize,
pub tx_fifo_errors: usize,
pub tx_heartbeat_errors: usize,
pub tx_packets: usize,
pub tx_window_errors: usize,
}
impl UnixNetworkDeviceStatistics {
pub fn new(device_name: &String) -> UnixNetworkDeviceStatistics {
let read_value = |statistic_name: &str| -> io::Result<usize> {
let stat_path = path::Path::new("/sys/class/net")
.join(device_name)
.join("statistics")
.join(statistic_name);
let value = fs::read_to_string(stat_path.as_path())?;
usize::from_str(value.trim())
.map_err(|e| io::Error::new(io::ErrorKind::InvalidInput, e))
};
UnixNetworkDeviceStatistics {
collisions: read_value("collisions").unwrap_or_default(),
multicast: read_value("multicast").unwrap_or_default(),
rx_bytes: read_value("rx_bytes").unwrap_or_default(),
rx_compressed: read_value("rx_compressed").unwrap_or_default(),
rx_crc_errors: read_value("rx_crc_errors").unwrap_or_default(),
rx_dropped: read_value("rx_dropped").unwrap_or_default(),
rx_errors: read_value("rx_errors").unwrap_or_default(),
rx_fifo_errors: read_value("rx_fifo_errors").unwrap_or_default(),
rx_frame_errors: read_value("rx_frame_errors").unwrap_or_default(),
rx_length_errors: read_value("rx_length_errors").unwrap_or_default(),
rx_missed_errors: read_value("rx_missed_errors").unwrap_or_default(),
rx_nohandler: read_value("rx_nohandler").unwrap_or_default(),
rx_over_errors: read_value("rx_over_errors").unwrap_or_default(),
rx_packets: read_value("rx_packets").unwrap_or_default(),
tx_aborted_errors: read_value("tx_aborted_errors").unwrap_or_default(),
tx_bytes: read_value("tx_bytes").unwrap_or_default(),
tx_carrier_errors: read_value("tx_carrier_errors").unwrap_or_default(),
tx_compressed: read_value("tx_compressed").unwrap_or_default(),
tx_dropped: read_value("tx_dropped").unwrap_or_default(),
tx_errors: read_value("tx_errors").unwrap_or_default(),
tx_fifo_errors: read_value("tx_fifo_errors").unwrap_or_default(),
tx_heartbeat_errors: read_value("tx_heartbeat_errors").unwrap_or_default(),
tx_packets: read_value("tx_packets").unwrap_or_default(),
tx_window_errors: read_value("tx_window_errors").unwrap_or_default(),
}
}
}
impl From<UnixNetworkDevice> for OpnFiInformNetworkInterface {
fn from(value: UnixNetworkDevice) -> Self {
let interface = value.interface();
let stats = value.statistics();
let ip = interface
.ips
.iter()
.filter(|ip| ip.is_ipv4())
.next()
.unwrap();
Self {
drops: stats.rx_dropped + stats.tx_dropped,
enabled: true,
full_duplex: true,
gateways: vec![],
ip: ip.ip().to_string(),
latency: 1,
mac: value.mac().to_string(),
name: value.name().to_string(),
nameservers: vec![],
netmask: ip.mask().to_string(),
num_port: interface.index as usize,
rx_bytes: stats.rx_bytes,
rx_dropped: stats.rx_dropped,
rx_errors: stats.rx_errors,
rx_multicast: 0,
rx_packets: stats.rx_packets,
speed: 1000,
speedtest_lastrun: 0,
speedtest_ping: 0,
speedtest_status: "Idle".to_string(),
tx_bytes: stats.tx_bytes,
tx_dropped: stats.tx_dropped,
tx_errors: stats.tx_errors,
tx_packets: stats.tx_packets,
up: true,
uptime: 0,
xput_down: 0,
xput_up: 0,
}
}
}
fn main() {
let matches = clap::App::new("OpnFi Device")
.version("0.1.0")
.author("James Parks <jrjparks#zathera.com>")
.about("Emulates a UniFi device")
.arg(
clap::Arg::with_name("config")
.short("c")
.long("config")
.value_name("FILE")
.help("Sets a config file path to use")
.takes_value(true),
)
.arg(
clap::Arg::with_name("controller")
.long("controller")
.value_name("FILE")
.help("FQDN or IP Address of UniFi Controller")
.takes_value(true),
)
.arg(
clap::Arg::with_name("wan")
.short("w")
.long("wan")
.value_name("NIC")
.help("Set the nic to report for WAN")
.takes_value(true)
.default_value("eth0"),
)
.arg(
clap::Arg::with_name("lan")
.short("l")
.long("lan")
.value_name("NIC")
.help("Set the nic to report for LAN")
.takes_value(true)
.default_value("eth1"),
)
.arg(
clap::Arg::with_name("bri")
.short("b")
.long("bri")
.value_name("NIC")
.help("Set the nic to report for BRIDGE")
.takes_value(true)
.default_value("bri0"),
)
.get_matches();
let _wan_device = match matches.value_of("wan") {
Some(wan_name) => {
info!("Using {} as WAN device.", wan_name);
UnixNetworkDevice::new(&wan_name.to_string()).ok()
}
None => None,
};
let _lan_device = match matches.value_of("lan") {
Some(lan_name) => {
info!("Using {} as LAN device.", lan_name);
UnixNetworkDevice::new(&lan_name.to_string()).ok()
}
None => None,
};
let _bri_device = match matches.value_of("bri") {
Some(bri_name) => {
info!("Using {} as BRIDGE device.", bri_name);
UnixNetworkDevice::new(&bri_name.to_string()).ok()
}
None => None,
};
println!("wan_device {:?}", _wan_device);
println!("lan_device {:?}", _lan_device);
println!("bri_device {:?}", _bri_device);
// Interfaces
println!("Wan_Interface");
let _wan_interface: Option<OpnFiInformNetworkInterface> = match &mut _wan_device.as_ref()
{
Some(wan) => Some(wan.clone().into()),
_ => None,
};
println!("Lan_Interface");
let _lan_interface: Option<OpnFiInformNetworkInterface> = match &mut _lan_device.as_ref()
{
Some(lan) => Some(lan.clone().into()),
_ => None,
};
println!("Bri_Interface");
let _bri_interface: Option<OpnFiInformNetworkInterface> = match &mut _bri_device.as_ref()
{
Some(bri) => Some(bri.clone().into()),
_ => None,
};
}
[package]
name = "test_opnfi"
version = "0.0.1"
authors = ["Mike Schaffner <mcschaffner#gmail.com>"]
edition = "2018"
license = "Apache-2.0"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
lazy_static = "1.4.0"
toml = "0.5.5"
regex = "1.3.1"
rand = "0.7.2"
net2 = "0.2.33"
byteorder = "1.3.2"
serde = "1.0.103"
serde_json = "1.0.42"
sysinfo = "0.9.6"
reqwest = "0.9"
hex = "0.4.0"
pnet = "0.23.0"
clap = "2.33"
ctrlc = "3.1.3"
log = "0.4.8"
simple_logger = "1.3.0"
Your question hits many rust specific questions. Types, size of types, livetime, optional values to just name a few.
Please read Shepmaster's comment and update your question. To give you a direction that may help: https://gist.github.com/rust-play/2289bbcf50caabe7bc0fb397e073e581
// #[derive(Clone)] dosn't work because the unknown size of `value`
struct A {
value: [isize],
}
#[derive(Clone)]
struct B<'a> {
value: Option<&'a [isize]>,
}
fn main() {
//let a: A = A { value: [1] };
//let a1 = a.clone();
let b: B = B { value: None };
let b1 = b.clone();
}