A client for HTTP server push (streaming) in Rust? - rust

Question

For the lack of a better example, let's say I want to write a simple client with Rust that could establish a connection and receive data from Twitter's HTTP Streaming API. Is this possible yet? I've been keeping an eye on Iron and Nickel which seem like good frameworks, but I don't think they have this feature yet?

Answer 1

The http client hyper supports reading responses incrementally (as anything that implements rust's Reader trait), but I wasn't able to find anything to parse the response incrementally, or that implements twitter's particular protocol (to end objecs with \r\n).
That said, I was able to implement a quick'n'dirty proof of concept:
EDIT: See and play with it on github.
use rustc_serialize::json::Json;
use std::str;
pub trait JsonObjectStreamer {
fn json_objects(&mut self) -> JsonObjects<Self>;
}
impl<T: Buffer> JsonObjectStreamer for T {
fn json_objects(&mut self) -> JsonObjects<T> {
JsonObjects { reader: self }
}
}
pub struct JsonObjects<'a, B> where B: 'a {
reader: &'a mut B
}
impl<'a, B> Iterator for JsonObjects<'a, B> where B: Buffer + 'a {
type Item = Json;
fn next(&mut self) -> Option<Json> {
let mut line_bytes = match self.reader.read_until(b'\r') {
Ok(bytes) => bytes,
Err(_) => return None,
};
if line_bytes.last() == Some(&b'\r') {
// drop the \r
line_bytes.pop();
// skip the \n
match self.reader.read_char() {
Ok(_) => (),
Err(_) => return None,
}
}
let line = match str::from_utf8(&line_bytes) {
Ok(line) => line,
Err(_) => return None
};
Json::from_str(line).ok()
}
}
Usage: (assuming you have dropped it on a src/json_streamer.rs file on your project)
#![feature(io)]
extern crate hyper;
extern crate "rustc-serialize" as rustc_serialize;
mod json_streamer;
use hyper::Client;
use std::old_io::BufferedReader;
use json_streamer::JsonObjectStreamer;
fn main() {
let mut client = Client::new();
let res = client.get("http://localhost:4567/").send().unwrap();
for obj in BufferedReader::new(res).json_objects() {
println!("object arrived: {}", obj);
}
}
I've used this tiny sinatra app to test it:
require 'sinatra'
require 'json'
class Stream
def each
hash = { index: 0 }
loop do
hash[:index] += 1
yield hash.to_json + "\r\n"
sleep 0.5
end
end
end
get '/' do
Stream.new
end