How to write an asynchronous recursive walkdir function with an asynchronous callback - rust

Question

I'm trying to write an async function that will traverse the filesystem tree, recursively, and calls an asynchronous callback for each file found.
This is for a learning effort, I have no real use case.
Here is what I have so far:
use async_std::{
fs::{self, *},
path::*,
prelude::*,
}; // 1.5.0, features = ["unstable"]
use futures::{
executor::block_on,
future::{BoxFuture, FutureExt},
}; // 0.3.4
use std::{marker::Sync, pin::Pin};
fn main() {
fn walkdir<F>(path: String, cb: &'static F) -> BoxFuture<'static, ()>
where
F: Fn(&DirEntry) -> BoxFuture<()> + Sync + Send,
{
async move {
let mut entries = fs::read_dir(&path).await.unwrap();
while let Some(path) = entries.next().await {
let entry = path.unwrap();
let path = entry.path().to_str().unwrap().to_string();
if entry.path().is_file().await {
cb(&entry).await
} else {
walkdir(path, cb).await
}
}
}
.boxed()
}
let foo = async {
walkdir(".".to_string(), &|entry: &DirEntry| async {
async_std::println!(">> {}\n", &entry.path().to_str().unwrap()).await
})
.await
};
block_on(foo);
}
I get this far by some sort of trial and error, but now I'm stuck on async closure callback with this error
warning: unused import: `path::*`
--> src/main.rs:3:5
|
3 | path::*,
| ^^^^^^^
|
= note: `#[warn(unused_imports)]` on by default
warning: unused import: `pin::Pin`
--> src/main.rs:10:25
|
10 | use std::{marker::Sync, pin::Pin};
| ^^^^^^^^
error[E0308]: mismatched types
--> src/main.rs:33:54
|
33 | walkdir(".".to_string(), &|entry: &DirEntry| async {
| ______________________________________________________^
34 | | async_std::println!(">> {}\n", &entry.path().to_str().unwrap()).await
35 | | })
| |_________^ expected struct `std::pin::Pin`, found opaque type
|
= note: expected struct `std::pin::Pin<std::boxed::Box<dyn core::future::future::Future<Output = ()> + std::marker::Send>>`
found opaque type `impl core::future::future::Future`

Answer 1

use async_std::{
fs::{self, *},
path::*,
prelude::*,
}; // 1.5.0
use futures::{future::{Future, FutureExt, LocalBoxFuture}, executor}; // 0.3.4
fn main() {
async fn walkdir<R>(path: impl AsRef<Path>, mut cb: impl FnMut(DirEntry) -> R)
where
R: Future<Output = ()>,
{
fn walkdir_inner<'a, R>(path: &'a Path, cb: &'a mut dyn FnMut(DirEntry) -> R) -> LocalBoxFuture<'a, ()>
where
R: Future<Output = ()>,
{
async move {
let mut entries = fs::read_dir(path).await.unwrap();
while let Some(path) = entries.next().await {
let entry = path.unwrap();
let path = entry.path();
if path.is_file().await {
cb(entry).await
} else {
walkdir_inner(&path, cb).await
}
}
}.boxed_local()
}
walkdir_inner(path.as_ref(), &mut cb).await
}
executor::block_on({
walkdir(".", |entry| async move {
async_std::println!(">> {}", entry.path().display()).await
})
});
}
Notable changes:
Take in AsRef<Path> instead of a String and a generic closure instead of a trait object reference
Change the closure type to be FnMut as it's more permissive
The closure returns any type that is a future.
There's an inner implementation function that hides the ugly API required for recursive async functions.
The callback takes the DirEntry by value instead of by reference.
See also:
How to asynchronously explore a directory and its sub-directories?
How to using async fn callback in rust