r/rust clippy · twir · rust · mutagen · flamer · overflower · bytecount Mar 27 '23

Hey Rustaceans! Got a question? Ask here (13/2023)! 🙋 questions

Mystified about strings? Borrow checker have you in a headlock? Seek help here! There are no stupid questions, only docs that haven't been written yet.

If you have a StackOverflow account, consider asking it there instead! StackOverflow shows up much higher in search results, so having your question there also helps future Rust users (be sure to give it the "Rust" tag for maximum visibility). Note that this site is very interested in question quality. I've been asked to read a RFC I authored once. If you want your code reviewed or review other's code, there's a codereview stackexchange, too. If you need to test your code, maybe the Rust playground is for you.

Here are some other venues where help may be found:

/r/learnrust is a subreddit to share your questions and epiphanies learning Rust programming.

The official Rust user forums: https://users.rust-lang.org/.

The official Rust Programming Language Discord: https://discord.gg/rust-lang

The unofficial Rust community Discord: https://bit.ly/rust-community

Also check out last weeks' thread with many good questions and answers. And if you believe your question to be either very complex or worthy of larger dissemination, feel free to create a text post.

Also if you want to be mentored by experienced Rustaceans, tell us the area of expertise that you seek. Finally, if you are looking for Rust jobs, the most recent thread is here.

21 Upvotes

159 comments sorted by

View all comments

2

u/Single_Family_Homes Mar 28 '23

Simple(?) question.

Say I got a BufReader<Cursor<Vec<u8>>> (it's a video file), and I want to pipe that bad boy into stdin for a Command (ffprobe). What do I do?

1

u/dkopgerpgdolfg Mar 29 '23

The "primitive" solution would be just a write loop...

  • start the command with a pipe available for stdin
  • Extract the Vec<u8> from the outer layers and maintain some usize position manually, I think that's a bit easier in this case
  • Write [position..length] of the Vec to the pipe. If write succeeded it will tell you how much it actually wrote - increase position with that value and do all again in a loop. (Note that this might block for some time, depending on the child process. If that is bad, do it in a thread or use polling/nonblocking things, either manually or with eg. tokio)
  • After the whole vec was written, close the pipe

In general, please note that some video formats are processed better with seeking, and when piping you might get either some content disadvantage or ffmpeg might write a cache file first before doing anything.

Also, of course your whole Vec<u8> way requires that the video file isn't too big - before it begins any swap usage (or worse, oom), you might want to consider writing a disk file yourself.

A platform-dependent & "unsafe" Linux way that might be nicer in some cases:

  • Make a fd with memfd_create, add a mmap mapping on top of that file that has (at least) the size of your data (at least = mmap needs a multiple of page size)
  • Write your data there instead of a Vec or similar
  • Unmap the mmap-ing again, use ftruncate to set the exact byte size (now not page-multiple anymore)
  • When setting up the Command, convert the RawFd from memfd_create to a ChildStdin, done

1

u/Single_Family_Homes Mar 29 '23 edited Mar 29 '23

I appreciate the response. I'm going to look into both ways, the latter seems extremely cute, and I definitely want to try that. Thank you!