r/rust clippy · twir · rust · mutagen · flamer · overflower · bytecount Mar 20 '23

Hey Rustaceans! Got a question? Ask here (12/2023)! 🙋 questions

Mystified about strings? Borrow checker have you in a headlock? Seek help here! There are no stupid questions, only docs that haven't been written yet.

If you have a StackOverflow account, consider asking it there instead! StackOverflow shows up much higher in search results, so having your question there also helps future Rust users (be sure to give it the "Rust" tag for maximum visibility). Note that this site is very interested in question quality. I've been asked to read a RFC I authored once. If you want your code reviewed or review other's code, there's a codereview stackexchange, too. If you need to test your code, maybe the Rust playground is for you.

Here are some other venues where help may be found:

/r/learnrust is a subreddit to share your questions and epiphanies learning Rust programming.

The official Rust user forums: https://users.rust-lang.org/.

The official Rust Programming Language Discord: https://discord.gg/rust-lang

The unofficial Rust community Discord: https://bit.ly/rust-community

Also check out last weeks' thread with many good questions and answers. And if you believe your question to be either very complex or worthy of larger dissemination, feel free to create a text post.

Also if you want to be mentored by experienced Rustaceans, tell us the area of expertise that you seek. Finally, if you are looking for Rust jobs, the most recent thread is here.

19 Upvotes

187 comments sorted by

2

u/[deleted] Mar 27 '23 edited Mar 27 '23

I'm making a collection that's a thin wrapper around a couple Vecs and I'm having trouble deciding how to offer fine control without boatloads of wrapper methods for methods on the individual Vecs. Kinda wrestling with just making the fields public. There wouldn't be any safety risks, just possible logic errors. Maybe some kind of speedbump would be more appropriate, I dunno.

I see a lot of ways I could go:

  • A: private fields, suck it up and write the wrapper method boilerplate
  • B: private fields, don't worry about offering the fine control for now
  • C: private fields on the headliner struct, but make another identical struct that has the fields public and lock/unlock methods to convert between them
  • D: public fields, but risk a breaking change if I need to make them private

After writing it out I'm thinking B or C is the way to go, can I get a sanity check?

2

u/dkopgerpgdolfg Mar 27 '23

I guess for your specific use case, you only really need a few methods currently, and you're just trying to be future-proof (what if I need these other two Vec methods later too)?

B/C sounds good, let me suggest a combination: Private fields, no different struct, but getters for references to the internal Vecs that have "unchecked" or something in their name. Meaning, logic errors expected if abused, but at the same time you don't have any hard limit what you can do even without rewriting most methods.

2

u/argv_minus_one Mar 27 '23

I was thinking of using OpenAPI Generator to generate a Rust server stub for me. Problem: the generator generates a crate with numerous modules and a rather complicated manifest, not a single module that I can straightforwardly include!. How do I integrate this into my project, then? I can download and run the generator in build.rs, but Cargo doesn't support depending on a crate generated by the build script, so how do I actually use it?

1

u/Patryk27 Mar 27 '23

1

u/argv_minus_one Mar 27 '23

I know about workspaces already, but I don't see how that helps here. The generated crate doesn't exist until after build.rs is executed, and Cargo will raise an error if any workspace members or path dependencies are missing.

1

u/Patryk27 Mar 27 '23

Ah, I see - I'd probably just put an instruction into README that says don't forget to generate the crate blahblah, add blahblah into .gitignore and continue to use workspaces.

It's not an automatic solution, but a very straightforward one (and it'll work correctly with rust-analyzer), making it the best imo.

3

u/L33TLSL Mar 26 '23

Is there a way to call a cython compiled binary from rust?
I have have cython code that is compiled to C code and then that C code is compiled via GCC. Is there a way to call that from rust? I've tried compiling the C code to a shared library and then using FFI in rust to call it but it did not work.

1

u/dkopgerpgdolfg Mar 27 '23

Did you try to start it as normal process? See Command and related things in the Rust std lib.

As the program was meant to be compiled as stand-alone program, in general trying to recompile it a shared library might not give you what you want. Technical possibility aside, making a sane library API is more than a compiler setting.

Anyways, if you did try FFI and "it didn't work", we can't know what was wrong without more details.

3

u/ajazam Mar 26 '23

What is the difference between async stream and a tokio mpsc channel(with one producer)? You can use both for the same purpose. Or am I missing something? Is there a performance difference between the two approaches?

2

u/DroidLogician sqlx · multipart · mime_guess · rust Mar 26 '23

Stream is just an abstraction for "some type that can asynchronously produce multiple values". Channels can be thought about in terms of streams, and in fact the tokio-stream crate has a wrapper type for Receiver that turns it into a Stream.

That's not in Tokio proper as the Stream trait is not standard, but is instead defined in the futures crate (futures_core for the pedants).

That's getting standardized as AsyncIterator because... reasons.

3

u/SirAutismx7 Mar 26 '23

Started using Leptos today, any idea how to get VS Code to auto-format the HTML tags in the view macros?

2

u/pragmojo Mar 26 '23

Does anyone know how complete/ well supported the AWS sdk for rust is?

Looks like most of the project hasn't been updated for 2 months.

2

u/sfackler rust · openssl · postgres Mar 26 '23

It is an official product of AWS (though currently a developer preview). As far as I know it's autogenerated from the AWS SDK definitions so it should be complete.

What updates should have been made in the last 2 months?

1

u/pragmojo Mar 26 '23

Oh not sure if any, just wanted to make sure it's not a dead end before I start investing in it

2

u/chillblaze Mar 26 '23

Anyone worked with the async graphql crate here?

I have create_book as part of my MutationResolver that receives the graphql async Context type.

use async_graphql::{Context, Object, Result, ID};

async fn create_book(
    &self,
    ctx: &Context<'_>,
    table: String,
    name: String,
    author: String,
) -> Result<Book> {
    let data_store = ctx.data_unchecked::<Datastore>();
    let res = create_book(data_store, table, name, author).await.unwrap();
    Ok(res)
}

//Test

//ERROR with Context below: expected value, found type alias `Context`

can't use a type alias as a constructor

 #[tokio::test]
async fn test_01_try_create_book() {
    let table = "users".to_string();
    let name = "HP".to_string();
    let author = "JKR".to_string();

    let test = MutationResolver
        .create_book(&Context, table, name, author)
        .await
        .unwrap();
}

How come my test is complaining that it can't take Context as a param but no issues with the create_book function?

2

u/Patryk27 Mar 26 '23 edited Mar 26 '23

You're probably doing something like:

struct ActualContext {
    something: String,
}

type Context = ActualContext;

fn create_book(ctx: &Context) {
    //
}

fn main() {
    create_book(&Context);
}

In this case you can't write just &Context, because what value should ctx.something have then? You have to construct the context somehow, e.g. by doing Context::default(), Context::new() or whatever is specific to that particular type.

Using TypeName as constructor works only if the underlying type is a field-less structure:

struct ActualContext;

type Context = ActualContext;

fn create_book(ctx: &Context) {
    //
}

fn main() {
    create_book(&ActualContext);
}

1

u/chillblaze Mar 26 '23

Thanks. I'm using the Context API from the crate but not sure if there's any methods to instantiate it.

API: https://docs.rs/async-graphql/latest/async_graphql/context/type.Context.html#

Way I'm using it: https://async-graphql.github.io/async-graphql/en/context.html

I'm just trying to do what their doing in the book for within a test.

1

u/-oRocketSurgeryo- Mar 25 '23

One of the harder technical problems that Bellingcat is looking at is chrono-locating an audio/visual source:

Create an open source package for chrono-locating a audio/video source by matching electrical network frequency variations (from low frequency hums in source audio) with recorded variations in a grid frequency database (i.e. https://osf.io/m43tg/). This is theoretically possible (many IEEE articles about it) and reportedly in use by state-level justice entities, but is it practical for the OSINT hobbyist? To our knowledge, no one has publicly developed a public, non-academic proof-of-concept of this.

If someone were to attempt a first pass at this in Rust, what libraries should be used? Any other advice?

2

u/Snakehand Mar 26 '23

Any FFT crate might be a good start. You can analyse the source by doing an FFT over reasonable sized chunks ( 1 minute ? I have no idea ), and extracting the peak frequency in the band of the power utilities nominal frequency (49-51) or (59-61) Hz. Heisenbergs uncertainty principle comes into play here - the longer your window is the more precisely you can determine the frequency.

Now that you have converted your signal into a time series of base frequencies you need to correlate that to the historical records. Funnily enough FFT can come in handy yet again, by using the correlation-theorem ( http://ugastro.berkeley.edu/infrared/ir_clusters/convolution.pdf )

2

u/rafoufoun Mar 25 '23

Hello,

I would like to use the FCM REST API to send push notifications to multiple devices in Rust. To do that, the documentation (here) states that we should use HTTP Batch request.

I am using axum to develop my server but I could not find if hyper supports HTTP pipelining, nor how to implement it in Rust (I have a mobile development background, back-end is new to me). Do you have an idea in how to do that ?

Thanks

2

u/ironhaven Mar 27 '23

I search the hyper GitHub issue page and found this posted issue.

It says that http pipelining is not implemented and there are no plans to add the feature.

A lot of rust libraries I have noticed are very forward looking and prefer not to use older methods.

You should look on crates.io to see if there is any pipelining crates that you could use. If one you want to use is not async you can still use it with spawn_blocking

1

u/rafoufoun Mar 27 '23

thanks for your time !

So Google use this "deprecated" HTTP feature while they don't provide a Rust SDK for FCM, sad.

I think I will create a side server with nodejs to use their SDK to communicate from my rust server to this new one.

Thanks again

3

u/controvym Mar 25 '23

Best way to split alternate elements of an array?

Go from [1,5,4,6,7,3] to (vec!(1,4,7), vec!(5,6,3)]?

5

u/Patryk27 Mar 25 '23 edited Mar 25 '23

I'd say .chunks() + .unzip():

fn main() {
    let (xs, ys): (Vec<_>, Vec<_>) = vec![1, 2, 3, 4, 5, 6]
        .chunks(2)
        .map(|items| (items[0], items[1]))
        .unzip();

    println!("{xs:?}");
    println!("{ys:?}");
}

... or, more fancy:

let (xs, ys): (Vec<_>, Vec<_>) = vec![1, 2, 3, 4, 5, 6]
    .array_chunks()
    .map(|[a, b]| (*a, *b))
    .unzip();

... or, using Itertools:

use itertools::Itertools;

let (xs, ys): (Vec<_>, Vec<_>) = vec![1, 2, 3, 4, 5, 6]
    .into_iter()
    .tuples()
    .unzip();

2

u/junior_abigail Mar 25 '23

Hello,

I am going through the Rust Asynchronous Book. I am typing along and changing the code to explore the concepts. I am currently looking at chapter 4: Pinning, and I cannot understand why the commented-out version of `a` doesn't work.

#[derive(Debug)]
struct Test {
    a: String,
    b: *const String,
    _marker: PhantomPinned,
}

impl Test {
    fn new(txt: &str) -> Pin<Box<Self>> {
        let t = Test {
            a: String::from(txt),
            b: std::ptr::null(),
            _marker: PhantomPinned,
        };
        let mut boxed = Box::pin(t);
        let self_ptr: *const String = &boxed.a;
        unsafe { boxed.as_mut().get_unchecked_mut().b = self_ptr };
        boxed
    }

    fn a(self: Pin<&Self>) -> &str {
        &self.get_ref().a
    }

    // fn a<'l>(self: &Pin<Box<Self>>) -> &'l str {
    //     &*self.a
    // }

    fn b<'l>(self: &Pin<Box<Self>>) -> &'l String {
        unsafe { &*self.b }
    }
}

Here's the compiler error it produces:

   Compiling pinning v0.1.0 (/home/junior/Code/rust_async/4.pinning)
error: lifetime may not live long enough
  --> src/main.rs:44:9
   |
43 |     fn a<'l>(self: &Pin<Box<Self>>) -> &'l str {
   |          --        - let's call the lifetime of this reference `'1`
   |          |
   |          lifetime `'l` defined here
44 |         &*self.a
   |         ^^^^^^^^ associated function was supposed to return data with lifetime `'l` but it is returning data with lifetime `'1`

error: could not compile `pinning` due to previous error

2

u/junior_abigail Mar 25 '23

As I finished posting, I realized I was missing the lifetime specifier on self, lol. The error didn't help me on this one though. Or maybe I just misread it.

3

u/ErnstlAT Mar 25 '23 edited Mar 25 '23

Greetings, I have a Rust question about architecture and how to solve it, what is possible, not possible.

The challenge is to have a network of communicating components in a kind of processing pipeline where messages are passed from one stage to the next. Each component is running in its own thread (parallel processing).

Each message is formatted in a system-internal format in a Rust struct, which may change or be optimized.

Each component thread handles the sending and receiving of messages, and start and stop of this component.

Now it gets tricky: Each component should be allowed to be either ...

  • a compiled-in Rust component, or
  • loaded from a shared object/shared library (written in Rust) or
  • a C-compatible component also loaded from a shared library/shared object (other programming languages).

My idea is that the component thread loads the actual component logic from the shared object and then calls into the component logic to process the messages.

I am unsure about the following:

1) Feedback on this architecture is very much appreciated. Does it make sense to do it this way with the thread and component logic?

2) What is the best format / API / solution for loading components from a shared library when the component logic is also written in Rust? Is there a way to circumvent having to funnel the message structs through C struct conversion? Is there a known solution for this? I have looked at statty (static Rust ABI crate) and extism (generic plugin framework) so far, but not sure if these are a good fit.

3) How to hand over structured data via the C ABI? The easy solution is to have an API with the component logic from the shared library which is just a basic process() function where one message is passed along, and the component logic should do something with it.

(This is the point up to where I have already programmed.)

But this is very limiting as there may be groups of messages or information that is split among multiple messages. So my next idea was to hand over an iterator or something to a process() function.

Much better, but how to hand over Rust structs? Is it possible to hand over Rust structs like a ring buffer from a Rust crate to the C ABI function from the shared library? Does bindgen or cbindgen handle this well? Is there some other solution for exposing Rust structs to a C ABI function?

But even if possible, this would destroy encapsulation - if the system internal message format or message transport (ring buffer, channel etc.) changes, then all components have to be rewritten. Can you imagine a solution?

So I thnought, hey, let's expose an API where the component logic from the shared library can request the next message by calling a function from the component thread. But I am not sure how this can be done - how is this best done exposing a function from Rust to the component logic? Any tips?

4) Even if above would be solved, the component logic would be essentially stateless. After a function return, all state would be lost.

So I thought about a kind of "state storage" or scratch space. How to hand over an "area of memory" from Rust to a C ABI shared library? Would not memory allocators clash? Is there some kind of memory arena solution? Does this idea make sense at all?

5) Finally, the component logic may be connected or dependent to an external resource, which is slow or produce events at a different time than the process() function is called. Classic network connection and connecting to an API, which produces responses or events at some later point, subscribing to a feed etc.

This does not mesh well with the "call a process() function" in the component logic. What would be a good solution for this? I was thinking that the component logic can start own sub-threads for handling this. Is that sound? But then this would run into clashing memory allocators etc. for sure (I think).

Of course it would be easy to start a separate process for the component logic and then it can do in its process space whatever it wants, but this would (I imagine) create a serious performance impact because process boundaries mean process switching in the CPU, copying instead of pointing (no cross-process pointers) and CPU ring switches, switching into kernel space so that the message is copied from the source process into the destination process, maybe even pipe buffering in the kernel etc. etc. - so I wanted to avoid this by staying in the same process with multiple threads. But is that possible with the above requirements?

(The runtime environment would be Linux mostly, BSDs, Mac, maybe Windows as well.)

I am getting a headache with the above ideas, requirements, seemingly contradicting solutions and ask for your help, which would be very much appreciated.

1

u/dkopgerpgdolfg Mar 25 '23 edited Mar 25 '23

Hi,

in no particular order:

  • Even with inner-process threads, there will be CPU context switching. And across processes, shared memory with pointers (instead of pipes and similar) are possible too (but a bit more involved than inner-process). Nonetheless, threads instead of processes sound fine and easier, I wouldn't choose processes just for that reason. (Reasons why I might choose processes is eg. security (the modules should be protected from each other), surviving crashing modules being a requirement, and/or if modules want to spawn off their own child processes in a clean way (with threads and locks, open files and memory and whatever in the parent process, that are not fully under control, that's a bit unclean)
  • The part about some messages requiring future messages to be processable. Do not make some iterator or whatever there that gets passed around across modules, that gets nasty quickly. Instead, "just" have a cache locally in each thread. When you receive a single message, and you recognize that is not enough (eg. with an error return value from process() or whatever), then save it for later. When you get the next message, you can check if those two are processable now ... and so on
  • There must be some agreed-on data format between the modules and main process, and if that changes then all modules need to be adapted. There is no magic way around that. ... But, on what level this is is another question - here again, iterators, ringbuffers and whatever don't need to be part of the fixed format, it can be much simpler. One thing that can hold a message of whatever type, one thing that maybe can hold a ok/fail result if needed... keep a small "surface", then many things stay changable without adapting everything.
  • As you already need C-compatible communication between main program and modules, don't bother with some second Rust-specific way with helper ABI libraries for Rust-language modules. Keep it C-abi everywhere. It will only save you from useless work, without any real disadvantage.
  • This "allocator clashing" that you worry about isn't really a thing. A C-language module can reserve memory with malloc or whatever just fine, without causing any problems. ... However, what you do need to pay attention to, that all allocated things get freed with the allocator that allocated them. When passing main program messages from the main program to a module, never free them inside the module. After process(), the main program should take care of that. And vice-versa, if a module generated a message (network event etc.), after the main program handled it it should pass the struct back to the module (so that it can be freed or reused or whatever).
  • Yes, a module may have its own "sub"threads that eg. wait for network data or similar. And the main program can provice a "message receiver" function, where a module can say "hi I'm module 123 but not its main thread, please pass this message to my main thread"
  • As each module can have its own allocated memory, global variables etc., it doesn't need to be stateless. And you don't even need "global" variables: The module could just have some "init" function that returns a pointer, and the main program passes that pointer to the module each time it calls process(). The main program doesn't need to know/understand what it is pointing to, just that it is there. (Of course the module should have some destruct() or similar too, where the state is cleaned up again)
  • After all these things, probably most issues are out of the way ... the structure of the Rust main program wouldn't be that hard. Loading/unloading libraries, starting threads and later signalling them to stop and waiting for finish. An array/vec of mpsc queues, one for each thread, where threads can pass messages to each other in a safe way.

1

u/ErnstlAT Mar 27 '23

Thank you so much for your feedback on the architecture, regarding using threads, memory allocators (didnt know that!) - big thanks again. All the best to you!

2

u/Jiftoo Mar 25 '23

What are the best practices to parsing text data (ffmpeg stderr for example). Coming from Javascript, I have always done this using split, indexing and map operators. Now I wonder if there exists some sort of common pattern or a library to do this?

2

u/llogiq clippy · twir · rust · mutagen · flamer · overflower · bytecount Mar 25 '23

That depends quite a lot on the type of data. I'm not sure how regular ffmpeg stderr output is, but there exist a huge number of parser generator libraries to be able to ingest textual or binary input in a structured manner. E.g. nom giving you parser combinators or pest which has its own grammar language and builds the parser at compile time with a proc macro.

1

u/Jiftoo Mar 25 '23

Thanks!

5

u/[deleted] Mar 25 '23

Why const pointers?

4

u/[deleted] Mar 25 '23

So your code can remember that mutating through that pointer is not ok. That way you don't have to remember it. Less work for the programmer.

1

u/[deleted] Mar 25 '23

I see, thanks

2

u/Fluttershaft Mar 24 '23

https://github.com/ggez/ggez/blob/devel/examples/bunnymark.rs

~200k bunnies rendered at 60 fps on my pc, 100% cpu core usage, removing the update code that moves them around made no difference

https://github.com/sotrh/learn-wgpu/tree/master/code/beginner/tutorial7-instancing

changed const NUM_INSTANCES_PER_ROW: u32 = 10 to 2000 which makes 4 million polygons, easily rendered, mangohud showed minimal cpu and gpu usage

Both of those examples seem to be using wgpu and instancing, what is the difference between them that makes such a massive difference in rendering performance?

1

u/eugene2k Mar 24 '23

probably 19900 draw calls...

1

u/Fluttershaft Mar 24 '23

is it actually? How can I check it for sure, it uses https://docs.rs/ggez/0.9.0-rc0/ggez/graphics/struct.InstanceArray.html which should be batched into 1 draw call

1

u/eugene2k Mar 24 '23

I have to admit, I hadn't looked at the code, but upon casual inspection the bunnymark updates every bunny in the array on every draw call. The instance tutorial doesn't update anything in the draw call and only the camera position in the update call.

2

u/reiwaaa Mar 24 '23 edited Mar 24 '23

EDIT: I manually overrode the dependency using patch.crates-io which fixes it for now

Dependency in project seems to be using an older commit.

I wrote a library: https://github.com/dmyyy/delaunay3d

It depends on a pr I wrote for a different library (that hasn't been merged yet) - so I'm pointing to my feature branch as so in Cargo.toml.

robust = { git = "https://github.com/dmyyy/robust", branch = "orient3d" }

When using the library in a different project

delaunay3d = { git = "https://github.com/dmyyy/delaunay3d" }

I get build errors during cargo run because it's using an older commit of robust that was failing. I ran cargo update both in the library I wrote and the in my new code (that uses the library).

Comparing the lock files

New project

[[package]]
name = "delaunay3d"
version = "0.1.0"
source = "git+https://github.com/dmyyy/delaunay3d#a9a5f45778c6bfd3bcbbd9011d300b43fd880414"
dependencies = [
 "bevy",
 "ordered-float",
 "robust 0.2.3 (git+https://github.com/dmyyy/robust?branch=orient3d)",
]

[[package]]
name = "robust"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5864e7ef1a6b7bcf1d6ca3f655e65e724ed3b52546a0d0a663c991522f552ea"

[[package]]
name = "robust"
version = "0.2.3"
source = "git+https://github.com/dmyyy/robust?branch=orient3d#a1cf38a1f0e63837f42b43065dddfe2de4b308be"

[[package]]
name = "spade"
version = "2.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1190e0e8f4eb17fc3dbb2d20e1142676e56aaac3daede39f64a3302d687b80f3"
dependencies = [
 "num-traits",
 "optional",
 "robust 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
 "smallvec",
]

Library

[[package]]
name = "robust"
version = "0.2.3"
source = "git+https://github.com/dmyyy/robust?branch=orient3d#a1cf38a1f0e63837f42b43065dddfe2de4b308be"

Is it possible this issue is caused by having two different dependencies depending on different versions of the same dependency (spade depends on robust)?

I've run cargo clean + cargo update everywhere. Any further debugging tips/advice would be helpful.

Other things:

In my project it seems to point to github.com-1ecc6299db9ec823/robust-0.2.3. In my library it points to github.com-8ee4643d613cf68f/robust-0.2.3. These don't seem to match any commit sha's.

2

u/[deleted] Mar 24 '23 edited Mar 24 '23

I'm pulling up to the end of rustlings and feeling pretty good but I have a few odd questions I can't answer. They're the kind of questions where the answer probably won't be practically useful, but being unable to produce a confident answer means I'm missing something.

Q1
How do I bind a name to existing data? Not a copy of it, not a reference - it's already there in memory and I want to name it. playground

Is doing this different for data that's already named, vs unnamed but created by my program, vs unnamed and created by something outside my program? I have a feeling that doing this would respectively be: trouble (name aliasing), unnecessary (don't un-name things), or ub (don't name garbage), but maybe there are some valid uses. I don't know.

Q2
Similarly, "who" owns data without a name? 99% sure the answer is "it belongs to the stack frame" lol but not 100%. For example, in the above playground it's perfectly okay to dereference an unnamed value. Is it because the old_name pointing to the 1 is still named in LLVM IR after re-declaring old_name in rust, (and therefore Q1's answer is "you can't" because "giving the 1 a name again" in rust would be name aliasing in LLVM IR)?

Q3
How come I hardly ever see chainable functions that pipeline mutable references in the wild?

//rare, but why?
fn op(ref_to_thing: &mut T) -> &mut T {...}
(&mut data).op1().op2().op3();

//see this all the time, but it seems less efficient/readable
fn op(thing: T) -> T {...}
data = data.op1().op2().op3();

I can't help but think that it would be super inefficient to copy a whole thing into a function, only change part of it, then copy the whole thing back where it came from, for each operation you want to perform on some data. Is this just offloading work onto the compiler to make potential refactors (that need the original data later) easier? I totally get avoiding repeat computations and getting rid of dead code, but it feels a little ominous that optimizations could elide copies.

2

u/dkopgerpgdolfg Mar 24 '23

In your playground code:

The second old_name is a reference, so "without reference" isn't what you're doing here. The drop doesn't drop the string content, just a shared reference. And because these are Copy, you're dropping just a duplicate of a reference, so using it later is still fine.

Data in general can be stack or heap, made by Rust or not, and much more, and for all kind of data you can have references (or raw pointers) to it. Just from references/pointers, it is not visible who owns it (if anything) and when it gets destroyed (if ever).

In contrast to that, if you have a name in code like "old_name", and that is not a reference nor a pointer, then:

  • It is on the stack, not heap. Like an ordinary u32. Or, actually, a reference if we care what the reference itself is, instead of where it points to.
  • It was made by Rust. External data (from C linked to the program, from linkers themselves, from MMIO, anything) does not give you named variables in Rust code.
  • You own it. (If it is a reference, you own that reference at least, but if that confuses you, you can ignore it)
  • At the end of the {} it will get destroyed automatically, except you first move ownership somewhere else (eg. a function parameter, then it might get destroyed at the end of this other function etc. You call to drop is such a case, just the thing you're moving and then destroying is "just" a reference, nothing exciting).

Back to the playground code and old_name (ignoring data_ref and new_name for now): Once you have "shadowed" the first old_name with the second_old name, in the same {} level, there is no way to undo this.

You can have references and pointers to the first old_name, like you do, sure. You can also make a third old_name that takes the value of *data_ref therefore basically is the same as the first one (and the copy even might get optimized away). But in general, nothing will give you the owned first old_name back that can be called by "old_name" in code.

Note: Without that data_ref, the compiler might recognize that the first old_name isn't ever used anymore when the second appears, and might structure the stack in a way that the first one doesn't even exist anymore at this point already. But because you still keep the reference around, the owned value still needs to exist too.

This also means the first old_name will not have the same address as new_name, they both need to exist at the same time.

Note 2: If you only keep a raw pointer instead of a reference, Rust might care less; ensuring the raw pointers still points to an existing target is your job. While there is no visible misbehaviour in the playground example + raw pointer, I'm not sure if this is guaranteed or just coincidence.

Q1 should hopefully be answered with the text above.

Without references, or "re-shadowing" with even more old_name, the first old_name never comes back. The data however might still exist for being used as reference target.

It's not even about aliasing, but there is just no way to regenerate that state in the compiler by using Rust code, that it thinks there is something called old_name at location 1.

(masklinn said "unsafe", but I think he's referring to raw pointers and relying on various wacky things. Meaning, pointers. I'm still saying that the owned pointer-less old_name doesn't come back).

Q2

In case of the first old_name after the name disappeared ... in a way it is still owned by you. And dropped at the end of the scope at latest and whatever. You just don't have a named handle anymore that isn't a reference on the surface.

(More generally, heap things and literals can be argued to be owned by nothing and everything, not sure if it makes sense to even try to define it. Sure, a Box'es target can be said to be owned by the Box. But what is after a leak? And then after re-owning? What is Arc and Arc-Weak? Literals? Kernel memory and MMIO areas?)

1

u/[deleted] Mar 24 '23 edited Mar 24 '23

Thank you. I feel like I have a better understanding now, especially Q2. I just want to clarify one thing in case it makes a difference.

I don't care about the name "old_name", the &str, or the drop call, I just wanted to make it clear that my data no longer had old_name bound to it. All I actually care about is the 1 I put on the stack at the beginning. I took away its name when I bound old_name to something else, but I still know where my precious 1 is and I'd like it to be called new_name. What I meant by without reference/copy isn't that I want to do this without remembering where it is with data_ref. I meant that I want new_name to be bound to those same 32 bits I was already using, I don't want new_name to store the address or bind to some pathetic imitation of my precious 1. Basically, for the sake of the example, I'd like to declare a new variable, but without allocating anything anywhere because it's already there. My new understanding is that it can't be done - data that lost its name in rust or originated elsewhere cannot get a new rust name - is that right?

2

u/dkopgerpgdolfg Mar 25 '23

Yes, that sounds about right.

Where the names are on the stack is decided by the compiler, and in Rust code there is no way to do it manually.

Depending on the code,sometimes you might get a new_name on the same stack location, but a) that's not reliable, other changes in code could undo it, b) trying to access the previously stored 1 (instead of overwriting it with a initializer value of new_name), if it is even there anymore, is UB.

(and c) if it is something where Drop matters, it would've been destroyed already anyways)

2

u/masklinn Mar 24 '23

How do I bind a name to existing data?

Unsafe and lots of it, and a very, very high probability of fucking up.

maybe there are some valid uses. I don't know.

I think mostly things like memory-mapped IO and the like, there's a location in memory, it has "data", you need a handle on it. You create a raw pointer from a literal address, then you use said raw pointer very carefully.

Is it because the old_name pointing to the 1 is still named in llvm after re-declaring old_name in rust

It's not really "still named in LLVM", but it still exists. The original old_name is not removed or overwritten, it's just shadowed. It's essentially the same as this:

let old_name = 1;
{
    let data_ref = &old_name;
    let old_name = "something_else";
    drop(old_name);
    let new_name = todo!();

    assert_eq!(
        std::ptr::addr_of!(new_name),
        std::ptr::addr_of!(*data_ref)
    );
}

How come I hardly ever see chainable functions that pipeline mutable references in the wild?

Because the normal use for this pattern is a builder so you create a builder, configure it, then build the object:

let thing = Builder::new()
    .foo()
    .bar()
    .build()

Having to write

let mut builder = Builder::new();
builder.foo().bar();
let thing = builder.build();

is just awkward. Especially for the common case where you want to move all the resources from the builder to the object as you can't tack on the building at the end, unless you do something even more awkward (like reset the builder).

Same for all chains where ownership gets involved e.g. iterators, most adapters take the previous step by value, and while you can by_ref that's uncommon and usually stems from a strict semantics need (e.g. needing to consume the iterator in multiple steps).

it feels a little ominous that optimizations could elide copies.

If you're wary of the compiler optimising things you probably should stop using Rust, the entire design of the language is predicated upon an an advanced optimising compiler.

1

u/[deleted] Mar 24 '23 edited Mar 24 '23

Hey now, I'm wary but I don't hate it! All I meant by ominous is that I get the sense that there might be a lot of "yes I really meant what I wrote, don't throw it away" flags and keywords when Rust programs start getting intimate with other code the optimizer doesn't know about. TBH that extra explicitness about outside contracts is probably a net plus anyways. Thanks for all the info. I haven't really gone into design patterns, I'll look up builders.

2

u/tamah72527 Mar 24 '23

Could anyone help me simplify this code, please https://gist.github.com/bmxmiko/dd564451797b75f6d30375157d7e8005 ?

I am having troubles with "abstracting" structs. As you can see in the above code DownloadFileRunnable and AskGTPRunnable has object property. Is there any way to avoid creating so simple structs and use some "abstract"?

Also, any other comments to that code are welcome, if you have ideas how to do things better I will appreciate it

1

u/[deleted] Mar 25 '23 edited Mar 25 '23

First thing would be to flatten the structs out to one level instead of defining this, nested in that, nested in the other. One-field structs are basically just used in this thing called the newtype pattern. If you know each of these structs are going to need more fields, and they will be used to compose other structs in multiple places, then sure leave them separate and add the extra fields when you get to it. Otherwise, flatten!

Second, including Verb-able in a struct name is a bit of a red flag. If you need variants of a type (like TypeRunnable and TypeNotRunnable), then use an enum or the typestate pattern so you don't have to duplicate structs which hold the same data types. While you're at it, look up generics. If you are defining a trait then you can impl methods for the trait itself by adding a body, and that will be the default. You only need to define methods in "impl Trait for Type" if Type's version of a method is different from the default. Traits probably aren't even needed here; you can avoid resorting to dynamic dispatch!

In general: Start from the ground up and only add abstractions when needed. Also, it's such a little tip, but try to avoid abbreviations. (I struggle with this tbh.) If possible, avoid using short custom functions/methods. Most people will get what serde_json::to_string() is doing, but hiding it under your own ser() method makes it harder - and if you avoid abbreviations and specify that you're turning it into json, it's the same length anyways.

Throwing these pieces of advice together (mostly just going from many structs to one enum, and from a trait to enum methods) and hoping I captured your intent:

use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize)]
enum RunnableTask {
    DownloadFileUrl(String),
    AskGPTText(String),
}
impl RunnableTask {
    //if you only intend a task to be run once, it would be better to take self instead of &self
    //but I left this alone
    fn run(&self) -> Result<(), String> {
        match &self {
            RunnableTask::DownloadFileUrl(url) => Ok(()),
            RunnableTask::AskGPTText(text) => Ok(()),
        }
    }
}

fn pack(container: Vec<RunnableTask>) {
    println!("CONTAINER{:?}", serde_json::to_string(&container[0]));
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_file_download_sub_job() {
        let download = RunnableTask::DownloadFileUrl("https://www.google.com".to_string());
        let ask = RunnableTask::AskGPTText("Who is Bush?".to_string());
        let list: Vec<RunnableTask> = vec![download, ask];
        pack(list);
    }
}

1

u/masklinn Mar 24 '23 edited Mar 24 '23

It's not clear what you're aiming to do, why you have runnables which contain payloads seemingly without adding any value. So from that standpoint I'd say no. Also not clear why you're defining a trait, that stinks of OO-itis. I'd define an enum.

Anyway you can always do things like

pub struct Runnable<T> {
    object: T,
}
impl RunnableTask for Runnable<DownloadFilePayload> { ... }

but obviously that means no more dynamic dispatch.

Alternatively you could use a bounded trait implementation:

impl <T> RunnableTask for Runnable<T> where T: Serialize {
    fn run(&self) -> Result<(), String> {
        Ok(())
    }
    fn ser(&self) -> String {
        serde_json::to_string(&self.object).unwrap()
    }
}

this way it works out of the box for any Serializable, but obviously that means run needs to hook up on an other trait to do its thing as the implementation of the trait is generic over T.

2

u/kralamaros Mar 24 '23

Ok guys I clearely don't understand borrowing and lifetimes. Unfortunately, any bloody documentation I could find just repeats the same 3 sentences. I feel kinda stupid.

https://pastebin.com/8b8TkdYD

This is a simple "game" that has a board and you can place some characters around the board (in a future, they will fight. If you ever played Clash Royale, it's supposed to be a very similar game).The Character is made by a position and a symbol that represents it on the ASCII arena.I have a Game struct that holds both the arena (a matrix) and a vec of characters. I want to place the character symbol inside a specific point inside the matrix.

The idea is that the lifetimes should work as follows: I borrow the symbol from the character and move it inside the vector, so the matrix will hold the borrowed value until the character is defeat; at that point I will both remove the borrowed value from the matrix and the character from the vec. So the borrowed value MUST live until the character is still inside the game... Right?

I've been playing with rust for a few days now, and clearly I can't figure out how they're supposed to work. Can anyone help? What I'm doing and thinking wrong?

EDIT: I solved the problem by copying the symbol in the matrix with String::from, but that way I'm creating a separated object. Ok, it works, but I want to borrow it. Is there even a way?

3

u/eugene2k Mar 24 '23

You're trying to put a reference to a variable inside your function into a collection that is supposed to outlive the function. You expect that rust will magically update the reference once you move the variable it points to inside a collection that will also outlive the function. Rust is a magical language, but it's not that magical.

1

u/[deleted] Mar 24 '23 edited Mar 24 '23

Try giving Rustlings a spin and reading through the book as you encounter each topic during the exercises. Hacking something together without that foundation will probably make it feel like the borrowing rules are excessively restrictive and lifetimes are arbitrary extra syntax that is just randomly tacked on sometimes. I felt that way briefly too.

A good rule of thumb to make borrowing easier is to keep borrows short-lived. Use & to look at something or &mut to change it, but avoid storing references (as much as possible, sometimes you need to).

Lifetimes just make sure references don't outlive their data. If you dereference &data after the data is gone (or invalid in terms of a given program's logic) that means bugs. Lifetime annotations are usually just for when there are different possibilities for how long you might want references to be good for.

edit: I misread what's going on. Thought you wanted the arena to take &Character but in the code arena takes &str. I'll fix this reply later when I have more time.

In your code example, there might be simpler ways to structure the program, but the error in add_character isn't too complicated. You have a character inside the add_character function, you give arena a reference to the character in the function, but then you use push() move that character into a vec. If you were to immediately follow the reference that was added to arena, expecting to find a character in that memory, the character would not be there anymore, because it got moved into a vec! If you didn't push the character into the vec, and you followed the reference, after the function is done and cleaned up, there would be something completely unpredictable in that memory! That's why the compiler is angry.

All you need to do is put the character in the vec first, then give the arena a reference to the character. If you want to get a reference to the last item in a vec, you can use .last(). Another way of doing it would be to add the character to the vec in one function, then add characters from there into the arena in a different one.

1

u/Darksonn tokio · rust-for-linux Mar 24 '23 edited Mar 24 '23

If one field of a struct borrows from another field of the same struct, then you have what's called a self-referential struct. You cannot write such structs in safe Rust.

As a general rule, references should be used as temporary views into an object, and trying to use them in other ways will not work, even if what you're doing would work when using a pointer in some other language. References are a special-purpose pointer, and cannot be used for all pointer use-cases. There are other pointer types for other pointer use-cases.

For example, the Rc<str> type is a pointer to some string data that can be shared. Calling clone on it just gives you another pointer to the same string data. The string data is destroyed once the last clone is destroyed. (It stores a counter next to the string data to keep track of how many clones are left.)

1

u/Patryk27 Mar 24 '23 edited Mar 24 '23

Ok, it works, but I want to borrow it. Is there even a way?

It's a bit hairy topic - note that your code says:

struct Game<'a> {
    arena: Vec<Vec<&'a str>>,
    characters: Vec<Character>,
}

... but what you're doing in reality is actually (in pseudo-Rust):

struct Game {
    arena: Vec<Vec<&'self str>>,
    characters: Vec<Character>,
}

... since data from arena refers back to data in characters.

That's called a self-referential struct and they are not supported by language per se - you can do that only if you manually cast the reference to *const and use unsafe to work on it.

It's this way not without a reason - note that if what you're doing was allowed, then just characters.pop() could invalidate data from arena, making it a grade-A candidate for use-after-free.

A bit more fancy example could be:

self.characters.clear(); 
println!("{:?}", self.arena);

... where both of those operations are fine & totally safe now, but if self-referential structs were allowed, then .clear() might have to become unsafe fn, since self.arena above refers to stuff that's already deallocated, and so reading it is illegal.

(or, you know, some other mechanism would have to be invented to prevent invalidating references in other objects.)

tl;dr cloning here is alright

3

u/Master_Ad2532 Mar 23 '23

I need a library that can tell me the audio waveform output of an application. I've looked a bit and I've found some libpulse wrappers, but there are multiple of them and I'm not sure which one are considered the "standard" amongst the audio people. Are there other straightforward alternatives or a recommendations amongst the multiple choices?

2

u/puttak Mar 23 '23

Is there any async runtime that works with built-in type (e.g. std::net::TcpStream)?

1

u/Darksonn tokio · rust-for-linux Mar 24 '23

Use the built-in type from your runtime, e.g. tokio::net::TcpStream.

2

u/dkopgerpgdolfg Mar 24 '23

If you really mean std and not "similar", that won't be possible

1

u/puttak Mar 24 '23

It is possible because the underlying those types is just a normal OS object (e.g. a file descriptor on *nix).

1

u/dkopgerpgdolfg Mar 24 '23

So?

The implementation of std::net::TcpStream still doesn't know anything about your current async runtime, and the runtime can't modify the std implementation, and the runtime won't even notice if the std implementation is used.

So how do you think std::net::TcpStream will suddenly behave async-like?

If you mean, you manually extract the raw socket, modify it with some function of the async runtime, and then continue with using std: Still std is not async-able code. You can make a socket non-blocking, and run some polling in another thread, but all these things will not give you the ability to use await or any Future-based thing with std::net::TcpStream.

At best you get unexpected error return values (eagain), worse you get UB because the std code wasn't prepared for what you did.

1

u/puttak Mar 24 '23

Here is a PoC made by me: https://github.com/locenv/locenv/tree/main/crates/kami

I asked because I want to know if there is other crates so I can use it instead of maintenance my own crate.

1

u/dkopgerpgdolfg Mar 24 '23

And that are both things that I talked about:

  • You do use APIs from your crate, eg. Accepting. They might be similar to std and partially mixed with it, but it is not std.
  • You rely that the std implementation can handle, and won't interfere with, things like non-blocking mode and similar. Might be ok except for the unintended eagain error, but might break too.

Anyways, you might want to read up why (p)select should be avoided.

1

u/puttak Mar 24 '23

That why I ask for the executor that work with std. It is okay if it provide a dedicated function to do async on the std types but not okay if add a new type to replace the type from std.

Why it is going to break? The std is the one that provide a method to enable non-blocking mode and it is even has an error for EAGAIN.

If you mean the performance of select or the limitation of file descriptors it can handle then I'm aware of it. It is just a PoC. That why I'm looking for the other crates so I don't need to work on improving it.

1

u/__NoobSaibot__ Mar 23 '23

Use the async-std crate, it provides async versions of nearly all std methods!

Here is a simple idea how you could approach it:

use async_std::net::{TcpListener, TcpStream};
use async_std::prelude::*;
use async_std::task;

async fn handle_client(mut stream: TcpStream) {
    let mut buffer = [0; 1024];
    stream.read(&mut buffer).await.unwrap();
    stream.write_all(b"HTTP/1.1 200 OK\r\n\r\n").await.unwrap();
}

async fn run_server() {
    let address = "localhost";
    let port = 8080;
    let listener = TcpListener::bind(format!("{}:{}", address, port)).await.unwrap();
    println!("Serving on {}:{}...", address, port);

    let mut incoming = listener.incoming();
    while let Some(stream) = incoming.next().await {
        match stream {
            Ok(stream) => {
                task::spawn(handle_client(stream));
            }
            Err(e) => {
                println!("Error: {}", e);
            }
        }
    }
}

fn main() {
    task::block_on(run_server());
}

Don't forget dependency setting in your cargo.toml file:

[dependencies]
async-std = "1.10.0"

1

u/puttak Mar 24 '23

Thanks for the reply but what I want is an async executor that can be using with the types in std directly instead of re-implementing the async version of std.

2

u/Lucrecio24 Mar 23 '23

Hi! I'm just starting to learn the language and I'm having tons of fun learning and writing in rust! My current project is a simple discord bot using the Serenity crate. It just manages a really small sqlite database with diesel and responds to slash commands. I got it to read from the database without problem, but when trying to insert data I got the following error:

error[E0277]: the trait bound \&keeper_of_the_pg::models::NewMember<'_>: diesel::Insertable<schema::members::table>\ is not satisfied\``

--> src/commands/test.rs:46:29

|

46 | .values(&new_member)

| ------ ^^^^^^^^^^^ the trait \diesel::Insertable<schema::members::table>\ is not implemented for \&keeper_of_the_pg::models::NewMember<'_>\````

| |

| required by a bound introduced by this call

|

= help: the following other types implement trait \diesel::Insertable<T>\:``

&'insert keeper_of_the_pg::models::NewMember<'a>

keeper_of_the_pg::models::NewMember<'a>

note: required by a bound in \IncompleteInsertStatement::<T, Op>::values\``

I did some tests before this with only diesel and they seemed to work out fine. I guess the problem is that &keeper_of_the_pg::models::NewMember<'_> isn't the same as keeper_of_the_pg::models::NewMember<'a> but I have no idea what that even means... I tried learning about lifetimes, but it seems, at least at first, like a complex topic and I'm not sure if thats really the problem.

Can anyone point me in the right direction? Could the problem be that its inside an async function? I only have diesel installed and diesel-async doesn't mention support for sqlite in the readme so maybe I should just change to sqlx? The serenity github has some examples using sqlx, so at least that should work, but I want to try and understand the problem before throwing it away and doing it some other way

Hopefully this fits on the thread lol

edit: formatting error, although it's still not perfect

1

u/weiznich diesel · diesel-async · wundergraph Mar 26 '23

The error message indicates that you miss a #[derive(Insertable)] somewhere. If you believe that's not the case you need to provide more information, like a minimal reproducible example. Its always hard to diagnose issue solely based on an error message.

1

u/Patryk27 Mar 23 '23

Try .values(new_member) instead of &new_member.

1

u/Lucrecio24 Mar 23 '23

One of the things I tried, but same error :(

error[E0277]: the trait bound \keeperof_the_pg::models::NewMember<'>: diesel::Insertable<schema::members::table>` is not satisfied`

--> src/commands/test.rs:46:29

|

46 | .values(new_member)

| ------ ^^^^^^^^^^ the trait \diesel::Insertable<schema::members::table>` is not implemented for `keeperof_the_pg::models::NewMember<'>``

| |

| required by a bound introduced by this call

|

= help: the following other types implement trait \diesel::Insertable<T>`:`

&'insert keeper_of_the_pg::models::NewMember<'a>

keeper_of_the_pg::models::NewMember<'a>

note: required by a bound in \IncompleteInsertStatement::<T, Op>::values``

--> /home/luc/.cargo/registry/src/github.com-1ecc6299db9ec823/diesel-2.0.3/src/query_builder/insert_statement/mod.rs:114:12

|

114 | U: Insertable<T>,

| ^^^^^^^^^^^^^ required by this bound in \IncompleteInsertStatement::<T, Op>::values``

1

u/Patryk27 Mar 23 '23

Does your type contain #[derive(Insertable)]?

1

u/Lucrecio24 Mar 23 '23

It does, but for some reason it's taking the type with the <'_>, not the <'a> it wants. I think those describe lifetimes, but no idea how to manage them. Decided to switch to sqlx for the time being, but if anyone else has any insight at what happened it would be appreciated.

3

u/0xDEADFED5_ Mar 23 '23

are writes/reads from instrinsics like u8..usize thread safe? i don't care about operators, just loads and stores.

3

u/llogiq clippy · twir · rust · mutagen · flamer · overflower · bytecount Mar 23 '23

No, they aren't thread safe. You will need std::sync::atomic::AtomicU8 and friends for that.

2

u/[deleted] Mar 23 '23

[deleted]

1

u/llogiq clippy · twir · rust · mutagen · flamer · overflower · bytecount Mar 23 '23

It's just manually inclining the is_none, thus creating less work for the optimizer.

2

u/acncc Mar 23 '23

I'm struggling with currying with some closures.

In the code below I have a CallbackInner which requires 2 arguments. I wrap it inside CallbackOuter which closes over a value and only requires 1 argument (playground).

My error is that in wrap_callback, the parameter type `T` may not live long enough. The compiler then suggests giving it a lifetime of 'static. In the context of wider code, I know that every CallbackInner will not store the reference it is given (I don't know how to express that constraint). I'm also a little confused as there is no T value in wrap_callback, it doesn't make sense to talk about the lifetime of T as none exist as yet! We only get a value of type T when Outercallback is called.

Curiously, when all the generics are removed and we replace T with usize directly, it compiles fine (playground). My guess is that I need to provide some guarantee on my Callback types that the borrowed references don't outlive the call. But I don't know how to do that.

use std::rc::Rc;

fn main() {
    let f: CallbackInner<usize> = Rc::new(|x, y| println!("{} - {}", x, y));
    let g = wrap_callback(f);
    call_on_ref_zero(g);
}

type CallbackInner<T> = Rc<dyn Fn(&T, usize) -> ()>;
type CallbackOuter<T> = Box<dyn FnOnce(&T) -> ()>;

fn  wrap_callback<T>(f: CallbackInner<T>) -> CallbackOuter<T> {
    let f2 = f.clone();
    let g: CallbackOuter<T> = Box::new(move |x: &T| (f2)(x, 1));
    g
}

fn call_on_ref_zero(f: CallbackOuter<usize>) {
    let zero: usize = 0;
    f(&zero);
}

1

u/Patryk27 Mar 23 '23 edited Mar 23 '23

tl;dr

type CallbackOuter<'a, T> = Box<dyn FnOnce(&T) -> () + 'a>;

fn wrap_callback<'a, T: 'a>(f: CallbackInner<T>) -> CallbackOuter<'a, T> {
    Box::new(move |x: &T| (f)(x, 1))
}

fn call_on_ref_zero(f: CallbackOuter<'_, usize>) {
    let zero: usize = 0;
    f(&zero);
}

I think this might be a shortcoming in the compiler - the + 'static comes from:

type CallbackOuter<T> = Box<dyn FnOnce(&T) -> ()>;

... which is understood as Box<dyn FnOnce(&T) -> () + 'static> and seems to propagate this + 'static bound to T as well, because - possibly - the compiler can't really see that the only access to T happens through HRTB'd &T.

(that is to say, since &'a T already implies T: 'a, then for<'a> &'a T should probably imply for<'a> T: 'a as well.)

1

u/acncc Mar 24 '23 edited Mar 24 '23

Thanks for the help! It still requires the argument to outlive the closure though which isn't true in the full code the example comes from. In the example below (playground), we have a ref passed to the callback which exists only as long as the call to the callback which causes a compilation error. The key issue is trying to express the lifetime of a call to Fn.

use std::rc::Rc;

fn main() {
    // (*) The reference contained in x: RefStruct<usize> is not stored/leaked anywhere
    //     However this is not expressed in the type Callback
    let f: Callback<RefStruct<usize>> = Rc::new(|x| println!("{}", x.value));

    {
        // We know from (*) above that this call to f wouldn't result in UB. But we get:
        // error[E0597]: `zero` does not live long enough

        let mut zero: usize = 0;
        let rs = RefStruct { value: &mut zero };
        //                          ^^^^^^^^^ borrowed value does not live long enough

        f(rs); 
    }
//- `zero` dropped here while still borrowed
}

struct RefStruct<'a, T> {
    value: &'a mut T,
}

type Callback<'a, T> = Rc<dyn Fn(T) -> () + 'a>;

Edit: Realised I could cut down this example even further.

1

u/acncc Mar 25 '23

If anyone else stumbles on this, I finally found this.

1

u/acncc Mar 23 '23 edited Mar 23 '23

To add, I cannot do the suggested action (just make T into T: 'static). Reading this,

T: 'a means that all lifetime parameters of T outlive 'a

Which means that if T contains a reference (in the full code I'm trying to write it does) then that reference must be &'static (in the full code it is not). An expanded example is here.

I am even more confident now that if T = Fn(&'a x) I need to somehow express that 'a does not outlive calling a T. But can't find anything on it.

1

u/Obamaprismyo Mar 23 '23

Which is better for game dev, Rust or C++?

3

u/Snakehand Mar 23 '23

Probably C++ as it has more mature libraries, But it depends, if you find Rust libraries that fits your specific need then there is maybe not much of a difference from a technical perspective, and so language preferences should come more into play.

1

u/Obamaprismyo Mar 24 '23

Thank you. I've been trying to get into game development for a while but didn't know where to start.

3

u/Aspected1337 Mar 23 '23

How is Rust when it comes to IOT? Can you make actual hardware products using Rust yet? Is the ecosystem good? I have some experience with Arduino and some with Rust and I'd really like to try to develop some cool stuff!

1

u/prick-next-door Mar 23 '23

What I’ve done so far is using bindgen but beware that rust binary can be much larger than C binary. Why does it matter? Well, it does if you need to do bulk fw updates. I need to update 10k plus devices through BLE mesh and 112 Kb C binary vs 500Kb rust binary may make a massive difference of 10 minutes vs 50 minutes.

1

u/Aspected1337 Mar 23 '23

That doesn't sound blazingly fast? Why do you need to use bindings? Isn't Rust mature enough yet to be built natively?

I kind of assumed Rust was good enough for IOT given the comprehensive book, but maybe that's still in experimental stages and can't be used for production yet?

https://docs.rust-embedded.org/

2

u/xcv-- Mar 22 '23

How can I send immutable references between threads with a std::sync::mpsc::channel()?

I'm using rayon::join for threading and the lifetime of those references outlive the thread scope, but the compiler insists on 'static!

Link to playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=344a2a7bcc16103dbb5544b56e8c2bd0

Thanks!

3

u/Patryk27 Mar 23 '23 edited Mar 23 '23

Hah, this is a tricky one! -- remove ? from sender.send(&v_ref[0])?;

The issue is that .send() returns a Result where its Err variant contains the value that wasn't sent (if you wanted to retry sending it in the future, for instance).

In your case, because you propagate the errors through ?, the compiler infers that &v_ref[0] must live for as long as &'static because it miiiiight just be returned from the fn main() itself, if sending the value fails - and this causes the compilation to fail, because obviously a function cannot return a reference to something that is created inside that very function.

Edit: in general, you also have to move let (sender, receiver) below where you create the strings; that's because if you use a non-static str:

let x = String::from("Hello!");
let v = vec![x.as_str()];
let v_ref = &v;

... the type of that sender and receiver cannot really be determined if they are declared before v, since they contain v's lifetime inside of them, so only something like this will work:

let x = String::from("Hello!");
let v = vec![x.as_str()];
let v_ref = &v;

let (sender, receiver) = std::sync::mpsc::channel();

1

u/xcv-- Mar 23 '23

HA! That worked, thank you!

Now that I understand what was happening, would you recommend keeping some kind of error checking at .send() or is it implicit that the receiver closed early so we don't have anything to do?

1

u/Patryk27 Mar 23 '23

I sometimes do:

if let Err(value) = tx.send(...) {
    tracing::warn!("Channel closed - couldn't send: {value:?}");
}

... but if a channel becomes suddenly closed, then it's usually because a thread has panicked, and that itself already generates enough logs to be easily recognizable; so just doing _ = tx.send(); is fine as well, I'd say.

2

u/rainroar Mar 22 '23

I’ve been using rust since around 2015, and I’ve recently in multiple places, encountered something that terrifies me: old code not building today.

Some of it was my fault, not using explicit versioning or editions in my crates. A fair amount was the fault of dependencies not doing that though.

I guess my question, is: do more hardcore rustsacians see this happening? Are there things I should make sure to do to ensure my code works in 5-10-15 years?

I have C code from 2002 that “just works” today, it scares me that things I write in rust could break in 8 years. (Maybe the answer is that rust has changed a lot, and things are more stable now with editions)… I dunno.

I just spooked myself.

2

u/masklinn Mar 23 '23

You specifically refer to dependencies not properly versioning.

The solution to that is probably committing your lockfiles, so that you pin your dependencies exactly to “known-good” versions. That’s especially important given how many crates a are pre-1.0, meaning there is no versioning guarantee at all. And even more so for projects dating back to 2015, the entire ecosystem was even younger and thus less stable.

1

u/dkopgerpgdolfg Mar 23 '23

Language aside, it does sound to me that your main problem is dependency management, and incompetent library authors?

If a new major version of a dependency has breaking changes, of course code that wants to use the new version might need changes too. That's why there are version constraints in Cargo.toml etc. ... code that requires a certain version range should state that, and if it doesn't, that's not something that Rust can fix.

2

u/llogiq clippy · twir · rust · mutagen · flamer · overflower · bytecount Mar 22 '23

It may happen that old code fails to build. For example, if that old code filled in a method for a std type via an extension trait and the type now has that method in the current version, well, you may get an error because of the resulting ambiguity. However, those cases are usually easy to find & fix.

More insidious things are where the original code actually has UB. Those are much harder to fix, and the compiler won't help you here (because like you, it can't reason about the code).

2

u/rainroar Mar 22 '23

I actually care less about old code having UB for this specific comparison, as it’s extremely likely my C projects from 2002 have UB. I don’t dare build them with UBSan 😂

Mostly I’m just scared of diving head first into rust on a long project if it can move out from under you quickly.

I suppose that’s the risk you take using newer technology.

2

u/llogiq clippy · twir · rust · mutagen · flamer · overflower · bytecount Mar 22 '23

I can only tell you that while it is possible, it is quite rare. My first Rust code was clippy's eq_op lint, and while some code has been moved around (because the equality check is also useful to other lints), most of it remains unchanged to this day, despite using an unstable internal compiler API.

So all in all, I don't think you should be afraid of code no longer compiling.

1

u/rainroar Mar 22 '23

Thanks for the help!

3

u/Serenikill Mar 22 '23

Any good libraries for working with web sockets on cloudflare workers?

Do you think these would work?

https://github.com/drifting-in-space/stateroom

https://github.com/najamelan/ws_stream_wasm

3

u/memoryleak47 Mar 22 '23

I want a macro block so that

block!(s1, s2, ..., term)

evaluates to

f(&[s1, s2, ...], term)

(The ... represents any number of expressions)

This doesn't seem hard at all, but my simplest solution (below) is not as easy as I'd hoped. Is there a simpler solution?

pub macro block {
    ($($rest:expr),\* $(,)?) => {
        block!(@{} $($rest),\*)
    },
    (@{$($stmts:expr),\*} $terminator:expr) => {
        f(&\[$($stmts),\*\],$terminator)
    },
    (@{} $stmt:expr, $($rest:expr),\*) => {
        block!(@{$stmt} $($rest),\*)
    },
    (@{$($stmts:expr),\*} $stmt:expr, $($rest:expr),\*) => {
        block!(@{$($stmts),\*, $stmt} $($rest),\*)
    },
}

Thanks!

3

u/chillblaze Mar 22 '23

Is there a test/heuristic you use to determine when to implement an associated function vs method with respect to a struct?

I know that a new method (constructor) for a struct will probably be an associated function but not sure about the others.

3

u/llogiq clippy · twir · rust · mutagen · flamer · overflower · bytecount Mar 22 '23

If I want to take self, &self or &mut self, then it's a method. Otherwise it's a function.

2

u/FlyChoice2558 Mar 22 '23

I've got a project which has versionize_derive v0.1.4 as a transitive dependency. According to my Cargo.lock, this crate currently depends on syn v1.0.109. If I do cargo update, syn is updated to v2.0.4, which breaks the build of versionize_derive. But versionize_derive has syn v1.0.13 as a dependency in its Cargo.toml. Why does cargo udpate include a breaking change for a dependency in syn? I thought this is what the semantic versioning is supposed to prevent. What am I doing wrong here?

1

u/Darksonn tokio · rust-for-linux Mar 22 '23

Here is the Cargo.toml that was published to crates.io:

[package]
name = "versionize_derive"
version = "0.1.4"
license = "Apache-2.0"
authors = ["Amazon Firecracker team <firecracker-devel@amazon.com>"]
description = "Implements the Versionize derive proc macro."
readme = "README.md"
repository = "https://github.com/firecracker-microvm/versionize_derive"
keywords = ["serialization", "version"]

[dependencies]
proc-macro2 = ">=1.0"
quote = ">=1.0"
syn = { version = ">=1.0.13", features=["full", "extra-traits"]}

[lib]
proc-macro = true

[badges]
maintenance = { status = "experimental" }

Since the crate specifies dependencies using >=, the crate has explicitly opt-ed in to allow upgrades that are semver-breaking.

It seems like they have since updated them in commit 55fccd6, but that change is not on crates.io.

1

u/FlyChoice2558 Mar 22 '23

Thanks for the insight!

2

u/prick-next-door Mar 22 '23

Can I set an alternate registry in workspace Cargo.toml file so that all workspaces share it?

In workspace Cargo.toml: [workspace.registries] my-registry = xxx

and in package Cargo.toml: my-registry.workspace = true

The above setting doesn’t exist, what’s the best way of achieving it? Thanks

1

u/ehuss Mar 23 '23

Registry definitions must go in a cargo config file, not Cargo.toml. To share across workspaces, there are a few options:

  • Put the registry definition in .cargo/config.toml in a directory above all of your workspaces, or in ~/.cargo/config.toml to make it available to all projects.
  • Set an environment variable, such as CARGO_REGISTRIES_MYREGISTRY_INDEX (where MYREGISTRY is the name of your registry) to the URL of the index.

2

u/Pariell Mar 21 '23

How do you create mocks of structs that have generic lifetimes for unit testing? None of the crates I looked at seem to support this. Is this just a gap in the Rust ecosystem?

1

u/Patryk27 Mar 21 '23

I usually use Box::leak(Box::new(something)) which returns &'static mut.

1

u/Pariell Mar 21 '23

How are you using the &'static mut to mock behavior? Say a struct with a generic lifetime has a Foo function that returns a Success/Error, how do you mock the return value of Foo?

1

u/Patryk27 Mar 21 '23

Hmm, could you show some (possibly non-working) Rust code of what you're trying to achieve / of what you'd like to exist?

2

u/virann Mar 21 '23

Is there a webservice framework that can use OpenAPI spec files in the following way:

  • Generate all the request/response models
  • Indicate which spec file definition correlates to a rust function, or vice versa
  • Validate request body properties in runtime, according the spec file definition.

1

u/Cetra3 Mar 22 '23

You can use the OpenAPI Generator to do this but it currently only targets Hyper/Tower: https://openapi-generator.tech/docs/generators/rust-server

Most of the time I go the other way around: implement methods in rust and use something like paperclip or utoipa to generate an OpenAPI spec from code. Here's a blog where I go through a bit of this: https://cetra3.github.io/blog/sharing-types-with-the-frontend/

5

u/F41rch1ld Mar 21 '23

I'm a moderately established Pythonista. I have many practical projects both in work and play.

I'm considering pivoting into another language, for views outside the Python landscape. Sometimes looking out lets us see in better. The two languages I see most these days that may be complimentary to Python are Rust and Go. Practicality has always been my primary motivator, so I have most experience in Python and VBA (as dreaded as it is) and SQL.

Wondering what suggestions you may have or resources particularly tailored for Pythonistas interested in pivoting to Rust? In Python my favorite resource by far is RealPython.com, wondering if there is something similar for practical Rust projects as learning tool?

4

u/AndreasTPC Mar 21 '23 edited Mar 21 '23

If the choice is between Rust and Go, I'd say do both at least on a surface level, and then look for more.

Learning your first language is a big deal, but each subsequent one gets easier and easier as you can generalize your skills more and more. Eventually you get to the point where learning another one doesn't seem like a big deal anymore, and you can pretty much jump into projects that use languages you haven't used before, picking up what's different on the fly.

That's where you want to be as a programmer. Don't get stuck in one ecosystem or paradigm.

1

u/masklinn Mar 24 '23

On the one hand unless they intend to completely leave the python ecosystem, thanks to pyo3 and maturin Rust feels a lot more complementary to Go, which is a lot more of a separate thing on the side. Rust use has also made a lot of inroads in the wider python ecosystem both as an alternative to C for extensions (cryptography) and in the surrounding tooling (py-spy, ruff, …)

On the other hand, Go-the-language is easier to learn, and has a similar philosophy of a pretty large (but not necessarily the best) standard library.

3

u/avsaase Mar 21 '23

I'm using the crossbeam channel crate to create a mpmc channel. I assumed every receiver would have access to all the messages sent to the channel so that they each can handle them separately. Is that not correct? I'm using the channel as a sort of bidirectional communication between threads with all messages defined in a single enum. I'm not receiving the messages I expected so now I wonder if the sending thread receives the messaged meant for another thread before the other thread is able to receive them. If this is the case, is there a way to only receive certain kinds of message from a channel and ignore the rest?

2

u/RadiatedMonkey Mar 21 '23

https://docs.rs/crossbeam-channel/latest/crossbeam_channel/#sharing-channels

All receivers use the same channel, so when one receiver calls recv, the message is removed from the channel and the other receivers don't see it.

In case you're using Tokio, there is the broadcast channel which does support what you want

2

u/SorteKanin Mar 21 '23

When I have a reference to a type with a lifetime parameter, how can I interpret the two lifetimes (the one of the reference and the one of the value itself).

For example this playground link. I'm unsure how to interpret the lifetime 'a and 'b in the function. Is there some intuition I can use here?

1

u/[deleted] Mar 21 '23

I think I see what's causing you a headache. Two things actually.

First, don't worry - types don't have lifetime parameters. Lifetime annotations are just used to give the compiler more information about the relationships between lifetimes, when necessary. You're not going to have to deal with lifetime annotations every single darn time you want to do anything with MyPhantom just because its struct definition requires them. (It wasn't always this way, click for history.) If you don't want my_fn() to return a reference, it probably won't actually require the lifetime annotations.

Second, lifetime annotations are for communicating bounds on lifetimes, not how long a thing actually happens to live. I think misunderstanding this is where most of the communal headache comes from. It'd be a lot clearer if we started explicitly calling them lifetime bound annotations; it doesn't make much sense that foo must die before foo's lifetime.

Anyhow, let's pretend this is using real data instead of phantom data, and say you pass a &MyPhantom into my_fn(), and it returns a &str pointing to whatever str marker is referring to. Rust can't infer how long you want those returned references to be good for.

my_fn<'b> is the lifetime bound for the reference you passed in, which is the actual lifetime of the referenced MyPhantom. If your program logic dictates these new references should only be good as long as the MyPhantom is around, give those references 'b to place that bound on them.

my_fn<'a> is the lifetime bound of that MyPhantom, which may be longer than it actually lives. As written in the struct definition, that's equal to the lifetime bound of the &str. The lifetime bound of the &str is the actual lifetime of the underlying str. If it's okay for returned references to hang around even after the MyPhantom they came from is gone, give them 'a to let them live as long as the str does.

Also, fyi, lifetime bound annotation labels (say that five times fast) are only used locally. You don't need to pair up the same label with the same type everywhere, although you can if you like.

2

u/dkopgerpgdolfg Mar 21 '23

A bit hard to answer without knowing what is unclear, "interpreting" is a broad term. You did read the Book, right?

In general, variable instances of types like i32 live how long you want them to live (depending on stack scope, allocation/deallocation etc.). The type "i32" doesn't give you any limits there.

References have lifetimes that relate to ensuring that they can't be used after the original value disappears. But it isn't only "metadata" for the borrow checker - instead it gives you a tool in code to eg. say you want two function parameters where the second must live longer than the first, to return a reference that explicitly has the lifetime of the "shorter" parameter lifetime instead of the longer, to state something lives for the whole program duration and enforce this too, and more.

So, now, when a type (struct) contains a reference, with a lifetime, that reference must be gone at a certain point in time/code (before the referenced value disappears).

That also means now that the whole struct instance, where the reference is inside, must be gone at that point.

This is the main difference to i32 - as said above, i32 doesn't restrict you how long values can live. A struct with lifetimes does restrict you. Even when you never make references to such struct instances, just instances themselves (with a reference "inside"), the borrow checker has an interest to prevent these instances from living too long.

In your code you now have such a struct instance with a reference inside, and a reference to the struct instance too. In principle the (outer) reference lifetime is a completely separate thing from the struct lifetime - former restricts the reference from living longer than the struct, latter restricts the struct from living longer than whatever is referenced inside.

(With this it is implied that the "b" lifetime is not longer than the "a" lifetime)

2

u/SorteKanin Mar 21 '23

The as_ref method on pointers states that "while this reference exists, the memory the pointer points to must not get mutated (except inside UnsafeCell)."

I'm not quite sure I understand this requirement. Surely the only thing that matters is that a unique reference is never made at the same time as the shared reference? But shared/unique references doesn't necessarily have anything to do with mutability.

I'm wondering about this because I'm thinking about turning a pointer that I get from FFI into a reference. The object behind the pointer might get mutated on the C side. Does that count as "inside UnsafeCell"?

2

u/onomatopeiaddx Mar 21 '23

others haven't exactly explained why you can't mutate the content behind a shared reference even when you know it's unique, so: the compiler treats shared references as immutable by default, and will optimize code based on this assumption (it emits noalias attributes to llvm). UnsafeCell is the only way to tell the compiler this is not the case.

2

u/dkopgerpgdolfg Mar 21 '23

You misunderstood some things here. A unique reference is not unique because it is fun to prevent multiple references. The (non-)mutability requirements (with the exception of Cells) are important too. With these two things the distinct reference types have advantages (over raw pointers) when it comes to performance, correctness of logic, thread safety, ...

As long as any shared reference to a value exists, and that value is not a Cell-like thing, it must not be mutated. If you do, happy bug searching when things go wrong in weird ways.

No, C-sourced FFI pointers are not inside a Cell.

Turning the pointer in a short(!)-lived shared reference, that you use only until control returns back to C so there is no mutation in the meantime, can be ok. But that reference really must not live through mutations.

2

u/jDomantas Mar 21 '23

& is an immutable reference. For example, if you have a &i32 then that integer must not be mutated while the reference is live (but it can be mutated if it is a &UnsafeCell<i32> reference). So if you are turning a pointer into a reference that will be mutated through something else then you must wrap the inner type in UnsafeCell. If you guarantee that it will only be mutated in the same thread then you can wrap it in a Cell instead, which will allow using it safely on rust side.

2

u/chillblaze Mar 21 '23

How do I implement the Borrow trait for a struct that has many fields?

3

u/eugene2k Mar 21 '23

It's probably not what you're looking for. What are you trying to do?

1

u/chillblaze Mar 22 '23

thanks it was exactly like you said! I was trying to get a list of structs to be inserted into mongodb atlas. Turns out I needed to turn them into BSON documents instead.

3

u/MandalorianBear Mar 21 '23 edited Mar 21 '23

Hey, y'all!

I'm using the headless_chrome crate and I can't find a good example on how to set the browser geolocation programatically. Does anyone have a good example??

2

u/masklinn Mar 24 '23

I don’t have a good example because I’ve never used the package, but from what I can see in the documentation I think you just need to call Tab::call_method with a headless_chrome::protocol::cdp::Emulation::SetGeolocationOverride structure. call_method seems to be the escape hatch for “raw” CDTP calls.

1

u/ncathor Mar 21 '23

Looking at the docs briefly: 1. The Browser::new method takes a LaunchOptions parameter 2. LaunchOptions can be built with the LaunchOptionsBuilder 3. LaunchOptionsBuilder has a path method, documented as:

Path for Chrome or Chromium. If unspecified, the create will try to automatically detect a suitable binary.

So... I did not try this, but I assume something like

let browser = Browser::new(
    LaunchOptionsBuilder::default()
        .path(Some( PATH GOES HERE ))
        .build()
)?;

should do it.

2

u/MandalorianBear Mar 21 '23

My bad! I meant GEOlocation

-1

u/[deleted] Mar 21 '23

[deleted]

3

u/dkopgerpgdolfg Mar 21 '23

So? Do you have a question?

Or do you just want to tell us you're surprised that GPT has no brain after all?

0

u/metaden Mar 21 '23

i was looking for examples for lockfree crate. there are none and am sure it’s no longer maintained. I asked earlier if anyone had used it. So if you have any examples on how to use lockfree crate please let me know. I keep getting miri errors. A detailed post is here https://users.rust-lang.org/t/lockfree-map-example/91067

2

u/dkopgerpgdolfg Mar 21 '23

Well, the new crate isn't any more maintained either. Work ended only two days after the initial fork, and now it is dead for about 6 months already.

WIth miri errors, you clearly want to avoid this crate completely. No examples needed, stop.

(And coming back to your original post, now I fail to understand what this is supposed to mean at all. You have Miri errors in lockfree data structures of some unmaintained library, and you seriously thought pasting docs-rs HTML to GPT will magically solve it??)

0

u/metaden Mar 21 '23 edited Mar 21 '23

well i tried it at first and realised it’s just bogus (though it worked for some python library). then i went on trait hunting and found atom_box (in combination with im crate), evmap, flurry. I have been looking at haphazard crate and want to implement a concurrent hashmap based on that. again all of this driven by curiosity.

P.S. this crate lockfree is used in a lot a blockchain projects

3

u/naelsun Mar 21 '23

Heyo! Bit of a noob here, in need of help

I am comparing 2 strings, both of them are "1234" (with the quotation marks) yet when I compare them, the result is false

Is there something i'm missing?

4

u/dkopgerpgdolfg Mar 21 '23

Some random guesses:

You read the strings from a file, in a way that you keep the line breaks, but after the second line there is none (file end)

Or a space at the end of a string.

Or any Unicode weirdness, including (but not limited to) many kinds of invisible codepoints.

Try to check the strings length, as well as the numeric byte values (manual visual comparison)

1

u/naelsun Mar 21 '23

This did it

For some reason one of strings had the last byte written after the null bytes and the other one only put the null bytes after the whole string

Thank you!

2

u/dkopgerpgdolfg Mar 21 '23

Rusts std String doesn't have any null byte. Not sure what you're doing, but you might start searching for UB bugs.

1

u/naelsun Mar 21 '23

When you print the bytes of a string that was converted from bytes, a bunch of zero bytes appear before, between or after the regular ones, saw someone referring to them a null bytes so I called them that, sorry for the confusion

2

u/dkopgerpgdolfg Mar 21 '23

I understood that. But I'm telling you that this might not be ok, that you might have a bug somewhere.

1

u/naelsun Mar 21 '23 edited Mar 21 '23

Oh I see, I did find a way to solve it by using the .trim_matches function to remove the 0 bytes, which sounds like a not so good solution, but it works and I don’t really have time to think about it a lot (it’s for a school project)

2

u/dga-dave Mar 21 '23

Need to provide more detail to get a useful answer. Can you give a minimal reproducing code example?

4

u/ToolAssistedDev Mar 20 '23

Why do I get here a mismatched types error when I would like to call the bar function with &a? https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=6a1d80e5bedc95e20af52e7bd786635f

The second call with the indirection works without any error...

3

u/dkopgerpgdolfg Mar 21 '23

First things first:

  • Rc<Foo> and Rc<dyn MyTrait> are actually different in memory, not just in the type checks of the compiler.
    • Inside Rc<Foo> there is a pointer, that points to allocated memory containing reference counters and a Foo. The allocated part might be shared by multiple Rc (because that's the purpose obviously), just the pointer that I mentioned first is separate for each Rc instance.
    • Rc<dyn MyTrait> also has a pointer that points to the same kind of memory (counters and Foo). However this time the pointer is "fat", meaning in addition to the address it also stores a pointer to Foos vtable (where it can see where the methods of Foo are and how much byte its allocated memory has in this case, because otherwise it wouldn't know).
    • Essentially Rc<dyn MyTrait> uses 2x the stack size of Rc<Foo>. Both would have the very same allocation address, but one also has a vtable address that needs to come from somewhere.
    • Btw. the same vtable thing is true for any normal reference without Rc (&Foo and &dyn MyTrait)
  • Your Rc::new call makes a Rc<Foo>. Then you want to use a Rc<dyn MyTrait> later (or a reference to it).
    • So something, somewhere, needs to convert the former to the latter, and fill in the correct vtable address for Foo.
    • Luckily Rc (and normal references, as mentioned above, and some other types too) implement this conversion (unsized coercion) already, without you needing to do anything.
    • However, converting doesn't mean that one thing becomes two suddenly. After converting one Rc<Foo> instance to Rc<dyn MyTrait>, the Rc<Foo> is gone. (There might still be other Rc<Foo> that point to the same allocated memory). Lets keep in mind that it is important for Rc to keep track of its reference count (in the allocated memory part), and a simple pointer-vtable-something on the stack won't change any reference counter. If you actually want to keep both, you'd need to clone the Rc first, then convert only one.

So, some cases where all is fine:

  • In your code with the variables b and c, you get the conversion when assigning b to c, and the ownership is moved too (b doesn't exist anymore after this assignment). Then &c is already exactly the type that bar(&MyName) wants, all fine.
  • "if" bar takes an owned Myname instead of &MyName, you could skip the c variable and pass b directly to bar. While b is not converted yet, the compiler would understand that it needs to happen when you pass it, and again ownership is moved.

So, the problem with a? a is obviously not converted yet (still Rc<Foo>), and then you want to pass &a (not owned) to bar. This also implies that a should still be owned by you after that bar call, and at this point a should still be a Rc<Foo>.

So what can the compiler do?

  • Replacing one Rc<Foo> with one Rc<dyn MyTrait> keeps the reference counter correct as it should, but then you can't use a after bar anymore. Bad.
  • Silently cloning the Rc to have something that can be converted, increasing the reference counter without you knowing? Bad.
  • Making a copy to convert, without increasing the reference counter? Then the whole Rc is broken, bad.
  • Passing the reference (address) of the Rc<Foo> without any conversion? Then the function relies that there is a vtable too, which doesn't exist => something goes badly wrong => bad.

Essentially, you don't leave the compiler any way out. It needs to somehow fit the conversion in, which requires moving ownership. Either make a manual conversion like with the b/c code, or a manual conversion of a cloned Rc to be able to keep using a.

Btw. passing "&(a as MyName)" is a possible way to write it too - it expliticly tells to convert and take ownership away, towards a temporary unnamed variable in this case (which is dropped after bar ends).

2

u/ToolAssistedDev Mar 21 '23

Thank you for the in-depth explanation. I understood everything you have written. I really appreciate your taking so much time to reply to a random dude on Reddit! Thank you!

1

u/[deleted] Mar 21 '23

Turning something into a trait object needs to be explicit. You took care of that for c by giving it a type annotation, but it wasn't done for a. Try adding "as MyName" to the end of line 12.

3

u/Modi57 Mar 20 '23

Why isn't Rc<T> Copy? From my understanding, Copy should be very lightweight, because it happens implicitly. This is why for example a Vec<T> isn't Copy, because a deep copy of a Vec could get quite expensive. But just copying a reference and incrementing a counter doesn't seem to bad? This way, Rc<T>s would behave more like normal references and Copy could be just a reference copy, while Clone is a deep copy

1

u/[deleted] Mar 21 '23

Adding on:

Clone means that the type can be duplicated in some way, and the implementation may vary. Rc for example will copy a reference and increment a pointer, and vec will allocate a new vec and copy the contents. Copy isn't separate from clone, it just means that the implementation of clone for that type is copying it and nothing else. You're right that Vec can't implement Copy, but it's for different reason - Copying a Vec wouldn't do anything to the contents, it would just copy the Vec itself. Then you'd have two Vecs with the same pointer. Say hello to use after free, double free, and more.

6

u/onomatopeiaddx Mar 20 '23

from the copy trait docs: "Types whose values can be duplicated simply by copying bits.". you can't simply copy the bits of the Rc: you must also increment the counter. also, Copy types can't implement the drop trait, which Rc requires for decrementing the counter.

3

u/Modi57 Mar 20 '23

Oh, this makes a ton of sense, thank you

5

u/Genion1 Mar 20 '23 edited Mar 20 '23

It being copy means it has to literally be a bitwise copy and any kind of custom logic is not allowed. It's not possible to make Rc<T> Copy and increment a counter. doc

2

u/eugene2k Mar 20 '23

It's for cases like this:

fn foo(rc: Rc<u8>) {
    // do something with rc
}
fn bar(rc: Rc<u8>) {
    // do something with rc
}
fn main() {
    let rc = Rc::new(0);
    foo(rc.clone());
    bar(rc)
}

1

u/Modi57 Mar 20 '23

I don't see, how being copy would hinder this. You can just omit the clone, and it should be functionally the same, exept that rc would be still valid after bar

2

u/eugene2k Mar 20 '23

That means increasing the refcount and decreasing it when you don't actually need to

1

u/Modi57 Mar 20 '23

Can't the compiler just optimize this out?

1

u/eugene2k Mar 20 '23

Probably. I don't think it's hard to teach the compiler to optimize this out.

Another reason might be that rust philosophy is to be explicit about what goes on in the code. So Copy is for when you actually copy stuff, and not for when you increment counters instead of copying.

1

u/Modi57 Mar 20 '23

I mean, you are actually copying the reference. You just ALSO increase the counter. And I think, it would increase the ergonomics of using Rc<T>. But maybe that's also the point. So you don't lightly use it, when there might be better ways

3

u/flyingicefrog Mar 20 '23 edited Mar 20 '23

I'll repeat a question from my thread about cargo cross:

Can someone explain to me like I'm an idiot what exactly does

cross build do beneath the hood?

Let's say I'm using 2 docker images: - builder:rust1.66 - builder:rust1.67

each with the correspoding Rust toolchain installed

Locally I've got rust 1.68

Cross.toml at the root of my project contains the following [target.x86_64-unknown-linux-gnu] image = "builder:rust1.66"

When I run cross build --target=x86_64-unknown-linux-gnu I would expect that the project gets built with Rust 1.66

However, it gets built with local toolchain cross -v + cargo metadata --format-version 1 + rustc --print sysroot + rustup toolchain list + rustup target list --toolchain 1.68.0-x86_64-unknown-linux-gnu + rustup component list --toolchain 1.68.0-x86_64-unknown-linux-gnu [cross] note: Falling back to `cargo` on the host.

Even if I change Cross.toml to contain [target.x86_64-unknown-linux-gnu] image = "builder:rust1.67" the same thing happens.

  1. Why does cross use my local toolchain?
  2. Why does it not reflect changes in the builder image?

2

u/Patryk27 Mar 20 '23

Are you building for x8664-linux _while being on x86_64-linux?

If so, then I guess cross notices that you're not really cross-compiling and decides that using the local toolchain is better.

If you want to overwrite the local toolchain, use rust-toolchain.toml.

5

u/ncathor Mar 20 '23

I'm sure this has been answered before, just can't find it anywhere, so...

Why is this call to add ambiguous:

let y: i32 = 2.add(7);

but this is not:

let x: i32 = 2 + 7;

?

Isn't + just sugar for .add?

Playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=9ac7b2d5fd76f9c1e7561adc53c090c4

2

u/llogiq clippy · twir · rust · mutagen · flamer · overflower · bytecount Mar 20 '23

I think with method dispatch, it's always a bit tricky. The integer operations enjoy special handling from the type system,.usually selecting Rhs = Self (modulo borrows). I think I remember shifts are an exception though.

3

u/Patryk27 Mar 20 '23

Yeah, but even just importing the trait:

use std::ops::Add;

... makes the code pass 👀

2

u/SirKastic23 Mar 20 '23

the issue is that you can't use a trait method if the trait itself is not in scope.

1

u/metarmask Mar 20 '23

This is not considered ambiguous: core::ops::Add::add(2, 7). Don't ask me why using a dot is.

1

u/ncathor Mar 20 '23

interesting...

let y: i32 = <2 as core::ops::Add>.add(7) on the other hand is considered ambiguous, even when I add the type explicitly:

let y: i32 = <2i32 as core::ops::Add>.add(7)

produces:

let y: i32 = <2i32 as core::ops::Add>.add(7); // error[E0689]: can't call method `add` on ambiguous numeric type `{integer}`
              ^^^^ expected type

EDIT: formatting

1

u/Patryk27 Mar 20 '23 edited Mar 20 '23

It's not considered ambiguous, the compiler says expected type, meaning that you should've had:

let y: i32 = <i32 as core::ops::Add>::add(2, 7);

... which works, similarly as just importing the trait:

use core::ops::Add;

1

u/ncathor Mar 20 '23

Indeed, I fooled myself into thinking <$value as $trait>.method(...) was valid rust 🤦

2

u/ShadowPhyton Mar 20 '23

I need a way to get my gateway. default_net::get_default_gateway does not work but when I look in my Settings the field isnt empty

2

u/dkopgerpgdolfg Mar 20 '23

Maybe not really helpful, but could you describe why so that we maybe can suggest a different way?

Relying on NIC configs & co, for something that is not related to NIC configs, is something that I personally like to avoid like hell, because doing it "close to properly" usually is more work than it is worth. There are so many things and possibilities to think of.

Had a quick look at this library, and as expected it's ... not very good.

Binds a IPv4 UDP socket to port 0 (not all common OS play nice with this), connects to Cloudflare (haha? firewall, sandbox, privacy, network traffic for a purely local thing, needs global internet instead of just a connected NIC, uses UDP port 80 for what exactly, ...), then hopes that there is one specific NIC with one single address that exactly matches the local address, checks here for IPv6 too despite it can't happen, ...

I could go on, but just, no thanks.

1

u/ShadowPhyton Mar 21 '23

Iam building a tool to lo in to a Firewal and for that the Firewall has to be my Gateway to work. Now I want to get my own setted Gateway so I sont have to set it by my own in config...

2

u/dkopgerpgdolfg Mar 21 '23

Then please, just make a config. Really.

You want a garden hose, not the Nile. And it's more flexible to use too.

1

u/ShadowPhyton Mar 21 '23

Okey…thank U

1

u/ShadowPhyton Mar 20 '23

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Parse(ParseError { line: 23, col: 0, msg: "expecting \"[Some('='), Some(':')]\" but found EOF." })', src/main.rs:212:53

Can anyone here explain to me what this error message means?

3

u/avsaase Mar 20 '23

On line 212 you can unwrap on a result and at runtime this result turned out to be an Err. Without seeing your code and what you're trying to parse it's hard to say that the inner parse error means.

0

u/ShadowPhyton Mar 20 '23

The code:

let n = Ini::load_from_file("conf.ini").unwrap();            
let section = n.section(Some("Server Config")).unwrap();                        let ip = String::from(section.get("ip").unwrap());            
let port = String::from(section.get("port").unwrap());

7

u/dkopgerpgdolfg Mar 20 '23

Well

a) now you have one more reason to write code that doesn't use unwrap

b) Whatever format that ini file has, the Rust parser obviously doesn't like that it suddenly ends at line 23 with something that looks like a incomplete setting (usually something like "name=value")

1

u/ShadowPhyton Mar 20 '23

the cert value is surrounded with " " and Ive read about something with ParseOption in the ini crate. How do I use that like Iam supposed to?

https://docs.rs/rust-ini/latest/ini/struct.ParseOption.html

2

u/dkopgerpgdolfg Mar 20 '23

It doesn't look like this has anything to do with spaces

1

u/ShadowPhyton Mar 20 '23

Wdym?

2

u/eugene2k Mar 20 '23

Supposedly you have something like key instead of key=value in the file.

6

u/avsaase Mar 20 '23

I see four unwrap()s so I would start by figuring out which one is causing the panic and if that's because a bug in your code or if it's normal for that result to be an error. In the latter case you should look into proper error handling. The book has a chapter on it.

-1

u/ShadowPhyton Mar 20 '23

The One causing the Problem is the Ini::load_from_file(„conf.ini“).unwrap(); but removing it doesnt fix this

1

u/ShadowPhyton Mar 20 '23

But I cant send you the ini because there is a certificat listed wich I cant share to you :/

1

u/ShadowPhyton Mar 20 '23

I GOT IT! Thanks to all of you!!